From jwaters at openjdk.org Mon Apr 1 10:36:54 2024 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 1 Apr 2024 10:36:54 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v49] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: On Sun, 21 Jan 2024 19:50:16 GMT, Phil Race wrote: >> Fixed the formatting (at least in the marked cases), but am unsure what you mean by set directly? > >> Fixed the formatting (at least in the marked cases), but am unsure what you mean by set directly? > > See my comment > "like in my recoded case above, we no longer need the "pData" var which was there only because that name > is magically known to the macro." > > so skip (and get rid of) the pData var and just set the target var directly I've switched the marked cases to use direct setting and bypass pData, should I do the unmarked ones too? @prrace ------------- PR Comment: https://git.openjdk.org/jdk/pull/15096#issuecomment-2024592358 From jwaters at openjdk.org Mon Apr 1 10:36:54 2024 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 1 Apr 2024 10:36:54 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v70] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: On Thu, 28 Mar 2024 07:36:04 GMT, Julian Waters wrote: >> We should set the -permissive- flag for the Microsoft Visual C compiler, as was requested by the now backed out [JDK-8241499](https://bugs.openjdk.org/browse/JDK-8241499). Doing so makes the Visual C compiler much less accepting of ill formed code, which will improve code quality on Windows in the future. > > Julian Waters has updated the pull request incrementally with four additional commits since the last revision: > > - Labels to empty line in awt_Window.cpp > - Labels to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp Bumping ------------- PR Comment: https://git.openjdk.org/jdk/pull/15096#issuecomment-2029552898 From ihse at openjdk.org Mon Apr 1 20:32:26 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 1 Apr 2024 20:32:26 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v70] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: On Thu, 28 Mar 2024 07:36:04 GMT, Julian Waters wrote: >> We should set the -permissive- flag for the Microsoft Visual C compiler, as was requested by the now backed out [JDK-8241499](https://bugs.openjdk.org/browse/JDK-8241499). Doing so makes the Visual C compiler much less accepting of ill formed code, which will improve code quality on Windows in the future. > > Julian Waters has updated the pull request incrementally with four additional commits since the last revision: > > - Labels to empty line in awt_Window.cpp > - Labels to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp Yes, fix all pData in areas touched by your changes. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15096#issuecomment-2030503535 From mikael at openjdk.org Mon Apr 1 21:06:08 2024 From: mikael at openjdk.org (Mikael Vidstedt) Date: Mon, 1 Apr 2024 21:06:08 GMT Subject: RFR: 8320799: Bump minimum boot jdk to JDK 22 Message-ID: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> With the JDK 22 GA out it's time to bump the minimum boot JDK version for mainline/JDK 23. Testing: tier1-5, GHA ------------- Commit messages: - 8320799: Bump minimum boot jdk to JDK 22 Changes: https://git.openjdk.org/jdk/pull/18563/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18563&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320799 Stats: 12 lines in 3 files changed: 0 ins; 0 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/18563.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18563/head:pull/18563 PR: https://git.openjdk.org/jdk/pull/18563 From iris at openjdk.org Mon Apr 1 21:22:59 2024 From: iris at openjdk.org (Iris Clark) Date: Mon, 1 Apr 2024 21:22:59 GMT Subject: RFR: 8320799: Bump minimum boot jdk to JDK 22 In-Reply-To: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> References: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> Message-ID: On Mon, 1 Apr 2024 16:16:50 GMT, Mikael Vidstedt wrote: > With the JDK 22 GA out it's time to bump the minimum boot JDK version for mainline/JDK 23. > > Testing: tier1-5, GHA Thanks! ------------- Marked as reviewed by iris (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18563#pullrequestreview-1972143983 From erikj at openjdk.org Mon Apr 1 21:52:58 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 1 Apr 2024 21:52:58 GMT Subject: RFR: 8320799: Bump minimum boot jdk to JDK 22 In-Reply-To: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> References: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> Message-ID: On Mon, 1 Apr 2024 16:16:50 GMT, Mikael Vidstedt wrote: > With the JDK 22 GA out it's time to bump the minimum boot JDK version for mainline/JDK 23. > > Testing: tier1-5, GHA Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18563#pullrequestreview-1972191074 From prr at openjdk.org Tue Apr 2 04:38:27 2024 From: prr at openjdk.org (Phil Race) Date: Tue, 2 Apr 2024 04:38:27 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v70] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: On Thu, 28 Mar 2024 07:36:04 GMT, Julian Waters wrote: >> We should set the -permissive- flag for the Microsoft Visual C compiler, as was requested by the now backed out [JDK-8241499](https://bugs.openjdk.org/browse/JDK-8241499). Doing so makes the Visual C compiler much less accepting of ill formed code, which will improve code quality on Windows in the future. > > Julian Waters has updated the pull request incrementally with four additional commits since the last revision: > > - Labels to empty line in awt_Window.cpp > - Labels to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp I've been monitoring it but saw no reason to pay particular attention until everything I raised is addressed. Including the cases identical to the ones I explicitly identified (which I suppose is what is meant by your question "should I do the unmarked ones too ?"). At some point you do tire of typing "ditto" :-) It has been long enough that I will need a little time to remember and re-review properly once the requested changes are in place. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15096#issuecomment-2031068525 From jkern at openjdk.org Tue Apr 2 09:02:01 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 09:02:01 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Fri, 29 Mar 2024 05:23:57 GMT, Julian Waters wrote: > > The rest of the changes are needed because of using utilities/compilerWarnings_xlc.hpp the compiler is much more nagging about ill formatted printf > > Did you mean compilerWarnings_gcc.hpp? Yes, you're right. I fixed it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18536#issuecomment-2031447588 From jkern at openjdk.org Tue Apr 2 09:16:59 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 09:16:59 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Fri, 29 Mar 2024 07:59:05 GMT, Thomas Stuefe wrote: >> While looking at this, I noticed that my question in https://github.com/openjdk/jdk/pull/14146#discussion_r1207078176 and followups had never been answered. Do you know the answers now? >> >> Quoting myself: >> >>> So, we do this only for malloc? Not for calloc, posix_memalign, realloc etc? What about free? >>> Removing that define and hard-coding it here assumes ... pointers it returns work with the unchanged free() and realloc() the system provides, and will always do so. >>> I am basically worried that undefining malloc, even if it seems harmless now, exposes us to difficult-to-investigate problems down the road, since it depends on how the libc devs will reform those macros in the future. > > Other than that, and kind of depending on your answer: How important is it that we catch every use of the original malloc? Can be safely mix the original malloc with vec_malloc if logging is not involved? > > I am asking, because from that it depends whether this hunk needs to appear right behind `#include ` or whether we can move it into the middle of the file together with the other AIX stuff. > > Because, if we move it into the middle of the file, we may miss any uses of malloc that may happen in system headers (would be unusual for that to happen but with IBM one never knows). Hi Thomas, I would like to get totally rid of this, because as I mentioned IBM already modified the `stdlib.h` header not using `#define malloc vec_malloc` any more (and all the other vec_... defines). We have to ask the adoptium colleagues at IBM if they already have raised their build environment by the 2 SP levels needed. In principle we had to do the same workaround for `calloc, free,...` too, but they didn't show up as errors in the logging files. These lines where never meant to stay for long. Just to be able to compile until IBM fixes the issue, which is done now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547465986 From jkern at openjdk.org Tue Apr 2 09:21:59 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 09:21:59 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 09:14:10 GMT, Joachim Kern wrote: >> Other than that, and kind of depending on your answer: How important is it that we catch every use of the original malloc? Can be safely mix the original malloc with vec_malloc if logging is not involved? >> >> I am asking, because from that it depends whether this hunk needs to appear right behind `#include ` or whether we can move it into the middle of the file together with the other AIX stuff. >> >> Because, if we move it into the middle of the file, we may miss any uses of malloc that may happen in system headers (would be unusual for that to happen but with IBM one never knows). > > Hi Thomas, > I would like to get totally rid of this, because as I mentioned IBM already modified the `stdlib.h` header not using `#define malloc vec_malloc` any more (and all the other vec_... defines). We have to ask the adoptium colleagues at IBM if they already have raised their build environment by the 2 SP levels needed. > In principle we had to do the same workaround for `calloc, free,...` too, but they didn't show up as errors in the logging files. > These lines where never meant to stay for long. Just to be able to compile until IBM fixes the issue, which is done now. @suchismith1993 Hi Suchi, can you please tell me when you will raise your build environment from AIX 7.2 TL5 SP5 to SP7? I' am asking you, because I want to get rid of this nasty workaround. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547473723 From jkern at openjdk.org Tue Apr 2 10:28:59 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 10:28:59 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Fri, 29 Mar 2024 07:21:43 GMT, Thomas Stuefe wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > src/hotspot/os/aix/os_aix.cpp line 314: > >> 312: ErrnoPreserver ep; >> 313: log_trace(os, map)("disclaim failed: " RANGEFMT " errno=(%s)", >> 314: RANGEFMTARGS(p, (long)maxDisclaimSize), > > Wait, why are these casts needed? maxDisclaimSize is size_t, RANGEFMT uses SIZE_FORMAT. That should work without cast. Hi Thomas, `maxDisclaimSize` is of type `unsigned int`; therefore I get the following warning: os/aix/os_aix.cpp:314:42: error: format specifies type 'unsigned long' but the argument has type 'unsigned int' [-Werror,-Wformat] RANGEFMTARGS(p, maxDisclaimSize), ^~~~~~~~~~~~~~~ Should I keep the casts, or change the type of `maxDisclaimSize, numFullDisclaimsNeeded, lastDisclaimSize` to `const unsigned long`? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547578012 From jkern at openjdk.org Tue Apr 2 10:49:01 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 10:49:01 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: <8yq0NeIit-6Q3aB1IF3MqNZnT4B4hWd7LnKYlsX6NPk=.38fd4342-e553-4a12-8e69-ca95b3cccb09@github.com> References: <8yq0NeIit-6Q3aB1IF3MqNZnT4B4hWd7LnKYlsX6NPk=.38fd4342-e553-4a12-8e69-ca95b3cccb09@github.com> Message-ID: On Fri, 29 Mar 2024 07:19:33 GMT, Thomas Stuefe wrote: >> src/hotspot/os/aix/loadlib_aix.cpp line 120: >> >>> 118: (lm->is_in_vm ? '*' : ' '), >>> 119: (uintptr_t)lm->text, (uintptr_t)lm->text + lm->text_len, >>> 120: (uintptr_t)lm->data, (uintptr_t)lm->data + lm->data_len, >> >> Please don't cast, use `p2i()`. > > Check copyrights in this file and all others. Adapt SAP and Oracle copyrights. Done + will adopt copyrights >> src/hotspot/os/aix/os_aix.cpp line 651: >> >>> 649: lt.print("Thread is alive (tid: " UINTX_FORMAT ", kernel thread id: " UINTX_FORMAT >>> 650: ", stack [" PTR_FORMAT " - " PTR_FORMAT " (" SIZE_FORMAT "k using %luk pages)).", >>> 651: os::current_thread_id(), (uintx) kernel_thread_id, (uintptr_t)low_address, (uintptr_t)high_address, >> >> Use p2i, not cast > > Here, and in other places too where you cast a pointer to fit into PTR_FORMAT or INTPTR_FORMAT Done. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547607793 PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547606610 From mli at openjdk.org Tue Apr 2 10:49:04 2024 From: mli at openjdk.org (Hamlin Li) Date: Tue, 2 Apr 2024 10:49:04 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v4] In-Reply-To: References: <1knXD7Wc8heH83BTJEguqmlTAa70oXDw_nWI0hBjAm0=.bcf88238-2b63-4d14-b3a3-94eabca69586@github.com> <77VcfKMeUXxKjoQ3zeEcBJfDQYhPRSt08OXMqg9rQDo=.de753f60-5568-4dec-9b86-2ba86acebfaa@github.com> Message-ID: <2kzCAM7nbD5d4nnO5NN2ioUaGfPZh2minA0XDYJm-U8=.b42cf8cb-65ca-4075-8ca4-68eefe126cf8@github.com> On Thu, 28 Mar 2024 18:41:03 GMT, Paul Sandoz wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> fix jni includes > > Hamlin, thank you for working on this. I think integrating a sub-set of SLEEF is valuable (not all of it makes sense e.g., DFT part). My recommendation would be to focus on a PR that integrates the required source, rather than taking steps towards that. > > AFAICT from browsing prior comments "integrate the source" appears to be the generally preferred solution, but there is some understandable hesitancy about legal aspects. IIUC from what you say this is a technically feasible and maintainable solution. As said here: > >> We (Oracle Java Platform Group) can handle the required "paperwork > https://github.com/openjdk/jdk/pull/16234#issuecomment-1823335443 Thanks @PaulSandoz for your comment and suggestion. I will work on the solution which integrates the source of sleef into jdk. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18294#issuecomment-2031671597 From jkern at openjdk.org Tue Apr 2 10:49:02 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 10:49:02 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Fri, 29 Mar 2024 07:25:30 GMT, Thomas Stuefe wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > src/hotspot/os/aix/os_aix.cpp line 1212: > >> 1210: st->print_cr("physical free : " SIZE_FORMAT, (unsigned long)mi.real_free); >> 1211: st->print_cr("swap total : " SIZE_FORMAT, (unsigned long)mi.pgsp_total); >> 1212: st->print_cr("swap free : " SIZE_FORMAT, (unsigned long)mi.pgsp_free); > > A better way to do this would be to change AIX::meminfo to use size_t. We should have done this when introducing that API. Done. modified `os::Aix::meminfo_t` to use `size_t` instead of `long long` > src/hotspot/os/aix/os_aix.cpp line 1399: > >> 1397: os->print("[" PTR_FORMAT " - " PTR_FORMAT "] (" UINTX_FORMAT >> 1398: " bytes, %ld %s pages), %s", >> 1399: (uintptr_t)addr, (uintptr_t)addr + size - 1, size, size / pagesize, describe_pagesize(pagesize), > > p2i Done. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547603744 PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547606275 From jkern at openjdk.org Tue Apr 2 11:26:00 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 11:26:00 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Fri, 29 Mar 2024 07:39:06 GMT, Thomas Stuefe wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 62: > >> 60: #include >> 61: >> 62: #if defined(LINUX) || defined(_ALLBSD_SOURCE) || defined(_AIX) > > What else is left? Could we just remove this line altogether now? I cannot answer this question. If this line is now obsolete it was also obsolete before including AIX, because AIX didn't use this file beforehand. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547667349 From jkern at openjdk.org Tue Apr 2 11:26:02 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 11:26:02 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Thu, 28 Mar 2024 17:33:29 GMT, Martin Doerr wrote: >> src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 83: >> >>> 81: #error "xlc version not supported, macro __open_xl_version__ not found" >>> 82: #endif >>> 83: #endif // AIX >> >> This `#ifdef _AIX` might be obsolete, because configure will throw a warning if the compiler has a lower version, but it's only a warning. > > I'd prefer having less AIX specific parts in this file. Can this be moved somewhere else? Or maybe combine it with the AIX code above? My question is, do we need this block, because now already configure warns about an outdated compiler, or is a warning to weak and we want to force this error here? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547672502 From jkern at openjdk.org Tue Apr 2 11:31:01 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 11:31:01 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Fri, 29 Mar 2024 08:06:01 GMT, Thomas Stuefe wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 83: > >> 81: #error "xlc version not supported, macro __open_xl_version__ not found" >> 82: #endif >> 83: #endif // AIX > > Can probably be shortened like this: > > Suggestion: > > #ifdef _AIX > #if !defined(__open_xl_version__) || (__open_xl_version__ < 17) > #error "this xlc version is not supported" > #endif > #endif // AIX followed your proposal. > src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 103: > >> 101: #endif >> 102: >> 103: #if !defined(LINUX) && !defined(_ALLBSD_SOURCE) && !defined(_AIX) > > I believe this whole section can be removed now. > > At least I have no idea who this is for. What gcc versions does OpenJDK still support, then, beside these platforms. Also, any gcc platform not on linux or bsd would have hit the #error below at line 132. linux macos and now Aix use this file. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547677545 PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547681162 From jkern at openjdk.org Tue Apr 2 11:37:59 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 11:37:59 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 11:28:30 GMT, Joachim Kern wrote: >> src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 103: >> >>> 101: #endif >>> 102: >>> 103: #if !defined(LINUX) && !defined(_ALLBSD_SOURCE) && !defined(_AIX) >> >> I believe this whole section can be removed now. >> >> At least I have no idea who this is for. What gcc versions does OpenJDK still support, then, beside these platforms. Also, any gcc platform not on linux or bsd would have hit the #error below at line 132. > > linux macos and now Aix use this file. Who is able to explain if `#if defined(LINUX) || defined(_ALLBSD_SOURCE) || defined(_AIX)` in this file is equivalent to `#if 1` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1547692144 From ihse at openjdk.org Tue Apr 2 13:05:28 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 2 Apr 2024 13:05:28 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v70] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: On Thu, 28 Mar 2024 07:36:04 GMT, Julian Waters wrote: >> We should set the -permissive- flag for the Microsoft Visual C compiler, as was requested by the now backed out [JDK-8241499](https://bugs.openjdk.org/browse/JDK-8241499). Doing so makes the Visual C compiler much less accepting of ill formed code, which will improve code quality on Windows in the future. > > Julian Waters has updated the pull request incrementally with four additional commits since the last revision: > > - Labels to empty line in awt_Window.cpp > - Labels to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp > - Label to empty line in awt_Window.cpp src/java.desktop/windows/native/libawt/windows/awt_Canvas.cpp line 234: > 232: c->m_eraseBackgroundOnResize = doEraseOnResize; > 233: > 234: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Component.cpp line 6586: > 6584: component->GetInsets(gis->insets); > 6585: > 6586: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Frame.cpp line 1603: > 1601: } else { > 1602: f = (AwtFrame *) JNI_GET_PDATA(peer); > 1603: if (f == nullptr) { @prrace What is the current take on `NULL` vs `nullptr` in client code? I know Hotspot made an effort to completely purge all `NULL` references. The new code here uses a mix of `NULL` and `nullprt`. Should it: a) only use `NULL` b) only use `nullptr` c) remain as it is ? src/java.desktop/windows/native/libawt/windows/awt_PrintJob.cpp line 214: > 212: static const double POINTS_TO_LOMETRIC = (254.0 / 72.0); > 213: > 214: I recommend deleting one more line, since otherwise you will now have three consecutive blank lines. src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 1031: > 1029: window->RepositionSecurityWarning(env); > 1030: > 1031: ret: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 3159: > 3157: } > 3158: > 3159: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 3193: > 3191: } > 3192: > 3193: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 3224: > 3222: window->SetTranslucency(iOpacity, window->isOpaque()); > 3223: > 3224: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 3255: > 3253: window->SetTranslucency(window->getOpacity(), isOpaque); > 3254: > 3255: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 3293: > 3291: uws->hBitmap); > 3292: > 3293: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 3328: > 3326: window->setFullScreenExclusiveModeState(state != 0); > 3327: > 3328: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547824279 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547824797 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547829415 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547830886 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547833185 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547836998 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547837189 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547837344 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547837450 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547837625 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547837761 From ihse at openjdk.org Tue Apr 2 13:05:29 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 2 Apr 2024 13:05:29 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v52] In-Reply-To: <6_80radA3v1Fq_kuf_4GZ74gfwc8hgMPOyBczHItlKc=.9a2463ef-c106-42c4-b0a8-12f94568ea96@github.com> References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> <0p1UVaUOt2bs1mkuBV1liUxBYGRhnKnclf1x_-Y22GE=.2de6c2c4-8cf5-470d-bdb3-cd0dd0e13975@github.com> <6_80radA3v1Fq_kuf_4GZ74gfwc8hgMPOyBczHItlKc=.9a2463ef-c106-42c4-b0a8-12f94568ea96@github.com> Message-ID: On Fri, 22 Mar 2024 12:26:25 GMT, Magnus Ihse Bursie wrote: >> Julian Waters has updated the pull request incrementally with two additional commits since the last revision: >> >> - Revert Formatting in awt_Component.cpp >> - Revert Formatting in awt_Window.cpp > > src/java.desktop/windows/native/libawt/windows/awt_PrintJob.cpp line 779: > >> 777: } >> 778: >> 779: > > Suggestion: To be clear: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. > src/java.desktop/windows/native/libawt/windows/awt_PrintJob.cpp line 1316: > >> 1314: env->CallVoidMethod(newPaper, setImageableID, ix, iy, iw, ih); >> 1315: >> 1316: > > Suggestion: I recommend deleting this line, since it does not make sense to have two consecutive blank lines here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547831604 PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547832678 From ihse at openjdk.org Tue Apr 2 13:05:31 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 2 Apr 2024 13:05:31 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v46] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> <5WJnl6Z9RyYzZrDStf9p74gus07Stz6S6NHwVpGQO3M=.0a04a120-77b5-48c3-9ae8-7d37c9646463@github.com> Message-ID: On Sat, 20 Jan 2024 00:38:04 GMT, Phil Race wrote: >> Julian Waters has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 79 commits: >> >> - Merge branch 'openjdk:master' into patch-10 >> - Merge branch 'openjdk:master' into patch-10 >> - Fix awt_Window.cpp >> - Fix awt_PrintJob.cpp >> - -Zc:stringStrings no longer needed with -permissive- flags-cflags.m4 >> - Fix awt_Window.cpp >> - awt_Window.cpp >> - awt_PrintJob.cpp >> - awt_Frame.cpp >> - Whitespace awt_Component.cpp >> - ... and 69 more: https://git.openjdk.org/jdk/compare/35e96627...cbfbaee6 > > src/java.desktop/windows/native/libawt/windows/awt_Window.cpp line 3137: > >> 3135: >> 3136: PDATA pData = JNI_GET_PDATA(self); >> 3137: if (self == NULL) { > > Surely line 3136 above should have been deleted ? It is replaced by line 3143 below. > And then you can also directly set window, pData isn't needed, as discussed in similar cases above Yes. @TheShermanTanker This is in fact a serious bug. You try to do `pData = JNI_GET_PDATA(self)` before the null check of self, and this will crash if self is null. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15096#discussion_r1547836085 From ihse at openjdk.org Tue Apr 2 13:13:04 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 2 Apr 2024 13:13:04 GMT Subject: Integrated: 8329292: Fix missing cleanups in java.management and jdk.management In-Reply-To: References: Message-ID: On Thu, 28 Mar 2024 18:22:27 GMT, Magnus Ihse Bursie wrote: > For some reason, I missed adding the new standard header for SetupJdkLibrary in java.management and jdk.management. I also missed to remove a now superfluous CFLAGS_JDKLIB in libmanagement_ext. This pull request has now been integrated. Changeset: 5ae849d6 Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/5ae849d66f195e96fbae9dcf06a44d8aab659181 Stats: 5 lines in 2 files changed: 4 ins; 0 del; 1 mod 8329292: Fix missing cleanups in java.management and jdk.management Reviewed-by: erikj ------------- PR: https://git.openjdk.org/jdk/pull/18538 From ihse at openjdk.org Tue Apr 2 13:20:15 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 2 Apr 2024 13:20:15 GMT Subject: RFR: 8329289: Unify SetupJdkExecutable and SetupJdkLibrary [v2] In-Reply-To: References: Message-ID: <8oUigBPJU9_2gdYw8HuCTaP3KZcgYv21LKUMVsU3gmk=.b95c9bdf-26da-44c2-95bd-e0bc07b9809a@github.com> > Currently a lot of code is duplicated between SetupJdkExecutable and SetupJdkLibrary. Furthermore, some functionality is still missing from SetupJdkExecutable that is present in SetupJdkLibrary. These functions also have not had their documentation properly updated as they have evolved. This PR will fix all of this. Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Merge branch 'master' into unify-jdk-compilation - FIx documentation - Remove FindSrcDirsForLib - Unify and simplify code - Merge SetupJdkExecutable and SetupJdkLibrary into SetupJdkNativeCompilation ------------- Changes: https://git.openjdk.org/jdk/pull/18537/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18537&range=01 Stats: 170 lines in 1 file changed: 52 ins; 83 del; 35 mod Patch: https://git.openjdk.org/jdk/pull/18537.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18537/head:pull/18537 PR: https://git.openjdk.org/jdk/pull/18537 From ihse at openjdk.org Tue Apr 2 13:20:16 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 2 Apr 2024 13:20:16 GMT Subject: Integrated: 8329289: Unify SetupJdkExecutable and SetupJdkLibrary In-Reply-To: References: Message-ID: On Thu, 28 Mar 2024 17:59:10 GMT, Magnus Ihse Bursie wrote: > Currently a lot of code is duplicated between SetupJdkExecutable and SetupJdkLibrary. Furthermore, some functionality is still missing from SetupJdkExecutable that is present in SetupJdkLibrary. These functions also have not had their documentation properly updated as they have evolved. This PR will fix all of this. This pull request has now been integrated. Changeset: 5ac067f6 Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/5ac067f6d6e0b301b33fb287aa3f288d318df2ba Stats: 170 lines in 1 file changed: 52 ins; 83 del; 35 mod 8329289: Unify SetupJdkExecutable and SetupJdkLibrary Reviewed-by: erikj ------------- PR: https://git.openjdk.org/jdk/pull/18537 From jkern at openjdk.org Tue Apr 2 13:38:20 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 13:38:20 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v2] In-Reply-To: References: Message-ID: > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: Followed the proposals ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18536/files - new: https://git.openjdk.org/jdk/pull/18536/files/61fd0ff2..689b353d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=00-01 Stats: 35 lines in 9 files changed: 0 ins; 4 del; 31 mod Patch: https://git.openjdk.org/jdk/pull/18536.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18536/head:pull/18536 PR: https://git.openjdk.org/jdk/pull/18536 From sgehwolf at openjdk.org Tue Apr 2 14:08:08 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Tue, 2 Apr 2024 14:08:08 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: Message-ID: On Tue, 19 Mar 2024 16:55:14 GMT, Severin Gehwolf wrote: >> Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app being modularized). >> >> The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JRTArchive` class which has all the info of what constitutes the final jlinked runtime. >> >> Basic usage example: >> >> $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se) >> $ diff -u <(./bin/java --list-modules --limit-modules jdk.jlink) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink) >> $ ls ../linux-x86_64-server-release/images/jdk/jmods >> java.base.jmod java.net.http.jmod java.sql.rowset.jmod jdk.crypto.ec.jmod jdk.internal.opt.jmod jdk.jdi.jmod jdk.management.agent.jmod jdk.security.auth.jmod >> java.compiler.jmod java.prefs.jmod java.transaction.xa.jmod jdk.dynalink.jmod jdk.internal.vm.ci.jmod jdk.jdwp.agent.jmod jdk.management.jfr.jmod jdk.security.jgss.jmod >> java.datatransfer.jmod java.rmi.jmod java.xml.crypto.jmod jdk.editpad.jmod jdk.internal.vm.compiler.jmod jdk.jfr.jmod jdk.management.jmod jdk.unsupported.desktop.jmod >> java.desktop.jmod java.scripting.jmod java.xml.jmod jdk.hotspot.agent.jmod jdk.i... > > Severin Gehwolf has updated the pull request incrementally with one additional commit since the last revision: > > Move CreateLinkableRuntimePlugin to build folder > > Keep runtime link supporting classes in package > jdk.tools.jlink.internal.runtimelink I'm posting this for posterity, since I did some research on the `jimage` format in light of being able to re-create an existing `jimage` file with potentially a few resources being added. One use-case for this would be to add the "diff" data to an pre-existing, optimized `jimage` in `images/jdk/lib/modules`. The way that the `jimage` write algorithm works is based on the fact that it knows the resources and bytes at `jlink` time in **full**. Therefore, it can iterate over all resources, generate an index (and header) for them and then can serialize resource bytes as well as container (folder) information for them so as to support various iteration entry points. Since it's inherently relying on the header, index and container information at `jimage` read-time the format doesn't lend itself nicely for "appending". By adding just a few resources, the header, index, and container iteration information get invalidated. Therefore, a new `jimage` would need to be created, based on the old `jimage`'s resources inferring `ResourcePoolEntry`s and then working with them. The additional resources would then get added and header/index/container info generated afresh. In order to support this, the existing `jimage` file would need to have sufficient information in it, to re-constitute a "similar" jimage from it (at the infer `ResourcePoolEntry` step). There are two specific issues that need to be overcome to support this: resource ordering and resource compression. There might be more, that I'm missing. ## Ordering of Resources `jlink` supports ordering of resources. In other words, the `--order-resources` plugin instructs `jlink` to produce a `jimage` where resources will end up in the target `jimage` in a specified order. This can be observed by listing `jimage` contents sorted by content byte offsets. For example, the default JDK build for Linux adds this expression: `--order-resources=**module-info.class,@/path/to/build/output/support/link_opt/classlist,/java.base/java/**,/java.base/jdk/**,/java.base/sun/**,/java.base/com/**,/jdk.localedata/**` to the `jlink` invocation at build time. That is, `module-info.class` files come first in the resulting `jimage`, then class bytes as specified by CDS's `classlist` file, then `java.base/java` classes and so on. How would one re-constitute the desired ordering given only the `jimage` file as input? I think this could be solved by looking at the content byte offsets in the existing `jimage` for each resource. This would allow one to re-constitute the ordering in t he derived image. The question would be how ordering of the added resources should be handled in such a case. ## Compression of Resources `jlink` suppports compression of resources. Currently, only `zip` and `compact-cp` are supported. There is still code handling multiple compressions. In fact, the `jimage` format itself doesn't put a bound on the number of levels of compressions. In other words, a resource could be compressed with the `zip` compressor multiple times or use the `compact-cp` compressor in conjunction with the `zip` compressor. In my investigation, I've focused on the `zip` compressor for now. In my research I've not found a reliable way to detect the **input level** compaction level, like `zip-1` by looking only at a `jimage` file. `jlink` uses `Inflater` and `Deflater` classes internally to decompress and compress respectively, but the `zlib` format doesn't seem to expose the 10 input levels (`zip-0` to `zip-9`). It only has a notion of it's three compression levels that's exposed in the `zlib` header. Fastest compression, fast compression and max compression (default compression level being the fourt h). However, for example `zip-1` compaction level has the following config in terms of `good`, `lazy`, `nice` and `chain` configs: `4`, `4`, `8`, `4` together with `deflate_fast` compression function (compress level exposed in the `zlib` header). One possible way to overcome this would be to support some kind of pass-through compressor in `jlink` code. I haven't looked much into `compact-cp` code, but the challenges there might be similar. It's also worth noting that the current `jlink` code seems to support compression only for a select set of files. That would likely also need investigation how it could be handled. The "pass-through" idea could be viable for this too as compressed bytes would be "passed through" to the target `jimage`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2032140826 From rehn at openjdk.org Tue Apr 2 14:34:10 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Tue, 2 Apr 2024 14:34:10 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: Message-ID: On Thu, 21 Mar 2024 06:58:43 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > remove swap file How would you add jni.h ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2032202423 From mdoerr at openjdk.org Tue Apr 2 14:51:11 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 2 Apr 2024 14:51:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 11:22:54 GMT, Joachim Kern wrote: >> I'd prefer having less AIX specific parts in this file. Can this be moved somewhere else? Or maybe combine it with the AIX code above? > > My question is, do we need this block, because now already configure warns about an outdated compiler, or is a warning to weak and we want to force this error here? I think that building with xlc 16 is no longer possible because the old build pipeline is no longer supported and that is already caught by configure. So, can we even reach here with older xlc compilers? If not, this code can get removed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548043503 From jkern at openjdk.org Tue Apr 2 15:46:11 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 15:46:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 14:48:49 GMT, Martin Doerr wrote: >> My question is, do we need this block, because now already configure warns about an outdated compiler, or is a warning to weak and we want to force this error here? > > I think that building with xlc 16 is no longer possible because the old build pipeline is no longer supported and that is already caught by configure. So, can we even reach here with older xlc compilers? > If not, this code can get removed. Yes, of course you are right. All the compile statements will fail with xlc 16 or older. I will remove it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548134431 From ihse at openjdk.org Tue Apr 2 15:53:25 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 2 Apr 2024 15:53:25 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v70] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: On Mon, 1 Apr 2024 10:33:41 GMT, Julian Waters wrote: >> Julian Waters has updated the pull request incrementally with four additional commits since the last revision: >> >> - Labels to empty line in awt_Window.cpp >> - Labels to empty line in awt_Window.cpp >> - Label to empty line in awt_Window.cpp >> - Label to empty line in awt_Window.cpp > > Bumping @TheShermanTanker I tried to help you get this done. I added fixes to a copy of your branch on my personal fork, but then it turned out I could not push them to your branch. :-( It ended up with me creating a new PR, https://github.com/openjdk/jdk/pull/18584. As a bonus, I think it might be easier to review with a fresh start. This PR has grown quite heavy with lots of comments and commits. I hope you don't feel like I'm stealing this away from you. You have done a great job, and shown a lot of patience of carrying this all the way here. But I also got the impression that you would appreciate my assistance in getting the last pieces in place so we can integrate this. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15096#issuecomment-2032430773 From duke at openjdk.org Tue Apr 2 16:10:55 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Tue, 2 Apr 2024 16:10:55 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic Message-ID: Performance. Before: Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s Benchmark (isMontBench) Mode Cnt Score Error Units PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s Performance, no intrinsic: Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s Benchmark (isMontBench) Mode Cnt Score Error Units PolynomialP256Bench.benchMultiply true thrpt 3 1919.574 ? 10.591 ops/s Performance, **with intrinsics** Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 10384.591 ? 65.274 ops/s SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 9592.912 ? 236.411 ops/s SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 3479.494 ? 44.578 ops/s SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 3402.147 ? 26.772 ops/s Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 2527.678 ? 64.791 ops/s o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 2541.258 ? 66.634 ops/s Benchmark (isMontBench) Mode Cnt Score Error Units PolynomialP256Bench.benchMultiply true thrpt 3 3021.139 ? 98.289 ops/s Summary on design (see code for 'ASCII art', references and details on math): - Added a new `IntegerPolynomial` field (`MontgomeryIntegerPolynomialP256`) with 52-bit limbs - `getElement(*)/fromMontgomery()` to convert numbers into/out of the field - `ECOperations` is the primary use of the new field - flattened some extra deep nested class hierarchy (also in prep for further other field optimizations) - `forParameters()/multiply()/setSum()` generates numbers in the new field - `ProjectivePoint/Montgomery{Imm|M}utable.asAffine()` to convert out of the new field - Added Fuzz Testing and KAT verified with OpenSSL ------------- Commit messages: - remove trailing whitespace - Remeasure performance - Fix rebase typo - Address comments from Anas and thorough cleanup - conditionalAssign intrinsic - rebase Changes: https://git.openjdk.org/jdk/pull/18583/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8329538 Stats: 2335 lines in 34 files changed: 2037 ins; 162 del; 136 mod Patch: https://git.openjdk.org/jdk/pull/18583.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18583/head:pull/18583 PR: https://git.openjdk.org/jdk/pull/18583 From jkern at openjdk.org Tue Apr 2 16:14:12 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 2 Apr 2024 16:14:12 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: version check not needed anymore ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18536/files - new: https://git.openjdk.org/jdk/pull/18536/files/689b353d..ac1335e5 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=01-02 Stats: 6 lines in 1 file changed: 0 ins; 6 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18536.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18536/head:pull/18536 PR: https://git.openjdk.org/jdk/pull/18536 From alanb at openjdk.org Tue Apr 2 16:32:09 2024 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 2 Apr 2024 16:32:09 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 15:42:05 GMT, Volodymyr Paprotski wrote: > Performance. Before: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s > > Performance, no intrinsic: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply true thrpt 3 1919.574 ? 10.591 ops/s > > Performance, **with intrinsics*... src/java.base/share/classes/module-info.java line 265: > 263: jdk.jfr, > 264: jdk.unsupported, > 265: jdk.crypto.ec; jdk.crypto.ec has been hollowed out since JDK 22, the sun.security.ec are in java.base. So I don't think you need this qualified export. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1548199507 From kbarrett at openjdk.org Tue Apr 2 16:43:59 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 2 Apr 2024 16:43:59 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 11:20:49 GMT, Joachim Kern wrote: >> src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 62: >> >>> 60: #include >>> 61: >>> 62: #if defined(LINUX) || defined(_ALLBSD_SOURCE) || defined(_AIX) >> >> What else is left? Could we just remove this line altogether now? > > I cannot answer this question. > If this line is now obsolete it was also obsolete before including AIX, because AIX didn't use this file beforehand. There was at one time an attempt at a gcc/Solaris port, but I think it was never completed, and most vestiges removed. More recently, @TheShermanTanker has been doing stuff to permit clang/Windows, and clang-based builds use this file. I'm kind of surprised he hasn't encountered problems and done some cleanup here. (and ) and 64bit integer types are standard C++ now, so no longer need all this conditionalization. I suggest cleaning that up as a separate precursor. That would eliminate the two !defined blocks entirely. I wish the other conditional includes in this block were "where needed" rather than in globalDefinitions_gcc.hpp, but that's a different mess. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548225495 From kbarrett at openjdk.org Tue Apr 2 16:55:01 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 2 Apr 2024 16:55:01 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 16:41:40 GMT, Kim Barrett wrote: >> I cannot answer this question. >> If this line is now obsolete it was also obsolete before including AIX, because AIX didn't use this file beforehand. > > There was at one time an attempt at a gcc/Solaris port, but I think it was > never completed, and most vestiges removed. More recently, @TheShermanTanker > has been doing stuff to permit clang/Windows, and clang-based builds use this file. > I'm kind of surprised he hasn't encountered problems and done some cleanup here. > > (and ) and 64bit integer types are standard C++ now, > so no longer need all this conditionalization. I suggest cleaning that up as a > separate precursor. That would eliminate the two !defined blocks entirely. I > wish the other conditional includes in this block were "where needed" rather > than in globalDefinitions_gcc.hpp, but that's a different mess. https://bugs.openjdk.org/browse/JDK-8329546 - I can take this if nobody else grabs it soon. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548239737 From kbarrett at openjdk.org Tue Apr 2 17:04:11 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 2 Apr 2024 17:04:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 16:52:04 GMT, Kim Barrett wrote: >> There was at one time an attempt at a gcc/Solaris port, but I think it was >> never completed, and most vestiges removed. More recently, @TheShermanTanker >> has been doing stuff to permit clang/Windows, and clang-based builds use this file. >> I'm kind of surprised he hasn't encountered problems and done some cleanup here. >> >> (and ) and 64bit integer types are standard C++ now, >> so no longer need all this conditionalization. I suggest cleaning that up as a >> separate precursor. That would eliminate the two !defined blocks entirely. I >> wish the other conditional includes in this block were "where needed" rather >> than in globalDefinitions_gcc.hpp, but that's a different mess. > > https://bugs.openjdk.org/browse/JDK-8329546 - I can take this if nobody else grabs it soon. I'm waiting for a bunch of tests to complete, so decided to just take that issue. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548252193 From duke at openjdk.org Tue Apr 2 19:19:59 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Tue, 2 Apr 2024 19:19:59 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: > Performance. Before: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s > > Performance, no intrinsic: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply true thrpt 3 1919.574 ? 10.591 ops/s > > Performance, **with intrinsics*... Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: remove use of jdk.crypto.ec ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18583/files - new: https://git.openjdk.org/jdk/pull/18583/files/dbe6cd3b..82b6dae7 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=00-01 Stats: 6 lines in 2 files changed: 0 ins; 1 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/18583.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18583/head:pull/18583 PR: https://git.openjdk.org/jdk/pull/18583 From duke at openjdk.org Tue Apr 2 19:20:00 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Tue, 2 Apr 2024 19:20:00 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 16:29:07 GMT, Alan Bateman wrote: >> Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: >> >> remove use of jdk.crypto.ec > > src/java.base/share/classes/module-info.java line 265: > >> 263: jdk.jfr, >> 264: jdk.unsupported, >> 265: jdk.crypto.ec; > > jdk.crypto.ec has been hollowed out since JDK 22, the sun.security.ec are in java.base. So I don't think you need this qualified export. Thanks, fixed. (Started this when `jdk.crypto.ec` was still in use.. missed a few spots during rebase I guess) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1548460157 From kbarrett at openjdk.org Tue Apr 2 20:14:10 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 2 Apr 2024 20:14:10 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 17:01:07 GMT, Kim Barrett wrote: >> https://bugs.openjdk.org/browse/JDK-8329546 - I can take this if nobody else grabs it soon. > > I'm waiting for a bunch of tests to complete, so decided to just take that issue. https://github.com/openjdk/jdk/pull/18586 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548549705 From kbarrett at openjdk.org Tue Apr 2 20:14:10 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 2 Apr 2024 20:14:10 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 11:35:44 GMT, Joachim Kern wrote: >> linux macos and now Aix use this file. > > Who is able to explain if > `#if defined(LINUX) || defined(_ALLBSD_SOURCE) || defined(_AIX)` > in this file is equivalent to > `#if 1` See my other comments and https://bugs.openjdk.org/browse/JDK-8329546 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548550923 From ihse at openjdk.org Wed Apr 3 00:07:31 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 00:07:31 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler Message-ID: This is a retake on https://github.com/openjdk/jdk/pull/15096. I tried to fix the remaining issues in that PR, but could not push them. In the end, it seemed easier to create a new branch in my own personal fork. The majority of the work in this PR has been done by @TheShermanTanker. I have stepped in and fixed the remaining comments, reverted some additional unneeded changed code, and also tried to minimize code duplication in `awt_PrintJob.cpp`. ------------- Commit messages: - 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler Changes: https://git.openjdk.org/jdk/pull/18584/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18584&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8307160 Stats: 428 lines in 12 files changed: 284 ins; 50 del; 94 mod Patch: https://git.openjdk.org/jdk/pull/18584.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18584/head:pull/18584 PR: https://git.openjdk.org/jdk/pull/18584 From jwaters at openjdk.org Wed Apr 3 02:31:11 2024 From: jwaters at openjdk.org (Julian Waters) Date: Wed, 3 Apr 2024 02:31:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: <7xb5qfR2poiQyEhUY5A09POeM2EpYBVRQKNs4XX_nuM=.7ed52b15-8c69-42c2-8625-cd0fe2ada78b@github.com> On Tue, 2 Apr 2024 20:10:12 GMT, Kim Barrett wrote: >> I'm waiting for a bunch of tests to complete, so decided to just take that issue. > > https://github.com/openjdk/jdk/pull/18586 @kimbarrett I've been doing things to permit gcc/Windows, not clang. clang has too many different distributions on Windows for me to settle on one, and generalising all of them to be able to compile with any of the Windows clang distributions seamlessly and without issues sounds like a nightmare :P gcc on the other hand has just 2: MSYS2 MINGW64 with ucrt (Which is the one I'm working on) and standalone gcc Windows builds that link to ucrt I haven't sent a cleanup in this area because I thought my changes were too specific to gcc/Windows, and wouldn't help much with HotSpot in general. I've learnt from my mistakes in the past where I caused reviewers pain in reviewing my admittedly selfish changes to HotSpot :( That said, if it is requested of me, I can commit some cleanups to this file. What do you think? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548833671 From jwaters at openjdk.org Wed Apr 3 02:41:25 2024 From: jwaters at openjdk.org (Julian Waters) Date: Wed, 3 Apr 2024 02:41:25 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v70] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: On Mon, 1 Apr 2024 10:33:41 GMT, Julian Waters wrote: >> Julian Waters has updated the pull request incrementally with four additional commits since the last revision: >> >> - Labels to empty line in awt_Window.cpp >> - Labels to empty line in awt_Window.cpp >> - Label to empty line in awt_Window.cpp >> - Label to empty line in awt_Window.cpp > > Bumping > @TheShermanTanker I tried to help you get this done. I added fixes to a copy of your branch on my personal fork, but then it turned out I could not push them to your branch. :-( > > It ended up with me creating a new PR, #18584. As a bonus, I think it might be easier to review with a fresh start. This PR has grown quite heavy with lots of comments and commits. > > I hope you don't feel like I'm stealing this away from you. You have done a great job, and shown a lot of patience of carrying this all the way here. But I also got the impression that you would appreciate my assistance in getting the last pieces in place so we can integrate this. Not at all, I don't feel like you're stealing this from me. In fact, I should be the one apologising for giving you extra work! Thanks for taking this up, I once again apologise for making you do this instead, I've been very busy since Thursday (working on OpenJDK while in lectures at times), and during my breaks I've been too drained to continue, so i really appreciate your help :) I will keep this open until the other Pull Request has been integrated, in case this might still be needed ------------- PR Comment: https://git.openjdk.org/jdk/pull/15096#issuecomment-2033427284 From jwaters at openjdk.org Wed Apr 3 02:46:59 2024 From: jwaters at openjdk.org (Julian Waters) Date: Wed, 3 Apr 2024 02:46:59 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 15:45:59 GMT, Magnus Ihse Bursie wrote: > This is a retake on https://github.com/openjdk/jdk/pull/15096. > > I tried to fix the remaining issues in that PR, but could not push them. In the end, it seemed easier to create a new branch in my own personal fork. > > The majority of the work in this PR has been done by @TheShermanTanker. I have stepped in and fixed the remaining comments, reverted some additional unneeded changed code, and also tried to minimize code duplication in `awt_PrintJob.cpp`. Looks good. Thanks for taking this up for me! In the future I hope @prrace will change his mind and allow the use of C++ Libraries in awt (awt already depends on the C++ Standard Library on Windows, verifiable if you run dumpbin -DEPENDENTS on it), because I have a change in mind relying on RAII that would look much cleaner and neater ------------- Marked as reviewed by jwaters (Committer). PR Review: https://git.openjdk.org/jdk/pull/18584#pullrequestreview-1975384014 From gli at openjdk.org Wed Apr 3 03:14:20 2024 From: gli at openjdk.org (Guoxiong Li) Date: Wed, 3 Apr 2024 03:14:20 GMT Subject: RFR: 8329494: Serial: Merge GenMarkSweep into MarkSweep Message-ID: Hi all, This patch merges the class `GenMarkSweep` into `MarkSweep`. Just simply move. It may be better to merge the class `DeadSpacer` and `Compacter` into `MarkSweep` and then rename `MarkSweep`. The `MarkSweep` will be renamed at [JDK-8329521](https://bugs.openjdk.org/browse/JDK-8329521). And the `DeadSpacer` and `Compacter` may be refactored at [JDK-8305896](https://bugs.openjdk.org/browse/JDK-8305896) and [JDK-8305898](https://bugs.openjdk.org/browse/JDK-8305898), so I don't refactor them now. The tests `make test-tier1_gc` passed locally. Thanks for taking the time to review. Best Regards, -- Guoxiong ------------- Commit messages: - JDK-8329494 Changes: https://git.openjdk.org/jdk/pull/18589/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18589&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8329494 Stats: 1112 lines in 8 files changed: 513 ins; 589 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/18589.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18589/head:pull/18589 PR: https://git.openjdk.org/jdk/pull/18589 From kbarrett at openjdk.org Wed Apr 3 03:39:10 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 3 Apr 2024 03:39:10 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <7xb5qfR2poiQyEhUY5A09POeM2EpYBVRQKNs4XX_nuM=.7ed52b15-8c69-42c2-8625-cd0fe2ada78b@github.com> References: <7xb5qfR2poiQyEhUY5A09POeM2EpYBVRQKNs4XX_nuM=.7ed52b15-8c69-42c2-8625-cd0fe2ada78b@github.com> Message-ID: <_rlVcRu9Omy5cqIe8UveAchgM8o5XT6rU2zS8ruUouE=.b944dc89-33ce-462e-89d4-880a1d267dba@github.com> On Wed, 3 Apr 2024 02:28:08 GMT, Julian Waters wrote: >> https://github.com/openjdk/jdk/pull/18586 > > @kimbarrett I've been doing things to permit gcc/Windows, not clang. clang has too many different distributions on Windows for me to settle on one, and generalising all of them to be able to compile with any of the Windows clang distributions seamlessly and without issues sounds like a nightmare :P gcc on the other hand has just 2: MSYS2 MINGW64 with ucrt (Which is the one I'm working on) and standalone gcc Windows builds that link to ucrt > > I haven't sent a cleanup in this area because I thought my changes were too specific to gcc/Windows, and wouldn't help much with HotSpot in general. I've learnt from my mistakes in the past where I caused reviewers pain in reviewing my admittedly selfish changes to HotSpot :( > That said, if it is requested of me, I can commit some cleanups to this file. What do you think? @TheShermanTanker It depends on the details, of course. I think there are lots of possible cleanups in this vicinity that have little or nothing to do with gcc/Windows specifically, though might help there too. And yeah, I misremembered that it was not clang/Windows but rather gcc/Windows you were working on. But the same points largely apply here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1548868243 From ihse at openjdk.org Wed Apr 3 08:21:26 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 08:21:26 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v70] In-Reply-To: References: <7piLRto5nNbhYYYfENCr5ecm4M2xNtMkjkE8XhrLLQ0=.8fd1ac3a-46f8-47a8-ae37-a4abbf7757d9@github.com> Message-ID: <499T-1pPl2Ik4W3iYBP6hOkngjUD7qINFHaf8eM8lCI=.17952ffd-d13b-4702-a350-a6aaa0963007@github.com> On Wed, 3 Apr 2024 02:38:21 GMT, Julian Waters wrote: > I will keep this open until the other Pull Request has been integrated, in case this might still be needed I don't think it is necessary. You can always re-open a PR if it should be needed, and you can look at source code and comments (and even make new comments) on a closed PR. So I think it will just make it easier for everybody if there is only a single open PR for this issue. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15096#issuecomment-2033880103 From ihse at openjdk.org Wed Apr 3 09:47:11 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 09:47:11 GMT Subject: RFR: 8329494: Serial: Merge GenMarkSweep into MarkSweep In-Reply-To: References: Message-ID: On Wed, 3 Apr 2024 03:10:26 GMT, Guoxiong Li wrote: > Hi all, > > This patch merges the class `GenMarkSweep` into `MarkSweep`. Just simply move. It may be better to merge the class `DeadSpacer` and `Compacter` into `MarkSweep` and then rename `MarkSweep`. The `MarkSweep` will be renamed at [JDK-8329521](https://bugs.openjdk.org/browse/JDK-8329521). And the `DeadSpacer` and `Compacter` may be refactored at [JDK-8305896](https://bugs.openjdk.org/browse/JDK-8305896) and [JDK-8305898](https://bugs.openjdk.org/browse/JDK-8305898), so I don't refactor them now. > > The tests `make test-tier1_gc` passed locally. Thanks for taking the time to review. > > Best Regards, > -- Guoxiong Build changes are trivially fine. ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18589#pullrequestreview-1976202642 From ihse at openjdk.org Wed Apr 3 09:49:00 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 09:49:00 GMT Subject: RFR: 8320799: Bump minimum boot jdk to JDK 22 In-Reply-To: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> References: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> Message-ID: On Mon, 1 Apr 2024 16:16:50 GMT, Mikael Vidstedt wrote: > With the JDK 22 GA out it's time to bump the minimum boot JDK version for mainline/JDK 23. > > Testing: tier1-5, GHA Marked as reviewed by ihse (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18563#pullrequestreview-1976211014 From ihse at openjdk.org Wed Apr 3 09:49:01 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 09:49:01 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 19:19:59 GMT, Volodymyr Paprotski wrote: >> Performance. Before: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s >> >> Performance, no intrinsic: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply true thrpt 3 1919.57... > > Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: > > remove use of jdk.crypto.ec Build changes are trivially fine. ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18583#pullrequestreview-1976208258 From ihse at openjdk.org Wed Apr 3 09:52:09 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 09:52:09 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 16:14:12 GMT, Joachim Kern wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > version check not needed anymore The build change look trivially fine. And allow me to show my appreciation for the hotspot code cleanup! (But note that this is not a review of that part). ------------- PR Review: https://git.openjdk.org/jdk/pull/18536#pullrequestreview-1976222691 From ayang at openjdk.org Wed Apr 3 09:59:08 2024 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 3 Apr 2024 09:59:08 GMT Subject: RFR: 8329494: Serial: Merge GenMarkSweep into MarkSweep In-Reply-To: References: Message-ID: On Wed, 3 Apr 2024 03:10:26 GMT, Guoxiong Li wrote: > Hi all, > > This patch merges the class `GenMarkSweep` into `MarkSweep`. Just simply move. It may be better to merge the class `DeadSpacer` and `Compacter` into `MarkSweep` and then rename `MarkSweep`. The `MarkSweep` will be renamed at [JDK-8329521](https://bugs.openjdk.org/browse/JDK-8329521). And the `DeadSpacer` and `Compacter` may be refactored at [JDK-8305896](https://bugs.openjdk.org/browse/JDK-8305896) and [JDK-8305898](https://bugs.openjdk.org/browse/JDK-8305898), so I don't refactor them now. > > The tests `make test-tier1_gc` passed locally. Thanks for taking the time to review. > > Best Regards, > -- Guoxiong Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18589#pullrequestreview-1976243060 From ihse at openjdk.org Wed Apr 3 10:04:01 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 10:04:01 GMT Subject: RFR: JDK-8303689: javac -Xlint could/should report on "dangling" doc comments In-Reply-To: References: Message-ID: On Wed, 27 Mar 2024 22:04:30 GMT, Jonathan Gibbons wrote: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. The build changes look okay. Do you have any plan of going through all the Java modules and fixing the issues, or opening JBS issues to have them fixed? Or will these lint warnings remain disabled for the foreseeable future? ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18527#pullrequestreview-1976255818 From tschatzl at openjdk.org Wed Apr 3 10:11:08 2024 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 3 Apr 2024 10:11:08 GMT Subject: RFR: 8329494: Serial: Merge GenMarkSweep into MarkSweep In-Reply-To: References: Message-ID: <9B1I_T-EUnuUusLr2wLRoCoFNyLW9U8mkx3i7Pr_oYQ=.91b67484-4220-4058-bd1d-68d70b8bf442@github.com> On Wed, 3 Apr 2024 03:10:26 GMT, Guoxiong Li wrote: > Hi all, > > This patch merges the class `GenMarkSweep` into `MarkSweep`. Just simply move. It may be better to merge the class `DeadSpacer` and `Compacter` into `MarkSweep` and then rename `MarkSweep`. The `MarkSweep` will be renamed at [JDK-8329521](https://bugs.openjdk.org/browse/JDK-8329521). And the `DeadSpacer` and `Compacter` may be refactored at [JDK-8305896](https://bugs.openjdk.org/browse/JDK-8305896) and [JDK-8305898](https://bugs.openjdk.org/browse/JDK-8305898), so I don't refactor them now. > > The tests `make test-tier1_gc` passed locally. Thanks for taking the time to review. > > Best Regards, > -- Guoxiong seems good. ------------- Marked as reviewed by tschatzl (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18589#pullrequestreview-1976271613 From mli at openjdk.org Wed Apr 3 14:48:24 2024 From: mli at openjdk.org (Hamlin Li) Date: Wed, 3 Apr 2024 14:48:24 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF Message-ID: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Hi, Can you help to review the patch? This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. Besides of the code changes, one important task is to handle the legal process. Thanks! ------------- Commit messages: - minor - add maintenance nodes - merge master - remove unnecessary changes - resolve build erorrs - add [generated] src from sleef - fix jni includes - rename - resolve magicus's comments - fix variable name in github workflow - ... and 13 more: https://git.openjdk.org/jdk/compare/6ae1cf12...3ab4795d Changes: https://git.openjdk.org/jdk/pull/18605/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18605&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8312425 Stats: 14582 lines in 20 files changed: 14535 ins; 1 del; 46 mod Patch: https://git.openjdk.org/jdk/pull/18605.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18605/head:pull/18605 PR: https://git.openjdk.org/jdk/pull/18605 From mli at openjdk.org Wed Apr 3 14:52:20 2024 From: mli at openjdk.org (Hamlin Li) Date: Wed, 3 Apr 2024 14:52:20 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v4] In-Reply-To: References: <1knXD7Wc8heH83BTJEguqmlTAa70oXDw_nWI0hBjAm0=.bcf88238-2b63-4d14-b3a3-94eabca69586@github.com> <77VcfKMeUXxKjoQ3zeEcBJfDQYhPRSt08OXMqg9rQDo=.de753f60-5568-4dec-9b86-2ba86acebfaa@github.com> Message-ID: On Thu, 28 Mar 2024 18:41:03 GMT, Paul Sandoz wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> fix jni includes > > Hamlin, thank you for working on this. I think integrating a sub-set of SLEEF is valuable (not all of it makes sense e.g., DFT part). My recommendation would be to focus on a PR that integrates the required source, rather than taking steps towards that. > > AFAICT from browsing prior comments "integrate the source" appears to be the generally preferred solution, but there is some understandable hesitancy about legal aspects. IIUC from what you say this is a technically feasible and maintainable solution. As said here: > >> We (Oracle Java Platform Group) can handle the required "paperwork > https://github.com/openjdk/jdk/pull/16234#issuecomment-1823335443 > Thanks @PaulSandoz for your comment and suggestion. > > I will work on the solution which integrates the source of sleef into jdk. I've created a new pr at https://github.com/openjdk/jdk/pull/18605 which is to integrate the sleef source into jdk so to avoid depending on external sleef at build or run time. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18294#issuecomment-2034832611 From mli at openjdk.org Wed Apr 3 14:52:20 2024 From: mli at openjdk.org (Hamlin Li) Date: Wed, 3 Apr 2024 14:52:20 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v4] In-Reply-To: <77VcfKMeUXxKjoQ3zeEcBJfDQYhPRSt08OXMqg9rQDo=.de753f60-5568-4dec-9b86-2ba86acebfaa@github.com> References: <1knXD7Wc8heH83BTJEguqmlTAa70oXDw_nWI0hBjAm0=.bcf88238-2b63-4d14-b3a3-94eabca69586@github.com> <77VcfKMeUXxKjoQ3zeEcBJfDQYhPRSt08OXMqg9rQDo=.de753f60-5568-4dec-9b86-2ba86acebfaa@github.com> Message-ID: <9fK9M5yFWEfVRYjUr7S5Xu3cLu3yWiydif6TaUROZZg=.57dfbf78-50f6-4541-afa6-3b6b6f0286e7@github.com> On Fri, 15 Mar 2024 13:58:05 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review this patch? >> Thanks >> >> This is a continuation of work based on [1] by @XiaohongGong, most work was done in that pr. In this new pr, just rebased the code in [1], then added some minor changes (renaming of calling method, add libsleef as extra lib in CI cross-build on aarch64 in github workflow); I aslo tested the combination of following scenarios: >> * at build time >> * with/without sleef >> * with/without sve support >> * at runtime >> * with/without sleef >> * with/without sve support >> >> [1] https://github.com/openjdk/jdk/pull/16234 >> >> ## Regression Test >> * test/jdk/jdk/incubator/vector/ >> * test/hotspot/jtreg/compiler/vectorapi/ >> >> ## Performance Test >> Previously, @XiaohongGong has shared the data: https://github.com/openjdk/jdk/pull/16234#issuecomment-1767727028 > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > fix jni includes Hey everyone, Please review the change at https://github.com/openjdk/jdk/pull/18605 when you're available. This pr will be closed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18294#issuecomment-2034835163 From mli at openjdk.org Wed Apr 3 14:52:20 2024 From: mli at openjdk.org (Hamlin Li) Date: Wed, 3 Apr 2024 14:52:20 GMT Subject: Withdrawn: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF In-Reply-To: <1knXD7Wc8heH83BTJEguqmlTAa70oXDw_nWI0hBjAm0=.bcf88238-2b63-4d14-b3a3-94eabca69586@github.com> References: <1knXD7Wc8heH83BTJEguqmlTAa70oXDw_nWI0hBjAm0=.bcf88238-2b63-4d14-b3a3-94eabca69586@github.com> Message-ID: On Thu, 14 Mar 2024 08:48:11 GMT, Hamlin Li wrote: > Hi, > Can you help to review this patch? > Thanks > > This is a continuation of work based on [1] by @XiaohongGong, most work was done in that pr. In this new pr, just rebased the code in [1], then added some minor changes (renaming of calling method, add libsleef as extra lib in CI cross-build on aarch64 in github workflow); I aslo tested the combination of following scenarios: > * at build time > * with/without sleef > * with/without sve support > * at runtime > * with/without sleef > * with/without sve support > > [1] https://github.com/openjdk/jdk/pull/16234 > > ## Regression Test > * test/jdk/jdk/incubator/vector/ > * test/hotspot/jtreg/compiler/vectorapi/ > > ## Performance Test > Previously, @XiaohongGong has shared the data: https://github.com/openjdk/jdk/pull/16234#issuecomment-1767727028 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/18294 From mikael at openjdk.org Wed Apr 3 16:28:05 2024 From: mikael at openjdk.org (Mikael Vidstedt) Date: Wed, 3 Apr 2024 16:28:05 GMT Subject: RFR: 8320799: Bump minimum boot jdk to JDK 22 In-Reply-To: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> References: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> Message-ID: On Mon, 1 Apr 2024 16:16:50 GMT, Mikael Vidstedt wrote: > With the JDK 22 GA out it's time to bump the minimum boot JDK version for mainline/JDK 23. > > Testing: tier1-5, GHA Thank you for the reviews! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18563#issuecomment-2035049134 From mikael at openjdk.org Wed Apr 3 16:28:06 2024 From: mikael at openjdk.org (Mikael Vidstedt) Date: Wed, 3 Apr 2024 16:28:06 GMT Subject: Integrated: 8320799: Bump minimum boot jdk to JDK 22 In-Reply-To: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> References: <8AjAsoXaDLVV8gYoSwngDVQv_AE28nfCJVPj7TRT1JQ=.26a14e38-f922-4251-a54d-a246edce8512@github.com> Message-ID: On Mon, 1 Apr 2024 16:16:50 GMT, Mikael Vidstedt wrote: > With the JDK 22 GA out it's time to bump the minimum boot JDK version for mainline/JDK 23. > > Testing: tier1-5, GHA This pull request has now been integrated. Changeset: 023f7f17 Author: Mikael Vidstedt URL: https://git.openjdk.org/jdk/commit/023f7f176b32ffa38dd599ea110c2b9c18886b74 Stats: 12 lines in 3 files changed: 0 ins; 0 del; 12 mod 8320799: Bump minimum boot jdk to JDK 22 Reviewed-by: iris, erikj, ihse ------------- PR: https://git.openjdk.org/jdk/pull/18563 From ihse at openjdk.org Wed Apr 3 19:26:00 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 3 Apr 2024 19:26:00 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF In-Reply-To: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Wed, 3 Apr 2024 14:40:42 GMT, Hamlin Li wrote: > Hi, > Can you help to review the patch? > This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). > > Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. > Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. > > Besides of the code changes, one important task is to handle the legal process. > > Thanks! Just a quick question after giving this a glance: My understanding was that the normal libsleef build set a lot of compiler options, e.g. disabling built-in maths etc. You don't seem to set any of these. Have you determined that they were not needed? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2035409207 From mchung at openjdk.org Wed Apr 3 22:34:14 2024 From: mchung at openjdk.org (Mandy Chung) Date: Wed, 3 Apr 2024 22:34:14 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: Message-ID: On Tue, 19 Mar 2024 16:55:14 GMT, Severin Gehwolf wrote: >> Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app being modularized). >> >> The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JRTArchive` class which has all the info of what constitutes the final jlinked runtime. >> >> Basic usage example: >> >> $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se) >> $ diff -u <(./bin/java --list-modules --limit-modules jdk.jlink) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink) >> $ ls ../linux-x86_64-server-release/images/jdk/jmods >> java.base.jmod java.net.http.jmod java.sql.rowset.jmod jdk.crypto.ec.jmod jdk.internal.opt.jmod jdk.jdi.jmod jdk.management.agent.jmod jdk.security.auth.jmod >> java.compiler.jmod java.prefs.jmod java.transaction.xa.jmod jdk.dynalink.jmod jdk.internal.vm.ci.jmod jdk.jdwp.agent.jmod jdk.management.jfr.jmod jdk.security.jgss.jmod >> java.datatransfer.jmod java.rmi.jmod java.xml.crypto.jmod jdk.editpad.jmod jdk.internal.vm.compiler.jmod jdk.jfr.jmod jdk.management.jmod jdk.unsupported.desktop.jmod >> java.desktop.jmod java.scripting.jmod java.xml.jmod jdk.hotspot.agent.jmod jdk.i... > > Severin Gehwolf has updated the pull request incrementally with one additional commit since the last revision: > > Move CreateLinkableRuntimePlugin to build folder > > Keep runtime link supporting classes in package > jdk.tools.jlink.internal.runtimelink Thanks for the investigation w.r.t. extending jimage. It does not seem worth the effort in pursuing the support of adding resources to an existing jimage file. To me, putting the diff data under `lib` directory as a private file is a simpler and acceptable solution. It allows the second jlink invocation to avoid the plugin pipeline if the per-module metadata is generated in normal jlink invocation. (An observation: The current implementation requires the diffs to be generated as a plugin. For this approach, it may be possible to implement with a separate main class that calls JLink API and generates the diffs.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2035719255 From gli at openjdk.org Thu Apr 4 03:45:02 2024 From: gli at openjdk.org (Guoxiong Li) Date: Thu, 4 Apr 2024 03:45:02 GMT Subject: RFR: 8329494: Serial: Merge GenMarkSweep into MarkSweep In-Reply-To: References: Message-ID: On Wed, 3 Apr 2024 09:56:42 GMT, Albert Mingkun Yang wrote: >> Hi all, >> >> This patch merges the class `GenMarkSweep` into `MarkSweep`. Just simply move. It may be better to merge the class `DeadSpacer` and `Compacter` into `MarkSweep` and then rename `MarkSweep`. The `MarkSweep` will be renamed at [JDK-8329521](https://bugs.openjdk.org/browse/JDK-8329521). And the `DeadSpacer` and `Compacter` may be refactored at [JDK-8305896](https://bugs.openjdk.org/browse/JDK-8305896) and [JDK-8305898](https://bugs.openjdk.org/browse/JDK-8305898), so I don't refactor them now. >> >> The tests `make test-tier1_gc` passed locally. Thanks for taking the time to review. >> >> Best Regards, >> -- Guoxiong > > Marked as reviewed by ayang (Reviewer). @albertnetymk @tschatzl Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18589#issuecomment-2036116806 From gli at openjdk.org Thu Apr 4 03:45:03 2024 From: gli at openjdk.org (Guoxiong Li) Date: Thu, 4 Apr 2024 03:45:03 GMT Subject: Integrated: 8329494: Serial: Merge GenMarkSweep into MarkSweep In-Reply-To: References: Message-ID: On Wed, 3 Apr 2024 03:10:26 GMT, Guoxiong Li wrote: > Hi all, > > This patch merges the class `GenMarkSweep` into `MarkSweep`. Just simply move. It may be better to merge the class `DeadSpacer` and `Compacter` into `MarkSweep` and then rename `MarkSweep`. The `MarkSweep` will be renamed at [JDK-8329521](https://bugs.openjdk.org/browse/JDK-8329521). And the `DeadSpacer` and `Compacter` may be refactored at [JDK-8305896](https://bugs.openjdk.org/browse/JDK-8305896) and [JDK-8305898](https://bugs.openjdk.org/browse/JDK-8305898), so I don't refactor them now. > > The tests `make test-tier1_gc` passed locally. Thanks for taking the time to review. > > Best Regards, > -- Guoxiong This pull request has now been integrated. Changeset: 41966885 Author: Guoxiong Li URL: https://git.openjdk.org/jdk/commit/41966885b9c0b71bf34431714702a8245ce3130b Stats: 1111 lines in 8 files changed: 513 ins; 589 del; 9 mod 8329494: Serial: Merge GenMarkSweep into MarkSweep Reviewed-by: ihse, ayang, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/18589 From sgehwolf at openjdk.org Thu Apr 4 07:49:19 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 07:49:19 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: Message-ID: On Wed, 3 Apr 2024 22:31:39 GMT, Mandy Chung wrote: > Thanks for the investigation w.r.t. extending jimage. It does not seem worth the effort in pursuing the support of adding resources to an existing jimage file. To me, putting the diff data under `lib` directory as a private file is a simpler and acceptable solution. It allows the second jlink invocation to avoid the plugin pipeline if the per-module metadata is generated in normal jlink invocation. Agreed. I'm pushing an update of the patch later today which does this. There is only one jlink call producing the final `jdk` image. That image is then amended to add the delta file under `lib`. The file is generated as a build step. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2036428963 From sgehwolf at openjdk.org Thu Apr 4 11:12:43 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 11:12:43 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: References: Message-ID: > Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app b eing modularized). > > The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JRTArchive` class which has all the info of what constitutes the final jlinked runtime. > > Basic usage example: > > $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se) > $ diff -u <(./bin/java --list-modules --limit-modules jdk.jlink) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink) > $ ls ../linux-x86_64-server-release/images/jdk/jmods > java.base.jmod java.net.http.jmod java.sql.rowset.jmod jdk.crypto.ec.jmod jdk.internal.opt.jmod jdk.jdi.jmod jdk.management.agent.jmod jdk.security.auth.jmod > java.compiler.jmod java.prefs.jmod java.transaction.xa.jmod jdk.dynalink.jmod jdk.internal.vm.ci.jmod jdk.jdwp.agent.jmod jdk.management.jfr.jmod jdk.security.jgss.jmod > java.datatransfer.jmod java.rmi.jmod java.xml.crypto.jmod jdk.editpad.jmod jdk.internal.vm.compiler.jmod jdk.jfr.jmod jdk.management.jmod jdk.unsupported.desktop.jmod > java.desktop.jmod java.scripting.jmod java.xml.jmod jdk.hotspot.agent.jmod jdk.internal.vm.compiler.manage... Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 100 commits: - Fix coment - Fix comment - Fix typo - Revert some now unneded build changes - Follow build tools naming convention - Update to new build-time approach with delta in lib - Make generation of fs_$module_files unconditional - Merge branch 'master' into jdk-8311302-jmodless-link - Fix copyright year - Move CreateLinkableRuntimePlugin to build folder Keep runtime link supporting classes in package jdk.tools.jlink.internal.runtimelink - ... and 90 more: https://git.openjdk.org/jdk/compare/6ae1cf12...ce04f42a ------------- Changes: https://git.openjdk.org/jdk/pull/14787/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14787&range=23 Stats: 3384 lines in 36 files changed: 3247 ins; 106 del; 31 mod Patch: https://git.openjdk.org/jdk/pull/14787.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14787/head:pull/14787 PR: https://git.openjdk.org/jdk/pull/14787 From sgehwolf at openjdk.org Thu Apr 4 12:10:15 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 12:10:15 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v20] In-Reply-To: References: Message-ID: On Thu, 14 Mar 2024 17:46:19 GMT, Erik Joelsson wrote: >> Severin Gehwolf has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix comment in autoconf file > > make/Images.gmk line 126: > >> 124: RL_BUILD_MODULE_NAME := jdk.unsupported_jlink_runtime >> 125: RL_CREATE_PLUGIN_MOD_OUTPUT := $(SUPPORT_OUTPUTDIR)/$(RL_BUILD_MODULE_NAME) >> 126: JDK_RUN_TIME_IMAGE_SUPPORT_DIR := $(SUPPORT_OUTPUTDIR)/images/runtime-link-support > > Suggestion: > > JDK_RUNTIME_IMAGE_SUPPORT_DIR := $(SUPPORT_OUTPUTDIR)/images/runtime-link-support > > or just inline it as it's only used in one location. Obsolete now. > make/Images.gmk line 132: > >> 130: JLINK_RUNTIME_CREATE_ARG += -J--add-exports=java.base/jdk.internal.jimage=$(RL_BUILD_MODULE_NAME) >> 131: JLINK_RUNTIME_CREATE_ARG += -J--add-exports=jdk.jlink/jdk.tools.jlink.internal=$(RL_BUILD_MODULE_NAME) >> 132: JLINK_RUNTIME_CREATE_ARG += --create-linkable-runtime jimage=$(JDK_LINK_OUTPUT_DIR)/lib/modules:module-path=$(IMAGES_OUTPUTDIR)/jmods > > I would suggest using recommendation 17 from the [style guideline](https://openjdk.org/groups/build/doc/code-conventions.html) here. > Suggestion: > > JLINK_RUNTIME_CREATE_ARG := \ > -J--module-path=$(RL_CREATE_PLUGIN_MOD_OUTPUT) \ > -J--add-exports=java.base/jdk.internal.module=$(RL_BUILD_MODULE_NAME) \ > -J--add-exports=java.base/jdk.internal.jimage=$(RL_BUILD_MODULE_NAME) \ > -J--add-exports=jdk.jlink/jdk.tools.jlink.internal=$(RL_BUILD_MODULE_NAME) \ > --create-linkable-runtime jimage=$(JDK_LINK_OUTPUT_DIR)/lib/modules:module-path=$(IMAGES_OUTPUTDIR)/jmods \ > # > > or just inline as it's only used in one location. No longer applicable. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551541291 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551542046 From sgehwolf at openjdk.org Thu Apr 4 12:10:15 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 12:10:15 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: <2vB9N80N5vzYIQiyt9yb2bufOR-l_mH4g5C4qZ9_Jm0=.57d0cc34-50b5-4965-8229-2c046014088d@github.com> References: <2vB9N80N5vzYIQiyt9yb2bufOR-l_mH4g5C4qZ9_Jm0=.57d0cc34-50b5-4965-8229-2c046014088d@github.com> Message-ID: On Thu, 21 Mar 2024 15:29:45 GMT, Severin Gehwolf wrote: >> make/Images.gmk line 145: >> >>> 143: $(eval $(call SetupJavaCompilation, BUILD_JDK_RUNLINK_CLASSES, \ >>> 144: COMPILER := buildjdk, \ >>> 145: DISABLED_WARNINGS := try, \ >> >> Why do we get warnings in the java code? > > That's not needed anymore. There are some `try` warnings in the `JmodsReader` and `JimageDiffGenerator` classes which used to get compiled with this. It'll probably change again... No longer there. >> src/jdk.jlink/share/classes/jdk/tools/jlink/internal/runtimelink/JimageDiffGenerator.java line 40: >> >>> 38: public class JimageDiffGenerator { >>> 39: >>> 40: private static final boolean DEBUG = false; >> >> This seems like left-over debug code. If this should go into product code I suggest you either remove it, or alternatively make it possible to change at runtime, if the debug functionality will be needed. > > OK. Removed in the latest version. The debug static in the build tools class can be enabled with a property. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551544541 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551547273 From sgehwolf at openjdk.org Thu Apr 4 12:10:15 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 12:10:15 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: <2vB9N80N5vzYIQiyt9yb2bufOR-l_mH4g5C4qZ9_Jm0=.57d0cc34-50b5-4965-8229-2c046014088d@github.com> Message-ID: On Thu, 21 Mar 2024 15:30:01 GMT, Magnus Ihse Bursie wrote: >> Thanks! This will also likely change. I'm thinking of just generating the diff file and putting it into `lib/` of the JDK image. That avoids needing to call this build-time only jlink plugin and this `FixPath` magic. > > I see. I'm sorry if I did not fully understand how the ongoing discussions affected build source changes. > > Maybe I should just put the build part review of this PR on the backburner, and then you can ping me when you think you have a solution that will stick, and I can check how it holds up from a build perspective? I hope the latest version is where we're headed now. I'll ping you for a review once the dust settles. Thanks! ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551544078 From sgehwolf at openjdk.org Thu Apr 4 12:10:15 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 12:10:15 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: <2vB9N80N5vzYIQiyt9yb2bufOR-l_mH4g5C4qZ9_Jm0=.57d0cc34-50b5-4965-8229-2c046014088d@github.com> Message-ID: On Thu, 21 Mar 2024 15:35:06 GMT, Severin Gehwolf wrote: >> Ok, fine. Will the new solution still include a build-time only tool? > > Yes, but I'll likely go with the interim solution and that would only require the a single "driver" class in the build tree with a `main` method. Now it's no longer a module, since no jlink plugin is being used any more. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551545310 From mli at openjdk.org Thu Apr 4 12:37:00 2024 From: mli at openjdk.org (Hamlin Li) Date: Thu, 4 Apr 2024 12:37:00 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Wed, 3 Apr 2024 19:23:01 GMT, Magnus Ihse Bursie wrote: > Just a quick question after giving this a glance: My understanding was that the normal libsleef build set a lot of compiler options, e.g. disabling built-in maths etc. You don't seem to set any of these. Have you determined that they were not needed? Thanks for having a look and quick response. Good question. Per `disabling built-in maths`, my understanding is that maybe we don't need to care about it, as this built-in math functions in compilers are only for scalar version, but we're using sleef's simd versions only which use vector intrinsics I think. e.g. in `src/libm/sleefdp.c` there is `ENABLE_BUILTIN_MATH` check, but in `src/libm/sleefsimdsp.c` there is no such check, so when generating inline header files, I assume its value (whether enable/disable built-in math) does not impact the generated simd functions. Please correct me if I'm understanding it wrongly. For other compiler options, I tend to agree with you, but I'm not sure which might need, can you supply more information or point to some reference about `normal libsleef build`? BTW, what I refered to before was from sleef.org and sleef on github (including its github workflow). ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2037072600 From erikj at openjdk.org Thu Apr 4 13:13:18 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 4 Apr 2024 13:13:18 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: References: Message-ID: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> On Thu, 4 Apr 2024 11:12:43 GMT, Severin Gehwolf wrote: >> Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app being modularized). >> >> The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JRTArchive` class which has all the info of what constitutes the final jlinked runtime. >> >> Basic usage example: >> >> $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se) >> $ diff -u <(./bin/java --list-modules --limit-modules jdk.jlink) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink) >> $ ls ../linux-x86_64-server-release/images/jdk/jmods >> java.base.jmod java.net.http.jmod java.sql.rowset.jmod jdk.crypto.ec.jmod jdk.internal.opt.jmod jdk.jdi.jmod jdk.management.agent.jmod jdk.security.auth.jmod >> java.compiler.jmod java.prefs.jmod java.transaction.xa.jmod jdk.dynalink.jmod jdk.internal.vm.ci.jmod jdk.jdwp.agent.jmod jdk.management.jfr.jmod jdk.security.jgss.jmod >> java.datatransfer.jmod java.rmi.jmod java.xml.crypto.jmod jdk.editpad.jmod jdk.internal.vm.compiler.jmod jdk.jfr.jmod jdk.management.jmod jdk.unsupported.desktop.jmod >> java.desktop.jmod java.scripting.jmod java.xml.jmod jdk.hotspot.agent.jmod jdk.i... > > Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 100 commits: > > - Fix coment > - Fix comment > - Fix typo > - Revert some now unneded build changes > - Follow build tools naming convention > - Update to new build-time approach with delta in lib > - Make generation of fs_$module_files unconditional > - Merge branch 'master' into jdk-8311302-jmodless-link > - Fix copyright year > - Move CreateLinkableRuntimePlugin to build folder > > Keep runtime link supporting classes in package > jdk.tools.jlink.internal.runtimelink > - ... and 90 more: https://git.openjdk.org/jdk/compare/6ae1cf12...ce04f42a The new approach certainly makes the build part simpler, which I appreciate. Left some polishing comments. You don't need to address them until the general approach is accepted. make/CompileToolsJdk.gmk line 50: > 48: EXCLUDES := \ > 49: build/tools/classlist \ > 50: build/tools/runtimeimagelinkdeltaproducer \ This directory name is comically long. I guess that's not really a problem, but perhaps "linkdelta" would be descriptive enough given that the class has the full name anyway? make/Images.gmk line 114: > 112: ifeq ($(JLINK_PRODUCE_RUNTIME_LINK_JDK), true) > 113: RL_BUILD_CLASSES := runtimeimagelinkdeltaproducer-classes > 114: RL_DELTA_GEN_CLASSES := $(BUILDTOOLS_OUTPUTDIR)/$(RL_BUILD_CLASSES) With a shorter name, this could be just one line. make/Images.gmk line 119: > 117: RL_DIFFS_OUTPUT_FILE_ARG := $(JDK_IMAGE_DIR)/lib/runtime-image-link.delta > 118: RL_MOD_PATH_ARG := $(IMAGES_OUTPUTDIR)/jmods > 119: TOOL_RUNTIME_IMAGE_LINK_DELTA_PRODUCER := $(BUILD_JAVA) --add-modules jdk.jlink \ All of these are only used once so should rather be inlined. I think that makes it easier to understand and read. make/Images.gmk line 122: > 120: --add-exports=jdk.jlink/jdk.tools.jlink.internal.runtimelink=ALL-UNNAMED \ > 121: --add-exports=java.base/jdk.internal.module=ALL-UNNAMED \ > 122: --add-exports=java.base/jdk.internal.jimage=ALL-UNNAMED \ These three are repeated in both compilation and runtime so could potentially be set in a variable to avoid the duplication. make/Images.gmk line 208: > 206: WARN := Creating CDS$$($1_$2_DUMP_TYPE) archive for jdk image for $1, \ > 207: INFO := Using CDS flags for $1: $$($1_$2_CDS_DUMP_FLAGS), \ > 208: DEPS := $$(FINAL_JDK_JLINK), \ Does this actually interfere with the cds archive creation? I would assume they output to different files and they aren't even running `java` from the same image. If not, I would skip the whole `FINAL_JDK_JLINK` and just add `$(diff_runtime_jdk)` to `JDK_TARGETS`. ------------- PR Review: https://git.openjdk.org/jdk/pull/14787#pullrequestreview-1979812209 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551631105 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551639403 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551648048 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551652030 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551636954 From sgehwolf at openjdk.org Thu Apr 4 15:24:22 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 15:24:22 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> Message-ID: On Thu, 4 Apr 2024 13:00:33 GMT, Erik Joelsson wrote: >> Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 100 commits: >> >> - Fix coment >> - Fix comment >> - Fix typo >> - Revert some now unneded build changes >> - Follow build tools naming convention >> - Update to new build-time approach with delta in lib >> - Make generation of fs_$module_files unconditional >> - Merge branch 'master' into jdk-8311302-jmodless-link >> - Fix copyright year >> - Move CreateLinkableRuntimePlugin to build folder >> >> Keep runtime link supporting classes in package >> jdk.tools.jlink.internal.runtimelink >> - ... and 90 more: https://git.openjdk.org/jdk/compare/6ae1cf12...ce04f42a > > make/Images.gmk line 208: > >> 206: WARN := Creating CDS$$($1_$2_DUMP_TYPE) archive for jdk image for $1, \ >> 207: INFO := Using CDS flags for $1: $$($1_$2_CDS_DUMP_FLAGS), \ >> 208: DEPS := $$(FINAL_JDK_JLINK), \ > > Does this actually interfere with the cds archive creation? I would assume they output to different files and they aren't even running `java` from the same image. If not, I would skip the whole `FINAL_JDK_JLINK` and just add `$(diff_runtime_jdk)` to `JDK_TARGETS`. You are right. It no longer does (the old approach did, though). I'll clean it up. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551898463 From sgehwolf at openjdk.org Thu Apr 4 15:27:24 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 15:27:24 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> Message-ID: On Thu, 4 Apr 2024 12:56:41 GMT, Erik Joelsson wrote: >> Severin Gehwolf has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 100 commits: >> >> - Fix coment >> - Fix comment >> - Fix typo >> - Revert some now unneded build changes >> - Follow build tools naming convention >> - Update to new build-time approach with delta in lib >> - Make generation of fs_$module_files unconditional >> - Merge branch 'master' into jdk-8311302-jmodless-link >> - Fix copyright year >> - Move CreateLinkableRuntimePlugin to build folder >> >> Keep runtime link supporting classes in package >> jdk.tools.jlink.internal.runtimelink >> - ... and 90 more: https://git.openjdk.org/jdk/compare/6ae1cf12...ce04f42a > > make/CompileToolsJdk.gmk line 50: > >> 48: EXCLUDES := \ >> 49: build/tools/classlist \ >> 50: build/tools/runtimeimagelinkdeltaproducer \ > > This directory name is comically long. I guess that's not really a problem, but perhaps "linkdelta" would be descriptive enough given that the class has the full name anyway? FYI: This was trying to address a comment from @magicus https://github.com/openjdk/jdk/pull/14787#discussion_r1514894494 Happy to not follow that convention and/or rename the tool. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551909878 From erikj at openjdk.org Thu Apr 4 15:41:17 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 4 Apr 2024 15:41:17 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> Message-ID: <4NcIbddx8UZky7IIR9XW0yRFXAARqA1HQyeH1eejk20=.e60906ea-8a4b-4932-99f9-0fb9e4ca0ca7@github.com> On Thu, 4 Apr 2024 15:24:53 GMT, Severin Gehwolf wrote: >> make/CompileToolsJdk.gmk line 50: >> >>> 48: EXCLUDES := \ >>> 49: build/tools/classlist \ >>> 50: build/tools/runtimeimagelinkdeltaproducer \ >> >> This directory name is comically long. I guess that's not really a problem, but perhaps "linkdelta" would be descriptive enough given that the class has the full name anyway? > > FYI: This was trying to address a comment from @magicus https://github.com/openjdk/jdk/pull/14787#discussion_r1514894494 Happy to not follow that convention and/or rename the tool. I was not aware of such a convention and I can't say I agree with it. It just seems redundant and unnecessary, but I suppose we should wait for Magnus to respond. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1551940327 From ihse at openjdk.org Thu Apr 4 16:13:33 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 16:13:33 GMT Subject: RFR: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation Message-ID: This patch will fix the few remaining places where a "raw" call to SetupNativeCompilation was made, instead of going via SetupJdkNativeCompilation. This include repairing the poor old X11Wrapper, which has been broken for some time. After this we can finally make SetupNativeCompilation an "inner macro" of SetupJdkNativeCompilation, and stop exposing two different ways of compiling native code. I have also added standard header comments to some places where I have missed it, and corrected an incorrect indentation (all things I found while double-checking all SetupJdkNativeCompilation calls). ------------- Commit messages: - 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation Changes: https://git.openjdk.org/jdk/pull/18631/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18631&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8329672 Stats: 107 lines in 15 files changed: 46 ins; 23 del; 38 mod Patch: https://git.openjdk.org/jdk/pull/18631.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18631/head:pull/18631 PR: https://git.openjdk.org/jdk/pull/18631 From aph at openjdk.org Thu Apr 4 16:33:08 2024 From: aph at openjdk.org (Andrew Haley) Date: Thu, 4 Apr 2024 16:33:08 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: Message-ID: <87t3reTHf8QtR506vIL1EY_FewcjebGG7JhLDdfBtkY=.b3f68cdf-5c34-467d-853f-f1cbc432b40d@github.com> On Mon, 25 Mar 2024 09:11:40 GMT, Magnus Ihse Bursie wrote: > > And neither should we compile or link it with "-fvisibility=hidden". That is the root of this problem. > > If you suggest that we should not compile hsdis with hidden visibility, I disagree. Yes, that's what I would do. > I have been working hard on unifying build of native libraries across the entire product, to fix holes where we have not used a consistent way of compiling and/or linking. There is no reason to tread hsdis differently. If I restore using hidden visibility as an option that all native libraries, except hsdis, must opt in to, then we are just back to square one, and suddenly someone will forget about it. Instead, now we set -fvisibility=hidden in configure so nobody can forget about it. OK, OK! So please can we get this fix in? > Robbin proposes to change this to > > ``` > #if defined(_WIN32) > __declspec(dllexport) > #elif defined(_GNU_SOURCE) > __attribute__ ((visibility ("default"))) > #endif > ``` > > My counter-proposal was to replace it with just `JNIEXPORT`. Surely you can't say that is a worse solution? JNIEXPORT is better, I guess, but it does mean that hsdis is no longer standalone, and IMO it should have a pathological dependency on some JVM header file. That's the problem here. But really, whatever works is good with me. The last thing I want to do is delay this fix any further. Robbin has asked "How would you add jni.h ?" Is anyone going to answer? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2037672904 From erikj at openjdk.org Thu Apr 4 16:45:13 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 4 Apr 2024 16:45:13 GMT Subject: RFR: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation In-Reply-To: References: Message-ID: On Thu, 4 Apr 2024 16:02:53 GMT, Magnus Ihse Bursie wrote: > This patch will fix the few remaining places where a "raw" call to SetupNativeCompilation was made, instead of going via SetupJdkNativeCompilation. This include repairing the poor old X11Wrapper, which has been broken for some time. > > After this we can finally make SetupNativeCompilation an "inner macro" of SetupJdkNativeCompilation, and stop exposing two different ways of compiling native code. > > I have also added standard header comments to some places where I have missed it, and corrected an incorrect indentation (all things I found while double-checking all SetupJdkNativeCompilation calls). Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18631#pullrequestreview-1980539107 From ihse at openjdk.org Thu Apr 4 16:46:00 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 16:46:00 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: Message-ID: On Thu, 21 Mar 2024 06:58:43 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > remove swap file Ideally, it should just be enough to add `#import "jni.h"`. This will definitely work for capstone and llvm. However, for binutils we disable our standard C include paths, so this will fail. (I'm not sure if this is really necessary, but that is how we currently do it) To solve it you can either set `HSDIS_TOOLCHAIN_DEFAULT_CFLAGS` to `true` for binutils, or you can add the include path to jni.h directly. The former would be the best, but in case that does not work, try adding `java.base:include` to `EXTRA_HEADER_DIRS`. I apologize for the late reply. I've been just working spotty hours due to spring break. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2037695449 PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2037696471 From sgehwolf at openjdk.org Thu Apr 4 16:46:32 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Thu, 4 Apr 2024 16:46:32 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v25] In-Reply-To: References: Message-ID: <2Hs5V_cOOhOtXxrKvM0ATWxJFO62xd3RtScyPEb5puI=.f16650ac-9812-40ed-a93c-a5dd4b396ad2@github.com> > Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app b eing modularized). > > The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JRTArchive` class which has all the info of what constitutes the final jlinked runtime. > > Basic usage example: > > $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se) > $ diff -u <(./bin/java --list-modules --limit-modules jdk.jlink) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink) > $ ls ../linux-x86_64-server-release/images/jdk/jmods > java.base.jmod java.net.http.jmod java.sql.rowset.jmod jdk.crypto.ec.jmod jdk.internal.opt.jmod jdk.jdi.jmod jdk.management.agent.jmod jdk.security.auth.jmod > java.compiler.jmod java.prefs.jmod java.transaction.xa.jmod jdk.dynalink.jmod jdk.internal.vm.ci.jmod jdk.jdwp.agent.jmod jdk.management.jfr.jmod jdk.security.jgss.jmod > java.datatransfer.jmod java.rmi.jmod java.xml.crypto.jmod jdk.editpad.jmod jdk.internal.vm.compiler.jmod jdk.jfr.jmod jdk.management.jmod jdk.unsupported.desktop.jmod > java.desktop.jmod java.scripting.jmod java.xml.jmod jdk.hotspot.agent.jmod jdk.internal.vm.compiler.manage... Severin Gehwolf has updated the pull request incrementally with three additional commits since the last revision: - Use shorter name for the build tool - Review feedback from Erik J. - Remove dependency on CDS which isn't needed anymore ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14787/files - new: https://git.openjdk.org/jdk/pull/14787/files/ce04f42a..84d4feff Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14787&range=24 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14787&range=23-24 Stats: 32 lines in 3 files changed: 3 ins; 16 del; 13 mod Patch: https://git.openjdk.org/jdk/pull/14787.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14787/head:pull/14787 PR: https://git.openjdk.org/jdk/pull/14787 From erikj at openjdk.org Thu Apr 4 16:49:08 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 4 Apr 2024 16:49:08 GMT Subject: RFR: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation In-Reply-To: References: Message-ID: On Thu, 4 Apr 2024 16:02:53 GMT, Magnus Ihse Bursie wrote: > This patch will fix the few remaining places where a "raw" call to SetupNativeCompilation was made, instead of going via SetupJdkNativeCompilation. This include repairing the poor old X11Wrapper, which has been broken for some time. > > After this we can finally make SetupNativeCompilation an "inner macro" of SetupJdkNativeCompilation, and stop exposing two different ways of compiling native code. > > I have also added standard header comments to some places where I have missed it, and corrected an incorrect indentation (all things I found while double-checking all SetupJdkNativeCompilation calls). Looks like windows build is failing in GHA. ------------- Changes requested by erikj (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18631#pullrequestreview-1980552191 From ihse at openjdk.org Thu Apr 4 16:50:08 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 16:50:08 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF In-Reply-To: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Wed, 3 Apr 2024 14:40:42 GMT, Hamlin Li wrote: > Hi, > Can you help to review the patch? > This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). > > Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. > Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. > > Besides of the code changes, one important task is to handle the legal process. > > Thanks! Build libsleef using their cmake system and look at the compile command line. (You do this by `VERBOSE=1 cmake` IIRC). Then you can see what flags they are using. This is what I was referring to as "normal libsleef build". I noticed there were a lot of compiler flags. I can't say if they are needed or not. In most cases, if it compilers, it's fine, but in this case, I guess some flags can be crucial to really get the kind of performance you need, and it might not be easy to spot that something is wrong if you get them incorrect. I assume one way to make sure is to run microbenchmarks with an externally built libsleef and compare it with the one you build within the JDK. If there is no noticeable difference, then I guess it is fine. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2037703774 From ihse at openjdk.org Thu Apr 4 16:53:01 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 16:53:01 GMT Subject: RFR: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation In-Reply-To: References: Message-ID: <--W0YPlHfGG9ZgCW-Ayt3kPqzj8ILvHmjz-aIeUqgMA=.e0fdd9d6-777a-49df-9aa9-1063d2391879@github.com> On Thu, 4 Apr 2024 16:46:39 GMT, Erik Joelsson wrote: > Looks like windows build is failing in GHA. I see. I thought it passed on my local Windows testing, but I might have done some changes (that I thought were "trivial") afterwards, since I see that it breaks there now as well. :-( ------------- PR Comment: https://git.openjdk.org/jdk/pull/18631#issuecomment-2037708913 From ihse at openjdk.org Thu Apr 4 20:50:25 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 20:50:25 GMT Subject: RFR: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation [v2] In-Reply-To: References: Message-ID: > This patch will fix the few remaining places where a "raw" call to SetupNativeCompilation was made, instead of going via SetupJdkNativeCompilation. This include repairing the poor old X11Wrapper, which has been broken for some time. > > After this we can finally make SetupNativeCompilation an "inner macro" of SetupJdkNativeCompilation, and stop exposing two different ways of compiling native code. > > I have also added standard header comments to some places where I have missed it, and corrected an incorrect indentation (all things I found while double-checking all SetupJdkNativeCompilation calls). Magnus Ihse Bursie has updated the pull request incrementally with three additional commits since the last revision: - Update documentation - Do not add LIBCXX for tests - Add ability to disable DEFAULT_VERSIONINFO_RESOURCE ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18631/files - new: https://git.openjdk.org/jdk/pull/18631/files/a592e19f..ecd86477 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18631&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18631&range=00-01 Stats: 15 lines in 3 files changed: 11 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/18631.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18631/head:pull/18631 PR: https://git.openjdk.org/jdk/pull/18631 From ihse at openjdk.org Thu Apr 4 20:59:09 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 20:59:09 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v25] In-Reply-To: <2Hs5V_cOOhOtXxrKvM0ATWxJFO62xd3RtScyPEb5puI=.f16650ac-9812-40ed-a93c-a5dd4b396ad2@github.com> References: <2Hs5V_cOOhOtXxrKvM0ATWxJFO62xd3RtScyPEb5puI=.f16650ac-9812-40ed-a93c-a5dd4b396ad2@github.com> Message-ID: On Thu, 4 Apr 2024 16:46:32 GMT, Severin Gehwolf wrote: >> Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app being modularized). >> >> The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JRTArchive` class which has all the info of what constitutes the final jlinked runtime. >> >> Basic usage example: >> >> $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se) >> $ diff -u <(./bin/java --list-modules --limit-modules jdk.jlink) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink) >> $ ls ../linux-x86_64-server-release/images/jdk/jmods >> java.base.jmod java.net.http.jmod java.sql.rowset.jmod jdk.crypto.ec.jmod jdk.internal.opt.jmod jdk.jdi.jmod jdk.management.agent.jmod jdk.security.auth.jmod >> java.compiler.jmod java.prefs.jmod java.transaction.xa.jmod jdk.dynalink.jmod jdk.internal.vm.ci.jmod jdk.jdwp.agent.jmod jdk.management.jfr.jmod jdk.security.jgss.jmod >> java.datatransfer.jmod java.rmi.jmod java.xml.crypto.jmod jdk.editpad.jmod jdk.internal.vm.compiler.jmod jdk.jfr.jmod jdk.management.jmod jdk.unsupported.desktop.jmod >> java.desktop.jmod java.scripting.jmod java.xml.jmod jdk.hotspot.agent.jmod jdk.i... > > Severin Gehwolf has updated the pull request incrementally with three additional commits since the last revision: > > - Use shorter name for the build tool > - Review feedback from Erik J. > - Remove dependency on CDS which isn't needed anymore Overall, I think the build system changes in the current revision of the patch look good, but please let me know for a deeper review when things have stabilized. src/jdk.jlink/share/classes/jdk/tools/jlink/resources/jlink.properties line 165: > 163: > 164: runtime.link.info=Linking based on the current run-time image. > 165: runtime.link.jprt.path.extra=(run-time image) Missing newline at last line. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2038205036 PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1552425957 From ihse at openjdk.org Thu Apr 4 20:59:10 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 20:59:10 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: <4NcIbddx8UZky7IIR9XW0yRFXAARqA1HQyeH1eejk20=.e60906ea-8a4b-4932-99f9-0fb9e4ca0ca7@github.com> References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> <4NcIbddx8UZky7IIR9XW0yRFXAARqA1HQyeH1eejk20=.e60906ea-8a4b-4932-99f9-0fb9e4ca0ca7@github.com> Message-ID: On Thu, 4 Apr 2024 15:38:36 GMT, Erik Joelsson wrote: >> FYI: This was trying to address a comment from @magicus https://github.com/openjdk/jdk/pull/14787#discussion_r1514894494 Happy to not follow that convention and/or rename the tool. > > I was not aware of such a convention and I can't say I agree with it. It just seems redundant and unnecessary, but I suppose we should wait for Magnus to respond. Just to clarify: I did not say the name needed to be long, just that many (if not all) tools has used the convention of using the package name `build.tools.` and the class name `.java`. I think the new name sounds ? . ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1552430689 From ihse at openjdk.org Thu Apr 4 21:22:00 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 21:22:00 GMT Subject: RFR: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation [v2] In-Reply-To: References: Message-ID: On Thu, 4 Apr 2024 20:50:25 GMT, Magnus Ihse Bursie wrote: >> This patch will fix the few remaining places where a "raw" call to SetupNativeCompilation was made, instead of going via SetupJdkNativeCompilation. This include repairing the poor old X11Wrapper, which has been broken for some time. >> >> After this we can finally make SetupNativeCompilation an "inner macro" of SetupJdkNativeCompilation, and stop exposing two different ways of compiling native code. >> >> I have also added standard header comments to some places where I have missed it, and corrected an incorrect indentation (all things I found while double-checking all SetupJdkNativeCompilation calls). > > Magnus Ihse Bursie has updated the pull request incrementally with three additional commits since the last revision: > > - Update documentation > - Do not add LIBCXX for tests > - Add ability to disable DEFAULT_VERSIONINFO_RESOURCE Just to confirm; this PR now passes with byte-by-byte identical builds compared to master. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18631#issuecomment-2038245442 From prr at openjdk.org Thu Apr 4 21:31:02 2024 From: prr at openjdk.org (Phil Race) Date: Thu, 4 Apr 2024 21:31:02 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 15:45:59 GMT, Magnus Ihse Bursie wrote: > This is a retake on https://github.com/openjdk/jdk/pull/15096. > > I tried to fix the remaining issues in that PR, but could not push them. In the end, it seemed easier to create a new branch in my own personal fork. > > The majority of the work in this PR has been done by @TheShermanTanker. I have stepped in and fixed the remaining comments, reverted some additional unneeded changed code, and also tried to minimize code duplication in `awt_PrintJob.cpp`. Marked as reviewed by prr (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18584#pullrequestreview-1981217028 From mikael at openjdk.org Thu Apr 4 21:59:12 2024 From: mikael at openjdk.org (Mikael Vidstedt) Date: Thu, 4 Apr 2024 21:59:12 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF In-Reply-To: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Wed, 3 Apr 2024 14:40:42 GMT, Hamlin Li wrote: > Hi, > Can you help to review the patch? > This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). > > Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. > Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. > > Besides of the code changes, one important task is to handle the legal process. > > Thanks! make/modules/jdk.incubator.vector/Lib.gmk line 44: > 42: $(eval $(call SetupJdkLibrary, BUILD_LIBVECTORMATH, \ > 43: NAME := vectormath, \ > 44: CFLAGS := $(CFLAGS_JDKLIB) -Wno-error=unused-function, \ Should the unused-function be passed in using `DISABLE_WARNINGS_*` instead? src/hotspot/cpu/aarch64/stubGenerator_aarch64.cpp line 8601: > 8599: } > 8600: } else { > 8601: log_info(library)("Failed to load native vector math library!"); Include the `ebuf` message? The corresponding x86_64 code could also use a log message for the error case. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18605#discussion_r1552502695 PR Review Comment: https://git.openjdk.org/jdk/pull/18605#discussion_r1552499482 From ihse at openjdk.org Thu Apr 4 22:47:18 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 4 Apr 2024 22:47:18 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v2] In-Reply-To: References: Message-ID: > This is a retake on https://github.com/openjdk/jdk/pull/15096. > > I tried to fix the remaining issues in that PR, but could not push them. In the end, it seemed easier to create a new branch in my own personal fork. > > The majority of the work in this PR has been done by @TheShermanTanker. I have stepped in and fixed the remaining comments, reverted some additional unneeded changed code, and also tried to minimize code duplication in `awt_PrintJob.cpp`. Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge branch 'master' into awt-permissive-minus - 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18584/files - new: https://git.openjdk.org/jdk/pull/18584/files/e9780f0c..868dc56d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18584&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18584&range=00-01 Stats: 9831 lines in 307 files changed: 3321 ins; 4776 del; 1734 mod Patch: https://git.openjdk.org/jdk/pull/18584.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18584/head:pull/18584 PR: https://git.openjdk.org/jdk/pull/18584 From erikj at openjdk.org Fri Apr 5 00:09:00 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 5 Apr 2024 00:09:00 GMT Subject: RFR: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation [v2] In-Reply-To: References: Message-ID: On Thu, 4 Apr 2024 20:50:25 GMT, Magnus Ihse Bursie wrote: >> This patch will fix the few remaining places where a "raw" call to SetupNativeCompilation was made, instead of going via SetupJdkNativeCompilation. This include repairing the poor old X11Wrapper, which has been broken for some time. >> >> After this we can finally make SetupNativeCompilation an "inner macro" of SetupJdkNativeCompilation, and stop exposing two different ways of compiling native code. >> >> I have also added standard header comments to some places where I have missed it, and corrected an incorrect indentation (all things I found while double-checking all SetupJdkNativeCompilation calls). > > Magnus Ihse Bursie has updated the pull request incrementally with three additional commits since the last revision: > > - Update documentation > - Do not add LIBCXX for tests > - Add ability to disable DEFAULT_VERSIONINFO_RESOURCE Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18631#pullrequestreview-1981425382 From jwaters at openjdk.org Fri Apr 5 05:51:09 2024 From: jwaters at openjdk.org (Julian Waters) Date: Fri, 5 Apr 2024 05:51:09 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v2] In-Reply-To: References: Message-ID: On Thu, 4 Apr 2024 22:47:18 GMT, Magnus Ihse Bursie wrote: >> This is a retake on https://github.com/openjdk/jdk/pull/15096. >> >> I tried to fix the remaining issues in that PR, but could not push them. In the end, it seemed easier to create a new branch in my own personal fork. >> >> The majority of the work in this PR has been done by @TheShermanTanker. I have stepped in and fixed the remaining comments, reverted some additional unneeded changed code, and also tried to minimize code duplication in `awt_PrintJob.cpp`. > > Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: > > - Merge branch 'master' into awt-permissive-minus > - 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler src/java.desktop/windows/native/libawt/windows/awt_Component.cpp line 6370: > 6368: AwtComponent *awtParent = NULL; > 6369: > 6370: if (self == NULL) { I had missed this in my earlier sweep of the changes, but didn't Phil request that both the check for self and parent be merged into self == NULL || parent == NULL? (Or as I'd prefer, nullptr) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18584#discussion_r1552961774 From ihse at openjdk.org Fri Apr 5 08:58:12 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 08:58:12 GMT Subject: Integrated: 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation In-Reply-To: References: Message-ID: <_5HjIAHdoS7BIG3EpH8szS0fk4oqSRMg-2l2ecqgp-s=.7e8b399b-0cd4-41f5-a62a-5ee58394a740@github.com> On Thu, 4 Apr 2024 16:02:53 GMT, Magnus Ihse Bursie wrote: > This patch will fix the few remaining places where a "raw" call to SetupNativeCompilation was made, instead of going via SetupJdkNativeCompilation. This include repairing the poor old X11Wrapper, which has been broken for some time. > > After this we can finally make SetupNativeCompilation an "inner macro" of SetupJdkNativeCompilation, and stop exposing two different ways of compiling native code. > > I have also added standard header comments to some places where I have missed it, and corrected an incorrect indentation (all things I found while double-checking all SetupJdkNativeCompilation calls). This pull request has now been integrated. Changeset: 3f4b167c Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/3f4b167c974881f5f7ea1c621c7efe2f550cb60c Stats: 122 lines in 15 files changed: 57 ins; 23 del; 42 mod 8329672: Only call SetupNativeCompilation from SetupJdkNativeCompilation Reviewed-by: erikj ------------- PR: https://git.openjdk.org/jdk/pull/18631 From sgehwolf at openjdk.org Fri Apr 5 08:58:16 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Fri, 5 Apr 2024 08:58:16 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> <4NcIbddx8UZky7IIR9XW0yRFXAARqA1HQyeH1eejk20=.e60906ea-8a4b-4932-99f9-0fb9e4ca0ca7@github.com> Message-ID: <0c6Kwv_iDi97U1H1jtBktZjgo2a6lC-WCT17TFBQm1g=.da11a706-3c9f-4e55-bf6f-b9afa3d28952@github.com> On Thu, 4 Apr 2024 20:56:02 GMT, Magnus Ihse Bursie wrote: >> I was not aware of such a convention and I can't say I agree with it. It just seems redundant and unnecessary, but I suppose we should wait for Magnus to respond. > > Just to clarify: I did not say the name needed to be long, just that many (if not all) tools has used the convention of using the package name `build.tools.` and the class name `.java`. > > I think the new name sounds ? . Thanks. Yes, the long name was my doing. Sorry. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1553173105 From ihse at openjdk.org Fri Apr 5 08:58:16 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 08:58:16 GMT Subject: RFR: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler [v2] In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 05:48:40 GMT, Julian Waters wrote: >> Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: >> >> - Merge branch 'master' into awt-permissive-minus >> - 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler > > src/java.desktop/windows/native/libawt/windows/awt_Component.cpp line 6370: > >> 6368: AwtComponent *awtParent = NULL; >> 6369: >> 6370: if (self == NULL) { > > I had missed this in my earlier sweep of the changes, but didn't Phil request that both the check for self and parent be merged into self == NULL || parent == NULL? (Or as I'd prefer, nullptr) I noticed Phil's comment, but I could not make it look nice. For instance, the NPE error message was unclear, and you could not see the connection between self->awtComponent and parent->awtParent, which was obvious in the original code. In the end, I thought it was preferable to use the exact same style when expanding the macro as elsewhere. That creates a common pattern across the codebase. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18584#discussion_r1553174771 From ihse at openjdk.org Fri Apr 5 08:58:18 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 08:58:18 GMT Subject: Integrated: 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 15:45:59 GMT, Magnus Ihse Bursie wrote: > This is a retake on https://github.com/openjdk/jdk/pull/15096. > > I tried to fix the remaining issues in that PR, but could not push them. In the end, it seemed easier to create a new branch in my own personal fork. > > The majority of the work in this PR has been done by @TheShermanTanker. I have stepped in and fixed the remaining comments, reverted some additional unneeded changed code, and also tried to minimize code duplication in `awt_PrintJob.cpp`. This pull request has now been integrated. Changeset: 8bc1867d Author: Julian Waters Committer: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/8bc1867da78ea0b7664892ee277af413ef503b61 Stats: 428 lines in 12 files changed: 284 ins; 50 del; 94 mod 8307160: Fix AWT/2D/A11Y to support the permissive- flag on the Microsoft Visual C compiler Co-authored-by: Magnus Ihse Bursie Reviewed-by: jwaters, prr ------------- PR: https://git.openjdk.org/jdk/pull/18584 From jbhateja at openjdk.org Fri Apr 5 09:20:10 2024 From: jbhateja at openjdk.org (Jatin Bhateja) Date: Fri, 5 Apr 2024 09:20:10 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 19:19:59 GMT, Volodymyr Paprotski wrote: >> Performance. Before: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s >> >> Performance, no intrinsic: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply true thrpt 3 1919.57... > > Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: > > remove use of jdk.crypto.ec Few early comments. Please update the copyright year of all the modified files. You can even consider splitting this into two patches, Java side changes in one and x86 optimized intrinsic in next one. src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp line 39: > 37: }; > 38: static address modulus_p256() { > 39: return (address)MODULUS_P256; Long constants should have UL suffix. src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp line 386: > 384: __ jcc(Assembler::equal, L_Length19); > 385: > 386: // Default copy loop Please add appropriate loop entry alignment. src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp line 394: > 392: __ lea(aLimbs, Address(aLimbs,8)); > 393: __ lea(bLimbs, Address(bLimbs,8)); > 394: __ jmp(L_DefaultLoop); Both sub and cmp are flag affecting instructions and are macro-fusible. By doing a loop rotation i.e. moving the length <= 0 check outside the loop and pushing the loop exit check at bottom you can save additional compare checks. ------------- PR Review: https://git.openjdk.org/jdk/pull/18583#pullrequestreview-1981555803 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1553056633 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1552710600 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1553110376 From ihse at openjdk.org Fri Apr 5 09:34:19 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 09:34:19 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: <0c6Kwv_iDi97U1H1jtBktZjgo2a6lC-WCT17TFBQm1g=.da11a706-3c9f-4e55-bf6f-b9afa3d28952@github.com> References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> <4NcIbddx8UZky7IIR9XW0yRFXAARqA1HQyeH1eejk20=.e60906ea-8a4b-4932-99f9-0fb9e4ca0ca7@github.com> <0c6Kwv_iDi97U1H1jtBktZjgo2a6lC-WCT17TFBQm1g=.da11a706-3c9f-4e55-bf6f-b9afa3d28952@github.com> Message-ID: On Fri, 5 Apr 2024 08:21:09 GMT, Severin Gehwolf wrote: >> Just to clarify: I did not say the name needed to be long, just that many (if not all) tools has used the convention of using the package name `build.tools.` and the class name `.java`. >> >> I think the new name sounds ? . > > Thanks. Yes, the long name was my doing. Sorry. Kind of aligning with the "Donaudampfschiffahrtsgesellschaftskapit?n" prejudice of German. ;-) (In Sweden, we have "flaggst?ngsknoppsf?rgyllare" so you are not alone) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1553253017 From sgehwolf at openjdk.org Fri Apr 5 09:34:20 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Fri, 5 Apr 2024 09:34:20 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> <4NcIbddx8UZky7IIR9XW0yRFXAARqA1HQyeH1eejk20=.e60906ea-8a4b-4932-99f9-0fb9e4ca0ca7@github.com> <0c6Kwv_iDi97U1H1jtBktZjgo2a6lC-WCT17TFBQm1g=.da11a706-3c9f-4e55-bf6f-b9afa3d28952@github.com> Message-ID: <8tI50-YtIdOPXzFlbhjmFNGkdK8-Tb6cSRwBmdAypqM=.270ddf3b-8acb-44c1-af71-d1274b29c3b3@github.com> On Fri, 5 Apr 2024 09:25:49 GMT, Magnus Ihse Bursie wrote: >> Thanks. Yes, the long name was my doing. Sorry. > > Kind of aligning with the "Donaudampfschiffahrtsgesellschaftskapit?n" prejudice of German. ;-) > > > > (In Sweden, we have "flaggst?ngsknoppsf?rgyllare" so you are not alone) Hah! My kids just recently informed me about "Donaudampfschiffahrtsgesellschaftskapit?nsm?tzenproductionsst?tte"... or whatever else you can come up with :) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1553260930 From rehn at openjdk.org Fri Apr 5 10:45:24 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Fri, 5 Apr 2024 10:45:24 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v3] In-Reply-To: References: Message-ID: > Hi, please consider. > > [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. > Tested with gcc and clang, and llvm and binutils backend. > > I didn't find any use of the "DLL_ENTRY", so I removed it. > > Thanks, Robbin Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: Use JNIEXPORT ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18400/files - new: https://git.openjdk.org/jdk/pull/18400/files/28862745..b4003e9f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18400&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18400&range=01-02 Stats: 22 lines in 5 files changed: 4 ins; 12 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/18400.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18400/head:pull/18400 PR: https://git.openjdk.org/jdk/pull/18400 From rehn at openjdk.org Fri Apr 5 10:45:24 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Fri, 5 Apr 2024 10:45:24 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: Message-ID: <2wol6Z3Rd2zNpWUl8GWlXiC_vRWMzcuVFQh_-E_j_w0=.35a4b254-ce8b-4bad-a03e-381d3c2ddae8@github.com> On Thu, 21 Mar 2024 06:58:43 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > remove swap file Thanks, updated. (Since windows build seems to need HSDIS_TOOLCHAIN_DEFAULT_CFLAGS, just adding java.base:include seemed a good easy way) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2039470504 From adinn at openjdk.org Fri Apr 5 11:18:16 2024 From: adinn at openjdk.org (Andrew Dinn) Date: Fri, 5 Apr 2024 11:18:16 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v24] In-Reply-To: <8tI50-YtIdOPXzFlbhjmFNGkdK8-Tb6cSRwBmdAypqM=.270ddf3b-8acb-44c1-af71-d1274b29c3b3@github.com> References: <14SkXHYJBCXlLvZoRJjvyg3uronlFaxUbM-tsCQURMQ=.8a5bf8d3-7774-4fec-af1a-049f8fc004ab@github.com> <4NcIbddx8UZky7IIR9XW0yRFXAARqA1HQyeH1eejk20=.e60906ea-8a4b-4932-99f9-0fb9e4ca0ca7@github.com> <0c6Kwv_iDi97U1H1jtBktZjgo2a6lC-WCT17TFBQm1g=.da11a706-3c9f-4e55-bf6f-b9afa3d28952@github.com> <8tI50-YtIdOPXzFlbhjmFNGkdK8-Tb6cSRwBmdAypqM=.270ddf3b-8acb-44c1-af71-d1274b29c3b3@github.com> Message-ID: <71UPW7nwAv56c-Kxn6IAeeDgdktknQbpSWs14sJ9nxM=.260aeb98-eba9-4e79-9729-e65acadc1359@github.com> On Fri, 5 Apr 2024 09:31:18 GMT, Severin Gehwolf wrote: >> Kind of aligning with the "Donaudampfschiffahrtsgesellschaftskapit?n" prejudice of German. ;-) >> >> >> >> (In Sweden, we have "flaggst?ngsknoppsf?rgyllare" so you are not alone) > > Hah! My kids just recently informed me about "Donaudampfschiffahrtsgesellschaftskapit?nsm?tzenproductionsst?tte"... or whatever else you can come up with :) Hmm, that's nothing. Look up Rhabarberbarbara on YouTube. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14787#discussion_r1553444805 From mli at openjdk.org Fri Apr 5 12:17:17 2024 From: mli at openjdk.org (Hamlin Li) Date: Fri, 5 Apr 2024 12:17:17 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: > Hi, > Can you help to review the patch? > This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). > > Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. > Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. > > Besides of the code changes, one important task is to handle the legal process. > > Thanks! Hamlin Li has updated the pull request incrementally with two additional commits since the last revision: - disable unused-function warnings; add log msg - minor ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18605/files - new: https://git.openjdk.org/jdk/pull/18605/files/3ab4795d..34529ff1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18605&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18605&range=00-01 Stats: 8 lines in 4 files changed: 5 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/18605.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18605/head:pull/18605 PR: https://git.openjdk.org/jdk/pull/18605 From mli at openjdk.org Fri Apr 5 12:17:17 2024 From: mli at openjdk.org (Hamlin Li) Date: Fri, 5 Apr 2024 12:17:17 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Thu, 4 Apr 2024 16:47:44 GMT, Magnus Ihse Bursie wrote: > Build libsleef using their cmake system and look at the compile command line. (You do this by `VERBOSE=1 cmake` IIRC). Then you can see what flags they are using. This is what I was referring to as "normal libsleef build". I noticed there were a lot of compiler flags. I can't say if they are needed or not. In most cases, if it compilers, it's fine, but in this case, I guess some flags can be crucial to really get the kind of performance you need, and it might not be easy to spot that something is wrong if you get them incorrect. I assume one way to make sure is to run microbenchmarks with an externally built libsleef and compare it with the one you build within the JDK. If there is no noticeable difference, then I guess it is fine. Thanks for the clarification and good suggestion. I will verify it and update here later. Just right now I have some trouble to get an aarch64 linux, I tried to get a graviton instance on AWS, but I failed to connect it when I create it. Previously I run all the test for correctness via qemu, but seems qemu is not for performance test. So I will update later when I get the environment ready. If someone got the easy environment to verify the performance, it's very welcome. :) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2039645463 From mli at openjdk.org Fri Apr 5 12:17:18 2024 From: mli at openjdk.org (Hamlin Li) Date: Fri, 5 Apr 2024 12:17:18 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Thu, 4 Apr 2024 21:55:54 GMT, Mikael Vidstedt wrote: >> Hamlin Li has updated the pull request incrementally with two additional commits since the last revision: >> >> - disable unused-function warnings; add log msg >> - minor > > make/modules/jdk.incubator.vector/Lib.gmk line 44: > >> 42: $(eval $(call SetupJdkLibrary, BUILD_LIBVECTORMATH, \ >> 43: NAME := vectormath, \ >> 44: CFLAGS := $(CFLAGS_JDKLIB) -Wno-error=unused-function, \ > > Should the unused-function be passed in using `DISABLE_WARNINGS_*` instead? Thanks! Good suggestion, it makes the output clean. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18605#discussion_r1553519958 From ihse at openjdk.org Fri Apr 5 12:47:13 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 12:47:13 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v3] In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 10:45:24 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Use JNIEXPORT Looks good to me now. Thanks for indulging me in doing it this way. :-) ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18400#pullrequestreview-1983072333 From ihse at openjdk.org Fri Apr 5 12:47:14 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 12:47:14 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: <2wol6Z3Rd2zNpWUl8GWlXiC_vRWMzcuVFQh_-E_j_w0=.35a4b254-ce8b-4bad-a03e-381d3c2ddae8@github.com> References: <2wol6Z3Rd2zNpWUl8GWlXiC_vRWMzcuVFQh_-E_j_w0=.35a4b254-ce8b-4bad-a03e-381d3c2ddae8@github.com> Message-ID: On Fri, 5 Apr 2024 10:42:31 GMT, Robbin Ehn wrote: > Since windows build seems to need HSDIS_TOOLCHAIN_DEFAULT_CFLAGS Riiiight. That is since we use a completely different compiler, gcc instead of cl. (Which is probably the worst hack in all of the JDK build system...) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2039711663 From jwaters at openjdk.org Fri Apr 5 14:04:12 2024 From: jwaters at openjdk.org (Julian Waters) Date: Fri, 5 Apr 2024 14:04:12 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: <2wol6Z3Rd2zNpWUl8GWlXiC_vRWMzcuVFQh_-E_j_w0=.35a4b254-ce8b-4bad-a03e-381d3c2ddae8@github.com> Message-ID: On Fri, 5 Apr 2024 12:43:30 GMT, Magnus Ihse Bursie wrote: > > Since windows build seems to need HSDIS_TOOLCHAIN_DEFAULT_CFLAGS > > Riiiight. That is since we use a completely different compiler, gcc instead of cl. (Which is probably the worst hack in all of the JDK build system...) I plan on adding support for using gcc to compile hsdis on Windows sometime soon (Like proper support for gcc as a toolchain on Windows, at least enough to support hsdis, so people can freely use any gcc instead of only being restricted to Cygwin), what might be an elegant solution to this issue, if you have any in mind? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2039888898 From ihse at openjdk.org Fri Apr 5 14:05:30 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 14:05:30 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS Message-ID: This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. ------------- Commit messages: - Fix missing include for libw2k_lsa_auth - Fix missing include for libsunmscapi - Do not link libjawt with libawt on macOS - 8329704: Implement framework for proper handling of JDK_LIBS Changes: https://git.openjdk.org/jdk/pull/18649/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8329704 Stats: 572 lines in 38 files changed: 205 ins; 266 del; 101 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From ihse at openjdk.org Fri Apr 5 14:05:31 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 14:05:31 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 09:56:48 GMT, Magnus Ihse Bursie wrote: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. The syntax for specifying JDK libraries can of course be discussed. I tried to align it with the current syntax for specifying source code, but it does not match 100%. The differences are: * For source code, the default module is the current module, since the most common use is to include additional source directories from the current module. For libs, the default module is "java.base", since the by far most commonly included libs are `libjvm` and `libjava`. * For source code, the full name of the directory is needed, so to include the `libjava` directory the name `libjava` needs to be specified. For libraries, the "lib-" prefix is dropped, and just the base name of the library is used, e.g. `java`. This started out as a gradual extension of the use of e.g. `-ljava`, but after working with this patch for quite some time, I am not convinced it is ideal. If we were to align the syntax of libraries completely with source code, then the code will typically look like this: JDK_LIBS := libawt java.base:libjvm java.base:libjava, \ instead of (as currently in the patch): JDK_LIBS := @:awt jvm java, \ It is longer, for sure, but maybe that does not matter as much. And it is definitely clearer. Comments are welcome! make/common/JdkNativeCompilation.gmk line 110: > 108: # $1: The name of the rule (namespace) > 109: # $2: The safe namespace of the library > 110: define ResolveLibPath This macro is a horrible mess. But, sort of the point of this PR was to collect all this horrible mess in a single place, instead of having this strewn out all over the code base. My intention is to make the placement of libraries during build more systematic in a future PR, so this function can be simplified. make/common/JdkNativeCompilation.gmk line 195: > 193: $1_$2_STATIC_LIBRARY := true > 194: endif > 195: # Ideally, we should not hardcode these This is also something I plan to address in an upcoming PR. (The poor handling of static libraries is basically what has driven this current round of refactorings.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2039414005 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1553336636 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1553337734 From ihse at openjdk.org Fri Apr 5 14:14:35 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 14:14:35 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v2] In-Reply-To: References: Message-ID: <9PqBTNNbqMJunV0Ms2cxejNjkObUdICDvJWHBc3l7io=.daec7bca-2c10-4a05-b092-373a43c9575b@github.com> > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Fix libfallbackLinker ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18649/files - new: https://git.openjdk.org/jdk/pull/18649/files/cdee250d..6e737795 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=00-01 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From ihse at openjdk.org Fri Apr 5 14:19:10 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 14:19:10 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v2] In-Reply-To: <9PqBTNNbqMJunV0Ms2cxejNjkObUdICDvJWHBc3l7io=.daec7bca-2c10-4a05-b092-373a43c9575b@github.com> References: <9PqBTNNbqMJunV0Ms2cxejNjkObUdICDvJWHBc3l7io=.daec7bca-2c10-4a05-b092-373a43c9575b@github.com> Message-ID: On Fri, 5 Apr 2024 14:14:35 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix libfallbackLinker I apologize for bouncing this one back and forth between open and draft. I thought I had verified everything but then I discovered two missing includes on Windows. And then GHA failed on linux-x86 for libfallbackLinker. Now these are fixed; I have confirmed that Windows builds properly on Oracle's CI system, and that libfallbackLinker works locally on a linux-x86 build, so I'm just waiting for GHA to confirm this. I have also run tier1 + tier2 on Oracle's CI system for windows x64, linux x64/aarch64 and macos x64/aarch64, without failures. (The Windows run is still waiting for a few tests to finish.) Furthermore I have checked with COMPARE_BUILD, and the only differences I can see is related to library linking: On macOS and Linux, most libraries linked with "-ljvm -ljava" as default before, regardless of if they were needed or not. Now we only link with these libraries if needed, so this changes the bit-by-bit comparison for almost all libraries. On Windows, there was just a few libraries that had dependency changes. As far as I can tell, all changes discovered were related to the change in dynamic library dependencies. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2039920478 From ihse at openjdk.org Fri Apr 5 15:18:11 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 5 Apr 2024 15:18:11 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: <2wol6Z3Rd2zNpWUl8GWlXiC_vRWMzcuVFQh_-E_j_w0=.35a4b254-ce8b-4bad-a03e-381d3c2ddae8@github.com> Message-ID: On Fri, 5 Apr 2024 14:01:29 GMT, Julian Waters wrote: > I plan on adding support for using gcc to compile hsdis on Windows sometime soon (Like proper support for gcc as a toolchain on Windows, at least enough to support hsdis, so people can freely use any gcc instead of only being restricted to Cygwin), what might be an elegant solution to this issue, if you have any in mind? The proper solution is, I think, to encapsulate toolchain variables/definitions into a package, and update the build system to not assume anything about toolchains. In a proper language, I'd create like a `class Toolchain` with everything the system would need to know to use a toolchain, and then pass a `Toolchain toolchain` to all invocations that need to do anything with a toolchain. Now this is not possible with make, but I hope it can be simulated using "fake namespaces" and something like a `$(TOOLCHAIN)_COMPILER` kind of setup. That would require some major surgery of the build system, but I think it will come out better in the end. After that is done, fixing so that hsdis/binutils can be compiled with gcc on Windows will be a trivial side-effect. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2040063933 From duke at openjdk.org Fri Apr 5 17:47:03 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Fri, 5 Apr 2024 17:47:03 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 19:19:59 GMT, Volodymyr Paprotski wrote: >> Performance. Before: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s >> >> Performance, no intrinsic: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply true thrpt 3 1919.57... > > Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: > > remove use of jdk.crypto.ec @ascarpino Hi Tony, this is the ECC P256 PR we talked about last year, would appreciate your feedback. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18583#issuecomment-2040325424 From erikj at openjdk.org Fri Apr 5 18:38:02 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 5 Apr 2024 18:38:02 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v2] In-Reply-To: <9PqBTNNbqMJunV0Ms2cxejNjkObUdICDvJWHBc3l7io=.daec7bca-2c10-4a05-b092-373a43c9575b@github.com> References: <9PqBTNNbqMJunV0Ms2cxejNjkObUdICDvJWHBc3l7io=.daec7bca-2c10-4a05-b092-373a43c9575b@github.com> Message-ID: On Fri, 5 Apr 2024 14:14:35 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix libfallbackLinker This is a nice improvement! make/common/JdkNativeCompilation.gmk line 107: > 105: ) > 106: > 107: # Create a proper LIBPATH for the given library. Suggestion: # Create a proper LIBPATH for the given library. Sets result in $1_$2_LIBPATH. make/common/JdkNativeCompilation.gmk line 139: > 137: else > 138: $1_$2_JVM_VARIANT_PATH := $(JVM_VARIANT_MAIN) > 139: endif I think this needs to be moved before lin 127. make/common/JdkNativeCompilation.gmk line 206: > 204: $$(eval $$(call ResolveLibPath,$1,$2)) > 205: > 206: $1_EXTRA_HEADER_DIRS += $$($1_$2_MODULE):lib$$($1_$2_NAME) I think the top level comment need to be clearer about JDK_LIB implicitly setting EXTRA_HEADER_DIRS. make/common/JdkNativeCompilation.gmk line 262: > 260: # be specified either as an absolute path, or relative directory names which > 261: # are preprocessed like SRC, or in the format :, which > 262: # will be processed like SRC but for the given module. The name "*:libjvm" When I first read this, it wasn't obvious that the `*` was actually a wildcard. Looking through all uses of :libjvm below, you are always doing `java.base:libjvm`. Is there a need to allow any prefix for the special libjvm case, or should we just enforce `java.base:libjvm` to maybe make things clearer? make/common/JdkNativeCompilation.gmk line 269: > 267: # These take the form :. If the module is java.base, > 268: # the entire java.base: prefix can be omitted. If the module is @, this will > 269: # be replaced with the current module. The gtest module is a virtual module I understand wanting to optimize for the common case, which is linking against libjava and libjvm in java.base, but I'm wondering if it wouldn't be better to be more coherent with the EXTRA_HEADER_DIRS argument. No prefix there means current module, while no prefix here means java.base. I think we will be tripping over this inconsistency in the future. I also note that headers are referred to as `libfoo` while libraries are only `foo`. This is harder to do anything about as on references a directory name, while the other is the build library. Still something to consider. It would be neat if these options could take the same syntax when appropriate. make/common/JdkNativeCompilation.gmk line 275: > 273: # EXTRA_RCFLAGS -- additional RCFLAGS to append. > 274: # RC_FILEDESC -- override the default FILEDESC for Windows version.rc > 275: # such as "java.base:headers". I don't think this line belongs here. It doesn't make sense for RC_FILEDESC. Probably got misplaced in an earlier patch. make/modules/jdk.internal.le/Lib.gmk line 39: > 37: EXTRA_HEADER_DIRS := \ > 38: java.base:libjava \ > 39: java.base:libjvm, \ Indentation. ------------- PR Review: https://git.openjdk.org/jdk/pull/18649#pullrequestreview-1983816521 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1554098311 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1554108796 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1554115371 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1554066430 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1554070533 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1554073121 PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1554085354 From erikj at openjdk.org Fri Apr 5 19:48:59 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 5 Apr 2024 19:48:59 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 10:11:02 GMT, Magnus Ihse Bursie wrote: > The syntax for specifying JDK libraries can of course be discussed. I tried to align it with the current syntax for specifying source code, but it does not match 100%. The differences are: > > * For source code, the default module is the current module, since the most common use is to include additional source directories from the current module. For libs, the default module is "java.base", since the by far most commonly included libs are `libjvm` and `libjava`. > * For source code, the full name of the directory is needed, so to include the `libjava` directory the name `libjava` needs to be specified. For libraries, the "lib-" prefix is dropped, and just the base name of the library is used, e.g. `java`. > > This started out as a gradual extension of the use of e.g. `-ljava`, but after working with this patch for quite some time, I am not convinced it is ideal. If we were to align the syntax of libraries completely with source code, then the code will typically look like this: > > ``` > JDK_LIBS := libawt java.base:libjvm java.base:libjava, \ > ``` > > instead of (as currently in the patch): > > ``` > JDK_LIBS := @:awt jvm java, \ > ``` > > It is longer, for sure, but maybe that does not matter as much. And it is definitely clearer. > > Comments are welcome! I missed this comment when I first reviewed the code and already wrote something about this. Now I see you already suggested more or less the same change I did. I would prefer the longer format. It's more verbose, but it becomes more readable, due to consistency with the headers parameter. It's also less error prone when writing for the same reason. I'm not a big fan of introducing special meaning to characters like `@`. Having the default be the current module feels natural to me. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2040524133 From aph at openjdk.org Sat Apr 6 13:22:08 2024 From: aph at openjdk.org (Andrew Haley) Date: Sat, 6 Apr 2024 13:22:08 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: Message-ID: On Thu, 4 Apr 2024 16:43:18 GMT, Magnus Ihse Bursie wrote: > I apologize for the late reply. I've been just working spotty hours due to spring break. I apologize for my bad temper. ? Thanks to everyone working on this. I still think that hsdis ought not to have any dependencies on HotSpot, but I'm not going to be fanatical about it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2041080237 From rehn at openjdk.org Mon Apr 8 06:19:09 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Mon, 8 Apr 2024 06:19:09 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v2] In-Reply-To: References: Message-ID: On Sat, 6 Apr 2024 13:18:54 GMT, Andrew Haley wrote: > Thanks to everyone working on this. I still think that hsdis ought not to have any dependencies on HotSpot, but I'm not going to be fanatical about it. I agree, good :) If everyone is 'okay' with this, can someone do the second review? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2041943627 From djelinski at openjdk.org Mon Apr 8 13:27:32 2024 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 8 Apr 2024 13:27:32 GMT Subject: RFR: 8324673: javacserver failed during build: RejectedExecutionException Message-ID: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> The RejectedExecutionException was thrown when the thread executing `Server.start` managed to shut down the `compilerThreadPool` before the thread executing `Server.handleRequest` submitted the compilation task. This patch removes the extra thread used for `Server.handleRequest`, and executes that method directly in the thread pool. All `compilerThreadPool` uses happen on the `Server.start` thread now, and no new tasks are submitted after the thread pool is shut down. In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. With that change the problem was occasionally reproducible without the patch from this PR. With the patch, the `RejectedExecutionException` problem did not reproduce. No new regression test. Existing langtools tests continue to pass. ------------- Commit messages: - Use threadpool for socket communication Changes: https://git.openjdk.org/jdk/pull/18672/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18672&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8324673 Stats: 19 lines in 2 files changed: 1 ins; 10 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/18672.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18672/head:pull/18672 PR: https://git.openjdk.org/jdk/pull/18672 From cstein at openjdk.org Mon Apr 8 13:45:09 2024 From: cstein at openjdk.org (Christian Stein) Date: Mon, 8 Apr 2024 13:45:09 GMT Subject: RFR: 8324673: javacserver failed during build: RejectedExecutionException In-Reply-To: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> References: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> Message-ID: On Mon, 8 Apr 2024 09:20:44 GMT, Daniel Jeli?ski wrote: > The RejectedExecutionException was thrown when the thread executing `Server.start` managed to shut down the `compilerThreadPool` before the thread executing `Server.handleRequest` submitted the compilation task. > > This patch removes the extra thread used for `Server.handleRequest`, and executes that method directly in the thread pool. All `compilerThreadPool` uses happen on the `Server.start` thread now, and no new tasks are submitted after the thread pool is shut down. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. With that change the problem was occasionally reproducible without the patch from this PR. With the patch, the `RejectedExecutionException` problem did not reproduce. > > No new regression test. Existing langtools tests continue to pass. Less code, less threads: cleaner and less race-conditions. Looks good to me. ------------- Marked as reviewed by cstein (Committer). PR Review: https://git.openjdk.org/jdk/pull/18672#pullrequestreview-1986496132 From erikj at openjdk.org Mon Apr 8 14:04:11 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Apr 2024 14:04:11 GMT Subject: RFR: 8324673: javacserver failed during build: RejectedExecutionException In-Reply-To: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> References: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> Message-ID: On Mon, 8 Apr 2024 09:20:44 GMT, Daniel Jeli?ski wrote: > The RejectedExecutionException was thrown when the thread executing `Server.start` managed to shut down the `compilerThreadPool` before the thread executing `Server.handleRequest` submitted the compilation task. > > This patch removes the extra thread used for `Server.handleRequest`, and executes that method directly in the thread pool. All `compilerThreadPool` uses happen on the `Server.start` thread now, and no new tasks are submitted after the thread pool is shut down. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. With that change the problem was occasionally reproducible without the patch from this PR. With the patch, the `RejectedExecutionException` problem did not reproduce. > > No new regression test. Existing langtools tests continue to pass. My understanding of the existing model is that we wanted to read the command on the socket without delay and then potentially wait until there was a free executor thread in the pool. With the new model, we won't read the input on the socket until there is a free thread. I don't know if this could ever be a problem, but it seems plausible. The default size of the thread pool is number of CPUs in the system. The default number of jobs in the makefiles is never larger than this number, so in reality, we won't get more compile requests than number of CPUs from the makefiles when using default values. However, if the user manually increases the JOBS number in the makefiles, we could. Perhaps it would be worth at least trying running make with a much larger JOBS value with this patch to see that it doesn't fall apart? ------------- PR Review: https://git.openjdk.org/jdk/pull/18672#pullrequestreview-1986545820 From djelinski at openjdk.org Mon Apr 8 14:30:10 2024 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Mon, 8 Apr 2024 14:30:10 GMT Subject: RFR: 8324673: javacserver failed during build: RejectedExecutionException In-Reply-To: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> References: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> Message-ID: <-b30Ht4CtG_MmghcNBlO4J5B6VfPZ5BzVfejyJJkp00=.987af7b2-7dc2-4843-97df-14b292aaab10@github.com> On Mon, 8 Apr 2024 09:20:44 GMT, Daniel Jeli?ski wrote: > The RejectedExecutionException was thrown when the thread executing `Server.start` managed to shut down the `compilerThreadPool` before the thread executing `Server.handleRequest` submitted the compilation task. > > This patch removes the extra thread used for `Server.handleRequest`, and executes that method directly in the thread pool. All `compilerThreadPool` uses happen on the `Server.start` thread now, and no new tasks are submitted after the thread pool is shut down. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. With that change the problem was occasionally reproducible without the patch from this PR. With the patch, the `RejectedExecutionException` problem did not reproduce. > > No new regression test. Existing langtools tests continue to pass. It won't be a problem. The client side does not set timeout on socket read/write operations, so there's no risk of the operation timing out. Also, the OS usually buffers reads and writes, so the client write call probably won't block even if the server doesn't read. Other than that, we're using TCP sockets for communication, so no data will ever be lost. Tried running make with JOBS=512, no problems observed. I might try that on mach5 too, just in case. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18672#issuecomment-2042901814 From erikj at openjdk.org Mon Apr 8 16:28:10 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Apr 2024 16:28:10 GMT Subject: RFR: 8324673: javacserver failed during build: RejectedExecutionException In-Reply-To: <-b30Ht4CtG_MmghcNBlO4J5B6VfPZ5BzVfejyJJkp00=.987af7b2-7dc2-4843-97df-14b292aaab10@github.com> References: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> <-b30Ht4CtG_MmghcNBlO4J5B6VfPZ5BzVfejyJJkp00=.987af7b2-7dc2-4843-97df-14b292aaab10@github.com> Message-ID: On Mon, 8 Apr 2024 14:27:28 GMT, Daniel Jeli?ski wrote: > It won't be a problem. The client side does not set timeout on socket read/write operations, so there's no risk of the operation timing out. Also, the OS usually buffers reads and writes, so the client write call probably won't block even if the server doesn't read. Other than that, we're using TCP sockets for communication, so no data will ever be lost. > > Tried running make with JOBS=512, no problems observed. I might try that on mach5 too, just in case. Thanks for confirming. Then this looks good to me! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18672#issuecomment-2043177438 From erikj at openjdk.org Mon Apr 8 16:28:10 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 8 Apr 2024 16:28:10 GMT Subject: RFR: 8324673: javacserver failed during build: RejectedExecutionException In-Reply-To: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> References: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> Message-ID: On Mon, 8 Apr 2024 09:20:44 GMT, Daniel Jeli?ski wrote: > The RejectedExecutionException was thrown when the thread executing `Server.start` managed to shut down the `compilerThreadPool` before the thread executing `Server.handleRequest` submitted the compilation task. > > This patch removes the extra thread used for `Server.handleRequest`, and executes that method directly in the thread pool. All `compilerThreadPool` uses happen on the `Server.start` thread now, and no new tasks are submitted after the thread pool is shut down. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. With that change the problem was occasionally reproducible without the patch from this PR. With the patch, the `RejectedExecutionException` problem did not reproduce. > > No new regression test. Existing langtools tests continue to pass. Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18672#pullrequestreview-1986906076 From jjg at openjdk.org Mon Apr 8 21:12:42 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Mon, 8 Apr 2024 21:12:42 GMT Subject: RFR: JDK-8298405: Implement JEP 467: Markdown Documentation Comments [v55] In-Reply-To: References: Message-ID: <48hbLsx6MHdQQNNWrPRs4C-7sB18MyPOLE0QeTTPSDg=.1c0bd28d-8bd0-4733-a257-2c0f60bef2eb@github.com> > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: add support for JDK-8329296: Update Elements for '///' documentation comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16388/files - new: https://git.openjdk.org/jdk/pull/16388/files/37646287..21f5b004 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=54 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=53-54 Stats: 331 lines in 10 files changed: 291 ins; 20 del; 20 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388 From prr at openjdk.org Mon Apr 8 23:22:00 2024 From: prr at openjdk.org (Phil Race) Date: Mon, 8 Apr 2024 23:22:00 GMT Subject: RFR: 8315113: Print request Chromaticity.MONOCHROME attribute does not work on macOS In-Reply-To: References: Message-ID: <9NaQ7MzCph2B2CYYqk_o_yMqLtOz45sWsq8fbCk67Es=.d278e2e4-2dfe-499f-913f-b7ec47e577b9@github.com> On Mon, 11 Mar 2024 13:54:02 GMT, Alexander Scherbatiy wrote: > The fix provides ability to print Black & White pages on macOS. > > Cocoa API has [PMSetColorMode](https://developer.apple.com/documentation/applicationservices/core_printing/1805783-pmsetcolormode) function but it is marked as deprecated and really does nothing. > > There is no replacement; this function was included to facilitate porting legacy applications to macOS, > but it serves no useful purpose. > > Dumping `NSPrintInfo` print settings which were set by the native print dialog on macOS shows that the keys and values used for Black & White printing depend on the used printer type. > For example, the tested > `HP Color LaserJet MFP M180n` printer uses `ColorModel` key and`Gray` value, and > `HP Ink Tank 110 series` uses `HPColorMode` key and `grayscale` value. > > Printing all `NSPrintInfo` presets shows that they do not contain key/value pairs for Black&White settings. > This is the code snippet used to print `NSPrintInfo` presets: > > PMPrinter pr; > PMPrintSession printSession = (PMPrintSession)[printInfo PMPrintSession]; > OSStatus status = PMSessionGetCurrentPrinter(printSession, &pr); > CFArrayRef presetsList = nil; > status = PMPrinterCopyPresets(pr, &presetsList); > CFIndex arrayCount = CFArrayGetCount(presetsList); > > for (CFIndex index = 0; index < arrayCount; index++) { > PMPreset preset = (PMPreset)CFArrayGetValueAtIndex(presetsList, index); > CFStringRef presetName = nil; > if (PMPresetCopyName(preset, &presetName) == noErr && CFStringGetLength(presetName) > 0) { > NSLog(@" presetName: '%@'", presetName); > NSDictionary* dict = nil; > if (PMPresetGetAttributes(preset, (CFDictionaryRef*)(&dict)) == noErr) { > // print preset dict > > > The idea of the proposed fix is to store printer dependent key/value pairs in the `/conf/printer.properties` property file. > > The property file has the format: > > print-attribute.print-attribute-value.key=value > > where `print-attribute` is the java print attribute, `print-attribute-value` is the corresponding attribute value, and `key` and `value` is the key/value pair used by a specific printer. > > For example, for `Chromaticity` attribute the property file could look like: > > Chromaticity.MONOCHROME.ColorModel=Gray > Chromaticity.COLOR.ColorModel=CMYK > Chromaticity.MONOCHROME.HPColorMode=grayscale > Chromaticity.COLOR.HPColorMode=color > > where `Chromaticity.MONOCHROME` key prefix corresponds to `Chromaticity.MONOCHOROME` print attribute constant with (... Since the user can always select greyscale in the user dialog, I presume you have some requirement to do this printing without user intervention. Perhaps there's no user there (server-side printing), or the app would like to make sure the user always defaults to B&W. As far as I can tell, this isn't supported via any kind of API on macOS, so it is not a Java problem. You could be printing from an Objective-C app written directly to Cocoa APIs and still have the same problem. So a Java-specific workaround like this seems inappropriate. I've poked around to help me understand what is going on. When I do "Print to File" on macOS - meaning use the native printer dialog's "Save as PDF" option, then the generated PDF is always colour even if I select the printer dialog's "Grayscale Printing" option. This isn't just true for Java apps, the same happens if I print a web page from Safari. And when I query the Chromaticity support for a couple of colour printers - one Canon, one HP, both report they support it but only the value COLOR. So I conclude the grayscale printing option is something handled by the printer driver at the time it is converted to the printer-specific format and sent to the physical printer and it isn't changing the rendering. So in general without an API, to get greyscale it requires the end-user to set an option in a print dialog and as mentioned above, what you are trying o enable isn't possible even if you wrote the app in Objective-C or Swift directly as a macOS native app. But what works for me to get greyscale by default is to set that up as the default for the print queue. It is easy to add a printer twice - with two queues for it. One has the default set to greyscale, the other to colour. Then the app just needs to select the required queue. This is clearly more site-specific than having the app just specify MONOCHROME but is a lot lower cost all round. I don't understand why PMSetColorMode is deprecated and non-functional since it seems like it would be the right option here, but then internally macOS would need to be able to map it to the right option. Doubtless that would be easier if MONOCHROME were supported by all these drivers and I don't understand that omission either. In fact if it were we probably could just specify that directly without macOS needing to understand anything - like I expect it doesn't understand the key/value pairs your code is sending it. I note that you send ALL known key/value pairs and hope one works .. and that points out that all it takes is some vendor API change or a different printer and it just won't work again. These points and the implied maintenance cost are some of my bigger concerns with this. Perhaps you should engage with Apple and get them to properly support MONOCHROME. Or maybe it would be enough if the printer vendors do ? FWIW, I don't think it is an Apple-specific issue, the free drivers for Linux behave similarly. Perhaps there's a clue in there somewhere. But similarly you can set up a separate queue. The other idea that crossed my mind is that instead of relying on a printing solution, we could explicitly render in greyscale. Then we could always declare MONOCHROME support and just emulate it. But I am not sure if this is really possible - it would require changes to the rendering code and I'm not sure if or how if we'd be able to do that properly and safely. And the risks of that clearly extend beyond printing. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18195#issuecomment-2043813832 From luhenry at openjdk.org Tue Apr 9 08:09:10 2024 From: luhenry at openjdk.org (Ludovic Henry) Date: Tue, 9 Apr 2024 08:09:10 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v3] In-Reply-To: References: Message-ID: <42T5rhl-oMvCAsHetMt7e44qpi4YmEh-JZasLQcMfRI=.00f60086-d586-46ff-be03-02e6d9aec719@github.com> On Fri, 5 Apr 2024 10:45:24 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Use JNIEXPORT Marked as reviewed by luhenry (Committer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18400#pullrequestreview-1988392854 From rehn at openjdk.org Tue Apr 9 09:04:10 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Tue, 9 Apr 2024 09:04:10 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v3] In-Reply-To: <42T5rhl-oMvCAsHetMt7e44qpi4YmEh-JZasLQcMfRI=.00f60086-d586-46ff-be03-02e6d9aec719@github.com> References: <42T5rhl-oMvCAsHetMt7e44qpi4YmEh-JZasLQcMfRI=.00f60086-d586-46ff-be03-02e6d9aec719@github.com> Message-ID: On Tue, 9 Apr 2024 08:06:19 GMT, Ludovic Henry wrote: >> Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: >> >> Use JNIEXPORT > > Marked as reviewed by luhenry (Committer). Thank you @luhenry ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2044496543 From ihse at openjdk.org Tue Apr 9 09:17:09 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 9 Apr 2024 09:17:09 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v3] In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 10:45:24 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Use JNIEXPORT FYI: The 2-reviewer rule only applies to Hotspot (and some other specific parts of the code base), and not generally for JDK fixes. So you had been fine to push with just one reviewer. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2044521203 From rehn at openjdk.org Tue Apr 9 09:42:02 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Tue, 9 Apr 2024 09:42:02 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v3] In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 10:45:24 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Use JNIEXPORT Yea, I know, but if hsdis is not an external library I considered it part of hotspot as it is the only use, hence I want two reviews. :) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18400#issuecomment-2044575768 From sgehwolf at openjdk.org Tue Apr 9 09:48:15 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Tue, 9 Apr 2024 09:48:15 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: Message-ID: <44E_TpjVa9rDhYIHl_1BYjpR721_p4xrAvOBYc95m3w=.4882f3ae-076b-46d1-9fe6-3201d04305e2@github.com> On Wed, 3 Apr 2024 22:31:39 GMT, Mandy Chung wrote: >> Severin Gehwolf has updated the pull request incrementally with one additional commit since the last revision: >> >> Move CreateLinkableRuntimePlugin to build folder >> >> Keep runtime link supporting classes in package >> jdk.tools.jlink.internal.runtimelink > > Thanks for the investigation w.r.t. extending jimage. It does not seem worth the effort in pursuing the support of adding resources to an existing jimage file. To me, putting the diff data under `lib` directory as a private file is a simpler and acceptable solution. It allows the second jlink invocation to avoid the plugin pipeline if the per-module metadata is generated in normal jlink invocation. > > (An observation: The current implementation requires the diffs to be generated as a plugin. For this approach, it may be possible to implement with a separate main class that calls JLink API and generates the diffs.) @mlchung @AlanBateman Any thoughts on this latest version? Is this going into the direction you had envisioned? Any more blockers? The CSR should be up-to-date and is open for review as well. If no more blockers I'll go over this patch once more and clean it up to prepare for integration. Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2044593102 From mli at openjdk.org Tue Apr 9 10:09:11 2024 From: mli at openjdk.org (Hamlin Li) Date: Tue, 9 Apr 2024 10:09:11 GMT Subject: RFR: 8328614: hsdis: dlsym can't find decode symbol [v3] In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 10:45:24 GMT, Robbin Ehn wrote: >> Hi, please consider. >> >> [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. >> Tested with gcc and clang, and llvm and binutils backend. >> >> I didn't find any use of the "DLL_ENTRY", so I removed it. >> >> Thanks, Robbin > > Robbin Ehn has updated the pull request incrementally with one additional commit since the last revision: > > Use JNIEXPORT Looks good. Although I'm not the right person to review this, thanks for explanation and discussion. ------------- Marked as reviewed by mli (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18400#pullrequestreview-1988652730 From prappo at openjdk.org Tue Apr 9 11:11:18 2024 From: prappo at openjdk.org (Pavel Rappo) Date: Tue, 9 Apr 2024 11:11:18 GMT Subject: RFR: JDK-8298405: Implement JEP 467: Markdown Documentation Comments [v55] In-Reply-To: <48hbLsx6MHdQQNNWrPRs4C-7sB18MyPOLE0QeTTPSDg=.1c0bd28d-8bd0-4733-a257-2c0f60bef2eb@github.com> References: <48hbLsx6MHdQQNNWrPRs4C-7sB18MyPOLE0QeTTPSDg=.1c0bd28d-8bd0-4733-a257-2c0f60bef2eb@github.com> Message-ID: <1uPEw9BTg1J92Zcw5bKKXcvc7Ula0WFLjFz0moN50U4=.752eb9c2-0a19-422d-adbb-79b71505be7f@github.com> On Mon, 8 Apr 2024 21:12:42 GMT, Jonathan Gibbons wrote: >> Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. >> >> Notable features: >> >> * support for `///` documentation comments in `JavaTokenizer` >> * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library >> * updates to `DocCommentParser` to treat `///` comments as Markdown >> * updates to the standard doclet to render Markdown comments in HTML > > Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: > > add support for JDK-8329296: Update Elements for '///' documentation comments Changes for 21f5b00 mostly look good, but I'm not a compiler person. src/jdk.compiler/share/classes/com/sun/tools/javac/model/JavacElements.java line 443: > 441: @DefinedBy(Api.LANGUAGE_MODEL) > 442: public CommentKind getDocCommentKind(Element e) { > 443: return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); Nit: Suggestion: return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); src/jdk.compiler/share/classes/com/sun/tools/javac/model/JavacElements.java line 443: > 441: @DefinedBy(Api.LANGUAGE_MODEL) > 442: public CommentKind getDocCommentKind(Element e) { > 443: return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); Again: Suggestion: return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); src/jdk.compiler/share/classes/com/sun/tools/javac/tree/DocCommentTable.java line 55: > 53: > 54: /** > 55: * Get the plain text of the doc comment, if any, for a tree node. This is likely a copy-pasted comment. ------------- PR Review: https://git.openjdk.org/jdk/pull/16388#pullrequestreview-1988489764 PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1557248565 PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1557249290 PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1557444619 From hannesw at openjdk.org Tue Apr 9 13:14:18 2024 From: hannesw at openjdk.org (Hannes =?UTF-8?B?V2FsbG7DtmZlcg==?=) Date: Tue, 9 Apr 2024 13:14:18 GMT Subject: RFR: JDK-8329617: Update stylesheet for specs and tool documentation Message-ID: Please review an update to the `jdk-default.css` stylesheet used for specifications and tool guides. The original purpose was to make use of the Dejavu web fonts provided by the API docs and to update the navigation bar to match the one in the API docs. However, the updates include some other fixes and improvements also described below. - The change to use the DejaVu web fonts consists only of the `@import` statement in line 16 as the stylesheet already used DejaVu web fonts as first choice in its `font-family` rules. - The changes to make the navigation bar match the one in the API docs are mostly located at the end of the file (beyond line 160). However, this also includes setting the `margin` property to '0' in the `body` element and adding a `margin` in the `main` and `footer` elements instead. - To set the horizontal margin for page content elements outside the `main` element which occur in some pages, a margin is set explicitly on those elements in lines 48-50. While this is a bit awkward, I think it's still better than working with negative margins in the header to offset the margin in the `body` element. - Most of the remaining changes (lines 53-110) are changes are to redefine the styles in simpler terms, such as leaving out declarations that are equal to browser defaults, and removing the units from `0`-length values. The changes are intended to preserve the layout of the pages, including the body font size which is slightly different from the one used in API docs (`10pt` vs `14px`). I can provide before/after snapshots of the rendered documentation if desired. ------------- Commit messages: - JDK-8329617: Update stylesheet for specs and tool documentation Changes: https://git.openjdk.org/jdk/pull/18694/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18694&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8329617 Stats: 73 lines in 1 file changed: 27 ins; 19 del; 27 mod Patch: https://git.openjdk.org/jdk/pull/18694.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18694/head:pull/18694 PR: https://git.openjdk.org/jdk/pull/18694 From djelinski at openjdk.org Tue Apr 9 14:14:04 2024 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Tue, 9 Apr 2024 14:14:04 GMT Subject: RFR: 8324673: javacserver failed during build: RejectedExecutionException In-Reply-To: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> References: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> Message-ID: On Mon, 8 Apr 2024 09:20:44 GMT, Daniel Jeli?ski wrote: > The RejectedExecutionException was thrown when the thread executing `Server.start` managed to shut down the `compilerThreadPool` before the thread executing `Server.handleRequest` submitted the compilation task. > > This patch removes the extra thread used for `Server.handleRequest`, and executes that method directly in the thread pool. All `compilerThreadPool` uses happen on the `Server.start` thread now, and no new tasks are submitted after the thread pool is shut down. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. With that change the problem was occasionally reproducible without the patch from this PR. With the patch, the `RejectedExecutionException` problem did not reproduce. > > No new regression test. Existing langtools tests continue to pass. Thanks for the reviews! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18672#issuecomment-2045276658 From djelinski at openjdk.org Tue Apr 9 14:14:05 2024 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Tue, 9 Apr 2024 14:14:05 GMT Subject: Integrated: 8324673: javacserver failed during build: RejectedExecutionException In-Reply-To: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> References: <1W72je45JGhBSNtpEHMr3SyWKf570x9AWKbGZkV0roE=.db6cd36e-3f8c-464a-bc4c-16697b3482fe@github.com> Message-ID: On Mon, 8 Apr 2024 09:20:44 GMT, Daniel Jeli?ski wrote: > The RejectedExecutionException was thrown when the thread executing `Server.start` managed to shut down the `compilerThreadPool` before the thread executing `Server.handleRequest` submitted the compilation task. > > This patch removes the extra thread used for `Server.handleRequest`, and executes that method directly in the thread pool. All `compilerThreadPool` uses happen on the `Server.start` thread now, and no new tasks are submitted after the thread pool is shut down. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. With that change the problem was occasionally reproducible without the patch from this PR. With the patch, the `RejectedExecutionException` problem did not reproduce. > > No new regression test. Existing langtools tests continue to pass. This pull request has now been integrated. Changeset: 3b6629ce Author: Daniel Jeli?ski URL: https://git.openjdk.org/jdk/commit/3b6629cec7a2ecec8dcb5b94d8ed3e169483aa97 Stats: 19 lines in 2 files changed: 1 ins; 10 del; 8 mod 8324673: javacserver failed during build: RejectedExecutionException Reviewed-by: cstein, erikj ------------- PR: https://git.openjdk.org/jdk/pull/18672 From rehn at openjdk.org Tue Apr 9 15:04:19 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Tue, 9 Apr 2024 15:04:19 GMT Subject: Integrated: 8328614: hsdis: dlsym can't find decode symbol In-Reply-To: References: Message-ID: On Wed, 20 Mar 2024 16:17:36 GMT, Robbin Ehn wrote: > Hi, please consider. > > [8327045](https://bugs.openjdk.org/browse/JDK-8327045) hide these symbols. > Tested with gcc and clang, and llvm and binutils backend. > > I didn't find any use of the "DLL_ENTRY", so I removed it. > > Thanks, Robbin This pull request has now been integrated. Changeset: 1e02a13a Author: Robbin Ehn URL: https://git.openjdk.org/jdk/commit/1e02a13a7f02a6fe9aac38b93935bcc238f7d227 Stats: 18 lines in 5 files changed: 8 ins; 8 del; 2 mod 8328614: hsdis: dlsym can't find decode symbol Reviewed-by: ihse, luhenry, mli ------------- PR: https://git.openjdk.org/jdk/pull/18400 From mli at openjdk.org Tue Apr 9 15:36:13 2024 From: mli at openjdk.org (Hamlin Li) Date: Tue, 9 Apr 2024 15:36:13 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: <9utr-LKgycFDqg9bdjf6zlINHJcnOhn-uLvU7u9Ls9E=.a1679bbe-c7cd-49cb-a168-4fa14852cee5@github.com> On Fri, 5 Apr 2024 12:17:17 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review the patch? >> This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). >> >> Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. >> Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. >> >> Besides of the code changes, one important task is to handle the legal process. >> >> Thanks! > > Hamlin Li has updated the pull request incrementally with two additional commits since the last revision: > > - disable unused-function warnings; add log msg > - minor Just a quick update, this pr introduces some performance regression compared with previous version (https://github.com/openjdk/jdk/pull/18294) for some math functions (e.g. Double256Vector.COS), and no regression for some others (e.g. Double256Vector.ACOS). I'm investigating. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2045495626 From acobbs at openjdk.org Tue Apr 9 15:36:27 2024 From: acobbs at openjdk.org (Archie Cobbs) Date: Tue, 9 Apr 2024 15:36:27 GMT Subject: RFR: 8317376: Minor improvements to the 'this' escape analyzer [v4] In-Reply-To: References: Message-ID: > Please review several fixes and improvements to the `this-escape` lint warning analyzer. > > The goal here is to apply some relatively simple logical fixes that improve the precision and accuracy of the analyzer, and capture the remaining low-hanging fruit so we can consider the analyzer relatively complete with respect to what's feasible with its current design. > > Although the changes are small from a logical point of view, they generate a fairly large patch due to impact of refactoring (sorry!). Most of the patch derives from the first two changes listed below. > > The changes are summarized here: > > #### 1. Generalize how we categorize references > > The `Ref` class hierarchy models the various ways in which, at any point during the execution of a constructor or some other method/constructor that it invokes, there can be live references to the original object under construction lying around. We then look for places where one of these `Ref`'s might be passed to a subclass method. In other words, the analyzer keeps track of these references and watches what happens to them as the code executes so it can catch them trying to "escape". > > Previously the `Ref` categories were: > * `ThisRef` - The current instance of the (non-static) method or constructor being analyzed > * `OuterRef` - The current outer instance of the (non-static) method or constructor being analyzed > * `VarRef` - A local variable or method parameter currently in scope > * `ExprRef` - An object reference sitting on top of the Java execution stack > * `YieldRef` - The current switch expression's yield value(s) > * `ReturnRef` - The current method's return value(s) > > For each of those types, we further classified the "indirection" of the reference, i.e., whether the reference was direct (from the thing itself) or indirect (from something the thing referenced). > > The problem with that hierarchy is that we could only track outer instance references that happened to be associated with the current instance. So we might know that `this` had an outer instance reference, but if we said `var x = this` we wouldn't know that `x` had an outer instance reference. > > In other words, we should be treating "via an outer instance" as just another flavor of indirection along with "direct" and "indirect". > > As a result, with this patch the `OuterRef` class goes away and a new `Indirection` enum has been created, with values `DIRECT`, `INDIRECT`, and `OUTER`. > > #### 2. Track the types of all references > > By keeping track of the actual type of... Archie Cobbs has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: - Merge branch 'master' into JDK-8317376 - Merge branch 'master' into JDK-8317376 - Merge branch 'master' into JDK-8317376 - Merge branch 'master' into JDK-8317376 - Merge branch 'master' into JDK-8317376 - Javadoc++ - Merge branch 'master' into JDK-8317376 - Several improvements to the 'this' escape analyzer. - Track direct, indirect, and outer references for all Ref types. - Keep type information about all references to improve tracking precision. - Track enhanced for() invocations of iterator(), hasNext(), and next(). - Don't report an escape of a non-public outer instances as a leak. - Fix omitted tracking of references from newly instantiated instances. - Fix omitted tracking of leaks via lambda return values. - Remove unneccesary suppressions of this-escape lint warning. ------------- Changes: https://git.openjdk.org/jdk/pull/16208/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16208&range=03 Stats: 869 lines in 19 files changed: 513 ins; 146 del; 210 mod Patch: https://git.openjdk.org/jdk/pull/16208.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16208/head:pull/16208 PR: https://git.openjdk.org/jdk/pull/16208 From stuefe at openjdk.org Tue Apr 9 17:02:09 2024 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 9 Apr 2024 17:02:09 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 10:26:42 GMT, Joachim Kern wrote: >> src/hotspot/os/aix/os_aix.cpp line 314: >> >>> 312: ErrnoPreserver ep; >>> 313: log_trace(os, map)("disclaim failed: " RANGEFMT " errno=(%s)", >>> 314: RANGEFMTARGS(p, (long)maxDisclaimSize), >> >> Wait, why are these casts needed? maxDisclaimSize is size_t, RANGEFMT uses SIZE_FORMAT. That should work without cast. > > Hi Thomas, `maxDisclaimSize` is of type `unsigned int`; therefore I get the following warning: > > os/aix/os_aix.cpp:314:42: error: format specifies type 'unsigned long' but the argument has type 'unsigned int' [-Werror,-Wformat] > RANGEFMTARGS(p, maxDisclaimSize), > ^~~~~~~~~~~~~~~ > > Should I keep the casts, or change the type of `maxDisclaimSize, numFullDisclaimsNeeded, lastDisclaimSize` to `const unsigned long`? I would change them to size_t. Thanks for doing this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558012122 From stuefe at openjdk.org Tue Apr 9 17:08:10 2024 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 9 Apr 2024 17:08:10 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: <60xqHKyBKIqrzMqVisUO5M_lQLCNt7OYZ6XcovISOc0=.f4bc36e0-c3f7-4e40-b6b7-69ed46ca37e8@github.com> On Tue, 2 Apr 2024 16:14:12 GMT, Joachim Kern wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > version check not needed anymore src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp line 440: > 438: st->print("pc =" INTPTR_FORMAT " ", (unsigned long)uc->uc_mcontext.jmp_context.iar); > 439: st->print("lr =" INTPTR_FORMAT " ", (unsigned long)uc->uc_mcontext.jmp_context.lr); > 440: st->print("ctr=" INTPTR_FORMAT " ", (unsigned long)uc->uc_mcontext.jmp_context.ctr); p2i src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp line 443: > 441: st->cr(); > 442: for (int i = 0; i < 32; i++) { > 443: st->print("r%-2d=" INTPTR_FORMAT " ", i, (unsigned long)uc->uc_mcontext.jmp_context.gpr[i]); p2i ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558017408 PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558017827 From stuefe at openjdk.org Tue Apr 9 17:08:11 2024 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 9 Apr 2024 17:08:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 09:19:16 GMT, Joachim Kern wrote: >> Hi Thomas, >> I would like to get totally rid of this, because as I mentioned IBM already modified the `stdlib.h` header not using `#define malloc vec_malloc` any more (and all the other vec_... defines). We have to ask the adoptium colleagues at IBM if they already have raised their build environment by the 2 SP levels needed. >> In principle we had to do the same workaround for `calloc, free,...` too, but they didn't show up as errors in the logging files. >> These lines where never meant to stay for long. Just to be able to compile until IBM fixes the issue, which is done now. > > @suchismith1993 > Hi Suchi, can you please tell me when you will raise your build environment from AIX 7.2 TL5 SP5 to SP7? > I' am asking you, because I want to get rid of this nasty workaround. Pinging @sxa - what build environment does temurin use for AIX? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558020493 From duke at openjdk.org Tue Apr 9 17:28:09 2024 From: duke at openjdk.org (Stewart X Addison) Date: Tue, 9 Apr 2024 17:28:09 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 17:01:59 GMT, Thomas Stuefe wrote: >> @suchismith1993 >> Hi Suchi, can you please tell me when you will raise your build environment from AIX 7.2 TL5 SP5 to SP7? >> I' am asking you, because I want to get rid of this nasty workaround. > > Pinging @sxa - what build environment does temurin use for AIX? Currently XLC16 but looking to upgrade to XLC17 on the minimum supported level for it (So it wouldn't be SP7 at present). Thanks for the ping - we have no current plans to increase to SP7. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558053537 From kbarrett at openjdk.org Tue Apr 9 19:24:11 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 9 Apr 2024 19:24:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 16:14:12 GMT, Joachim Kern wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > version check not needed anymore Changes requested by kbarrett (Reviewer). src/hotspot/share/utilities/byteswap.hpp line 2: > 1: /* > 2: * Copyright (c) 2023, Google and/or its affiliates. All rights reserved. Don't drop the creation year. src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 36: > 34: #if defined(_AIX) > 35: #include > 36: #endif I would much rather see this include added in the few places it was actually needed, rather than being added here. ------------- PR Review: https://git.openjdk.org/jdk/pull/18536#pullrequestreview-1989864573 PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558124034 PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558172309 From erikj at openjdk.org Tue Apr 9 19:48:30 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 9 Apr 2024 19:48:30 GMT Subject: RFR: 8329970: Update autoconf build-aux files with latest from 2024-01-01 Message-ID: Since we are now able to update the autoconf build-aux files, I think it's prudent to semi regularly do just that. I'm not aware of any particular platform that has been improved that would affect OpenJDK and I don't think we can further trim down our wrappers this time. This is more about staying with the latest. As of the filing of this bug, that is 2024-01-01, found here: https://git.savannah.gnu.org/cgit/config.git/ I have verified this with the platforms we build for at Oracle. I would encourage other OpenJDK distributors to verify on any platforms not covered by us. I will leave this open for a few days. ------------- Commit messages: - JDK-8329970 Changes: https://git.openjdk.org/jdk/pull/18704/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18704&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8329970 Stats: 224 lines in 2 files changed: 126 ins; 24 del; 74 mod Patch: https://git.openjdk.org/jdk/pull/18704.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18704/head:pull/18704 PR: https://git.openjdk.org/jdk/pull/18704 From mikael at openjdk.org Tue Apr 9 20:13:01 2024 From: mikael at openjdk.org (Mikael Vidstedt) Date: Tue, 9 Apr 2024 20:13:01 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Fri, 5 Apr 2024 12:17:17 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review the patch? >> This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). >> >> Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. >> Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. >> >> Besides of the code changes, one important task is to handle the legal process. >> >> Thanks! > > Hamlin Li has updated the pull request incrementally with two additional commits since the last revision: > > - disable unused-function warnings; add log msg > - minor Thank you for the update and for working on this in general. I've started working on JDK-8329816, preparing the change for the SLEEF specific part of the change. Specifically, I'm currently planning on including the three SLEEF header files, the README and a legal/sleef.md file in that change. Let me know if you have any thoughts/concerns. Also, just for my understanding, would love to understand your thoughts on the future here (I apologize if this was already discussed elsewhere): It seem like SLEEF is (sort of) limited to linux at this point (the SLEEF README mentions that "Due to limited test capacities, SLEEF is currently only officially supported on Linux with gcc or llvm/clang." ). That same README does, however, indicate good test coverage on several architectures in addition to aarch64 (including x86_64, PPC, RISC-V). With that in mind, it looks like we could potentially use SLEEF for other architectures on linux in the future? And potentially additional operating systems as well? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2045972249 From kbarrett at openjdk.org Wed Apr 10 00:54:17 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 10 Apr 2024 00:54:17 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 19:20:22 GMT, Kim Barrett wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> version check not needed anymore > > src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 36: > >> 34: #if defined(_AIX) >> 35: #include >> 36: #endif > > I would much rather see this include added in the few places it was actually needed, rather than being > added here. Do we even need to include ? >From the Linux man page for alloca: By necessity, alloca() is a compiler built-in, also known as __builtin_alloca(). By default, modern compilers automatically translate all uses of alloca() into the built-in, but this is forbidden if standards conformance is requested (-ansi, -std=c*), in which case is required, lest a symbol dependency be emitted. There are uses of it in shared code where there isn't an applicable include, other than from globalDefinitions_xlc.hpp. So it appears all other supported compilers do treat it as a built-in with the options we are providing, and don't need the include. Maybe that's true for the new xlc compiler too? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1558565268 From jkern at openjdk.org Wed Apr 10 09:20:12 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 09:20:12 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 16:59:39 GMT, Thomas Stuefe wrote: >> Hi Thomas, `maxDisclaimSize` is of type `unsigned int`; therefore I get the following warning: >> >> os/aix/os_aix.cpp:314:42: error: format specifies type 'unsigned long' but the argument has type 'unsigned int' [-Werror,-Wformat] >> RANGEFMTARGS(p, maxDisclaimSize), >> ^~~~~~~~~~~~~~~ >> >> Should I keep the casts, or change the type of `maxDisclaimSize, numFullDisclaimsNeeded, lastDisclaimSize` to `const unsigned long`? > > I would change them to size_t. Thanks for doing this. Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559121239 From jkern at openjdk.org Wed Apr 10 09:26:14 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 09:26:14 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <60xqHKyBKIqrzMqVisUO5M_lQLCNt7OYZ6XcovISOc0=.f4bc36e0-c3f7-4e40-b6b7-69ed46ca37e8@github.com> References: <60xqHKyBKIqrzMqVisUO5M_lQLCNt7OYZ6XcovISOc0=.f4bc36e0-c3f7-4e40-b6b7-69ed46ca37e8@github.com> Message-ID: On Tue, 9 Apr 2024 17:00:56 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> version check not needed anymore > > src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp line 440: > >> 438: st->print("pc =" INTPTR_FORMAT " ", (unsigned long)uc->uc_mcontext.jmp_context.iar); >> 439: st->print("lr =" INTPTR_FORMAT " ", (unsigned long)uc->uc_mcontext.jmp_context.lr); >> 440: st->print("ctr=" INTPTR_FORMAT " ", (unsigned long)uc->uc_mcontext.jmp_context.ctr); > > p2i I had tried this, but got following error: .../src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp:438:40: error: no matching function for call to 'p2i' st->print("pc =" INTPTR_FORMAT " ", p2i(uc->uc_mcontext.jmp_context.iar)); ^~~ .../src/hotspot/share/utilities/globalDefinitions.hpp:179:17: note: candidate function not viable: no known conversion from 'const unsigned long long' to 'const volatile void *' for 1st argument; take the address of the argument with & inline intptr_t p2i(const volatile void* p) { ^ .../src/hotspot/share/oops/oopsHierarchy.hpp:169:17: note: candidate function not viable: no known conversion from 'const unsigned long long' to 'narrowOop' for 1st argument inline intptr_t p2i(narrowOop o) { ^ ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559128609 From mli at openjdk.org Wed Apr 10 09:29:05 2024 From: mli at openjdk.org (Hamlin Li) Date: Wed, 10 Apr 2024 09:29:05 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Tue, 9 Apr 2024 20:10:36 GMT, Mikael Vidstedt wrote: > Thank you for the update and for working on this in general. > > I've started working on JDK-8329816, preparing the change for the SLEEF specific part of the change. Specifically, I'm currently planning on including the three SLEEF header files, the README and a legal/sleef.md file in that change. Let me know if you have any thoughts/concerns. > Thanks a lot, that's a great news. Please go ahead to integrate the files via JDK-8329816. :) Besides of the performance issue currently found out, I have no other concerns. > Also, just for my understanding, would love to understand your thoughts on the future here (I apologize if this was already discussed elsewhere): > > It seem like SLEEF is (sort of) limited to linux at this point (the SLEEF README mentions that "Due to limited test capacities, SLEEF is currently only officially supported on Linux with gcc or llvm/clang." ). That same README does, however, indicate good test coverage on several architectures in addition to aarch64 (including x86_64, PPC, RISC-V). With that in mind, it looks like we could potentially use SLEEF for other architectures on linux in the future? And potentially additional operating systems as well? There are more informantion at https://sleef.org/compile.xhtml, seems it could be formally supported in the future, but I'm not sure about it. Maybe others have more information could help to comment here. Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2047008550 From jkern at openjdk.org Wed Apr 10 09:36:10 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 09:36:10 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 18:32:04 GMT, Kim Barrett wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> version check not needed anymore > > src/hotspot/share/utilities/byteswap.hpp line 2: > >> 1: /* >> 2: * Copyright (c) 2023, Google and/or its affiliates. All rights reserved. > > Don't drop the creation year. Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559142128 From jkern at openjdk.org Wed Apr 10 09:43:11 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 09:43:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 00:51:22 GMT, Kim Barrett wrote: >> src/hotspot/share/utilities/globalDefinitions_gcc.hpp line 36: >> >>> 34: #if defined(_AIX) >>> 35: #include >>> 36: #endif >> >> I would much rather see this include added in the few places it was actually needed, rather than being >> added here. > > Do we even need to include ? > > From the Linux man page for alloca: > > By necessity, alloca() is a compiler built-in, also known as > __builtin_alloca(). By default, modern compilers automatically > translate all uses of alloca() into the built-in, but this is > forbidden if standards conformance is requested (-ansi, -std=c*), > in which case is required, lest a symbol dependency be > emitted. > > There are uses of it in shared code where there isn't an applicable include, > other than from globalDefinitions_xlc.hpp. So it appears all other supported > compilers do treat it as a built-in with the options we are providing, and > don't need the include. Maybe that's true for the new xlc compiler too? If I omit this #include I get compiler errors of the following kind .../src/hotspot/share/runtime/javaThread.cpp:2222:24: error: use of undeclared identifier 'alloca' char* p1 = (char*) alloca(1); ^ Of course I can do this include in every nagging file, but I thought it is simpler to keep it in the central header. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559150964 From rehn at openjdk.org Wed Apr 10 09:48:16 2024 From: rehn at openjdk.org (Robbin Ehn) Date: Wed, 10 Apr 2024 09:48:16 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: <33a-yu0_4j-ciPcJ_NtgbjHIrganBjKktC3REG8nyOc=.ace97c4f-b738-4049-a92b-60d1e8584e49@github.com> On Wed, 10 Apr 2024 09:24:09 GMT, Hamlin Li wrote: > With that in mind, it looks like we could potentially use SLEEF for other architectures on linux in the future? And potentially additional operating systems as well? Hi Mikael(@vidmik ) ! :) Thanks for looking into the legal stuff! We are pushing for this as we can leverage these changes when adding sleef to risc-v. Cross-fingers about legal! /Robbin ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2047049521 From jkern at openjdk.org Wed Apr 10 09:55:23 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 09:55:23 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v4] In-Reply-To: References: Message-ID: <8JetMf0mf_mYybUtWzB0YwfDi76Qul9d6x8ge58zklc=.207771b4-e02c-4a5c-85dd-f9cc97942761@github.com> > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: cosmetic changes ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18536/files - new: https://git.openjdk.org/jdk/pull/18536/files/ac1335e5..815974f5 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=02-03 Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/18536.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18536/head:pull/18536 PR: https://git.openjdk.org/jdk/pull/18536 From mdoerr at openjdk.org Wed Apr 10 10:03:12 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 10 Apr 2024 10:03:12 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 09:40:16 GMT, Joachim Kern wrote: >> Do we even need to include ? >> >> From the Linux man page for alloca: >> >> By necessity, alloca() is a compiler built-in, also known as >> __builtin_alloca(). By default, modern compilers automatically >> translate all uses of alloca() into the built-in, but this is >> forbidden if standards conformance is requested (-ansi, -std=c*), >> in which case is required, lest a symbol dependency be >> emitted. >> >> There are uses of it in shared code where there isn't an applicable include, >> other than from globalDefinitions_xlc.hpp. So it appears all other supported >> compilers do treat it as a built-in with the options we are providing, and >> don't need the include. Maybe that's true for the new xlc compiler too? > > If I omit this #include > I get compiler errors of the following kind > > .../src/hotspot/share/runtime/javaThread.cpp:2222:24: error: use of undeclared identifier 'alloca' > char* p1 = (char*) alloca(1); > ^ > > > Of course I can do this include in every nagging file, but I thought it is simpler to keep it in the central header. Is the comment in front of https://github.com/openjdk/jdk/blob/51ed69a586105b707ae616f9eba898449bf9fba7/src/hotspot/os/aix/os_aix.cpp#L28 still correct? Seems like it isn't followed everywhere. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559175426 From mdoerr at openjdk.org Wed Apr 10 10:10:13 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 10 Apr 2024 10:10:13 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: Message-ID: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> On Wed, 10 Apr 2024 10:00:02 GMT, Martin Doerr wrote: >> If I omit this #include >> I get compiler errors of the following kind >> >> .../src/hotspot/share/runtime/javaThread.cpp:2222:24: error: use of undeclared identifier 'alloca' >> char* p1 = (char*) alloca(1); >> ^ >> >> >> Of course I can do this include in every nagging file, but I thought it is simpler to keep it in the central header. > > Is the comment in front of https://github.com/openjdk/jdk/blob/51ed69a586105b707ae616f9eba898449bf9fba7/src/hotspot/os/aix/os_aix.cpp#L28 still correct? Seems like it should get replaced. See https://www.ibm.com/docs/en/openxl-c-and-cpp-aix/17.1.1?topic=pragmas-pragma-alloca-c-only Can `-Dalloca=__builtin_alloca` replace `#include `? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559183757 From jkern at openjdk.org Wed Apr 10 10:16:10 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 10:16:10 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 10:07:02 GMT, Martin Doerr wrote: >> Is the comment in front of https://github.com/openjdk/jdk/blob/51ed69a586105b707ae616f9eba898449bf9fba7/src/hotspot/os/aix/os_aix.cpp#L28 still correct? Seems like it should get replaced. See https://www.ibm.com/docs/en/openxl-c-and-cpp-aix/17.1.1?topic=pragmas-pragma-alloca-c-only > > Can `-Dalloca=__builtin_alloca` replace `#include `? Yes I believe. I will remove the `#pragma alloca` everywhere, I will remove the `#include ` everywhere and I will add `-Dalloca=__builtin_alloca` to the compile commands. If it works I will update the PR. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559191851 From jkern at openjdk.org Wed Apr 10 10:46:35 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 10:46:35 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v5] In-Reply-To: References: Message-ID: > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf Joachim Kern has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Merge master - cosmetic changes - version check not needed anymore - Followed the proposals - JDK-8329257 ------------- Changes: https://git.openjdk.org/jdk/pull/18536/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=04 Stats: 257 lines in 14 files changed: 11 ins; 208 del; 38 mod Patch: https://git.openjdk.org/jdk/pull/18536.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18536/head:pull/18536 PR: https://git.openjdk.org/jdk/pull/18536 From jkern at openjdk.org Wed Apr 10 11:45:25 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 11:45:25 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v6] In-Reply-To: References: Message-ID: <58IY2j850mbxA8NTbzOfjHfYX5_n9aib_e8amY7MVNo=.673fa8c2-ad22-4384-aef7-f5954e110a94@github.com> > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: replaced pragma alloca and include alloca by compiler define ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18536/files - new: https://git.openjdk.org/jdk/pull/18536/files/302ea6a7..801cfb54 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=04-05 Stats: 8 lines in 3 files changed: 0 ins; 7 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18536.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18536/head:pull/18536 PR: https://git.openjdk.org/jdk/pull/18536 From jkern at openjdk.org Wed Apr 10 12:15:34 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 12:15:34 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v7] In-Reply-To: References: Message-ID: > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: saver solution ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18536/files - new: https://git.openjdk.org/jdk/pull/18536/files/801cfb54..a8d85924 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=05-06 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18536.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18536/head:pull/18536 PR: https://git.openjdk.org/jdk/pull/18536 From djelinski at openjdk.org Wed Apr 10 12:27:29 2024 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Wed, 10 Apr 2024 12:27:29 GMT Subject: RFR: 8201183: sjavac build failures: "Connection attempt failed: Connection refused" Message-ID: The "Connection attempt failed: Connection refused" error may happen if the client tries to connect to a server that is no longer listening, which in turn may happen if the server shuts down without removing the port file. I added a check that the delete operation succeeded, and retry as necessary. I removed the comment about asynchronous deletes on Windows. I don't think it's correct; it's more likely that the file existed because the delete operation failed. I added a 1 second delay after deleting the port file; this delay is intended to allow any clients that managed to read the port file before it was deleted to finish connecting. It should also take care of the "IOException caught during compilation: Connection reset" issue. And finally, the portfile is now closed when not in use. This was necessary to fix the failures on Windows, where the file cannot be deleted as long as it is open in any process. In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. Without the changes from this PR this resulted in at least a few failures in every mach5 run. With this PR I was able to build tier1-5 with no failures. ------------- Commit messages: - Close port file after use - Check delete result Changes: https://git.openjdk.org/jdk/pull/18712/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18712&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8201183 Stats: 32 lines in 1 file changed: 12 ins; 8 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/18712.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18712/head:pull/18712 PR: https://git.openjdk.org/jdk/pull/18712 From ihse at openjdk.org Wed Apr 10 12:59:19 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 12:59:19 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v25] In-Reply-To: <2Hs5V_cOOhOtXxrKvM0ATWxJFO62xd3RtScyPEb5puI=.f16650ac-9812-40ed-a93c-a5dd4b396ad2@github.com> References: <2Hs5V_cOOhOtXxrKvM0ATWxJFO62xd3RtScyPEb5puI=.f16650ac-9812-40ed-a93c-a5dd4b396ad2@github.com> Message-ID: On Thu, 4 Apr 2024 16:46:32 GMT, Severin Gehwolf wrote: >> Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use `jlink` even though your JDK install might not come with the packaged modules (directory `jmods`). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of `~25%`. Such a base JDK container could then be used to `jlink` application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image *or* separate bundles, depending on the app being modularized). >> >> The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (`lib/modules`). This enables producing a `JRTArchive` class which has all the info of what constitutes the final jlinked runtime. >> >> Basic usage example: >> >> $ diff -u <(./bin/java --list-modules --limit-modules java.se) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules java.se) >> $ diff -u <(./bin/java --list-modules --limit-modules jdk.jlink) <(../linux-x86_64-server-release/images/jdk/bin/java --list-modules --limit-modules jdk.jlink) >> $ ls ../linux-x86_64-server-release/images/jdk/jmods >> java.base.jmod java.net.http.jmod java.sql.rowset.jmod jdk.crypto.ec.jmod jdk.internal.opt.jmod jdk.jdi.jmod jdk.management.agent.jmod jdk.security.auth.jmod >> java.compiler.jmod java.prefs.jmod java.transaction.xa.jmod jdk.dynalink.jmod jdk.internal.vm.ci.jmod jdk.jdwp.agent.jmod jdk.management.jfr.jmod jdk.security.jgss.jmod >> java.datatransfer.jmod java.rmi.jmod java.xml.crypto.jmod jdk.editpad.jmod jdk.internal.vm.compiler.jmod jdk.jfr.jmod jdk.management.jmod jdk.unsupported.desktop.jmod >> java.desktop.jmod java.scripting.jmod java.xml.jmod jdk.hotspot.agent.jmod jdk.i... > > Severin Gehwolf has updated the pull request incrementally with three additional commits since the last revision: > > - Use shorter name for the build tool > - Review feedback from Erik J. > - Remove dependency on CDS which isn't needed anymore Marked as reviewed by ihse (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/14787#pullrequestreview-1991612359 From erikj at openjdk.org Wed Apr 10 13:18:09 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Apr 2024 13:18:09 GMT Subject: RFR: 8201183: sjavac build failures: "Connection attempt failed: Connection refused" In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 11:32:43 GMT, Daniel Jeli?ski wrote: > The "Connection attempt failed: Connection refused" error may happen if the client tries to connect to a server that is no longer listening, which in turn may happen if the server shuts down without removing the port file. I added a check that the delete operation succeeded, and retry as necessary. > > I removed the comment about asynchronous deletes on Windows. I don't think it's correct; it's more likely that the file existed because the delete operation failed. > > I added a 1 second delay after deleting the port file; this delay is intended to allow any clients that managed to read the port file before it was deleted to finish connecting. It should also take care of the "IOException caught during compilation: Connection reset" issue. > > And finally, the portfile is now closed when not in use. This was necessary to fix the failures on Windows, where the file cannot be deleted as long as it is open in any process. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. Without the changes from this PR this resulted in at least a few failures in every mach5 run. With this PR I was able to build tier1-5 with no failures. Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18712#pullrequestreview-1991663738 From mdoerr at openjdk.org Wed Apr 10 13:22:11 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 10 Apr 2024 13:22:11 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v7] In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 17:25:04 GMT, Stewart X Addison wrote: >> Pinging @sxa - what build environment does temurin use for AIX? > > Currently XLC16 but looking to upgrade to XLC17 on the minimum supported level for it (So it wouldn't be SP7 at present). Thanks for the ping - we have no current plans to increase to SP7. Seems like we need to keep it. This is unfortunate. I wouldn't risk mixing malloc and vec_malloc. Who knows what kind of problems this could cause? What happens if we try to build this code on AIX 7.2 TL5 SP7? Will the compiler complain because `malloc` is no longer defined? Should we check `defined(malloc)` in addition? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559425371 From jkern at openjdk.org Wed Apr 10 13:33:14 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 10 Apr 2024 13:33:14 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v7] In-Reply-To: References: Message-ID: <2KbsDhW7N8i7j6hn7zL9msQxCyw6SRCBuJgjsQf-o4Y=.88840d2a-c8c1-4d86-ada0-dfc53226b01f@github.com> On Wed, 10 Apr 2024 13:19:50 GMT, Martin Doerr wrote: >> Currently XLC16 but looking to upgrade to XLC17 on the minimum supported level for it (So it wouldn't be SP7 at present). Thanks for the ping - we have no current plans to increase to SP7. > > Seems like we need to keep it. This is unfortunate. I wouldn't risk mixing malloc and vec_malloc. Who knows what kind of problems this could cause? > What happens if we try to build this code on AIX 7.2 TL5 SP7? Will the compiler complain because `malloc` is no longer defined? Should we check `defined(malloc)` in addition? We already built this code since months on AIX 7.2 TL5 SP7, because we raised the OS. This code is needed on SP5 and does not hurt SP7. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559441769 From jwaters at openjdk.org Wed Apr 10 13:38:15 2024 From: jwaters at openjdk.org (Julian Waters) Date: Wed, 10 Apr 2024 13:38:15 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 10:13:37 GMT, Joachim Kern wrote: >> Can `-Dalloca=__builtin_alloca` replace `#include `? > > Yes I believe. I will remove the `#pragma alloca` everywhere, I will remove the `#include ` everywhere and I will add `-Dalloca=__builtin_alloca` to the compile commands. If it works I will update the PR. In my humble opinion the inclusion of alloca.h was slightly cleaner, but I guess it doesn't matter. Out of curiosity, why do you guys prefer not including it? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559450230 From ihse at openjdk.org Wed Apr 10 13:40:37 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 13:40:37 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v3] In-Reply-To: References: Message-ID: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: - Set JVM_VARIANT_PATH before use - Explain ResolveLibPath result Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18649/files - new: https://git.openjdk.org/jdk/pull/18649/files/6e737795..320f12d2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=01-02 Stats: 13 lines in 1 file changed: 6 ins; 6 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From ihse at openjdk.org Wed Apr 10 13:44:42 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 13:44:42 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v4] In-Reply-To: References: Message-ID: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Clarify how JDK_LIBS affect other arguments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18649/files - new: https://git.openjdk.org/jdk/pull/18649/files/320f12d2..245663fc Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=02-03 Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From ihse at openjdk.org Wed Apr 10 13:44:42 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 13:44:42 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v2] In-Reply-To: References: <9PqBTNNbqMJunV0Ms2cxejNjkObUdICDvJWHBc3l7io=.daec7bca-2c10-4a05-b092-373a43c9575b@github.com> Message-ID: On Fri, 5 Apr 2024 18:32:54 GMT, Erik Joelsson wrote: >> Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix libfallbackLinker > > make/common/JdkNativeCompilation.gmk line 206: > >> 204: $$(eval $$(call ResolveLibPath,$1,$2)) >> 205: >> 206: $1_EXTRA_HEADER_DIRS += $$($1_$2_MODULE):lib$$($1_$2_NAME) > > I think the top level comment need to be clearer about JDK_LIB implicitly setting EXTRA_HEADER_DIRS. It is documented on line 175, for AddJdkLibrary. I added it to SetupJdkNativeCompilation as well, not sure if that was what you meant. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1559462557 From mdoerr at openjdk.org Wed Apr 10 13:49:12 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 10 Apr 2024 13:49:12 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 13:35:39 GMT, Julian Waters wrote: >> Yes I believe. I will remove the `#pragma alloca` everywhere, I will remove the `#include ` everywhere and I will add `-Dalloca=__builtin_alloca` to the compile commands. If it works I will update the PR. > > In my humble opinion the inclusion of alloca.h was slightly cleaner, but I guess it doesn't matter. Out of curiosity, why do you guys prefer not including it? When only looking at AIX code, I think the inclusion of alloca.h was cleaner. Agreed. The new code makes AIX behave like other platforms and avoids the AIX specific part in shared code. I could live with either version. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559470659 From erikj at openjdk.org Wed Apr 10 13:51:30 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Apr 2024 13:51:30 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v2] In-Reply-To: References: <9PqBTNNbqMJunV0Ms2cxejNjkObUdICDvJWHBc3l7io=.daec7bca-2c10-4a05-b092-373a43c9575b@github.com> Message-ID: On Wed, 10 Apr 2024 13:41:53 GMT, Magnus Ihse Bursie wrote: >> make/common/JdkNativeCompilation.gmk line 206: >> >>> 204: $$(eval $$(call ResolveLibPath,$1,$2)) >>> 205: >>> 206: $1_EXTRA_HEADER_DIRS += $$($1_$2_MODULE):lib$$($1_$2_NAME) >> >> I think the top level comment need to be clearer about JDK_LIB implicitly setting EXTRA_HEADER_DIRS. > > It is documented on line 175, for AddJdkLibrary. I added it to SetupJdkNativeCompilation as well, not sure if that was what you meant. That is what I meant, as that is the public API. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1559473879 From ihse at openjdk.org Wed Apr 10 13:51:30 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 13:51:30 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v5] In-Reply-To: References: Message-ID: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. Magnus Ihse Bursie has updated the pull request incrementally with three additional commits since the last revision: - Clarify libjvm virtual library - Fix indentation - Remove misplaced line ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18649/files - new: https://git.openjdk.org/jdk/pull/18649/files/245663fc..3ce7793c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=03-04 Stats: 6 lines in 2 files changed: 1 ins; 1 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From clanger at openjdk.org Wed Apr 10 13:59:10 2024 From: clanger at openjdk.org (Christoph Langer) Date: Wed, 10 Apr 2024 13:59:10 GMT Subject: RFR: 8329970: Update autoconf build-aux files with latest from 2024-01-01 In-Reply-To: References: Message-ID: <_mMw3A9xMPWA_aUUmV8Y2KkCL_KQCz3e5gyusDEnPdg=.8fb908d7-90c2-4aa7-b36c-e346b42c4eba@github.com> On Tue, 9 Apr 2024 19:43:27 GMT, Erik Joelsson wrote: > Since we are now able to update the autoconf build-aux files, I think it's prudent to semi regularly do just that. I'm not aware of any particular platform that has been improved that would affect OpenJDK and I don't think we can further trim down our wrappers this time. This is more about staying with the latest. As of the filing of this bug, that is 2024-01-01, found here: > > https://git.savannah.gnu.org/cgit/config.git/ > > I have verified this with the platforms we build for at Oracle. I would encourage other OpenJDK distributors to verify on any platforms not covered by us. I will leave this open for a few days. Will run it through the tests at SAP... ------------- PR Comment: https://git.openjdk.org/jdk/pull/18704#issuecomment-2047622086 From jwaters at openjdk.org Wed Apr 10 14:07:12 2024 From: jwaters at openjdk.org (Julian Waters) Date: Wed, 10 Apr 2024 14:07:12 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v5] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 13:51:30 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with three additional commits since the last revision: > > - Clarify libjvm virtual library > - Fix indentation > - Remove misplaced line Marked as reviewed by jwaters (Committer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18649#pullrequestreview-1991798951 From stuefe at openjdk.org Wed Apr 10 14:24:16 2024 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 10 Apr 2024 14:24:16 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v7] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 12:15:34 GMT, Joachim Kern wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > saver solution This looks good to me now, provided Martin likes it too. Thanks for incorporating my suggestions. ------------- Marked as reviewed by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18536#pullrequestreview-1991847641 From stuefe at openjdk.org Wed Apr 10 14:24:17 2024 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 10 Apr 2024 14:24:17 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 13:46:11 GMT, Martin Doerr wrote: >> In my humble opinion the inclusion of alloca.h was slightly cleaner, but I guess it doesn't matter. Out of curiosity, why do you guys prefer not including it? > > When only looking at AIX code, I think the inclusion of alloca.h was cleaner. Agreed. The new code makes AIX behave like other platforms and avoids the AIX specific part in shared code. > I could live with either version. I can live with either, too. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1559536724 From mdoerr at openjdk.org Wed Apr 10 14:28:16 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 10 Apr 2024 14:28:16 GMT Subject: RFR: JDK-8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v7] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 12:15:34 GMT, Joachim Kern wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > saver solution Yes, I like it too ? Thanks, Thomas, for your helpful feedback! ------------- Marked as reviewed by mdoerr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18536#pullrequestreview-1991857617 From ihse at openjdk.org Wed Apr 10 14:38:31 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 14:38:31 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v6] In-Reply-To: References: Message-ID: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 14 additional commits since the last revision: - Merge branch 'master' into jdk-libs-framework - Update parsing to new syntax, and add missing "lib" prefix for gtest. - Use new syntax - Clarify libjvm virtual library - Fix indentation - Remove misplaced line - Clarify how JDK_LIBS affect other arguments - Set JVM_VARIANT_PATH before use - Explain ResolveLibPath result Co-authored-by: Erik Joelsson <37597443+erikj79 at users.noreply.github.com> - Fix libfallbackLinker - ... and 4 more: https://git.openjdk.org/jdk/compare/20f7321c...b8bedc8b ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18649/files - new: https://git.openjdk.org/jdk/pull/18649/files/3ce7793c..b8bedc8b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=04-05 Stats: 12308 lines in 380 files changed: 6978 ins; 3460 del; 1870 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From ihse at openjdk.org Wed Apr 10 14:43:26 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 14:43:26 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v7] In-Reply-To: References: Message-ID: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Remove extra blank line ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18649/files - new: https://git.openjdk.org/jdk/pull/18649/files/b8bedc8b..14cf0a02 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=05-06 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From ihse at openjdk.org Wed Apr 10 14:50:01 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 14:50:01 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v7] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 14:43:26 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Remove extra blank line I'm glad we agreed on the new syntax. It aligns nicely with the existing SRC syntax, and there is no ambiguity or symbols like `@` with special meaning (I was not fond of that either, but could not figure out a better way to mimic the old syntax). It is a bit longer, and a line like this is a bit hard to read: JDK_LIBS := java.base:libjava java.base:libjli java.base:libjvm, \ but I guess that is a price we just have to pay. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2047760454 From ascarpino at openjdk.org Wed Apr 10 17:22:09 2024 From: ascarpino at openjdk.org (Anthony Scarpino) Date: Wed, 10 Apr 2024 17:22:09 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 19:19:59 GMT, Volodymyr Paprotski wrote: >> Performance. Before: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s >> >> Performance, no intrinsic: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply true thrpt 3 1919.57... > > Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: > > remove use of jdk.crypto.ec In `ECOperations.java`, if I understand this correctly, it is to replace the existing `PointMultiplier` with montgomery-based PointMuliplier. But when I look at the code, I see both are still options. If I read this correctly, it checks for the old `IntegerFieldModuloP`, then looks for the new `IntegerMontgomeryFieldModuloP`. It appears to use the new one always. Why doesn't it just replace the old implementation entry in the `fields` Map? Is there a reason to keep it around? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18583#issuecomment-2048090075 From duke at openjdk.org Wed Apr 10 18:05:10 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Wed, 10 Apr 2024 18:05:10 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 17:18:55 GMT, Anthony Scarpino wrote: > In `ECOperations.java`, if I understand this correctly, it is to replace the existing `PointMultiplier` with montgomery-based PointMuliplier. But when I look at the code, I see both are still options. If I read this correctly, it checks for the old `IntegerFieldModuloP`, then looks for the new `IntegerMontgomeryFieldModuloP`. It appears to use the new one always. Why doesn't it just replace the old implementation entry in the `fields` Map? Is there a reason to keep it around? Hmm, thats a good point I haven't fully considered; i.e. (if I read correctly) "for `CurveDB.P_256` remove the fallback path to non-montgomery entirely".. that might also help in cleaning a few things up in the construction. Maybe even get rid of this nested ECOperations inside ECOperations.. Perhaps nesting isnt a big deal, but all attempts to make the ECC stack clearer is positive! One functional reason that might justify keeping it as-is, is fuzz-testing; with the fallback available, I am able to write the included Fuzz tests and have them check the values against the existing implementation. While I also included a few KAT tests using openssl-generated values, the fuzz tests check millions of values and it does add a lot more certainty about correctness of this code. Can it be removed? For the operations that do not involve multiplication (i.e. `setSum(*)`), montgomery is expensive. I think I did go through the uses of this code some time back (i.e. ECDHE, ECDSA and KeyGeneration) and existing IntegerPolynomialP256 is no longer used (I should verify that again) and only P256OrderField remains non-montgomery. So removing references to IntegerPolynomialP256 in ECOperations should be possible and cleaner. Removing IntegerPolynomialP256 from MontgomeryIntegerPolynomialP256 is harder (fromMontgomery() uses IntegerPolynomialP256) but perhaps also worth some thought.. I tend to like `ECOperationsFuzzTest.java` and would prefer to keep it, but it could also be chucked up as part of 'scaffolding' and removed in name of code quality? Thanks @ascarpino PS: Perhaps there is some middle ground, remove the `ECOperations montgomeryOps` nesting, and construct (somehow?? singleton makes most things inaccessible..) the reference ECOperations in the fuzz test instead.. not sure how yet, but perhaps worth a further thought.. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18583#issuecomment-2048159645 From erikj at openjdk.org Wed Apr 10 18:46:11 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 10 Apr 2024 18:46:11 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v7] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 14:43:26 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Remove extra blank line This looks good to me now. Thank you for changing the format. ------------- Marked as reviewed by erikj (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18649#pullrequestreview-1992433321 From mli at openjdk.org Wed Apr 10 18:49:10 2024 From: mli at openjdk.org (Hamlin Li) Date: Wed, 10 Apr 2024 18:49:10 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Wed, 10 Apr 2024 09:24:09 GMT, Hamlin Li wrote: > > Thank you for the update and for working on this in general. > > I've started working on JDK-8329816, preparing the change for the SLEEF specific part of the change. Specifically, I'm currently planning on including the three SLEEF header files, the README and a legal/sleef.md file in that change. Let me know if you have any thoughts/concerns. > > Thanks a lot, that's a great news. Please go ahead to integrate the files via JDK-8329816. :) Besides of the performance issue currently found out, I have no other concerns. I found the root cause of the performance regression, and have a draft solution for it, I'm running a thorough benchmark to see if it works for all sleef functions we use in jdk. So, basically this solution is good. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2048217671 From duke at openjdk.org Wed Apr 10 20:47:00 2024 From: duke at openjdk.org (Tres Finocchiaro) Date: Wed, 10 Apr 2024 20:47:00 GMT Subject: RFR: 8315113: Print request Chromaticity.MONOCHROME attribute does not work on macOS In-Reply-To: <9NaQ7MzCph2B2CYYqk_o_yMqLtOz45sWsq8fbCk67Es=.d278e2e4-2dfe-499f-913f-b7ec47e577b9@github.com> References: <9NaQ7MzCph2B2CYYqk_o_yMqLtOz45sWsq8fbCk67Es=.d278e2e4-2dfe-499f-913f-b7ec47e577b9@github.com> Message-ID: On Mon, 8 Apr 2024 23:19:13 GMT, Phil Race wrote: >> The fix provides ability to print Black & White pages on macOS. >> >> Cocoa API has [PMSetColorMode](https://developer.apple.com/documentation/applicationservices/core_printing/1805783-pmsetcolormode) function but it is marked as deprecated and really does nothing. >> >> There is no replacement; this function was included to facilitate porting legacy applications to macOS, >> but it serves no useful purpose. >> >> Dumping `NSPrintInfo` print settings which were set by the native print dialog on macOS shows that the keys and values used for Black & White printing depend on the used printer type. >> For example, the tested >> `HP Color LaserJet MFP M180n` printer uses `ColorModel` key and`Gray` value, and >> `HP Ink Tank 110 series` uses `HPColorMode` key and `grayscale` value. >> >> Printing all `NSPrintInfo` presets shows that they do not contain key/value pairs for Black&White settings. >> This is the code snippet used to print `NSPrintInfo` presets: >> >> PMPrinter pr; >> PMPrintSession printSession = (PMPrintSession)[printInfo PMPrintSession]; >> OSStatus status = PMSessionGetCurrentPrinter(printSession, &pr); >> CFArrayRef presetsList = nil; >> status = PMPrinterCopyPresets(pr, &presetsList); >> CFIndex arrayCount = CFArrayGetCount(presetsList); >> >> for (CFIndex index = 0; index < arrayCount; index++) { >> PMPreset preset = (PMPreset)CFArrayGetValueAtIndex(presetsList, index); >> CFStringRef presetName = nil; >> if (PMPresetCopyName(preset, &presetName) == noErr && CFStringGetLength(presetName) > 0) { >> NSLog(@" presetName: '%@'", presetName); >> NSDictionary* dict = nil; >> if (PMPresetGetAttributes(preset, (CFDictionaryRef*)(&dict)) == noErr) { >> // print preset dict >> >> >> The idea of the proposed fix is to store printer dependent key/value pairs in the `/conf/printer.properties` property file. >> >> The property file has the format: >> >> print-attribute.print-attribute-value.key=value >> >> where `print-attribute` is the java print attribute, `print-attribute-value` is the corresponding attribute value, and `key` and `value` is the key/value pair used by a specific printer. >> >> For example, for `Chromaticity` attribute the property file could look like: >> >> Chromaticity.MONOCHROME.ColorModel=Gray >> Chromaticity.COLOR.ColorModel=CMYK >> Chromaticity.MONOCHROME.HPColorMode=grayscale >> Chromaticity.COLOR.HPColorMode=color >> >> where `Chromaticity.MO... > > Since the user can always select greyscale in the user dialog, I presume you have some requirement > to do this printing without user intervention. Perhaps there's no user there (server-side printing), or > the app would like to make sure the user always defaults to B&W. > > As far as I can tell, this isn't supported via any kind of API on macOS, so it is not a Java problem. > You could be printing from an Objective-C app written directly to Cocoa APIs and still have the same problem. > So a Java-specific workaround like this seems inappropriate. > > I've poked around to help me understand what is going on. > > When I do "Print to File" on macOS - meaning use the native printer dialog's "Save as PDF" option, > then the generated PDF is always colour even if I select the printer dialog's "Grayscale Printing" option. > This isn't just true for Java apps, the same happens if I print a web page from Safari. > > And when I query the Chromaticity support for a couple of colour printers - one Canon, one HP, > both report they support it but only the value COLOR. > > So I conclude the grayscale printing option is something handled by the printer driver at the time it is converted to > the printer-specific format and sent to the physical printer and it isn't changing the rendering. > > So in general without an API, to get greyscale it requires the end-user to set an option in a print dialog > and as mentioned above, what you are trying o enable isn't possible even if you wrote the app in > Objective-C or Swift directly as a macOS native app. > > But what works for me to get greyscale by default is to set that up as the default for the print queue. > It is easy to add a printer twice - with two queues for it. > One has the default set to greyscale, the other to colour. Then the app just needs to select the required queue. > This is clearly more site-specific than having the app just specify MONOCHROME but is a lot lower cost all round. > > I don't understand why PMSetColorMode is deprecated and non-functional since it seems like it would > be the right option here, but then internally macOS would need to be able to map it to the right option. > Doubtless that would be easier if MONOCHROME were supported by all these drivers and I don't understand > that omission either. In fact if it were we probably could just specify that directly without macOS needing > to understand anything - like I expect it doesn't understand the key/value pairs your code is sending it. > I note that you send ALL known key/value pairs and hope one ... @prrace thanks for the detailed thoughts on this problem. As the stakeholder in this issue, I wanted to offer some feedback to your thoughts. I understand that the desire for OpenJDK is to NOT maintain a list of proprietary printer settings, but since many of the thoughts above get into use-case scenarios, I feel compelled to share my perspective. I hope these are well received. > When I do "Print to File" on macOS - meaning use the native printer dialog's "Save as PDF" option, then the generated PDF is always colour even if I select the printer dialog's "Grayscale Printing" option. This isn't just true for Java apps, the same happens if I print a web page from Safari. Several PDF printers (on several OSs) ignore grayscale/b&w settings. I'm not sure why, but I've observed the same. > As far as I can tell, this isn't supported via any kind of API on macOS, so it is not a Java problem. > You could be printing from an Objective-C app written directly to Cocoa APIs and still have the same problem. Agreed. > So a Java-specific workaround like this seems inappropriate. As the stakeholder (biased) it's hard for me to agree, however allowing the implementing application to maintain (and thus offer) this workaround would certainly suffice. Since AWT (quite graciously) abstracts these attributes away from the user, "wontfix" status naturally causes a feature gap, albeit not the fault of OpenJDK, but a parity that -- at a bare minimum -- could live on as -- for example -- a workaround on Stackoverflow, etc. > But what works for me to get greyscale by default is to set that up as the default for the print queue. > It is easy to add a printer twice - with two queues for it. Easy is subjective (and prescriptive). Again, biased, but setting up additional printer queues is a viable workaround, but comes at a factor of 2 queues for each color printer added to a system. This also offers support redundancies when removing and re-adding a printer (e.g. IP, port or driver change). > I don't understand why `PMSetColorMode` is deprecated and non-functional since it seems like it would be the right option here, but then internally macOS would need to be able to map it to the right option. Agreed. > [...] all it takes is some vendor API change or a different printer and it just won't work again. These points and the implied maintenance cost are some of my bigger concerns with this. Understood. > The other idea that crossed my mind is that instead of relying on a printing solution, we could explicitly render in greyscale. Then we could always declare MONOCHROME support and just emulate it. But I am not sure if this is really possible - it would require changes to the rendering code and I'm not sure if or how if we'd be able to do that properly and safely. And the risks of that clearly extend beyond printing. Since printers can emulate grayscale using color cartridges, I do not believe this will yield the same results as expected on all printers, but I do believe this to be a creative stop-gap/workaround which may be suitable for some use-cases. Naturally, I would expect some performance implications as a result. As the stakeholder, I can say that this will be trickier with prints using vector graphics than those already rastered/pixels. Speculating, vector graphics will likely be less performance-affecting, but require diving into the document modifications (e.g. PDFBOX, JavaFX) albeit more performant, will require some creative code to desaturate all color data in all objects prior to sending (or requests to add APIs thereo to each specific project). > Perhaps there's no user there (server-side printing), or the app would like to make sure the user always defaults to B&W. I'm not sure how much interest there is in understanding the use-case that discovered this issue, but it's part of a (pretty popular) Java app that helps print documents. It does so through a WebSocket interface... so in that sense it is server-like (effectively headless), although it's predominantly used by ordinary, non-technical end-users and runs as a sort of background service. Much like AWT offers a (rather) standardized API for printing, so does the Websocket interface, so "Black & White" or "Monochrome" are offered as common, selectable options. "Always default to B&W" would be a case-by-case situation, depending on how the websocket API is offered up to the end-user. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18195#issuecomment-2048405524 From ihse at openjdk.org Wed Apr 10 21:10:27 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 10 Apr 2024 21:10:27 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v8] In-Reply-To: References: Message-ID: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Fix missing lib prefix ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18649/files - new: https://git.openjdk.org/jdk/pull/18649/files/14cf0a02..d40e00db Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18649&range=06-07 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18649.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18649/head:pull/18649 PR: https://git.openjdk.org/jdk/pull/18649 From kbarrett at openjdk.org Wed Apr 10 22:16:44 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 10 Apr 2024 22:16:44 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 14:19:59 GMT, Thomas Stuefe wrote: >> When only looking at AIX code, I think the inclusion of alloca.h was cleaner. Agreed. The new code makes AIX behave like other platforms and avoids the AIX specific part in shared code. >> I could live with either version. > > I can live with either, too. That build failure in shared code does not happen with Xcode clang, gcc, or Visual Studio, even though none of them appear to have a relevant define or include. So the clang variant being used for AIX is different from the Xcode clang variant (and maybe others) in its treatment of alloca. Weird! I can also live with either the macro or the includes where needed. I dislike conditionally adding the include in globalDefinitions_gcc.hpp. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1560113548 From kbarrett at openjdk.org Wed Apr 10 22:16:45 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 10 Apr 2024 22:16:45 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 22:12:42 GMT, Kim Barrett wrote: >> I can live with either, too. > > That build failure in shared code does not happen with Xcode clang, gcc, or > Visual Studio, even though none of them appear to have a relevant define or > include. So the clang variant being used for AIX is different from the Xcode > clang variant (and maybe others) in its treatment of alloca. Weird! > > I can also live with either the macro or the includes where needed. I dislike > conditionally adding the include in globalDefinitions_gcc.hpp. Should also remove the `#pragma alloca` in os_aix.cpp. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1560114722 From vlivanov at openjdk.org Wed Apr 10 23:34:42 2024 From: vlivanov at openjdk.org (Vladimir Ivanov) Date: Wed, 10 Apr 2024 23:34:42 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: <70tJacyO4JrzIFiCc9rxO3P46sC5ieEJEdWbSRuKsaU=.73481033-6a81-4ced-a328-f82cd2794ac7@github.com> On Fri, 5 Apr 2024 12:17:17 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review the patch? >> This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). >> >> Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. >> Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. >> >> Besides of the code changes, one important task is to handle the legal process. >> >> Thanks! > > Hamlin Li has updated the pull request incrementally with two additional commits since the last revision: > > - disable unused-function warnings; add log msg > - minor Nice work, Hamlin and Xiaohong. I'm glad to see progress on incorporating SLEEF library into the JDK. (Somehow I missed all previous PRs you posted before.) I'm not a lawyer, so won't comment on 3rd party library sources under Boost Software License in OpenJDK. >From engineering perspective, I believe that bundling vector math library with the JDK is the right thing to do, but it doesn't imply the sources should be part of JDK. There are already examples of optional dependencies on external native libraries in HotSpot (e.g., hsdis tool w/ binutils, capstone, and llvm backends). Speaking of HotSpot-specific changes, IMO it desperately needs a cross-platform interface between vector math libraries and JVM. Most of the changes in `StubGenerator` are library-specific and are irrelevant in the context of the JVM. I do see that you try to replicate SVML logic, but SVML support didn't set a precedent to follow here. For background, SVML stubs were initially contributed to Panama as assembly stubs statically linked into libjvm.so. It was acceptable for experimentation purposes, but not for mainline JDK (even for functionality in incubating module). The compromise was to bundle the stubs as a dynamic library and link against them. And that's how it stayed until today. IMO in order to get SLEEF in, the interaction between JVM and backend native library should be unified. And it should affect both SLEEF and SVML stubs. In particular, I'd like to see all those named lookups to go away from the JVM code. A single call into the library during compiler/VM initialization can produce a fully populated table of function pointers (`StubRoutines::_vector_[fd]_math` now) for C2 to use later. FTR there were other alternatives discussed (use Panama FFI or rewrite the stubs in Vector API itself). The latter (complete rewrite) is still something for a distant future, but Foreign Function API is public API now, so once it supports vector calling conventions, it should become fully capable of satisfying Vector API implementation needs to interact with vector math library. IMO that what we should keep in mind when designing new interface. There's no inherent need to keep vector stub support in the JVM. Once Foreign Function API gains vector support, it should be replaced with a pure Java FFI-based implementation. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2048599089 From duke at openjdk.org Wed Apr 10 23:59:41 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Wed, 10 Apr 2024 23:59:41 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: <48md2WEAhqPyuVf4AYOxBQDykUiOaEL0PQb-ki0_TYM=.6c25bf41-b0ae-49ec-b606-236deb4561e3@github.com> On Fri, 5 Apr 2024 09:17:18 GMT, Jatin Bhateja wrote: > Few early comments. > > Please update the copyright year of all the modified files. > > You can even consider splitting this into two patches, Java side changes in one and x86 optimized intrinsic in next one. Thanks Jatin, will fix! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18583#issuecomment-2048618452 From kbarrett at openjdk.org Thu Apr 11 00:52:42 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Thu, 11 Apr 2024 00:52:42 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v7] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 12:15:34 GMT, Joachim Kern wrote: >> As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). >> Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. >> This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. >> The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > saver solution Looks good. ------------- Marked as reviewed by kbarrett (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18536#pullrequestreview-1992964955 From aph at openjdk.org Thu Apr 11 09:11:42 2024 From: aph at openjdk.org (Andrew Haley) Date: Thu, 11 Apr 2024 09:11:42 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Fri, 5 Apr 2024 12:17:17 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review the patch? >> This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). >> >> Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. >> Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. >> >> Besides of the code changes, one important task is to handle the legal process. >> >> Thanks! > > Hamlin Li has updated the pull request incrementally with two additional commits since the last revision: > > - disable unused-function warnings; add log msg > - minor > Nice work, Hamlin and Xiaohong. I'm glad to see progress on incorporating SLEEF library into the JDK. (Somehow I > From engineering perspective, I believe that bundling vector math library with the JDK is the right thing to do, but it doesn't imply the sources should be part of JDK. There are already examples of optional dependencies on external native libraries in HotSpot (e.g., hsdis tool w/ binutils, capstone, and llvm backends). No, it doesn't imply that the sources should be part of JDK, but practical reasons to do with the way that OpenJDK is built and shipped by various parties strongly suggests that we should integrate the SLEEF library into the JDK source tree. If we don't, there will be skew between OpenJDK versions shipped by different vendors. Also, I believe that there is less work for all of us if we integrate rather than having communicate to everyone building the JDK. And finally, Mark Reinhold has stated that the JDK is not downstream of any other project. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2049256172 From ihse at openjdk.org Thu Apr 11 09:44:47 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 09:44:47 GMT Subject: Integrated: 8329704: Implement framework for proper handling of JDK_LIBS In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 09:56:48 GMT, Magnus Ihse Bursie wrote: > This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. This pull request has now been integrated. Changeset: f0cd866a Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/f0cd866a375082e14c69ccd3bf5e3d4d18edaebf Stats: 578 lines in 38 files changed: 211 ins; 266 del; 101 mod 8329704: Implement framework for proper handling of JDK_LIBS Reviewed-by: erikj, jwaters ------------- PR: https://git.openjdk.org/jdk/pull/18649 From jkern at openjdk.org Thu Apr 11 09:48:56 2024 From: jkern at openjdk.org (Joachim Kern) Date: Thu, 11 Apr 2024 09:48:56 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v8] In-Reply-To: References: Message-ID: <6-WH9_Hbwi2jmndNw-AfJUDEdEFd5wqtE1dDiB6lasQ=.d5f99193-75c8-4d50-8a77-7a7126bd5b2d@github.com> > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: my_disclaim64 already removed by other PR ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18536/files - new: https://git.openjdk.org/jdk/pull/18536/files/a8d85924..030de164 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18536&range=06-07 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/18536.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18536/head:pull/18536 PR: https://git.openjdk.org/jdk/pull/18536 From ihse at openjdk.org Thu Apr 11 10:20:41 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 10:20:41 GMT Subject: RFR: 8201183: sjavac build failures: "Connection attempt failed: Connection refused" In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 11:32:43 GMT, Daniel Jeli?ski wrote: > The "Connection attempt failed: Connection refused" error may happen if the client tries to connect to a server that is no longer listening, which in turn may happen if the server shuts down without removing the port file. I added a check that the delete operation succeeded, and retry as necessary. > > I removed the comment about asynchronous deletes on Windows. I don't think it's correct; it's more likely that the file existed because the delete operation failed. > > I added a 1 second delay after deleting the port file; this delay is intended to allow any clients that managed to read the port file before it was deleted to finish connecting. It should also take care of the "IOException caught during compilation: Connection reset" issue. > > And finally, the portfile is now closed when not in use. This was necessary to fix the failures on Windows, where the file cannot be deleted as long as it is open in any process. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. Without the changes from this PR this resulted in at least a few failures in every mach5 run. With this PR I was able to build tier1-5 with no failures. Thank you very much for spending the time and effort of fixing these intermittent javac server issues! ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18712#pullrequestreview-1993815955 From mli at openjdk.org Thu Apr 11 10:36:03 2024 From: mli at openjdk.org (Hamlin Li) Date: Thu, 11 Apr 2024 10:36:03 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v3] In-Reply-To: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: > Hi, > Can you help to review the patch? > This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). > > Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. > Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. > > Besides of the code changes, one important task is to handle the legal process. > > Thanks! Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: fix performance issue ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18605/files - new: https://git.openjdk.org/jdk/pull/18605/files/34529ff1..cd70f5a9 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18605&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18605&range=01-02 Stats: 4 lines in 4 files changed: 1 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/18605.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18605/head:pull/18605 PR: https://git.openjdk.org/jdk/pull/18605 From mli at openjdk.org Thu Apr 11 10:38:42 2024 From: mli at openjdk.org (Hamlin Li) Date: Thu, 11 Apr 2024 10:38:42 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v2] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Tue, 9 Apr 2024 20:10:36 GMT, Mikael Vidstedt wrote: >> Hamlin Li has updated the pull request incrementally with two additional commits since the last revision: >> >> - disable unused-function warnings; add log msg >> - minor > > Thank you for the update and for working on this in general. > > I've started working on JDK-8329816, preparing the change for the SLEEF specific part of the change. Specifically, I'm currently planning on including the three SLEEF header files, the README and a legal/sleef.md file in that change. Let me know if you have any thoughts/concerns. > > Also, just for my understanding, would love to understand your thoughts on the future here (I apologize if this was already discussed elsewhere): > > It seem like SLEEF is (sort of) limited to linux at this point (the SLEEF README mentions that "Due to limited test capacities, SLEEF is currently only officially supported on Linux with gcc or llvm/clang." ). That same README does, however, indicate good test coverage on several architectures in addition to aarch64 (including x86_64, PPC, RISC-V). With that in mind, it looks like we could potentially use SLEEF for other architectures on linux in the future? And potentially additional operating systems as well? Hey, @vidmik I've fixed the performance issue, and update the sleef inline headers and README. It's good for you to integrate these files via JDK-8329816. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2049400793 From ihse at openjdk.org Thu Apr 11 10:44:44 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 10:44:44 GMT Subject: RFR: 8329970: Update autoconf build-aux files with latest from 2024-01-01 In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 19:43:27 GMT, Erik Joelsson wrote: > Since we are now able to update the autoconf build-aux files, I think it's prudent to semi regularly do just that. I'm not aware of any particular platform that has been improved that would affect OpenJDK and I don't think we can further trim down our wrappers this time. This is more about staying with the latest. As of the filing of this bug, that is 2024-01-01, found here: > > https://git.savannah.gnu.org/cgit/config.git/ > > I have verified this with the platforms we build for at Oracle. I would encourage other OpenJDK distributors to verify on any platforms not covered by us. I will leave this open for a few days. Marked as reviewed by ihse (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18704#pullrequestreview-1993854457 From mli at openjdk.org Thu Apr 11 10:45:46 2024 From: mli at openjdk.org (Hamlin Li) Date: Thu, 11 Apr 2024 10:45:46 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v3] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Thu, 11 Apr 2024 10:36:03 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review the patch? >> This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). >> >> Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. >> Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. >> >> Besides of the code changes, one important task is to handle the legal process. >> >> Thanks! > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > fix performance issue Thanks everyone for discussion about the direction (integrate source or lib). We did have some implementation for integrating sleef lib into jdk, but seems previously the most strong opinion is to integrate the sleef source into jdk. I know there are cons and pros for every solution, but I will stick to current solution unless everyone can reach another agreement. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2049410731 From ihse at openjdk.org Thu Apr 11 11:14:57 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 11:14:57 GMT Subject: RFR: 8326947: Minimize MakeBase.gmk [v2] In-Reply-To: References: Message-ID: > This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. > > This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 14 commits: - Merge branch 'master' into minimize-makebase - Whitespace fix - MakeBase.gmk should not include MakeIO.gmk anymore - MakeBase.gmk should not include CopyFiles.gmk anymore - Reorder BaseUtils.gmk to make more sense - Move some more functionality to BaseUtils.gmk - Create BaseUtils.gmk with the most basic stuff - Move all file stuff from Utils.gmk to FileUtils.gmk - Document the purpose of MakeBase - Move SOURCE_REVISION_TRACKER to where it is used. This unfortunately creates a duplication, but there is no suitable place to put this information, and it does not belong in MakeBase. - ... and 4 more: https://git.openjdk.org/jdk/compare/9acce7a6...36bb38f0 ------------- Changes: https://git.openjdk.org/jdk/pull/18041/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=01 Stats: 1099 lines in 43 files changed: 607 ins; 480 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/18041.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18041/head:pull/18041 PR: https://git.openjdk.org/jdk/pull/18041 From mli at openjdk.org Thu Apr 11 11:31:44 2024 From: mli at openjdk.org (Hamlin Li) Date: Thu, 11 Apr 2024 11:31:44 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v3] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Thu, 11 Apr 2024 10:36:03 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review the patch? >> This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). >> >> Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. >> Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. >> >> Besides of the code changes, one important task is to handle the legal process. >> >> Thanks! >> >> ## Performance >> NOTE: >> * `Src` means implementation in this pr, i.e. without depenency on external sleef. >> * `Disabled` means disable intrinsics by `-XX:-UseVectorStubs` >> * `system_sleef` means implementation in [previous pr 18294](https://github.com/openjdk/jdk/pull/18294), i.e. build and run jdk with depenency on external sleef. >> >> Basically, the perf data below shows that >> * this implementation has better performance than previous version in [pr 18294](https://github.com/openjdk/jdk/pull/18294), >> * and both sleef versions has much better performance compared with non-sleef version. >> >> |Benchmark |(size)|Src |Units|system_sleef|(system_sleef-Src)/Src|Diabled |(Disable-Src)/Src| >> |------------------------------|------|---------|-----|------------|----------------------|---------|-----------------| >> |3472:Double128Vector.ACOS |1024 |8546.842 |ns/op|8516.007 |-0.004 |16799.273|0.966 | >> |3473:Double128Vector.ASIN |1024 |6864.656 |ns/op|6987.328 |0.018 |16602.442|1.419 | >> |3474:Double128Vector.ATAN |1024 |11489.255|ns/op|12261.800 |0.067 |26329.320|1.292 | >> |3475:Double128Vector.ATAN2 |1024 |16661.170|ns/op|17234.472 |0.034 |42084.100|1.526 | >> |3476:Double128Vector.CBRT |1024 |18999.387|ns/op|20298.458 |0.068 |35998.688|0.895 | >> |3477:Double128Vector.COS |1024 |14081.857|ns/op|14846.117 |0.054 |24420.692|0.734 | >> |3478:Double128Vector.COSH |1024 |12202.306|ns/op|12237.772 |0.003 |21343.863|0.749 | >> |3479:Double128Vector.EXP |1024 |4553.108 |ns/op|4777.638 ... > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > fix performance issue I've also updated the pr description with performance data, it shows that * this implementation has better performance than previous version in https://github.com/openjdk/jdk/pull/18294, * and both sleef versions has much better performance compared with non-sleef version. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2049482861 From ihse at openjdk.org Thu Apr 11 11:34:10 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 11:34:10 GMT Subject: RFR: 8326947: Minimize MakeBase.gmk [v3] In-Reply-To: References: Message-ID: > This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. > > This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Move SOURCE_REVISION_TRACKER to spec.gmk ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18041/files - new: https://git.openjdk.org/jdk/pull/18041/files/36bb38f0..47278b69 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=01-02 Stats: 12 lines in 3 files changed: 4 ins; 8 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18041.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18041/head:pull/18041 PR: https://git.openjdk.org/jdk/pull/18041 From ihse at openjdk.org Thu Apr 11 11:39:12 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 11:39:12 GMT Subject: RFR: 8326947: Minimize MakeBase.gmk [v4] In-Reply-To: References: Message-ID: <44xq9fehAkxbvkPc2LUEx99STbtEZ4uCjmY2OM9FLWg=.d8608832-63d2-4553-a786-9e29f661c99b@github.com> > This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. > > This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Inline BaseUtils.gmk into Utils.gmk ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18041/files - new: https://git.openjdk.org/jdk/pull/18041/files/47278b69..18bcf569 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=02-03 Stats: 357 lines in 3 files changed: 163 ins; 194 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18041.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18041/head:pull/18041 PR: https://git.openjdk.org/jdk/pull/18041 From ihse at openjdk.org Thu Apr 11 12:25:09 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 12:25:09 GMT Subject: RFR: 8326947: Minimize MakeBase.gmk [v5] In-Reply-To: References: Message-ID: > This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. > > This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Restore copyright year for untouched file ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18041/files - new: https://git.openjdk.org/jdk/pull/18041/files/18bcf569..bba3672d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=03-04 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18041.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18041/head:pull/18041 PR: https://git.openjdk.org/jdk/pull/18041 From djelinski at openjdk.org Thu Apr 11 12:45:51 2024 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Thu, 11 Apr 2024 12:45:51 GMT Subject: RFR: 8201183: sjavac build failures: "Connection attempt failed: Connection refused" In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 11:32:43 GMT, Daniel Jeli?ski wrote: > The "Connection attempt failed: Connection refused" error may happen if the client tries to connect to a server that is no longer listening, which in turn may happen if the server shuts down without removing the port file. I added a check that the delete operation succeeded, and retry as necessary. > > I removed the comment about asynchronous deletes on Windows. I don't think it's correct; it's more likely that the file existed because the delete operation failed. > > I added a 1 second delay after deleting the port file; this delay is intended to allow any clients that managed to read the port file before it was deleted to finish connecting. It should also take care of the "IOException caught during compilation: Connection reset" issue. > > And finally, the portfile is now closed when not in use. This was necessary to fix the failures on Windows, where the file cannot be deleted as long as it is open in any process. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. Without the changes from this PR this resulted in at least a few failures in every mach5 run. With this PR I was able to build tier1-5 with no failures. Thanks for the reviews! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18712#issuecomment-2049609810 From djelinski at openjdk.org Thu Apr 11 12:45:51 2024 From: djelinski at openjdk.org (Daniel =?UTF-8?B?SmVsacWEc2tp?=) Date: Thu, 11 Apr 2024 12:45:51 GMT Subject: Integrated: 8201183: sjavac build failures: "Connection attempt failed: Connection refused" In-Reply-To: References: Message-ID: <9jYQacqsc5L9f7pVydZRkwocvfo8nvdnRPLLhrmgGc8=.87798cc5-54f7-41df-b654-9ab4d69b689e@github.com> On Wed, 10 Apr 2024 11:32:43 GMT, Daniel Jeli?ski wrote: > The "Connection attempt failed: Connection refused" error may happen if the client tries to connect to a server that is no longer listening, which in turn may happen if the server shuts down without removing the port file. I added a check that the delete operation succeeded, and retry as necessary. > > I removed the comment about asynchronous deletes on Windows. I don't think it's correct; it's more likely that the file existed because the delete operation failed. > > I added a 1 second delay after deleting the port file; this delay is intended to allow any clients that managed to read the port file before it was deleted to finish connecting. It should also take care of the "IOException caught during compilation: Connection reset" issue. > > And finally, the portfile is now closed when not in use. This was necessary to fix the failures on Windows, where the file cannot be deleted as long as it is open in any process. > > In order to verify the fix, I modified `IdleMonitor.KEEPALIVE` to 1 second. Without the changes from this PR this resulted in at least a few failures in every mach5 run. With this PR I was able to build tier1-5 with no failures. This pull request has now been integrated. Changeset: ecc603ca Author: Daniel Jeli?ski URL: https://git.openjdk.org/jdk/commit/ecc603ca9b441cbb7ad27fbc2529fcb0b1da1992 Stats: 32 lines in 1 file changed: 12 ins; 8 del; 12 mod 8201183: sjavac build failures: "Connection attempt failed: Connection refused" Reviewed-by: erikj, ihse ------------- PR: https://git.openjdk.org/jdk/pull/18712 From ihse at openjdk.org Thu Apr 11 12:48:01 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 12:48:01 GMT Subject: RFR: 8326947: Minimize MakeBase.gmk [v6] In-Reply-To: References: Message-ID: > This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. > > This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Fix dependency problem from inlining BaseUtils ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18041/files - new: https://git.openjdk.org/jdk/pull/18041/files/bba3672d..8c25c87c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=04-05 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18041.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18041/head:pull/18041 PR: https://git.openjdk.org/jdk/pull/18041 From ihse at openjdk.org Thu Apr 11 13:01:01 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 13:01:01 GMT Subject: RFR: 8326947: Minimize MakeBase.gmk [v7] In-Reply-To: References: Message-ID: <06tkBJ4LEs7dJHdjbbAJUoYRJYP5xIrjlsSmc2nKImY=.5d0646bf-97e6-4194-9203-4f6b27da46b4@github.com> > This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. > > This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Fix test-make ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18041/files - new: https://git.openjdk.org/jdk/pull/18041/files/8c25c87c..b6059e1a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18041&range=05-06 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18041.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18041/head:pull/18041 PR: https://git.openjdk.org/jdk/pull/18041 From erikj at openjdk.org Thu Apr 11 13:02:43 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 11 Apr 2024 13:02:43 GMT Subject: RFR: 8326947: Minimize MakeBase.gmk [v6] In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 12:48:01 GMT, Magnus Ihse Bursie wrote: >> This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. >> >> This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix dependency problem from inlining BaseUtils Looks good now. ------------- Marked as reviewed by erikj (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18041#pullrequestreview-1994127692 From ihse at openjdk.org Thu Apr 11 13:59:15 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 13:59:15 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files Message-ID: The file to build most of the java.desktop native libraries, t Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). ------------- Commit messages: - 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files Changes: https://git.openjdk.org/jdk/pull/18743/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18743&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330107 Stats: 1707 lines in 4 files changed: 865 ins; 841 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18743.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18743/head:pull/18743 PR: https://git.openjdk.org/jdk/pull/18743 From ihse at openjdk.org Thu Apr 11 13:59:15 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 13:59:15 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 13:53:23 GMT, Magnus Ihse Bursie wrote: > The file to build most of the java.desktop native libraries, t Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. > > I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). The default github review screen makes this a bit difficult to understand, especially AwtLibraries which it sees as a diff from the old Awt2d one. I recommend viewing the new file in its entirety instead: https://github.com/openjdk/jdk/blob/8180274092fa955cb7ff002bbdf3c32053dd337e/make/modules/java.desktop/lib/AwtLibraries.gmk Also to clarify: I have only moved the individual library compilations around, and made no other changes. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2049752669 From ihse at openjdk.org Thu Apr 11 14:01:41 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 14:01:41 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 13:53:23 GMT, Magnus Ihse Bursie wrote: > The file to build most of the java.desktop native libraries, t Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. > > I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). @prrace I will need your assistance in confirming that my understanding about the AWT vs 2D split is correct. In particular, `libosxui` gave me some headache, but after trying to dig into the code my understanding ended up being that this is part of Swing which is considered part of AWT, not 2D. I also looked at `libosx` and `libosxapp` which are located in the general `Lib.gmk` file. I could not easily see that they should have been placed in either 2d or Awt, so my assumption is that they are correctly placed outside these two new files. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2049760109 From ihse at openjdk.org Thu Apr 11 14:18:46 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 11 Apr 2024 14:18:46 GMT Subject: Integrated: 8326947: Minimize MakeBase.gmk In-Reply-To: References: Message-ID: <0Fh8-ZpyfJSMZYhFT29cCpWnvibks2x9JHGkOARTI-4=.cb58ac25-9f12-4e77-932a-2988c8ae97c0@github.com> On Wed, 28 Feb 2024 11:24:06 GMT, Magnus Ihse Bursie wrote: > This is part of a general "spring cleaning" of the build system, addressing old code that has bit-rotted, been subject to lava flow, or just had bad or smelly code that we've never gotten around to fix. > > This particular patch tries to make MakeBase truly minimal; only including the core parts of the build system that all makefiles will need. This is now limited to essential functionality for named parameter functions, variable dependency, tool execution, logging and fixpath functionality. MakeBase still includes Utils.gmk and FileUtils.gmk, and thus "provides" this functionality as well. Separating these out as well will be the subject of a future patch. This pull request has now been integrated. Changeset: 16061874 Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/16061874ffdd1b018fe1cad7e6d8ba8bdbdbbee1 Stats: 981 lines in 43 files changed: 499 ins; 404 del; 78 mod 8326947: Minimize MakeBase.gmk Reviewed-by: erikj ------------- PR: https://git.openjdk.org/jdk/pull/18041 From clanger at openjdk.org Thu Apr 11 16:47:41 2024 From: clanger at openjdk.org (Christoph Langer) Date: Thu, 11 Apr 2024 16:47:41 GMT Subject: RFR: 8329970: Update autoconf build-aux files with latest from 2024-01-01 In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 19:43:27 GMT, Erik Joelsson wrote: > Since we are now able to update the autoconf build-aux files, I think it's prudent to semi regularly do just that. I'm not aware of any particular platform that has been improved that would affect OpenJDK and I don't think we can further trim down our wrappers this time. This is more about staying with the latest. As of the filing of this bug, that is 2024-01-01, found here: > > https://git.savannah.gnu.org/cgit/config.git/ > > I have verified this with the platforms we build for at Oracle. I would encourage other OpenJDK distributors to verify on any platforms not covered by us. I will leave this open for a few days. SAP testing didn't show problems. ------------- Marked as reviewed by clanger (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18704#pullrequestreview-1994700270 From ascarpino at openjdk.org Thu Apr 11 17:17:44 2024 From: ascarpino at openjdk.org (Anthony Scarpino) Date: Thu, 11 Apr 2024 17:17:44 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 18:02:38 GMT, Volodymyr Paprotski wrote: > > In `ECOperations.java`, if I understand this correctly, it is to replace the existing `PointMultiplier` with montgomery-based PointMuliplier. But when I look at the code, I see both are still options. If I read this correctly, it checks for the old `IntegerFieldModuloP`, then looks for the new `IntegerMontgomeryFieldModuloP`. It appears to use the new one always. Why doesn't it just replace the old implementation entry in the `fields` Map? Is there a reason to keep it around? > > Hmm, thats a good point I haven't fully considered; i.e. (if I read correctly) "for `CurveDB.P_256` remove the fallback path to non-montgomery entirely".. that might also help in cleaning a few things up in the construction. Maybe even get rid of this nested ECOperations inside ECOperations.. Perhaps nesting isnt a big deal, but all attempts to make the ECC stack clearer is positive! > > One functional reason that might justify keeping it as-is, is fuzz-testing; with the fallback available, I am able to write the included Fuzz tests and have them check the values against the existing implementation. While I also included a few KAT tests using openssl-generated values, the fuzz tests check millions of values and it does add a lot more certainty about correctness of this code. I hadn't looked at your fuzz test until you mentioned it. I see you are using reflection to change the values. Is that what you mean by "fallback"? I'm assuming there is no to access the older implementation without reflection. > > Can it be removed? For the operations that do not involve multiplication (i.e. `setSum(*)`), montgomery is expensive. I think I did go through the uses of this code some time back (i.e. ECDHE, ECDSA and KeyGeneration) and existing IntegerPolynomialP256 is no longer used (I should verify that again) and only P256OrderField remains non-montgomery. So removing references to IntegerPolynomialP256 in ECOperations should be possible and cleaner. Removing IntegerPolynomialP256 from MontgomeryIntegerPolynomialP256 is harder (fromMontgomery() uses IntegerPolynomialP256) but perhaps also worth some thought.. > > I tend to like `ECOperationsFuzzTest.java` and would prefer to keep it, but it could also be chucked up as part of 'scaffolding' and removed in name of code quality? I wouldn't rip out the old implementation. I have been wondering if we should make the older implementation available, maybe by security property. I was looking at the static Maps at the top of `ECOperations`, `forParameters`, and the constructors where it checks if the `montgomeryOps` was null or set. It would be nice if we could have one set of `fields` Maps by putting the montgomery entry into the `fields` to replace it. I think that should work because `IntegerMontgomeryFieldModuloP` extends `IntegerFieldModuloP`. `instanceof` or other `montgomeryOps` checks would still need to exist because not all the `fields` support mongomery, and the older implementation would still be accessible for your fuzz tester. At least that is my theory. > > Thanks @ascarpino > > PS: Perhaps there is some middle ground, remove the `ECOperations montgomeryOps` nesting, and construct (somehow?? singleton makes most things inaccessible..) the reference ECOperations in the fuzz test instead.. not sure how yet, but perhaps worth a further thought.. It would be nice to remove the nesting and it would be nice to be a singleton. Maybe some combination of what I mentioned above chance can help that. I haven't fully thought this out either. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18583#issuecomment-2050148475 From jjg at openjdk.org Thu Apr 11 17:42:57 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 11 Apr 2024 17:42:57 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v55] In-Reply-To: <1uPEw9BTg1J92Zcw5bKKXcvc7Ula0WFLjFz0moN50U4=.752eb9c2-0a19-422d-adbb-79b71505be7f@github.com> References: <48hbLsx6MHdQQNNWrPRs4C-7sB18MyPOLE0QeTTPSDg=.1c0bd28d-8bd0-4733-a257-2c0f60bef2eb@github.com> <1uPEw9BTg1J92Zcw5bKKXcvc7Ula0WFLjFz0moN50U4=.752eb9c2-0a19-422d-adbb-79b71505be7f@github.com> Message-ID: On Tue, 9 Apr 2024 08:53:10 GMT, Pavel Rappo wrote: >> Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: >> >> add support for JDK-8329296: Update Elements for '///' documentation comments > > src/jdk.compiler/share/classes/com/sun/tools/javac/model/JavacElements.java line 443: > >> 441: @DefinedBy(Api.LANGUAGE_MODEL) >> 442: public CommentKind getDocCommentKind(Element e) { >> 443: return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); > > Nit: > Suggestion: > > return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); Fixed > src/jdk.compiler/share/classes/com/sun/tools/javac/model/JavacElements.java line 443: > >> 441: @DefinedBy(Api.LANGUAGE_MODEL) >> 442: public CommentKind getDocCommentKind(Element e) { >> 443: return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); > > Again: > Suggestion: > > return getDocCommentItem(e, ((docCommentTable, tree) -> docCommentTable.getCommentKind(tree))); Fixed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1561382358 PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1561382835 From jjg at openjdk.org Thu Apr 11 17:52:14 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 11 Apr 2024 17:52:14 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v56] In-Reply-To: References: Message-ID: > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: address review feedback for updates to Elements and friends ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16388/files - new: https://git.openjdk.org/jdk/pull/16388/files/21f5b004..d4c2c73b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=55 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=54-55 Stats: 9 lines in 2 files changed: 0 ins; 0 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388 From jjg at openjdk.org Thu Apr 11 20:55:24 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 11 Apr 2024 20:55:24 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v2] In-Reply-To: References: Message-ID: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: adjust call for `saveDanglingDocComments` for enum members ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18527/files - new: https://git.openjdk.org/jdk/pull/18527/files/3d6f1f95..56d6dcac Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=00-01 Stats: 5 lines in 1 file changed: 3 ins; 2 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From magnus.ihse.bursie at oracle.com Fri Apr 12 11:52:00 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 12 Apr 2024 13:52:00 +0200 Subject: Questions about the Hermetic Java project In-Reply-To: References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> Message-ID: On 2024-04-02 21:16, Jiangli Zhou wrote: > Hi?Magnus, > > In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) > discussed how to move forward contributing the static Java related > changes (additional runtime fixes/enhancements?on top of the existing > static support in JDK) from > https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK > mainline. > > Just a bit more details/context below, which may be useful for others > reading this thread. > > The https://github.com/openjdk/leyden/tree/hermetic-java-runtime > branch currently contains following for supporting hermetic Java > (without the launcher work for runtime support): > > 1. Build change for linking the Java launcher (as bin/javastatic) with > JDK/hotspot static libraries (.a), mainly in > https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. > The part for creating the complete sets of static libraries (including > libjvm.a) has already been included in the mainline since last year. > https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk > is in a very raw state and is intended to demonstrate the capability > of building a static Java launcher. Indeed. It is nowhere near being able to be integrated. > > 2. Additional runtime fixes/enhancements?on top of the existing static > support in JDK, e.g. support further lookup dynamic native library if > the built-in native library cannot be found. > > 3. Some initial (prototype) work on supporting hermetic JDK resource > files in the jimage (JDK modules image). > > To move forward, one of the earliest items needed is to add the > capability of building the fully statically linked Java launcher in > JDK mainline. The other static Java runtime changes can be followed up > after the launcher linking part, so they can be built and tested as > individual PRs created for the JDK mainline. Magnus, you have > expressed interest in helping get the launcher linking part (refactor > from > https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) > into JDK mainline. What's your thought on prioritizing the launcher > static linking part before other makefile clean ups for static libraries? Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). This, in turn, require several changes: 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. /Magnus > > Thanks! > Jiangli > > On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou > wrote: > > On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou > wrote: > > > > Hi Magnus, > > > > Thanks for looking into this from the build perspective. > > > > On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie > > wrote: > > > > > > First some background for build-dev: I have spent some time > looking at > > > the build implications of the Hermetic Java effort, which is > part of > > > Project Leyden. A high-level overview is available here: > > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the > current source > > > code is here: > https://github.com/openjdk/leyden/tree/hermetic-java-runtime. > > > > Some additional hermetic Java related references that are also > useful: > > > > - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug > that > > links to the issues for resolving static linking issues so far > > - https://github.com/openjdk/jdk21/pull/26 is the enhancement for > > building the complete set of static libraries in JDK/VM, > particularly > > including libjvm.a > > > > > > > > Hermetic Java faces several challenges, but the part that is > relevant > > > for the build system is the ability to create static > libraries. We've > > > had this functionality (in three different ways...) for some > time, but > > > it is rather badly implemented. > > > > > > As a result of my investigations, I have a bunch of questions. > :-) I > > > have gotten some answers in private discussion, but for the > sake of > > > transparency I will repeat them here, to foster an open dialogue. > > > > > > 1. Am I correct in understanding that the ultimate goal of > this exercise > > > is to be able to have jmods which include static libraries > (*.a) of the > > > native code which the module uses, and that the user can then > run a > > > special jlink command to have this linked into a single executable > > > binary (which also bundles the *.class files and any additional > > > resources needed)? > > > > > > 2. If so, is the idea to create special kinds of static jmods, > like > > > java.base-static.jmod, that contains *.a files instead of > lib*.so files? > > > Or is the idea that the normal jmod should contain both? > > > > > > 3. Linking .o and .a files into an executable is a formidable > task. Is > > > the intention to have jlink call a system-provided ld, or to > bundle ld > > > with jlink, or to reimplement this functionality in Java? > > > > I have a similar view as Alan responded in your other email thread. > > Things are still in the early stage for the general solution. > > > > In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime > > branch, when configuring JDK with --with-static-java=yes, the JDK > > binary contains the following extra artifacts: > > > > - static-libs/*.a: The complete set of JDK/VM static libraries > > - jdk/bin/javastatic: A demo Java launcher fully statically linked > > with the selected JDK .a libraries (e.g. it currently statically > link > > with the headless) and libjvm.a. It's the standard Java launcher > > without additional work for hermetic Java. > > > > In our prototype for hermetic Java, we build the hermetic executable > > image (a single image) from the following input (see description on > > singlejar packaging tool in > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf): > > > > - A customized launcher (with additional work for hermetic) > executable > > fully statically linked with JDK/VM static libraries (.a files), > > application natives and dependencies (e.g. in .a static libraries) > > - JDK lib/modules, JDK resource files > > - Application classes and resource files > > > > Including a JDK library .a into the corresponding .jmod would > require > > extracting the .a for linking with the executable. In some systems > > that may cause memory overhead due to the extracted copy of the .a > > files. I think we should consider the memory overhead issue. > > > > One possibility (as Alan described in his response) is for jlink to > > invoke the ld on the build system. jlink could pass the needed JDK > > static libraries and libjvm.a (provided as part of the JDK > binary) to > > ld based on the modules required for the application. > > > > I gave a bit more thoughts on this one. For jlink to trigger ld, it > would need to know the complete linker options and inputs. Those > include options and inputs related to the application part as well. In > some usages, it might be easier to handle native linking separately > and pass the linker output, the executable to jlink directly. Maybe we > could consider supporting different modes for various usages > requirements, from static libraries and native linking point of view: > > Mode #1 > Support .jmod packaged natives static libraries, for both JDK/VM .a > and application natives and dependencies. If the inputs to jlink > include .jmods, jlink can extract the .a libraries and pass the > information to ld to link the executable. > > Mode #2 > Support separate .a as jlink input. Jlink could pass the path > information to the .a libraries and other linker options to ld to > create the executable. > > For both mode #1 and #2, jlink would then use the linker output > executable to create the final hermetic image. > > Mode #3 > Support a fully linked executable as a jlink input. When a linked > executable is given to jlink, it can process it directly with other > JDK data/files to create the final image, without native linking step. > > Any other thoughts and considerations? > > Best, > Jiangli > > > > > > > 4. Is the intention is to allow users to create their own > jmods with > > > static libraries, and have these linked in as well? This seems > to be the > > > case. > > > > An alternative with less memory overhead could be using application > > modular JAR and separate .a as the input for jlink. > > > > > If that is so, then there will always be the risk for name > > > collisions, and we can only minimize the risk by making sure > any global > > > names are as unique as possible. > > > > Part of the current effort includes resolving the discovered symbol > > collision issues with static linking. Will respond to your other > email > > on the symbol issue separately later. > > > > > > > > 5. The original implementation of static builds in the JDK, > created for > > > the Mobile project, used a configure flag, > --enable-static-builds, to > > > change the entire behavior of the build system to only produce > *.a files > > > instead of lib*.so. In contrast, the current system is using a > special > > > target instead. > > > > I think we would need both configure flag and special target for the > > static builds. > > > > > In my eyes, this is a much worse solution. Apart from > > > the conceptual principle (if the build should generate static > or dynamic > > > libraries is definitely a property of what a "configuration" > means), > > > this makes it much harder to implement efficiently, since we > cannot make > > > changes in NativeCompilation.gmk, where they are needed. > > > > For the potential objcopy work to resolve symbol issues, we can add > > that conditionally in NativeCompilation.gmk if STATIC_LIBS is > true. We > > have an internal prototype (not included in > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime > yet) done > > by one of colleagues for localizing symbols in libfreetype using > > objcopy. > > > > > > > > That was not as much a question as a statement. ? But here is the > > > question: Do you think it would be reasonable to restore the old > > > behavior but with the new methods, so that we don't use > special targets, > > > but instead tells configure to generate static libraries? I'm > thinking > > > we should have a flag like "--with-library-type=" that can > have values > > > "dynamic" (which is default), "static" or "both". > > > > If we want to also build a fully statically linked launcher, maybe > > --with-static-java? Being able to configure either dynamic, > static or > > both as you suggested also seems to be a good idea. > > > > > I am not sure if "both" are needed, but if we want to bundle > both lib*.so and *.a files > > > into a single jmod file (see question 2 above), then it > definitely is. > > > In general, the cost of producing two kinds of libraries are quite > > > small, compared to the cost of compiling the source code to > object files. > > > > Completely agree. It would be good to avoid recompiling the .o file > > for static and dynamic builds. As proposed in > > https://bugs.openjdk.org/browse/JDK-8303796: > > > > It's beneficial to be able to build both .so and .a from the > same set > > of .o files. That would involve some changes to handle the > dynamic JDK > > and static JDK difference at runtime, instead of relying on the > > STATIC_BUILD macro. > > > > > > > > Finally, I have looked at how to manipulate symbol visibility. > There > > > seems many ways forward, so I feel confident that we can find > a good > > > solution. > > > > > > One way forward is to use objcopy to manipulate symbol status > > > (global/local). There is an option --localize-symbol in > objcopy, that > > > has been available in objcopy since at least 2.15, which was > released > > > 2004, so it should be safe to use. But ideally we should avoid > using > > > objcopy and do this as part of the linking process. This should be > > > possible to do, given that we make changes in > NativeCompilation.gmk -- > > > see question 5 above. > > > > > > As a fallback, it is also possible to rename symbols, either > piecewise > > > or wholesale, using objcopy. There are many ways to do this, using > > > --prefix-symbols, --redefine-sym or --redefine-syms (note the > -s, this > > > takes a file with a list of symbols). Thus we can always > introduce a > > > "post factum namespace" by renaming symbols. > > > > Renaming or redefining the symbol at build time could cause > confusions > > with debugging. That's a concern raised in > > https://github.com/openjdk/jdk/pull/17456 discussions. > > > > Additionally, redefining symbols using tools like objcopy may not > > handle member names referenced in string literals. For example, in > > https://github.com/openjdk/jdk/pull/17456 additional changes are > > needed in assembling and SA to reflect the symbol change. > > > > > > > > So in the end, I think it will be fully possible to produce .a > files > > > that only has global symbols for the functions that are part > of the API > > > exposed by that library, and have all other symbols local, and > make this > > > is in a way that is consistent with the rest of the build system. > > > > > > Finally, a note on Hotspot. Due to debugging reasons, we export > > > basically all symbols in hotspot as global. This is not > reasonable to do > > > for a static build. The effect of not exporting those symbols > will be > > > that SA will not function to 100%. On the other hand, I have > no idea if > > > SA works at all with a static build. Have you tested this? Is > this part > > > of the plan to support, or will it be officially dropped for > Hermetic Java? > > > > We have done some testing with jtreg SA related tests for the fully > > statically linked `javastatic`. > > > > If we use objcopy to localize symbols in hotspot, it's not yet clear > > what's the impact on SA. We could do some tests. The other question > > that I raised is the supported gcc versions (for partial linking) > > related to the solution. > > > > Best, > > Jiangli > > > > > > > > /Magnus > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ihse at openjdk.org Fri Apr 12 12:30:49 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 12 Apr 2024 12:30:49 GMT Subject: RFR: 8330110: AIX build fails after JDK-8329704 - issue with libjli.a Message-ID: Unfortunately, after [JDK-8329704](https://bugs.openjdk.org/browse/JDK-8329704) AIX fails to build. The reason is that libjli is specially treated on AIX, and built like a static library. I tried to compensate for this (and had tested on an earlier draft), but some late-minute changes in JDK-8329704 made this check fail. At this point, I will not add any more checks for `$1_$2_STATIC_LIBRARY` in `ResolveLibPath`. From reading the code, it become apparent that building with `STATIC_BUILD` (n.b. -- not `STATIC_LIBS`!) will likely fail. This has on the other hand not been tested for a long time, and is likely bit-rotten anyway. Also, I will very soon address all of these shortcomings in the upcoming rewrite for proper static builds. ------------- Commit messages: - 8330110: AIX build fails after JDK-8329704 - issue with libjli.a Changes: https://git.openjdk.org/jdk/pull/18759/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18759&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330110 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18759.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18759/head:pull/18759 PR: https://git.openjdk.org/jdk/pull/18759 From mbaesken at openjdk.org Fri Apr 12 14:07:43 2024 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 12 Apr 2024 14:07:43 GMT Subject: RFR: 8330110: AIX build fails after JDK-8329704 - issue with libjli.a In-Reply-To: References: Message-ID: On Fri, 12 Apr 2024 12:26:06 GMT, Magnus Ihse Bursie wrote: > Unfortunately, after [JDK-8329704](https://bugs.openjdk.org/browse/JDK-8329704) AIX fails to build. The reason is that libjli is specially treated on AIX, and built like a static library. I tried to compensate for this (and had tested on an earlier draft), but some late-minute changes in JDK-8329704 made this check fail. > > At this point, I will not add any more checks for `$1_$2_STATIC_LIBRARY` in `ResolveLibPath`. From reading the code, it become apparent that building with `STATIC_BUILD` (n.b. -- not `STATIC_LIBS`!) will likely fail. This has on the other hand not been tested for a long time, and is likely bit-rotten anyway. Also, I will very soon address all of these shortcomings in the upcoming rewrite for proper static builds. This resolves the AIX build problem. ------------- Marked as reviewed by mbaesken (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18759#pullrequestreview-1997232221 From mdoerr at openjdk.org Fri Apr 12 14:11:42 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 12 Apr 2024 14:11:42 GMT Subject: RFR: 8330110: AIX build fails after JDK-8329704 - issue with libjli.a In-Reply-To: References: Message-ID: On Fri, 12 Apr 2024 12:26:06 GMT, Magnus Ihse Bursie wrote: > Unfortunately, after [JDK-8329704](https://bugs.openjdk.org/browse/JDK-8329704) AIX fails to build. The reason is that libjli is specially treated on AIX, and built like a static library. I tried to compensate for this (and had tested on an earlier draft), but some late-minute changes in JDK-8329704 made this check fail. > > At this point, I will not add any more checks for `$1_$2_STATIC_LIBRARY` in `ResolveLibPath`. From reading the code, it become apparent that building with `STATIC_BUILD` (n.b. -- not `STATIC_LIBS`!) will likely fail. This has on the other hand not been tested for a long time, and is likely bit-rotten anyway. Also, I will very soon address all of these shortcomings in the upcoming rewrite for proper static builds. Thanks for fixing it so quickly! ------------- Marked as reviewed by mdoerr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18759#pullrequestreview-1997252234 From jjg at openjdk.org Fri Apr 12 17:24:43 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Fri, 12 Apr 2024 17:24:43 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v2] In-Reply-To: References: Message-ID: On Wed, 3 Apr 2024 10:01:37 GMT, Magnus Ihse Bursie wrote: >> Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: >> >> adjust call for `saveDanglingDocComments` for enum members > > The build changes look okay. > > Do you have any plan of going through all the Java modules and fixing the issues, or opening JBS issues to have them fixed? Or will these lint warnings remain disabled for the foreseeable future? @magicus > The build changes look okay. > > Do you have any plan of going through all the Java modules and fixing the issues, or opening JBS issues to have them fixed? Or will these lint warnings remain disabled for the foreseeable future? The plan is to create an umbrella bug to clean up the individual modules. There is interest to clean up `java.base`, to keep that one free of any warnings, and I can see that the lang tools modules will get cleaner up as well. It will be up to other component teams to decide if and when to clean up other parts of the system. Once this work has been integrated, it is relatively easy to enable the warnings for a module and to fix the ensuing issues. Since any changes "only" involve comments, it should be reasonably easy to fix them, unlike some pervasive other warnings, like `this-escape`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18527#issuecomment-2052174696 From darcy at openjdk.org Fri Apr 12 17:41:56 2024 From: darcy at openjdk.org (Joe Darcy) Date: Fri, 12 Apr 2024 17:41:56 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v56] In-Reply-To: References: Message-ID: <5BLnTYx2hjFyOvw3_RQWLDcPwvREkTII1AaLhdQUB7I=.84043d72-429a-4b9a-a119-57f45d239950@github.com> On Thu, 11 Apr 2024 17:52:14 GMT, Jonathan Gibbons wrote: >> Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. >> >> Notable features: >> >> * support for `///` documentation comments in `JavaTokenizer` >> * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library >> * updates to `DocCommentParser` to treat `///` comments as Markdown >> * updates to the standard doclet to render Markdown comments in HTML > > Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: > > address review feedback for updates to Elements and friends There should be some quick testing of the new default method on Elements using the VacuousElements implementation; see `test/langtools/tools/javac/processing/model/util/elements` for some examples. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16388#issuecomment-2052196439 From ihse at openjdk.org Fri Apr 12 21:00:46 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Fri, 12 Apr 2024 21:00:46 GMT Subject: Integrated: 8330110: AIX build fails after JDK-8329704 - issue with libjli.a In-Reply-To: References: Message-ID: On Fri, 12 Apr 2024 12:26:06 GMT, Magnus Ihse Bursie wrote: > Unfortunately, after [JDK-8329704](https://bugs.openjdk.org/browse/JDK-8329704) AIX fails to build. The reason is that libjli is specially treated on AIX, and built like a static library. I tried to compensate for this (and had tested on an earlier draft), but some late-minute changes in JDK-8329704 made this check fail. > > At this point, I will not add any more checks for `$1_$2_STATIC_LIBRARY` in `ResolveLibPath`. From reading the code, it become apparent that building with `STATIC_BUILD` (n.b. -- not `STATIC_LIBS`!) will likely fail. This has on the other hand not been tested for a long time, and is likely bit-rotten anyway. Also, I will very soon address all of these shortcomings in the upcoming rewrite for proper static builds. This pull request has now been integrated. Changeset: 68f86dcc Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/68f86dccce601ec10111dc3e535d28ce9fc80928 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod 8330110: AIX build fails after JDK-8329704 - issue with libjli.a Reviewed-by: mbaesken, mdoerr ------------- PR: https://git.openjdk.org/jdk/pull/18759 From jjg at openjdk.org Fri Apr 12 21:04:06 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Fri, 12 Apr 2024 21:04:06 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v3] In-Reply-To: References: Message-ID: <3Ynu_D2CcFh3usjCkWQVk7VFhTlxVzD4f2nVhvZrP50=.f1c29740-ff73-4a0c-a63b-3e00ece05bbf@github.com> > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: call `saveDanglingDocComments` for local variable declarations ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18527/files - new: https://git.openjdk.org/jdk/pull/18527/files/56d6dcac..3f745431 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=01-02 Stats: 10 lines in 1 file changed: 5 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From prr at openjdk.org Fri Apr 12 22:47:51 2024 From: prr at openjdk.org (Phil Race) Date: Fri, 12 Apr 2024 22:47:51 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 13:59:30 GMT, Magnus Ihse Bursie wrote: > @prrace I will need your assistance in confirming that my understanding about the AWT vs 2D split is correct. In particular, `libosxui` gave me some headache, but after trying to dig into the code my understanding ended up being that this is part of Swing which is considered part of AWT, not 2D. > > I also looked at `libosx` and `libosxapp` which are located in the general `Lib.gmk` file. I could not easily see that they should have been placed in either 2d or Awt, so my assumption is that they are correctly placed outside these two new files. Swing isn't part of AWT. They are separate entities. Swing is built on top of 2D as much as it is built on top of AWT. Any hope of finding a neat dividing line will be arbitrary and not really reflecting reality And in some ways AWT is more closely tied to AWT than 2D is. And things are not the same across platforms either. You've determined splashscreen is 2D functionality which it most certainly is not. Awtlibraries is setting up the shaders which for the Metal rendering pipeline which is 100% 2D. And it seems to be building OpenGL, etc as well .. and generally I see so manay references to 2D in the AWT file and vice versa. it isn't even true that "libawt" is awt. Libawt is mostly a misnomer, in one case more than mostly. It ought to be called "lib2d" So I think this change will just cause confusion and doesn't seem right to me. It ought to "not happen". ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2052648551 From jwaters at openjdk.org Mon Apr 15 04:22:53 2024 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 15 Apr 2024 04:22:53 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v8] In-Reply-To: References: Message-ID: <-iew2xjrFUfYcZ_q3d0nF1hrxDwBcZ-p1Mgz-B-Ijk0=.c045f6c4-336a-4e52-b23e-cb917d719f2b@github.com> On Wed, 10 Apr 2024 21:10:27 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix missing lib prefix make/common/JdkNativeCompilation.gmk line 212: > 210: > 211: ifneq ($(STATIC_LIBS), true) > 212: ifeq ($$(call isTargetOs, windows), true) I should've looked through this more carefully. The selection on whether to use -L and -libpath: should've been based on the compiler, not the target OS ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18649#discussion_r1565162835 From jwaters at openjdk.org Mon Apr 15 04:26:47 2024 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 15 Apr 2024 04:26:47 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v8] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 21:10:27 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix missing lib prefix Quick Question: On Windows, does it set LIBPATH to the path of the .lib import library? If so, are the import libraries in the same directories as the compiled .dll files? Reason is that gcc does not depend on .lib import libraries and directly links to the dll file, and if the path given is to the non-existent .lib file when compiling with gcc, linking will fail ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2054974650 From ihse at openjdk.org Mon Apr 15 07:45:47 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 07:45:47 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v8] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 21:10:27 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix missing lib prefix In general, I have not been considering anything but the microsoft toolchain on Windows when designing this (sorry about that!), but on the other hand, I *have* had in mind that it should be easy to change if we ever get to support some other toolchain on Windows. One of the upcoming changes I am planning is to improve handling of import libraries. I can try to make it more "microsoft"-ish rather than "windows"-ish, but then again, there are a lot of Windows checks already, and if I can hook into them I probably will. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2055933013 From ihse at openjdk.org Mon Apr 15 07:56:44 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 07:56:44 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 13:53:23 GMT, Magnus Ihse Bursie wrote: > The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. > > I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). But the current solution is no good, either. For instance, the ordering is strange. awt_xawt and awt_lwawt is placed in opposite ends of the file, even though they are in some sense just platform specific variants of the same thing. And the entire file is huge and hard to navigate. > and generally I see so manay references to 2D in the AWT file and vice versa. > it isn't even true that "libawt" is awt. Libawt is mostly a misnomer, in one case more than mostly. It ought to be called "lib2d" If this is the case, then it is clearly hopeless to try and split up the native code based on which area the source code used to compile it came from. So let's go back to another approach: My starting point for this split was to take everything with "awt" in the name, and move it to a separate file. (I naively assumed that libs named "awt" belonged to AWT.) After reviewing the other libraries, my conclusion was to move yet another library there. But let's get that away from there. So what if we keep the libraries named "awt" in a separate file, and all other in another file -- mostly this separation, but we agree that the reason for separation is not due to source code provenance, but just the name of the resulting libraries, and it does not need to signify anything but a need to be able to navigate in complex make files. And let's rename the "2dLibraries.gmk" file to `ClientLibraries.gmk` instead, so it is clear that there is no pretending that this is supposed to represent a 2D/AWT split. Would that sound okay to you? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2055999383 From jwaters at openjdk.org Mon Apr 15 08:16:46 2024 From: jwaters at openjdk.org (Julian Waters) Date: Mon, 15 Apr 2024 08:16:46 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v8] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 21:10:27 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix missing lib prefix No worries, I can clean it up on my end, since it's not that big of a change. I just needed to know whether the LIBPATH pointed to the import library or library itself when linking (Also another question, AddJdkLibrary _is_ what adds the library path and library itself to the LDFLAGS and LIBS respectively, right?) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2056094573 From jkern at openjdk.org Mon Apr 15 08:49:53 2024 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 15 Apr 2024 08:49:53 GMT Subject: Integrated: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc In-Reply-To: References: Message-ID: On Thu, 28 Mar 2024 16:50:20 GMT, Joachim Kern wrote: > As of [JDK-8325880](https://bugs.openjdk.org/browse/JDK-8325880), building the JDK requires version 17 of IBM Open XL C/C++ (xlc). This is in effect clang by another name, and it uses the clang toolchain in the JDK build. Thus the old xlc toolchain was removed by [JDK-8327701](https://bugs.openjdk.org/browse/JDK-8327701). > Now we also switch the HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc, removing the last xlc rudiment. > This means merging the AIX specific content of utilities/globalDefinitions_xlc.hpp and utilities/compilerWarnings_xlc.hpp into the corresponding gcc files on the on side and removing the defined(TARGET_COMPILER_xlc) blocks in the code, because the defined(TARGET_COMPILER_gcc) blocks work out of the box for the new AIX compiler. > The rest of the changes are needed because of using utilities/compilerWarnings_gcc.hpp the compiler is much more nagging about ill formatted printf This pull request has now been integrated. Changeset: 3f1d9c44 Author: Joachim Kern Committer: Martin Doerr URL: https://git.openjdk.org/jdk/commit/3f1d9c441ea98910d9483e133bccfac784db393d Stats: 256 lines in 15 files changed: 8 ins; 212 del; 36 mod 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc Reviewed-by: jwaters, stuefe, kbarrett, mdoerr ------------- PR: https://git.openjdk.org/jdk/pull/18536 From ihse at openjdk.org Mon Apr 15 11:37:40 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 11:37:40 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 13:53:23 GMT, Magnus Ihse Bursie wrote: > The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. > > I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). To rephrase myself: The goal here is to bring a huge Makefile into manageable chunks. I thought this could be done by separating AWT and 2D, but if that is not possible then any kind of predictable split is fine. I could just as well order libraries alphabetically by name and split the file half way down. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2056613043 From ihse at openjdk.org Mon Apr 15 11:48:50 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 11:48:50 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 22:14:33 GMT, Kim Barrett wrote: >> That build failure in shared code does not happen with Xcode clang, gcc, or >> Visual Studio, even though none of them appear to have a relevant define or >> include. So the clang variant being used for AIX is different from the Xcode >> clang variant (and maybe others) in its treatment of alloca. Weird! >> >> I can also live with either the macro or the includes where needed. I dislike >> conditionally adding the include in globalDefinitions_gcc.hpp. > > Should also remove the `#pragma alloca` in os_aix.cpp. It was too bad that I did not see and review this change in the makefiles. :-( While you guys could have gone either way, I strongly dislike the choice to include a redefinition in the makefiles. If this really should be done, we should introduced a new variable to carry such changes, instead of piggybacking it with the OS defines. :-( But, I don't think it should be done at all. There are several reasons why this is a inferior solution: 1) It does not follow prior examples. We have tried hard before not do things like this, but rather pass flags as defines (e.g. `-DREDEFINE_ALLOCA` had been better) 2) It does not scale. If we start in effect allowing code in the command line, there is no clear limit anymore what should be placed in the source code files and what should be placed on the command line. 3) It messes up command lines. Keeping command lines as short as reasonable possible is a goal we try to strive for. In this case, there is also the `'` inside them (which I don't understand why), which is just begging for quoting/escaping problems, making command lines hard to copy/paste, send to different systems (like logging) etc. I'd really like to see a follow-up PR that moves this away from the command line define and into a source code file instead. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1565646578 From ihse at openjdk.org Mon Apr 15 11:52:49 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 11:52:49 GMT Subject: RFR: 8329704: Implement framework for proper handling of JDK_LIBS [v8] In-Reply-To: References: Message-ID: On Wed, 10 Apr 2024 21:10:27 GMT, Magnus Ihse Bursie wrote: >> This is the pinnacle of the recent stream of refactorings in the build system. This patch introduces a more abstract concept of "JDK_LIBS", where only the library name (e.g. "java" or "java.desktop:jawt") is specified, and the build system turns this into suitable linker flags: `-ljawt -L` or `jawt.lib -libpath:`, depending on linker. It will also automatically create proper dependencies. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix missing lib prefix > (Also another question, AddJdkLibrary is what adds the library path and library itself to the LDFLAGS and LIBS respectively, right?) Yes. I was hoping that would be rather clearly documented... Any way, my plan is to make a clear separate directory `import-libs` or something like that, so we can clearly separate import libraries from "proper" static libraries, which is not really done right now. I think and hope that after I'm done, everything will be much clearer, so whatever changes you need to do are obvious and simple. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18649#issuecomment-2056641541 From ihse at openjdk.org Mon Apr 15 12:01:49 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 12:01:49 GMT Subject: RFR: 8325163: Enable -Wpedantic on clang [v2] In-Reply-To: References: Message-ID: On Mon, 5 Feb 2024 10:58:17 GMT, Magnus Ihse Bursie wrote: >> Inspired by (the later backed-out) [JDK-8296115](https://bugs.openjdk.org/browse/JDK-8296115), I propose to enable `-Wpedantic` for clang. This has already found some irregularities in the code, like mistakenly using `#import` instead of `#include`. In this patch, I disable warnings for these individual buggy or badly written files, but I intend to post follow-up issues on the respective teams to have them properly fixed. >> >> Unfortunately, it is not possible to enable `-Wpedantic` on gcc, since individual warnings in `-Wpedantic` cannot be disabled. This means that code like this: >> >> >> #define DEBUG_ONLY(code) code; >> >> DEBUG_ONLY(foo()); >> >> >> will result in a `; ;`. This breaks the C standard, but is benign, and we use it all over the place. On clang, we can ignore this by `-Wno-extra-semi`, but this is not available on gcc. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > FIx dtrace build I'll close this now. I guess if we want to make progress on this, we will probably have to evaluate every individual warning separately, and add those which does make sense (e.g. `import-preprocessor-directive-pedantic`), and ignore the rest. I personally will not have time nor interest in driving this forward, at least not right now. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17687#issuecomment-2056659089 From ihse at openjdk.org Mon Apr 15 12:01:49 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 12:01:49 GMT Subject: Withdrawn: 8325163: Enable -Wpedantic on clang In-Reply-To: References: Message-ID: On Fri, 2 Feb 2024 15:22:03 GMT, Magnus Ihse Bursie wrote: > Inspired by (the later backed-out) [JDK-8296115](https://bugs.openjdk.org/browse/JDK-8296115), I propose to enable `-Wpedantic` for clang. This has already found some irregularities in the code, like mistakenly using `#import` instead of `#include`. In this patch, I disable warnings for these individual buggy or badly written files, but I intend to post follow-up issues on the respective teams to have them properly fixed. > > Unfortunately, it is not possible to enable `-Wpedantic` on gcc, since individual warnings in `-Wpedantic` cannot be disabled. This means that code like this: > > > #define DEBUG_ONLY(code) code; > > DEBUG_ONLY(foo()); > > > will result in a `; ;`. This breaks the C standard, but is benign, and we use it all over the place. On clang, we can ignore this by `-Wno-extra-semi`, but this is not available on gcc. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/17687 From ihse at openjdk.org Mon Apr 15 12:39:49 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 12:39:49 GMT Subject: RFR: 8330261: Clean up linking steps Message-ID: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. This PR also contains some additional code cleanup/fixes. ------------- Commit messages: - Also add NAME check - Wrap all non-trivial executions in ExecuteWithLog - Inline calls to tools in link recipe Changes: https://git.openjdk.org/jdk/pull/18783/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330261 Stats: 112 lines in 4 files changed: 48 ins; 35 del; 29 mod Patch: https://git.openjdk.org/jdk/pull/18783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18783/head:pull/18783 PR: https://git.openjdk.org/jdk/pull/18783 From erikj at openjdk.org Mon Apr 15 12:56:41 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Apr 2024 12:56:41 GMT Subject: RFR: 8330261: Clean up linking steps In-Reply-To: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: On Mon, 15 Apr 2024 12:34:53 GMT, Magnus Ihse Bursie wrote: > Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. > > This PR also contains some additional code cleanup/fixes. The indentation for the first line after ExecuteWithLog is done with 2 spaces everywhere in this patch, but as that's a macro call it should be 4, right? Looks like we usually do 4 in other places. ------------- PR Review: https://git.openjdk.org/jdk/pull/18783#pullrequestreview-2001028930 From mdoerr at openjdk.org Mon Apr 15 13:12:50 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 15 Apr 2024 13:12:50 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Mon, 15 Apr 2024 11:46:25 GMT, Magnus Ihse Bursie wrote: >> Should also remove the `#pragma alloca` in os_aix.cpp. > > It was too bad that I did not see and review this change in the makefiles. :-( > > While you guys could have gone either way, I strongly dislike the choice to include a redefinition in the makefiles. If this really should be done, we should introduced a new variable to carry such changes, instead of piggybacking it with the OS defines. :-( But, I don't think it should be done at all. > > There are several reasons why this is a inferior solution: > > 1) It does not follow prior examples. We have tried hard before not do things like this, but rather pass flags as defines (e.g. `-DREDEFINE_ALLOCA` had been better) > 2) It does not scale. If we start in effect allowing code in the command line, there is no clear limit anymore what should be placed in the source code files and what should be placed on the command line. > 3) It messes up command lines. Keeping command lines as short as reasonable possible is a goal we try to strive for. In this case, there is also the `'` inside them (which I don't understand why), which is just begging for quoting/escaping problems, making command lines hard to copy/paste, send to different systems (like logging) etc. > > I'd really like to see a follow-up PR that moves this away from the command line define and into a source code file instead. Can we unconditionally `#include ` in all files which use `alloca`? Or does that disturb any platform? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1565770190 From ihse at openjdk.org Mon Apr 15 13:39:08 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 13:39:08 GMT Subject: RFR: 8330261: Clean up linking steps [v2] In-Reply-To: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: > Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. > > This PR also contains some additional code cleanup/fixes. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Restore missing ) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18783/files - new: https://git.openjdk.org/jdk/pull/18783/files/abb44ff0..8cefc3b4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18783/head:pull/18783 PR: https://git.openjdk.org/jdk/pull/18783 From ihse at openjdk.org Mon Apr 15 13:43:53 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 13:43:53 GMT Subject: RFR: 8330261: Clean up linking steps [v3] In-Reply-To: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: > Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. > > This PR also contains some additional code cleanup/fixes. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Fix ExecuteWithLog indentation ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18783/files - new: https://git.openjdk.org/jdk/pull/18783/files/8cefc3b4..65765e49 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=01-02 Stats: 16 lines in 2 files changed: 0 ins; 1 del; 15 mod Patch: https://git.openjdk.org/jdk/pull/18783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18783/head:pull/18783 PR: https://git.openjdk.org/jdk/pull/18783 From ihse at openjdk.org Mon Apr 15 13:46:40 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 13:46:40 GMT Subject: RFR: 8330261: Clean up linking steps [v3] In-Reply-To: References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: On Mon, 15 Apr 2024 12:54:22 GMT, Erik Joelsson wrote: > The indentation for the first line after ExecuteWithLog is done with 2 spaces everywhere in this patch, but as that's a macro call it should be 4, right? Looks like we usually do 4 in other places. Good catch! I checked some other places with ExecuteWithLog. We do consistently use 4 spaces (except two places for static linking; I fixed those) for the first line. However, we have not really been consistent in how we have treated multi-line commands. Usually we break them with 4 spaces additional indentation, but when they are seen as individual commands we have not always done so. Maybe we should do a pass over the code base at some point to make sure we are consistent. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18783#issuecomment-2056899777 From erikj at openjdk.org Mon Apr 15 14:05:41 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 15 Apr 2024 14:05:41 GMT Subject: RFR: 8330261: Clean up linking steps [v3] In-Reply-To: References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: On Mon, 15 Apr 2024 13:43:53 GMT, Magnus Ihse Bursie wrote: >> Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. >> >> This PR also contains some additional code cleanup/fixes. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix ExecuteWithLog indentation Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18783#pullrequestreview-2001212672 From ihse at openjdk.org Mon Apr 15 14:21:08 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 15 Apr 2024 14:21:08 GMT Subject: RFR: 8330261: Clean up linking steps [v4] In-Reply-To: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: > Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. > > This PR also contains some additional code cleanup/fixes. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: MACOSX_CODESIGN_MODE was not always properly set ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18783/files - new: https://git.openjdk.org/jdk/pull/18783/files/65765e49..df3f64a2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=02-03 Stats: 7 lines in 1 file changed: 2 ins; 3 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/18783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18783/head:pull/18783 PR: https://git.openjdk.org/jdk/pull/18783 From darcy at openjdk.org Mon Apr 15 19:07:31 2024 From: darcy at openjdk.org (Joe Darcy) Date: Mon, 15 Apr 2024 19:07:31 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 Message-ID: Get JDK 24 underway. ------------- Commit messages: - JDK-8330182: Start of release updates for JDK 24 Changes: https://git.openjdk.org/jdk/pull/18787/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18787&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330182 Stats: 101 lines in 37 files changed: 43 ins; 3 del; 55 mod Patch: https://git.openjdk.org/jdk/pull/18787.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18787/head:pull/18787 PR: https://git.openjdk.org/jdk/pull/18787 From darcy at openjdk.org Mon Apr 15 19:09:59 2024 From: darcy at openjdk.org (Joe Darcy) Date: Mon, 15 Apr 2024 19:09:59 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 In-Reply-To: References: Message-ID: On Mon, 15 Apr 2024 19:01:08 GMT, Joe Darcy wrote: > Get JDK 24 underway. This initial version of the PR intentionally excludes the creation of the new symbol files so that the fundamental code aspects of the update are easier to see. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18787#issuecomment-2057615983 From iris at openjdk.org Mon Apr 15 20:00:43 2024 From: iris at openjdk.org (Iris Clark) Date: Mon, 15 Apr 2024 20:00:43 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 In-Reply-To: References: Message-ID: <7kb9r7YhLGd33spkclLkTdBCaN0d6nZ7h1z52Hhv4rs=.bffedb51-7144-4706-a4a8-34460959534a@github.com> On Mon, 15 Apr 2024 19:01:08 GMT, Joe Darcy wrote: > Get JDK 24 underway. The copyright year was not updated in all files *14.java. I assume that's intentional. I've also Reviewed the associated CSRs. make/conf/version-numbers.conf line 36: > 34: DEFAULT_VERSION_EXTRA2=0 > 35: DEFAULT_VERSION_EXTRA3=0 > 36: DEFAULT_VERSION_DATE=2024-03-17 "2024-03-17"? I thought that the expected date was "2025-03-18"? ------------- Marked as reviewed by iris (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18787#pullrequestreview-2002002355 PR Review Comment: https://git.openjdk.org/jdk/pull/18787#discussion_r1566336599 From darcy at openjdk.org Mon Apr 15 20:23:41 2024 From: darcy at openjdk.org (Joe Darcy) Date: Mon, 15 Apr 2024 20:23:41 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 In-Reply-To: <7kb9r7YhLGd33spkclLkTdBCaN0d6nZ7h1z52Hhv4rs=.bffedb51-7144-4706-a4a8-34460959534a@github.com> References: <7kb9r7YhLGd33spkclLkTdBCaN0d6nZ7h1z52Hhv4rs=.bffedb51-7144-4706-a4a8-34460959534a@github.com> Message-ID: On Mon, 15 Apr 2024 19:57:49 GMT, Iris Clark wrote: > The copyright year was not updated in all files *14.java. I assume that's intentional. I'll run my copyright update script before pushing. > I've also Reviewed the associated CSRs. Thanks. > make/conf/version-numbers.conf line 36: > >> 34: DEFAULT_VERSION_EXTRA2=0 >> 35: DEFAULT_VERSION_EXTRA3=0 >> 36: DEFAULT_VERSION_DATE=2024-03-17 > > "2024-03-17"? I thought that the expected date was "2025-03-18"? Thanks for catching that; yes, I'll correct and double-check the expected GA date in the next push. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18787#issuecomment-2057735741 PR Review Comment: https://git.openjdk.org/jdk/pull/18787#discussion_r1566383373 From duke at openjdk.org Mon Apr 15 22:12:30 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Mon, 15 Apr 2024 22:12:30 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v3] In-Reply-To: References: Message-ID: <-64Xlhk6ln43-xTmlv_cvloS-gzDrKMyiPUdPbMNlIM=.2b524654-ca5b-4a7a-a7da-316e99cfea35@github.com> > Performance. Before: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s > > Performance, no intrinsic: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply true thrpt 3 1919.574 ? 10.591 ops/s > > Performance, **with intrinsics*... Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: Comments from Jatin and Tony ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18583/files - new: https://git.openjdk.org/jdk/pull/18583/files/82b6dae7..6f9ac046 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=01-02 Stats: 97 lines in 20 files changed: 4 ins; 36 del; 57 mod Patch: https://git.openjdk.org/jdk/pull/18583.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18583/head:pull/18583 PR: https://git.openjdk.org/jdk/pull/18583 From duke at openjdk.org Mon Apr 15 22:12:30 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Mon, 15 Apr 2024 22:12:30 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: <48md2WEAhqPyuVf4AYOxBQDykUiOaEL0PQb-ki0_TYM=.6c25bf41-b0ae-49ec-b606-236deb4561e3@github.com> References: <48md2WEAhqPyuVf4AYOxBQDykUiOaEL0PQb-ki0_TYM=.6c25bf41-b0ae-49ec-b606-236deb4561e3@github.com> Message-ID: On Wed, 10 Apr 2024 23:56:52 GMT, Volodymyr Paprotski wrote: > Few early comments. > > Please update the copyright year of all the modified files. > > You can even consider splitting this into two patches, Java side changes in one and x86 optimized intrinsic in next one. Fixed all copyright years git diff da8a095a19c90e7ee2b45fab9b533a1092887023 | lsdiff -p1 | while read line; do echo $line =========================; grep Copyright $line | grep -v 2024; done | less Re splitting.. probably too late for that now.. (did consider it initially.. got hard to manage two changes while developing. And easier to justify the change when the entire patch is presented? but yes, far more code to review.. ) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18583#issuecomment-2057892691 From duke at openjdk.org Mon Apr 15 22:12:30 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Mon, 15 Apr 2024 22:12:30 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 17:15:21 GMT, Anthony Scarpino wrote: >>> In `ECOperations.java`, if I understand this correctly, it is to replace the existing `PointMultiplier` with montgomery-based PointMuliplier. But when I look at the code, I see both are still options. If I read this correctly, it checks for the old `IntegerFieldModuloP`, then looks for the new `IntegerMontgomeryFieldModuloP`. It appears to use the new one always. Why doesn't it just replace the old implementation entry in the `fields` Map? Is there a reason to keep it around? >> >> Hmm, thats a good point I haven't fully considered; i.e. (if I read correctly) "for `CurveDB.P_256` remove the fallback path to non-montgomery entirely".. that might also help in cleaning a few things up in the construction. Maybe even get rid of this nested ECOperations inside ECOperations.. Perhaps nesting isnt a big deal, but all attempts to make the ECC stack clearer is positive! >> >> One functional reason that might justify keeping it as-is, is fuzz-testing; with the fallback available, I am able to write the included Fuzz tests and have them check the values against the existing implementation. While I also included a few KAT tests using openssl-generated values, the fuzz tests check millions of values and it does add a lot more certainty about correctness of this code. >> >> Can it be removed? For the operations that do not involve multiplication (i.e. `setSum(*)`), montgomery is expensive. I think I did go through the uses of this code some time back (i.e. ECDHE, ECDSA and KeyGeneration) and existing IntegerPolynomialP256 is no longer used (I should verify that again) and only P256OrderField remains non-montgomery. So removing references to IntegerPolynomialP256 in ECOperations should be possible and cleaner. Removing IntegerPolynomialP256 from MontgomeryIntegerPolynomialP256 is harder (fromMontgomery() uses IntegerPolynomialP256) but perhaps also worth some thought.. >> >> I tend to like `ECOperationsFuzzTest.java` and would prefer to keep it, but it could also be chucked up as part of 'scaffolding' and removed in name of code quality? >> >> Thanks @ascarpino >> >> PS: Perhaps there is some middle ground, remove the `ECOperations montgomeryOps` nesting, and construct (somehow?? singleton makes most things inaccessible..) the reference ECOperations in the fuzz test instead.. not sure how yet, but perhaps worth a further thought.. > >> > In `ECOperations.java`, if I understand this correctly, it is to replace the existing `PointMultiplier` with montgomery-based PointMuliplier. But when I look at the code, I see both are still options. If I read this correctly, it checks for the old `IntegerFieldModuloP`, then looks for the new `IntegerMontgomeryFieldModuloP`. It appears to use the new one always. Why doesn't it just replace the old implementation entry in the `fields` Map? Is there a reason to keep it around? >> >> Hmm, thats a good point I haven't fully considered; i.e. (if I read correctly) "for `CurveDB.P_256` remove the fallback path to non-montgomery entirely".. that might also help in cleaning a few things up in the construction. Maybe even get rid of this nested ECOperations inside ECOperations.. Perhaps nesting isnt a big deal, but all attempts to make the ECC stack clearer is positive! >> >> One functional reason that might justify keeping it as-is, is fuzz-testing; with the fallback available, I am able to write the included Fuzz tests and have them check the values against the existing implementation. While I also included a few KAT tests using openssl-generated values, the fuzz tests check millions of values and it does add a lot more certainty about correctness of this code. > > I hadn't looked at your fuzz test until you mentioned it. I see you are using reflection to change the values. Is that what you mean by "fallback"? I'm assuming there is no to access the older implementation without reflection. > >> >> Can it be removed? For the operations that do not involve multiplication (i.e. `setSum(*)`), montgomery is expensive. I think I did go through the uses of this code some time back (i.e. ECDHE, ECDSA and KeyGeneration) and existing IntegerPolynomialP256 is no longer used (I should verify that again) and only P256OrderField remains non-montgomery. So removing references to IntegerPolynomialP256 in ECOperations should be possible and cleaner. Removing IntegerPolynomialP256 from MontgomeryIntegerPolynomialP256 is harder (fromMontgomery() uses IntegerPolynomialP256) but perhaps also worth some thought.. >> >> I tend to like `ECOperationsFuzzTest.java` and would prefer to keep it, but it could also be chucked up as part of 'scaffolding' and removed in name of code quality? > > I wouldn't rip out the old implementation. I have been wondering if we should make the older implementation available, maybe by security property. I was looking at the static Maps at the top of `ECOperatio... @ascarpino Fixed as suggested... actually.. that was _waaay_ easier then I thought it would be (I saw singleton and assumed private constructor.. nope, ECOperations() is public, no reflection required!! Ended up with cleaner implementation _and_ cleaner tests! Thanks!) ------------- PR Comment: https://git.openjdk.org/jdk/pull/18583#issuecomment-2057895950 From duke at openjdk.org Mon Apr 15 22:12:31 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Mon, 15 Apr 2024 22:12:31 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Fri, 5 Apr 2024 07:19:28 GMT, Jatin Bhateja wrote: >> Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: >> >> remove use of jdk.crypto.ec > > src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp line 39: > >> 37: }; >> 38: static address modulus_p256() { >> 39: return (address)MODULUS_P256; > > Long constants should have UL suffix. Properly ULL, but good point, fixed > src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp line 386: > >> 384: __ jcc(Assembler::equal, L_Length19); >> 385: >> 386: // Default copy loop > > Please add appropriate loop entry alignment. This is actually a 'switch statement default'. The default should never happen (See "Known Length comment on line 335"), but added because java code has that behavior. (i.e. in the unlikely case NIST adds a new elliptic curve to the existing standard?) > src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp line 394: > >> 392: __ lea(aLimbs, Address(aLimbs,8)); >> 393: __ lea(bLimbs, Address(bLimbs,8)); >> 394: __ jmp(L_DefaultLoop); > > Both sub and cmp are flag affecting instructions and are macro-fusible. > By doing a loop rotation i.e. moving the length <= 0 check outside the loop and pushing the loop exit check at bottom you can save additional compare checks. Per-above, this is a switch statement (`UNLIKELY`) fallback. I can still add alignment and loop rotation, but being a fallback figured its more important to keep it small&readable... ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1566486768 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1566486717 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1566486848 From jianglizhou at google.com Tue Apr 16 01:01:53 2024 From: jianglizhou at google.com (Jiangli Zhou) Date: Mon, 15 Apr 2024 18:01:53 -0700 Subject: Questions about the Hermetic Java project In-Reply-To: References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> Message-ID: Magnus, thanks for the response. Please see comments inlined below. On Fri, Apr 12, 2024 at 4:52?AM Magnus Ihse Bursie wrote: > > On 2024-04-02 21:16, Jiangli Zhou wrote: > > Hi Magnus, > > In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) discussed how to move forward contributing the static Java related changes (additional runtime fixes/enhancements on top of the existing static support in JDK) from https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK mainline. > > Just a bit more details/context below, which may be useful for others reading this thread. > > The https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch currently contains following for supporting hermetic Java (without the launcher work for runtime support): > > 1. Build change for linking the Java launcher (as bin/javastatic) with JDK/hotspot static libraries (.a), mainly in https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. The part for creating the complete sets of static libraries (including libjvm.a) has already been included in the mainline since last year. https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk is in a very raw state and is intended to demonstrate the capability of building a static Java launcher. > > Indeed. It is nowhere near being able to be integrated. > The main purpose of StaticLink.gmk is to support the static-java-image make target, which can be used to perform the actual static linking step using libjvm.a and JDK static libraries. That currently doesn't exist in the JDK mainline. Creating a "fully" statically linked Java launcher is the first step (out of many) towards supporting static/hermetic Java. As part of cleaning/refactoring/integrating for the static linking step, we want to agree and decide/accept on the following: - Support the "fully" statically linked java launcher for testing and demoing the capability of static JDK support, e.g. - Support running jtreg testing using the "fully" statically linked Java launcher - Set up tests in github workflow to help detect any breaking changes for static support, e.g. new symbol issues introduced by any changes. There were some earlier discussions on this with Ron and Alan during the zoom meetings. - Which JDK native libraries to be statically linked with the new launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, libawt_xawt.a, etc from statically linked with the launcher. - Do we want more than one statically linked launcher target, based on the set of linked native libraries? Based on the decisions of the above, the launcher static linking part would mostly be in a different shape when it's integrated into the mainline. That's why I referred to StaticLink.gmk as in a "very raw" state. Here is a high-level view of the state of things for static support: (I) What we already have in the JDK mainline: - Able to build a complete set of JDK/VM static libraries using `static-libs-image` make target (necessary for supporting static JDK) - Compilation for .o files are done separately for the static libraries and dynamic library (ok for now) (II) What missing: - Static linking step as mentioned above (III) What needs to be improved (require cleanups and refactoring, and you mentioned some of those in your response as well): - Support building both the static libraries and dynamic libraries using the same set of .o files, instead of separately compiled .o files. That helps improve build speed and reduce memory overhead for building JDK. Your current refactoring work aims to help that. - Clean up the usages of STATIC_BUILD macro. Most of the usages are in test code. - Other runtime fixes/enhancements in the leyden https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch I think most work mentioned in III has dependencies on II. We need a workable base to be able to build the "fully" statically linked launcher for building and testing the work mentioned in III, when integrating any of those to the JDK mainline. The makefile refactoring work can be done in parallel but does not need to be completed before we add the static linking step in JDK mainline. > > 2. Additional runtime fixes/enhancements on top of the existing static support in JDK, e.g. support further lookup dynamic native library if the built-in native library cannot be found. > > 3. Some initial (prototype) work on supporting hermetic JDK resource files in the jimage (JDK modules image). > > To move forward, one of the earliest items needed is to add the capability of building the fully statically linked Java launcher in JDK mainline. The other static Java runtime changes can be followed up after the launcher linking part, so they can be built and tested as individual PRs created for the JDK mainline. Magnus, you have expressed interest in helping get the launcher linking part (refactor from https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) into JDK mainline. What's your thought on prioritizing the launcher static linking part before other makefile clean ups for static libraries? > > Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. Thanks! > > I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? Please see my comments above. > > The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. > > The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. > > My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). As of today, the leyden https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch can build a "fully" statically linked Java launcher. The issue of compiling the dynamic and static libraries .o files separately is not a blocker. It's good to have it resolved at some point of time. > > This, in turn, require several changes: > > 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > > 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > > This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. Thumbs up! That seems to be a good direction. Currently in the leyden branch, it first looks up the unique JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in libraries, then search for the dynamic libraries using the conventional naming when necessary. e.g.: https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 When spec supports JNI_OnLoad_ and etc. for dynamic libraries, we may still need to support the conventional naming without the <_lib_name> part for existing libraries out there. > > And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. > > A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. Thank you for taking this on! Potentially we could consider taking the objcopy to localizing hotspot symbols on unix-like platforms, based on https://github.com/openjdk/jdk/pull/17456 discussions. Additional testing is still needed to verify the solution. > > My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. Most of the JDK resources are now supported as hermetic jimage (lib/modules) bundled in the https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch. The remaining sound.properties, ct.sym and .jfc files can be handled later. Overally, that part of the work has confirmed the hermetic jimage bundled solution is robust and helps resolve some of the difficult start-up sequence issues observed when the hermetic resource was implemented using JAR file based solution. It might be a good idea to follow up on the static linking discussion in tomorrow's zoom meeting (hope you'll be able to join tomorrow). Thanks! Jiangli > > /Magnus > > > > Thanks! > Jiangli > > On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou wrote: >> >> On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou wrote: >> > >> > Hi Magnus, >> > >> > Thanks for looking into this from the build perspective. >> > >> > On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie >> > wrote: >> > > >> > > First some background for build-dev: I have spent some time looking at >> > > the build implications of the Hermetic Java effort, which is part of >> > > Project Leyden. A high-level overview is available here: >> > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current source >> > > code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. >> > >> > Some additional hermetic Java related references that are also useful: >> > >> > - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug that >> > links to the issues for resolving static linking issues so far >> > - https://github.com/openjdk/jdk21/pull/26 is the enhancement for >> > building the complete set of static libraries in JDK/VM, particularly >> > including libjvm.a >> > >> > > >> > > Hermetic Java faces several challenges, but the part that is relevant >> > > for the build system is the ability to create static libraries. We've >> > > had this functionality (in three different ways...) for some time, but >> > > it is rather badly implemented. >> > > >> > > As a result of my investigations, I have a bunch of questions. :-) I >> > > have gotten some answers in private discussion, but for the sake of >> > > transparency I will repeat them here, to foster an open dialogue. >> > > >> > > 1. Am I correct in understanding that the ultimate goal of this exercise >> > > is to be able to have jmods which include static libraries (*.a) of the >> > > native code which the module uses, and that the user can then run a >> > > special jlink command to have this linked into a single executable >> > > binary (which also bundles the *.class files and any additional >> > > resources needed)? >> > > >> > > 2. If so, is the idea to create special kinds of static jmods, like >> > > java.base-static.jmod, that contains *.a files instead of lib*.so files? >> > > Or is the idea that the normal jmod should contain both? >> > > >> > > 3. Linking .o and .a files into an executable is a formidable task. Is >> > > the intention to have jlink call a system-provided ld, or to bundle ld >> > > with jlink, or to reimplement this functionality in Java? >> > >> > I have a similar view as Alan responded in your other email thread. >> > Things are still in the early stage for the general solution. >> > >> > In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime >> > branch, when configuring JDK with --with-static-java=yes, the JDK >> > binary contains the following extra artifacts: >> > >> > - static-libs/*.a: The complete set of JDK/VM static libraries >> > - jdk/bin/javastatic: A demo Java launcher fully statically linked >> > with the selected JDK .a libraries (e.g. it currently statically link >> > with the headless) and libjvm.a. It's the standard Java launcher >> > without additional work for hermetic Java. >> > >> > In our prototype for hermetic Java, we build the hermetic executable >> > image (a single image) from the following input (see description on >> > singlejar packaging tool in >> > https://cr.openjdk.org/~jiangli/hermetic_java.pdf): >> > >> > - A customized launcher (with additional work for hermetic) executable >> > fully statically linked with JDK/VM static libraries (.a files), >> > application natives and dependencies (e.g. in .a static libraries) >> > - JDK lib/modules, JDK resource files >> > - Application classes and resource files >> > >> > Including a JDK library .a into the corresponding .jmod would require >> > extracting the .a for linking with the executable. In some systems >> > that may cause memory overhead due to the extracted copy of the .a >> > files. I think we should consider the memory overhead issue. >> > >> > One possibility (as Alan described in his response) is for jlink to >> > invoke the ld on the build system. jlink could pass the needed JDK >> > static libraries and libjvm.a (provided as part of the JDK binary) to >> > ld based on the modules required for the application. >> > >> >> I gave a bit more thoughts on this one. For jlink to trigger ld, it >> would need to know the complete linker options and inputs. Those >> include options and inputs related to the application part as well. In >> some usages, it might be easier to handle native linking separately >> and pass the linker output, the executable to jlink directly. Maybe we >> could consider supporting different modes for various usages >> requirements, from static libraries and native linking point of view: >> >> Mode #1 >> Support .jmod packaged natives static libraries, for both JDK/VM .a >> and application natives and dependencies. If the inputs to jlink >> include .jmods, jlink can extract the .a libraries and pass the >> information to ld to link the executable. >> >> Mode #2 >> Support separate .a as jlink input. Jlink could pass the path >> information to the .a libraries and other linker options to ld to >> create the executable. >> >> For both mode #1 and #2, jlink would then use the linker output >> executable to create the final hermetic image. >> >> Mode #3 >> Support a fully linked executable as a jlink input. When a linked >> executable is given to jlink, it can process it directly with other >> JDK data/files to create the final image, without native linking step. >> >> Any other thoughts and considerations? >> >> Best, >> Jiangli >> >> > > >> > > 4. Is the intention is to allow users to create their own jmods with >> > > static libraries, and have these linked in as well? This seems to be the >> > > case. >> > >> > An alternative with less memory overhead could be using application >> > modular JAR and separate .a as the input for jlink. >> > >> > > If that is so, then there will always be the risk for name >> > > collisions, and we can only minimize the risk by making sure any global >> > > names are as unique as possible. >> > >> > Part of the current effort includes resolving the discovered symbol >> > collision issues with static linking. Will respond to your other email >> > on the symbol issue separately later. >> > >> > > >> > > 5. The original implementation of static builds in the JDK, created for >> > > the Mobile project, used a configure flag, --enable-static-builds, to >> > > change the entire behavior of the build system to only produce *.a files >> > > instead of lib*.so. In contrast, the current system is using a special >> > > target instead. >> > >> > I think we would need both configure flag and special target for the >> > static builds. >> > >> > > In my eyes, this is a much worse solution. Apart from >> > > the conceptual principle (if the build should generate static or dynamic >> > > libraries is definitely a property of what a "configuration" means), >> > > this makes it much harder to implement efficiently, since we cannot make >> > > changes in NativeCompilation.gmk, where they are needed. >> > >> > For the potential objcopy work to resolve symbol issues, we can add >> > that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. We >> > have an internal prototype (not included in >> > https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) done >> > by one of colleagues for localizing symbols in libfreetype using >> > objcopy. >> > >> > > >> > > That was not as much a question as a statement. ? But here is the >> > > question: Do you think it would be reasonable to restore the old >> > > behavior but with the new methods, so that we don't use special targets, >> > > but instead tells configure to generate static libraries? I'm thinking >> > > we should have a flag like "--with-library-type=" that can have values >> > > "dynamic" (which is default), "static" or "both". >> > >> > If we want to also build a fully statically linked launcher, maybe >> > --with-static-java? Being able to configure either dynamic, static or >> > both as you suggested also seems to be a good idea. >> > >> > > I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files >> > > into a single jmod file (see question 2 above), then it definitely is. >> > > In general, the cost of producing two kinds of libraries are quite >> > > small, compared to the cost of compiling the source code to object files. >> > >> > Completely agree. It would be good to avoid recompiling the .o file >> > for static and dynamic builds. As proposed in >> > https://bugs.openjdk.org/browse/JDK-8303796: >> > >> > It's beneficial to be able to build both .so and .a from the same set >> > of .o files. That would involve some changes to handle the dynamic JDK >> > and static JDK difference at runtime, instead of relying on the >> > STATIC_BUILD macro. >> > >> > > >> > > Finally, I have looked at how to manipulate symbol visibility. There >> > > seems many ways forward, so I feel confident that we can find a good >> > > solution. >> > > >> > > One way forward is to use objcopy to manipulate symbol status >> > > (global/local). There is an option --localize-symbol in objcopy, that >> > > has been available in objcopy since at least 2.15, which was released >> > > 2004, so it should be safe to use. But ideally we should avoid using >> > > objcopy and do this as part of the linking process. This should be >> > > possible to do, given that we make changes in NativeCompilation.gmk -- >> > > see question 5 above. >> > > >> > > As a fallback, it is also possible to rename symbols, either piecewise >> > > or wholesale, using objcopy. There are many ways to do this, using >> > > --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this >> > > takes a file with a list of symbols). Thus we can always introduce a >> > > "post factum namespace" by renaming symbols. >> > >> > Renaming or redefining the symbol at build time could cause confusions >> > with debugging. That's a concern raised in >> > https://github.com/openjdk/jdk/pull/17456 discussions. >> > >> > Additionally, redefining symbols using tools like objcopy may not >> > handle member names referenced in string literals. For example, in >> > https://github.com/openjdk/jdk/pull/17456 additional changes are >> > needed in assembling and SA to reflect the symbol change. >> > >> > > >> > > So in the end, I think it will be fully possible to produce .a files >> > > that only has global symbols for the functions that are part of the API >> > > exposed by that library, and have all other symbols local, and make this >> > > is in a way that is consistent with the rest of the build system. >> > > >> > > Finally, a note on Hotspot. Due to debugging reasons, we export >> > > basically all symbols in hotspot as global. This is not reasonable to do >> > > for a static build. The effect of not exporting those symbols will be >> > > that SA will not function to 100%. On the other hand, I have no idea if >> > > SA works at all with a static build. Have you tested this? Is this part >> > > of the plan to support, or will it be officially dropped for Hermetic Java? >> > >> > We have done some testing with jtreg SA related tests for the fully >> > statically linked `javastatic`. >> > >> > If we use objcopy to localize symbols in hotspot, it's not yet clear >> > what's the impact on SA. We could do some tests. The other question >> > that I raised is the supported gcc versions (for partial linking) >> > related to the solution. >> > >> > Best, >> > Jiangli >> > >> > > >> > > /Magnus >> > > From jbhateja at openjdk.org Tue Apr 16 02:31:02 2024 From: jbhateja at openjdk.org (Jatin Bhateja) Date: Tue, 16 Apr 2024 02:31:02 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Mon, 15 Apr 2024 22:04:14 GMT, Volodymyr Paprotski wrote: >> src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp line 394: >> >>> 392: __ lea(aLimbs, Address(aLimbs,8)); >>> 393: __ lea(bLimbs, Address(bLimbs,8)); >>> 394: __ jmp(L_DefaultLoop); >> >> Both sub and cmp are flag affecting instructions and are macro-fusible. >> By doing a loop rotation i.e. moving the length <= 0 check outside the loop and pushing the loop exit check at bottom you can save additional compare checks. > > Per-above, this is a switch statement (`UNLIKELY`) fallback. I can still add alignment and loop rotation, but being a fallback figured its more important to keep it small&readable... It's all part of intrinsic, no harm in polishing it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1566630667 From volker.simonis at gmail.com Tue Apr 16 08:28:48 2024 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 16 Apr 2024 10:28:48 +0200 Subject: Questions about the Hermetic Java project In-Reply-To: References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> Message-ID: Hi, and sorry, I'm completely new to this discussion. I just wonder how `static-libs-image` is related to the `graal-builder-image` target which is available in mainline since JDK 11 and builds a static version of all the native libraries excluding the Hotspot ones? @Magnus: will the new, static build you're working on also cover `graal-builder-image` or will it be independent of it? From a quick look it seems that currently both `static-libs-image` and `graal-builder-image` are handled in `make/StaticLibsImage.gmk` but I'm not sure if they use the same build functionality. I just wondered why `graal-builder-image` hasn't been mentioned in this thread but maybe it's something as obvious as that it already is the same functionality anway? Thank you and best regards, Volker On Tue, Apr 16, 2024 at 3:02?AM Jiangli Zhou wrote: > > Magnus, thanks for the response. Please see comments inlined below. > > On Fri, Apr 12, 2024 at 4:52?AM Magnus Ihse Bursie > wrote: > > > > On 2024-04-02 21:16, Jiangli Zhou wrote: > > > > Hi Magnus, > > > > In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) discussed how to move forward contributing the static Java related changes (additional runtime fixes/enhancements on top of the existing static support in JDK) from https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK mainline. > > > > Just a bit more details/context below, which may be useful for others reading this thread. > > > > The https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch currently contains following for supporting hermetic Java (without the launcher work for runtime support): > > > > 1. Build change for linking the Java launcher (as bin/javastatic) with JDK/hotspot static libraries (.a), mainly in https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. The part for creating the complete sets of static libraries (including libjvm.a) has already been included in the mainline since last year. https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk is in a very raw state and is intended to demonstrate the capability of building a static Java launcher. > > > > Indeed. It is nowhere near being able to be integrated. > > > > The main purpose of StaticLink.gmk is to support the static-java-image > make target, which can be used to perform the actual static linking > step using libjvm.a and JDK static libraries. That currently doesn't > exist in the JDK mainline. Creating a "fully" statically linked Java > launcher is the first step (out of many) towards supporting > static/hermetic Java. > > As part of cleaning/refactoring/integrating for the static linking > step, we want to agree and decide/accept on the following: > > - Support the "fully" statically linked java launcher for testing and > demoing the capability of static JDK support, e.g. > - Support running jtreg testing using the "fully" statically linked > Java launcher > - Set up tests in github workflow to help detect any breaking > changes for static support, e.g. new symbol issues introduced by any > changes. There were some earlier discussions on this with Ron and Alan > during the zoom meetings. > - Which JDK native libraries to be statically linked with the new > launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, > libawt_xawt.a, etc from statically linked with the launcher. > - Do we want more than one statically linked launcher target, based on > the set of linked native libraries? > > Based on the decisions of the above, the launcher static linking part > would mostly be in a different shape when it's integrated into the > mainline. That's why I referred to StaticLink.gmk as in a "very raw" > state. > > Here is a high-level view of the state of things for static support: > > (I) What we already have in the JDK mainline: > - Able to build a complete set of JDK/VM static libraries using > `static-libs-image` make target (necessary for supporting static JDK) > - Compilation for .o files are done separately for the static > libraries and dynamic library (ok for now) > > (II) What missing: > - Static linking step as mentioned above > > (III) What needs to be improved (require cleanups and refactoring, and > you mentioned some of those in your response as well): > - Support building both the static libraries and dynamic libraries > using the same set of .o files, instead of separately compiled .o > files. That helps improve build speed and reduce memory overhead for > building JDK. Your current refactoring work aims to help that. > - Clean up the usages of STATIC_BUILD macro. Most of the usages are in > test code. > - Other runtime fixes/enhancements in the leyden > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > > I think most work mentioned in III has dependencies on II. We need a > workable base to be able to build the "fully" statically linked > launcher for building and testing the work mentioned in III, when > integrating any of those to the JDK mainline. The makefile refactoring > work can be done in parallel but does not need to be completed before > we add the static linking step in JDK mainline. > > > > > 2. Additional runtime fixes/enhancements on top of the existing static support in JDK, e.g. support further lookup dynamic native library if the built-in native library cannot be found. > > > > 3. Some initial (prototype) work on supporting hermetic JDK resource files in the jimage (JDK modules image). > > > > To move forward, one of the earliest items needed is to add the capability of building the fully statically linked Java launcher in JDK mainline. The other static Java runtime changes can be followed up after the launcher linking part, so they can be built and tested as individual PRs created for the JDK mainline. Magnus, you have expressed interest in helping get the launcher linking part (refactor from https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) into JDK mainline. What's your thought on prioritizing the launcher static linking part before other makefile clean ups for static libraries? > > > > Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. > > Thanks! > > > > > I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? > > Please see my comments above. > > > > > The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. > > > > The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. > > > > My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). > > As of today, the leyden > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > can build a "fully" statically linked Java launcher. The issue of > compiling the dynamic and static libraries .o files separately is not > a blocker. It's good to have it resolved at some point of time. > > > > > This, in turn, require several changes: > > > > 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > > > > 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > > > > This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > > Thumbs up! That seems to be a good direction. Currently in the leyden > branch, it first looks up the unique > JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in > libraries, then search for the dynamic libraries using the > conventional naming when necessary. e.g.: > > https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a > https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 > > When spec supports JNI_OnLoad_ and etc. for dynamic > libraries, we may still need to support the conventional naming > without the <_lib_name> part for existing libraries out there. > > > > > And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. > > > > A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. > > Thank you for taking this on! Potentially we could consider taking the > objcopy to localizing hotspot symbols on unix-like platforms, based on > https://github.com/openjdk/jdk/pull/17456 discussions. Additional > testing is still needed to verify the solution. > > > > > My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. > > Most of the JDK resources are now supported as hermetic jimage > (lib/modules) bundled in the > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch. > The remaining sound.properties, ct.sym and .jfc files can be handled > later. Overally, that part of the work has confirmed the hermetic > jimage bundled solution is robust and helps resolve some of the > difficult start-up sequence issues observed when the hermetic resource > was implemented using JAR file based solution. > > It might be a good idea to follow up on the static linking discussion > in tomorrow's zoom meeting (hope you'll be able to join tomorrow). > > Thanks! > > Jiangli > > > > /Magnus > > > > > > > > Thanks! > > Jiangli > > > > On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou wrote: > >> > >> On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou wrote: > >> > > >> > Hi Magnus, > >> > > >> > Thanks for looking into this from the build perspective. > >> > > >> > On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie > >> > wrote: > >> > > > >> > > First some background for build-dev: I have spent some time looking at > >> > > the build implications of the Hermetic Java effort, which is part of > >> > > Project Leyden. A high-level overview is available here: > >> > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current source > >> > > code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. > >> > > >> > Some additional hermetic Java related references that are also useful: > >> > > >> > - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug that > >> > links to the issues for resolving static linking issues so far > >> > - https://github.com/openjdk/jdk21/pull/26 is the enhancement for > >> > building the complete set of static libraries in JDK/VM, particularly > >> > including libjvm.a > >> > > >> > > > >> > > Hermetic Java faces several challenges, but the part that is relevant > >> > > for the build system is the ability to create static libraries. We've > >> > > had this functionality (in three different ways...) for some time, but > >> > > it is rather badly implemented. > >> > > > >> > > As a result of my investigations, I have a bunch of questions. :-) I > >> > > have gotten some answers in private discussion, but for the sake of > >> > > transparency I will repeat them here, to foster an open dialogue. > >> > > > >> > > 1. Am I correct in understanding that the ultimate goal of this exercise > >> > > is to be able to have jmods which include static libraries (*.a) of the > >> > > native code which the module uses, and that the user can then run a > >> > > special jlink command to have this linked into a single executable > >> > > binary (which also bundles the *.class files and any additional > >> > > resources needed)? > >> > > > >> > > 2. If so, is the idea to create special kinds of static jmods, like > >> > > java.base-static.jmod, that contains *.a files instead of lib*.so files? > >> > > Or is the idea that the normal jmod should contain both? > >> > > > >> > > 3. Linking .o and .a files into an executable is a formidable task. Is > >> > > the intention to have jlink call a system-provided ld, or to bundle ld > >> > > with jlink, or to reimplement this functionality in Java? > >> > > >> > I have a similar view as Alan responded in your other email thread. > >> > Things are still in the early stage for the general solution. > >> > > >> > In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime > >> > branch, when configuring JDK with --with-static-java=yes, the JDK > >> > binary contains the following extra artifacts: > >> > > >> > - static-libs/*.a: The complete set of JDK/VM static libraries > >> > - jdk/bin/javastatic: A demo Java launcher fully statically linked > >> > with the selected JDK .a libraries (e.g. it currently statically link > >> > with the headless) and libjvm.a. It's the standard Java launcher > >> > without additional work for hermetic Java. > >> > > >> > In our prototype for hermetic Java, we build the hermetic executable > >> > image (a single image) from the following input (see description on > >> > singlejar packaging tool in > >> > https://cr.openjdk.org/~jiangli/hermetic_java.pdf): > >> > > >> > - A customized launcher (with additional work for hermetic) executable > >> > fully statically linked with JDK/VM static libraries (.a files), > >> > application natives and dependencies (e.g. in .a static libraries) > >> > - JDK lib/modules, JDK resource files > >> > - Application classes and resource files > >> > > >> > Including a JDK library .a into the corresponding .jmod would require > >> > extracting the .a for linking with the executable. In some systems > >> > that may cause memory overhead due to the extracted copy of the .a > >> > files. I think we should consider the memory overhead issue. > >> > > >> > One possibility (as Alan described in his response) is for jlink to > >> > invoke the ld on the build system. jlink could pass the needed JDK > >> > static libraries and libjvm.a (provided as part of the JDK binary) to > >> > ld based on the modules required for the application. > >> > > >> > >> I gave a bit more thoughts on this one. For jlink to trigger ld, it > >> would need to know the complete linker options and inputs. Those > >> include options and inputs related to the application part as well. In > >> some usages, it might be easier to handle native linking separately > >> and pass the linker output, the executable to jlink directly. Maybe we > >> could consider supporting different modes for various usages > >> requirements, from static libraries and native linking point of view: > >> > >> Mode #1 > >> Support .jmod packaged natives static libraries, for both JDK/VM .a > >> and application natives and dependencies. If the inputs to jlink > >> include .jmods, jlink can extract the .a libraries and pass the > >> information to ld to link the executable. > >> > >> Mode #2 > >> Support separate .a as jlink input. Jlink could pass the path > >> information to the .a libraries and other linker options to ld to > >> create the executable. > >> > >> For both mode #1 and #2, jlink would then use the linker output > >> executable to create the final hermetic image. > >> > >> Mode #3 > >> Support a fully linked executable as a jlink input. When a linked > >> executable is given to jlink, it can process it directly with other > >> JDK data/files to create the final image, without native linking step. > >> > >> Any other thoughts and considerations? > >> > >> Best, > >> Jiangli > >> > >> > > > >> > > 4. Is the intention is to allow users to create their own jmods with > >> > > static libraries, and have these linked in as well? This seems to be the > >> > > case. > >> > > >> > An alternative with less memory overhead could be using application > >> > modular JAR and separate .a as the input for jlink. > >> > > >> > > If that is so, then there will always be the risk for name > >> > > collisions, and we can only minimize the risk by making sure any global > >> > > names are as unique as possible. > >> > > >> > Part of the current effort includes resolving the discovered symbol > >> > collision issues with static linking. Will respond to your other email > >> > on the symbol issue separately later. > >> > > >> > > > >> > > 5. The original implementation of static builds in the JDK, created for > >> > > the Mobile project, used a configure flag, --enable-static-builds, to > >> > > change the entire behavior of the build system to only produce *.a files > >> > > instead of lib*.so. In contrast, the current system is using a special > >> > > target instead. > >> > > >> > I think we would need both configure flag and special target for the > >> > static builds. > >> > > >> > > In my eyes, this is a much worse solution. Apart from > >> > > the conceptual principle (if the build should generate static or dynamic > >> > > libraries is definitely a property of what a "configuration" means), > >> > > this makes it much harder to implement efficiently, since we cannot make > >> > > changes in NativeCompilation.gmk, where they are needed. > >> > > >> > For the potential objcopy work to resolve symbol issues, we can add > >> > that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. We > >> > have an internal prototype (not included in > >> > https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) done > >> > by one of colleagues for localizing symbols in libfreetype using > >> > objcopy. > >> > > >> > > > >> > > That was not as much a question as a statement. ? But here is the > >> > > question: Do you think it would be reasonable to restore the old > >> > > behavior but with the new methods, so that we don't use special targets, > >> > > but instead tells configure to generate static libraries? I'm thinking > >> > > we should have a flag like "--with-library-type=" that can have values > >> > > "dynamic" (which is default), "static" or "both". > >> > > >> > If we want to also build a fully statically linked launcher, maybe > >> > --with-static-java? Being able to configure either dynamic, static or > >> > both as you suggested also seems to be a good idea. > >> > > >> > > I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files > >> > > into a single jmod file (see question 2 above), then it definitely is. > >> > > In general, the cost of producing two kinds of libraries are quite > >> > > small, compared to the cost of compiling the source code to object files. > >> > > >> > Completely agree. It would be good to avoid recompiling the .o file > >> > for static and dynamic builds. As proposed in > >> > https://bugs.openjdk.org/browse/JDK-8303796: > >> > > >> > It's beneficial to be able to build both .so and .a from the same set > >> > of .o files. That would involve some changes to handle the dynamic JDK > >> > and static JDK difference at runtime, instead of relying on the > >> > STATIC_BUILD macro. > >> > > >> > > > >> > > Finally, I have looked at how to manipulate symbol visibility. There > >> > > seems many ways forward, so I feel confident that we can find a good > >> > > solution. > >> > > > >> > > One way forward is to use objcopy to manipulate symbol status > >> > > (global/local). There is an option --localize-symbol in objcopy, that > >> > > has been available in objcopy since at least 2.15, which was released > >> > > 2004, so it should be safe to use. But ideally we should avoid using > >> > > objcopy and do this as part of the linking process. This should be > >> > > possible to do, given that we make changes in NativeCompilation.gmk -- > >> > > see question 5 above. > >> > > > >> > > As a fallback, it is also possible to rename symbols, either piecewise > >> > > or wholesale, using objcopy. There are many ways to do this, using > >> > > --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this > >> > > takes a file with a list of symbols). Thus we can always introduce a > >> > > "post factum namespace" by renaming symbols. > >> > > >> > Renaming or redefining the symbol at build time could cause confusions > >> > with debugging. That's a concern raised in > >> > https://github.com/openjdk/jdk/pull/17456 discussions. > >> > > >> > Additionally, redefining symbols using tools like objcopy may not > >> > handle member names referenced in string literals. For example, in > >> > https://github.com/openjdk/jdk/pull/17456 additional changes are > >> > needed in assembling and SA to reflect the symbol change. > >> > > >> > > > >> > > So in the end, I think it will be fully possible to produce .a files > >> > > that only has global symbols for the functions that are part of the API > >> > > exposed by that library, and have all other symbols local, and make this > >> > > is in a way that is consistent with the rest of the build system. > >> > > > >> > > Finally, a note on Hotspot. Due to debugging reasons, we export > >> > > basically all symbols in hotspot as global. This is not reasonable to do > >> > > for a static build. The effect of not exporting those symbols will be > >> > > that SA will not function to 100%. On the other hand, I have no idea if > >> > > SA works at all with a static build. Have you tested this? Is this part > >> > > of the plan to support, or will it be officially dropped for Hermetic Java? > >> > > >> > We have done some testing with jtreg SA related tests for the fully > >> > statically linked `javastatic`. > >> > > >> > If we use objcopy to localize symbols in hotspot, it's not yet clear > >> > what's the impact on SA. We could do some tests. The other question > >> > that I raised is the supported gcc versions (for partial linking) > >> > related to the solution. > >> > > >> > Best, > >> > Jiangli > >> > > >> > > > >> > > /Magnus > >> > > From ihse at openjdk.org Tue Apr 16 08:37:02 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 08:37:02 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Mon, 15 Apr 2024 13:10:22 GMT, Martin Doerr wrote: >> It was too bad that I did not see and review this change in the makefiles. :-( >> >> While you guys could have gone either way, I strongly dislike the choice to include a redefinition in the makefiles. If this really should be done, we should introduced a new variable to carry such changes, instead of piggybacking it with the OS defines. :-( But, I don't think it should be done at all. >> >> There are several reasons why this is a inferior solution: >> >> 1) It does not follow prior examples. We have tried hard before not do things like this, but rather pass flags as defines (e.g. `-DREDEFINE_ALLOCA` had been better) >> 2) It does not scale. If we start in effect allowing code in the command line, there is no clear limit anymore what should be placed in the source code files and what should be placed on the command line. >> 3) It messes up command lines. Keeping command lines as short as reasonable possible is a goal we try to strive for. In this case, there is also the `'` inside them (which I don't understand why), which is just begging for quoting/escaping problems, making command lines hard to copy/paste, send to different systems (like logging) etc. >> >> I'd really like to see a follow-up PR that moves this away from the command line define and into a source code file instead. > > Can we unconditionally `#include ` in all files which use `alloca`? Or does that disturb any platform? I don't think it does. From the documentation I've checked, `#include ` is valid everywhere. I'm not sure why the fact that it is a compiler-builtin is relevant here. The man page for alloca on Linux says: SYNOPSIS #include void *alloca(size_t size); which I take it as this is the formally correct way to use alloca. If it happens to work without the include file, that's just coincidence, and not a sign that we should remove `#include ` everywhere. @kimbarrett What do you say? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1566964500 From jkern at openjdk.org Tue Apr 16 08:51:47 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 16 Apr 2024 08:51:47 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Wed, 10 Apr 2024 22:14:33 GMT, Kim Barrett wrote: >> That build failure in shared code does not happen with Xcode clang, gcc, or >> Visual Studio, even though none of them appear to have a relevant define or >> include. So the clang variant being used for AIX is different from the Xcode >> clang variant (and maybe others) in its treatment of alloca. Weird! >> >> I can also live with either the macro or the includes where needed. I dislike >> conditionally adding the include in globalDefinitions_gcc.hpp. > > Should also remove the `#pragma alloca` in os_aix.cpp. We can not use `#include ` in all files which use `alloca`, because windows does not know this header. Maybe we can use `#include ` unconditionally in globalDefinitions_gcc.hpp, if windows will never use this file. @kimbarrett What do you say? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1566988309 From ihse at openjdk.org Tue Apr 16 09:18:03 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 09:18:03 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> On Tue, 16 Apr 2024 09:14:25 GMT, Magnus Ihse Bursie wrote: >> We can not use `#include ` in all files which use `alloca`, because windows does not know this header. Maybe we can use `#include ` unconditionally in globalDefinitions_gcc.hpp, if windows will never use this file. @kimbarrett What do you say? > > That was kind of where the discussion started, and which Kim did not like. If I read him correctly, his suggestion was instead to place: > > #if defined(_AIX) > #include > #endif > > in the files where `alloca` is needed on AIX. (If some of these files happen to be files which are not compiled on Windows, I assume it will not hurt to drop the ifdef guard, but then again, it can certainly be kept as well for consistency.) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1567027594 From ihse at openjdk.org Tue Apr 16 09:18:03 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 09:18:03 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> Message-ID: On Tue, 16 Apr 2024 08:49:02 GMT, Joachim Kern wrote: >> Should also remove the `#pragma alloca` in os_aix.cpp. > > We can not use `#include ` in all files which use `alloca`, because windows does not know this header. Maybe we can use `#include ` unconditionally in globalDefinitions_gcc.hpp, if windows will never use this file. @kimbarrett What do you say? That was kind of where the discussion started, and which Kim did not like. If I read him correctly, his suggestion was instead to place: #if defined(_AIX) #include #endif in the files where `alloca` is needed on AIX. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1567026294 From ihse at openjdk.org Tue Apr 16 09:49:11 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 09:49:11 GMT Subject: RFR: 8330261: Clean up linking steps [v5] In-Reply-To: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: > Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. > > This PR also contains some additional code cleanup/fixes. Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: - ... and fix indentation - Remote ExecuteWithLog for the MT call ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18783/files - new: https://git.openjdk.org/jdk/pull/18783/files/df3f64a2..34422bc9 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18783&range=03-04 Stats: 4 lines in 1 file changed: 0 ins; 1 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/18783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18783/head:pull/18783 PR: https://git.openjdk.org/jdk/pull/18783 From ihse at openjdk.org Tue Apr 16 09:49:11 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 09:49:11 GMT Subject: RFR: 8330261: Clean up linking steps [v4] In-Reply-To: References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: On Mon, 15 Apr 2024 14:21:08 GMT, Magnus Ihse Bursie wrote: >> Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. >> >> This PR also contains some additional code cleanup/fixes. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > MACOSX_CODESIGN_MODE was not always properly set It turned out that the MT step failed on Windows, due to the special combination of `"`, `,` and `#`. I tried messing around a bit, but failed to find a solution for all three at the same time. Since the important part of this PR is to get the non-Microsoft linking steps cleaned up, I reverted that part for the time being. I have opened [JDK-8330341](https://bugs.openjdk.org/browse/JDK-8330341) to fix ExecuteWithLog. After that is done, we can take another stab at the MT command line. Now it should work properly. Sorry for the mess. :-( ------------- PR Comment: https://git.openjdk.org/jdk/pull/18783#issuecomment-2058520681 PR Comment: https://git.openjdk.org/jdk/pull/18783#issuecomment-2058682564 From ihse at openjdk.org Tue Apr 16 09:52:19 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 09:52:19 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files [v2] In-Reply-To: References: Message-ID: > The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. > > I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge branch 'master' into split-awt2d - 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18743/files - new: https://git.openjdk.org/jdk/pull/18743/files/81802740..49f1b4bf Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18743&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18743&range=00-01 Stats: 7258 lines in 263 files changed: 3001 ins; 2087 del; 2170 mod Patch: https://git.openjdk.org/jdk/pull/18743.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18743/head:pull/18743 PR: https://git.openjdk.org/jdk/pull/18743 From jwaters at openjdk.org Tue Apr 16 09:55:02 2024 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 16 Apr 2024 09:55:02 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> Message-ID: On Tue, 16 Apr 2024 09:15:19 GMT, Magnus Ihse Bursie wrote: >> That was kind of where the discussion started, and which Kim did not like. If I read him correctly, his suggestion was instead to place: >> >> #if defined(_AIX) >> #include >> #endif >> >> in the files where `alloca` is needed on AIX. > > (If some of these files happen to be files which are not compiled on Windows, I assume it will not hurt to drop the ifdef guard, but then again, it can certainly be kept as well for consistency.) Windows does use this file, in the unofficial Windows/gcc Port. That said, I am fairly sure the Windows distribution of gcc does recognise alloca.h. It would be a little strange to unconditionally include this if only AIX needs it, though ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1567076923 From ihse at openjdk.org Tue Apr 16 10:03:27 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 10:03:27 GMT Subject: RFR: 8330107: Split Awt2dLibraries.gmk into separate AWT and 2D files [v3] In-Reply-To: References: Message-ID: > The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. > > I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: - Make the split based on the name "awt" instead, and document the reason - Rename 2dLibraries.gmk to ClientLibraries.gmk ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18743/files - new: https://git.openjdk.org/jdk/pull/18743/files/49f1b4bf..de43e1e0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18743&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18743&range=01-02 Stats: 905 lines in 4 files changed: 459 ins; 446 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18743.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18743/head:pull/18743 PR: https://git.openjdk.org/jdk/pull/18743 From ihse at openjdk.org Tue Apr 16 10:05:43 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 10:05:43 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk In-Reply-To: References: Message-ID: On Fri, 12 Apr 2024 22:44:35 GMT, Phil Race wrote: >> @prrace I will need your assistance in confirming that my understanding about the AWT vs 2D split is correct. In particular, `libosxui` gave me some headache, but after trying to dig into the code my understanding ended up being that this is part of Swing which is considered part of AWT, not 2D. >> >> I also looked at `libosx` and `libosxapp` which are located in the general `Lib.gmk` file. I could not easily see that they should have been placed in either 2d or Awt, so my assumption is that they are correctly placed outside these two new files. > >> @prrace I will need your assistance in confirming that my understanding about the AWT vs 2D split is correct. In particular, `libosxui` gave me some headache, but after trying to dig into the code my understanding ended up being that this is part of Swing which is considered part of AWT, not 2D. >> >> I also looked at `libosx` and `libosxapp` which are located in the general `Lib.gmk` file. I could not easily see that they should have been placed in either 2d or Awt, so my assumption is that they are correctly placed outside these two new files. > > Swing isn't part of AWT. They are separate entities. > Swing is built on top of 2D as much as it is built on top of AWT. > > Any hope of finding a neat dividing line will be arbitrary and not really reflecting reality > > And in some ways AWT is more closely tied to AWT than 2D is. > > And things are not the same across platforms either. > You've determined splashscreen is 2D functionality which it most certainly is not. > > Awtlibraries is setting up the shaders which for the Metal rendering pipeline which is 100% 2D. > > And it seems to be building OpenGL, etc as well .. > > and generally I see so manay references to 2D in the AWT file and vice versa. > > it isn't even true that "libawt" is awt. Libawt is mostly a misnomer, in one case more than mostly. It ought to be called "lib2d" > > So I think this change will just cause confusion and doesn't seem right to me. It ought to "not happen". @prrace I tried implementing my suggestion: making the split based purely on having "awt" in the library name. I also clearly documented that the reason for the split do not imply anything about the source code. Would this be okay to you? ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2058714191 From ihse at openjdk.org Tue Apr 16 11:00:23 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 11:00:23 GMT Subject: RFR: 8330351: Remove the SHARED_LIBRARY and STATIC_LIBRARY macros Message-ID: The macros SHARED_LIBRARY and STATIC_LIBRARY are weird (they do not fit in with naming and placement of other macros), and almost never used. We should just get rid of them. ------------- Commit messages: - 8330351: Remove the SHARED_LIBRARY and STATIC_LIBRARY macros Changes: https://git.openjdk.org/jdk/pull/18793/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18793&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330351 Stats: 28 lines in 5 files changed: 1 ins; 19 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/18793.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18793/head:pull/18793 PR: https://git.openjdk.org/jdk/pull/18793 From alanb at openjdk.org Tue Apr 16 11:37:06 2024 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 16 Apr 2024 11:37:06 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: Message-ID: <2oYovRHOXaq7f1-_yuHOx8Q65i_Z5HVse63hkRyE0Ek=.aacaf8e9-defc-4dc7-a9d3-d0260755dfba@github.com> On Wed, 3 Apr 2024 22:31:39 GMT, Mandy Chung wrote: >> Severin Gehwolf has updated the pull request incrementally with one additional commit since the last revision: >> >> Move CreateLinkableRuntimePlugin to build folder >> >> Keep runtime link supporting classes in package >> jdk.tools.jlink.internal.runtimelink > > Thanks for the investigation w.r.t. extending jimage. It does not seem worth the effort in pursuing the support of adding resources to an existing jimage file. To me, putting the diff data under `lib` directory as a private file is a simpler and acceptable solution. It allows the second jlink invocation to avoid the plugin pipeline if the per-module metadata is generated in normal jlink invocation. > > (An observation: The current implementation requires the diffs to be generated as a plugin. For this approach, it may be possible to implement with a separate main class that calls JLink API and generates the diffs.) > @mlchung @AlanBateman Any thoughts on this latest version? Is this going into the direction you had envisioned? Any more blockers? The CSR should be up-to-date and is open for review as well. If no more blockers I'll go over this patch once more and clean it up to prepare for integration. Thanks! Thanks for all the efforts on this. I've looked through the latest version. I'm a little bit comfortable with LinkDeltaProducer. One reason it's build tool that is tied tied to code in the jdk.jlink module, and secondly is that it copies runtime-image-link.delta into the run-time image. Our long standing position is that the run-time image is created by jlink rather than a combination of jlink and extra stuff copied in by the build. The part that I like with the current proposal is lib/runtime-image-link.delta as it's less complicated that previous iterations that added a resource into the jimage container. What would you (and @mlchung) think of having jlink generate lib/runtime-image-link.delta as a step between the pipeline and image generation? ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2058876568 From sgehwolf at openjdk.org Tue Apr 16 12:01:51 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Tue, 16 Apr 2024 12:01:51 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: Message-ID: <_sbNFJXG7ghwsfNfxaGvvoDQ7-UtWpnhGnygKsFTsgQ=.623700db-7306-4b34-9e05-ca9059797e39@github.com> On Wed, 3 Apr 2024 22:31:39 GMT, Mandy Chung wrote: >> Severin Gehwolf has updated the pull request incrementally with one additional commit since the last revision: >> >> Move CreateLinkableRuntimePlugin to build folder >> >> Keep runtime link supporting classes in package >> jdk.tools.jlink.internal.runtimelink > > Thanks for the investigation w.r.t. extending jimage. It does not seem worth the effort in pursuing the support of adding resources to an existing jimage file. To me, putting the diff data under `lib` directory as a private file is a simpler and acceptable solution. It allows the second jlink invocation to avoid the plugin pipeline if the per-module metadata is generated in normal jlink invocation. > > (An observation: The current implementation requires the diffs to be generated as a plugin. For this approach, it may be possible to implement with a separate main class that calls JLink API and generates the diffs.) > > @mlchung @AlanBateman Any thoughts on this latest version? Is this going into the direction you had envisioned? Any more blockers? The CSR should be up-to-date and is open for review as well. If no more blockers I'll go over this patch once more and clean it up to prepare for integration. Thanks! > > Thanks for all the efforts on this. Thanks for looking at this, Alan! > I've looked through the latest version. I'm a little bit comfortable with LinkDeltaProducer. One reason it's build tool that is tied tied to code in the jdk.jlink module, and secondly is that it copies runtime-image-link.delta into the run-time image. Our long standing position is that the run-time image is created by jlink rather than a combination of jlink and extra stuff copied in by the build. > > The part that I like with the current proposal is lib/runtime-image-link.delta as it's less complicated that previous iterations that added a resource into the jimage container. > > What would you (and @mlchung) think of having jlink generate lib/runtime-image-link.delta as a step between the pipeline and image generation? If I understand you correctly, this would be no longer a build-time only approach to produce the "linkable runtime"? It would be some-kind of jlink-option driven approach (as it would run some code that should only run when producing a linkable runtime is requested)? Other than that, it should be fine as the previous iteration basically did that but at a different phase. Also note that the previous iteration used a build-only jlink plugin so as to satisfy the build-time only approach, yet it ran as part of the plugin-pipeline which wasn't desired either. But it was something similar to what you seem to be describing now. Did I get something wrong? ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2058919838 From erikj at openjdk.org Tue Apr 16 12:46:42 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 16 Apr 2024 12:46:42 GMT Subject: RFR: 8330261: Clean up linking steps [v5] In-Reply-To: References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: On Tue, 16 Apr 2024 09:49:11 GMT, Magnus Ihse Bursie wrote: >> Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. >> >> This PR also contains some additional code cleanup/fixes. > > Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: > > - ... and fix indentation > - Remote ExecuteWithLog for the MT call Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18783#pullrequestreview-2003522534 From erikj at openjdk.org Tue Apr 16 12:50:59 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 16 Apr 2024 12:50:59 GMT Subject: RFR: 8330351: Remove the SHARED_LIBRARY and STATIC_LIBRARY macros In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 09:31:19 GMT, Magnus Ihse Bursie wrote: > The macros SHARED_LIBRARY and STATIC_LIBRARY are weird (they do not fit in with naming and placement of other macros), and almost never used. We should just get rid of them. Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18793#pullrequestreview-2003532209 From ihse at openjdk.org Tue Apr 16 13:54:04 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 13:54:04 GMT Subject: Integrated: 8330351: Remove the SHARED_LIBRARY and STATIC_LIBRARY macros In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 09:31:19 GMT, Magnus Ihse Bursie wrote: > The macros SHARED_LIBRARY and STATIC_LIBRARY are weird (they do not fit in with naming and placement of other macros), and almost never used. We should just get rid of them. This pull request has now been integrated. Changeset: 61fa4d45 Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/61fa4d45b68ffbfb751471730518651b78b195aa Stats: 28 lines in 5 files changed: 1 ins; 19 del; 8 mod 8330351: Remove the SHARED_LIBRARY and STATIC_LIBRARY macros Reviewed-by: erikj ------------- PR: https://git.openjdk.org/jdk/pull/18793 From ihse at openjdk.org Tue Apr 16 13:54:45 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 13:54:45 GMT Subject: Integrated: 8330261: Clean up linking steps In-Reply-To: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> References: <-YAsmq_6lj3nFQ3vcvDByUEMstLtwIX32FzMgs8IOLU=.4b58a073-3a60-41c7-9d4a-012293b4d498@github.com> Message-ID: On Mon, 15 Apr 2024 12:34:53 GMT, Magnus Ihse Bursie wrote: > Instead of executing code directly in Link.gmk, we set variables that are then evaluated to get the code we want. This is non-transparent and makes it unnecessarily hard to work with the linking code base. > > This PR also contains some additional code cleanup/fixes. This pull request has now been integrated. Changeset: 6e77d918 Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/6e77d918e62a2eb77c3c1cc32b8ddad909036517 Stats: 120 lines in 5 files changed: 49 ins; 39 del; 32 mod 8330261: Clean up linking steps Reviewed-by: erikj ------------- PR: https://git.openjdk.org/jdk/pull/18783 From alanb at openjdk.org Tue Apr 16 13:57:51 2024 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 16 Apr 2024 13:57:51 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: <_sbNFJXG7ghwsfNfxaGvvoDQ7-UtWpnhGnygKsFTsgQ=.623700db-7306-4b34-9e05-ca9059797e39@github.com> References: <_sbNFJXG7ghwsfNfxaGvvoDQ7-UtWpnhGnygKsFTsgQ=.623700db-7306-4b34-9e05-ca9059797e39@github.com> Message-ID: On Tue, 16 Apr 2024 11:59:12 GMT, Severin Gehwolf wrote: > If I understand you correctly, this would be no longer a build-time only approach to produce the "linkable runtime"? It would be some-kind of jlink-option driven approach (as it would run some code that should only run when producing a linkable runtime is requested)? Other than that, it should be fine as the previous iteration basically did that but at a different phase. Also note that the previous iteration used a build-only jlink plugin so as to satisfy the build-time only approach, yet it ran as part of the plugin-pipeline which wasn't desired either. But it was something similar to what you seem to be describing now. Did I get something wrong? I think it continues to build time, it's just using some hidden jlink option. So yes, it similar to a previous iteration except that it doesn't run as a plugin the pipeline and the delta goes to the lib directory. Let's see what @mlchung says. You've put a lot of work into this so let's see if we can converge to avoid too many more rounds. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2059157584 From magnus.ihse.bursie at oracle.com Tue Apr 16 15:30:04 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 17:30:04 +0200 Subject: Questions about the Hermetic Java project In-Reply-To: References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> Message-ID: <8852ba23-2830-4d23-ae52-11c554d8d750@oracle.com> Jiangli, First of all: I tried building the leyden branch "hermetic-java-runtime" but failed. :-( I tried some various attempts to work around the failures, but the javastatic file I ended up with were unable to start. What kind of environment are you using to build this branch? (OS, compiler, libc version, etc) Do you have any special configure flags that are required? I would very much like to test it out myself to check this static java launcher that you are creating. On 2024-04-16 03:01, Jiangli Zhou wrote: > The main purpose of StaticLink.gmk is to support the static-java-image > make target, which can be used to perform the actual static linking > step using libjvm.a and JDK static libraries. That currently doesn't > exist in the JDK mainline. Creating a "fully" statically linked Java > launcher is the first step (out of many) towards supporting > static/hermetic Java. I am still not sure it is the best design to have this in a separate file. You are in some way "just" creating a new launcher. > As part of cleaning/refactoring/integrating for the static linking > step, we want to agree and decide/accept on the following: > > - Support the "fully" statically linked java launcher for testing and > demoing the capability of static JDK support, e.g. > - Support running jtreg testing using the "fully" statically linked > Java launcher So how do you want to test native jtreg tests? By compiling the jtreg tests to native .a files and statically link with those as well, or to make this a hybrid mode where a statically linked launcher loads the jtreg tests dynamically? > - Set up tests in github workflow to help detect any breaking > changes for static support, e.g. new symbol issues introduced by any > changes. There were some earlier discussions on this with Ron and Alan > during the zoom meetings. If we do this correct, we should not have to worry about symbol conflicts. (Local symbols should all be hidden before creation of the static libraries.) That said, creating static libraries needs to be regularly tested or it will break. > - Which JDK native libraries to be statically linked with the new > launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, > libawt_xawt.a, etc from statically linked with the launcher. That sounds rather arbitrarily. I would assume that a statically linked launcher should include native libraries from all included modules. If you are not interested in java.desktop native libraries, then you'd need to exclude java.desktop from the build. But my assumption here is that we need to build a solution that works with all modules included in the JDK, and that is what must be tested in GHA. > - Do we want more than one statically linked launcher target, based on > the set of linked native libraries? Do you mean that you want like "static-java-java.base-only", "static-java-eveything-but-java.desktop" etc? That does not sound like something that belongs in the makefiles. If you want to build a JDK that can only support a certain subset of modules, you need to specify those module in your build scripts as input to configure. > (II) What missing: > - Static linking step as mentioned above > > (III) What needs to be improved (require cleanups and refactoring, and > you mentioned some of those in your response as well): > - Support building both the static libraries and dynamic libraries > using the same set of .o files, instead of separately compiled .o > files. That helps improve build speed and reduce memory overhead for > building JDK. Your current refactoring work aims to help that. > - Clean up the usages of STATIC_BUILD macro. Most of the usages are in > test code. > - Other runtime fixes/enhancements in the leyden > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > > I think most work mentioned in III has dependencies on II. We need a > workable base to be able to build the "fully" statically linked > launcher for building and testing the work mentioned in III, when > integrating any of those to the JDK mainline. The makefile refactoring > work can be done in parallel but does not need to be completed before > we add the static linking step in JDK mainline. I'm not sure I understand why you consider (II) to be a prerequisite for the (III) issues. From my point of view, they are all mutually independent changes that needs to be done, and if anything, I think getting in code to create a statically linked launcher will probably be easier if we get further along on the other issues. In particular, I'd like to see changes to the STATIC_BUILD macro come in first. Ideally, we should not send in such a macro at all. The JNI_OnLoad stuff is bugging me mostly. > As of today, the leyden > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > can build a "fully" statically linked Java launcher. The issue of > compiling the dynamic and static libraries .o files separately is not > a blocker. It's good to have it resolved at some point of time. Weeeell, yes and no... It is not *technically* a blocker, but unless we can reuse the .o files, including static builds in the standard testing on GHA will effectively double the build time, and that is definitely not acceptable. And the alternative is to bring in static builds without having it regularly tested, which I don't think is acceptable either. So the conclusion is that getting static and dynamic libraries built from the same .o files is in effect a blocker for Hermetic Java. >> This, in turn, require several changes: >> >> 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. >> >> 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. >> >> This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > Thumbs up! That seems to be a good direction. Currently in the leyden > branch, it first looks up the unique > JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in > libraries, then search for the dynamic libraries using the > conventional naming when necessary. e.g.: > > https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a > https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 > > When spec supports JNI_OnLoad_ and etc. for dynamic > libraries, we may still need to support the conventional naming > without the <_lib_name> part for existing libraries out there. That looks interesting. Basically, this is very similar to what I imagine needs to be done in the mainline. However, I don't think we can just change this code like that without a CSR? Or are we allowed to treat JDK-internal libraries different than the specification states? I would say getting this part into mainline should be the main focus right now, not the StaticLink.gmk file. And that includes getting any CSR approvals etc. > Thank you for taking this on! Potentially we could consider taking the > objcopy to localizing hotspot symbols on unix-like platforms, based on > https://github.com/openjdk/jdk/pull/17456 discussions. Additional > testing is still needed to verify the solution. We don't need any additional testing on that; proof of concept shows it works. It just needs to be implemented. With JDK-8330261 (pushed today), the ground is now prepared to get it done. It is next on my todo-list. > It might be a good idea to follow up on the static linking discussion > in tomorrow's zoom meeting (hope you'll be able to join tomorrow). I'll join but might be somewhat late due to family commitments. /Magnus -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.ihse.bursie at oracle.com Tue Apr 16 15:40:09 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 17:40:09 +0200 Subject: Questions about the Hermetic Java project In-Reply-To: References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> Message-ID: <4bda696b-eb73-497c-ac9d-b128fc887514@oracle.com> On 2024-04-16 10:28, Volker Simonis wrote: > Hi, and sorry, I'm completely new to this discussion. > > I just wonder how `static-libs-image` is related to the > `graal-builder-image` target which is available in mainline since JDK > 11 and builds a static version of all the native libraries excluding > the Hotspot ones? That's a very good question. You can also ask how it relates to --enable-static-build from JDK-8136556 which has been in the JDK since 9. :-) Or, to put in other words, we already have three different ways to produce static libraries in the JDK, each of them created more or less independently of the other. And now Hermetic Java wants to add a fourth. This is not a good situation. My intention here is to basically retire static-libs-image, graal-builder-image and --enable-static-build, and replace them with a more general solution that supports all these existing use cases, and the new use case from the StaticLink.gmk file that the Hermetic Java project wants to integrate. Unfortunately, this requires some delicate reverse-engineering of specifications (i.e. "why the heck did they do it like this???") to understand what the different use cases want to achieve. That is the main driver behind my recent work in clearing out technical debt in the native linking part of the build system. /Magnus > > @Magnus: will the new, static build you're working on also cover > `graal-builder-image` or will it be independent of it? From a quick > look it seems that currently both `static-libs-image` and > `graal-builder-image` are handled in `make/StaticLibsImage.gmk` but > I'm not sure if they use the same build functionality. I just wondered > why `graal-builder-image` hasn't been mentioned in this thread but > maybe it's something as obvious as that it already is the same > functionality anway? > > Thank you and best regards, > Volker > > On Tue, Apr 16, 2024 at 3:02?AM Jiangli Zhou wrote: >> Magnus, thanks for the response. Please see comments inlined below. >> >> On Fri, Apr 12, 2024 at 4:52?AM Magnus Ihse Bursie >> wrote: >>> On 2024-04-02 21:16, Jiangli Zhou wrote: >>> >>> Hi Magnus, >>> >>> In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) discussed how to move forward contributing the static Java related changes (additional runtime fixes/enhancements on top of the existing static support in JDK) from https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK mainline. >>> >>> Just a bit more details/context below, which may be useful for others reading this thread. >>> >>> The https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch currently contains following for supporting hermetic Java (without the launcher work for runtime support): >>> >>> 1. Build change for linking the Java launcher (as bin/javastatic) with JDK/hotspot static libraries (.a), mainly in https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. The part for creating the complete sets of static libraries (including libjvm.a) has already been included in the mainline since last year. https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk is in a very raw state and is intended to demonstrate the capability of building a static Java launcher. >>> >>> Indeed. It is nowhere near being able to be integrated. >>> >> The main purpose of StaticLink.gmk is to support the static-java-image >> make target, which can be used to perform the actual static linking >> step using libjvm.a and JDK static libraries. That currently doesn't >> exist in the JDK mainline. Creating a "fully" statically linked Java >> launcher is the first step (out of many) towards supporting >> static/hermetic Java. >> >> As part of cleaning/refactoring/integrating for the static linking >> step, we want to agree and decide/accept on the following: >> >> - Support the "fully" statically linked java launcher for testing and >> demoing the capability of static JDK support, e.g. >> - Support running jtreg testing using the "fully" statically linked >> Java launcher >> - Set up tests in github workflow to help detect any breaking >> changes for static support, e.g. new symbol issues introduced by any >> changes. There were some earlier discussions on this with Ron and Alan >> during the zoom meetings. >> - Which JDK native libraries to be statically linked with the new >> launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, >> libawt_xawt.a, etc from statically linked with the launcher. >> - Do we want more than one statically linked launcher target, based on >> the set of linked native libraries? >> >> Based on the decisions of the above, the launcher static linking part >> would mostly be in a different shape when it's integrated into the >> mainline. That's why I referred to StaticLink.gmk as in a "very raw" >> state. >> >> Here is a high-level view of the state of things for static support: >> >> (I) What we already have in the JDK mainline: >> - Able to build a complete set of JDK/VM static libraries using >> `static-libs-image` make target (necessary for supporting static JDK) >> - Compilation for .o files are done separately for the static >> libraries and dynamic library (ok for now) >> >> (II) What missing: >> - Static linking step as mentioned above >> >> (III) What needs to be improved (require cleanups and refactoring, and >> you mentioned some of those in your response as well): >> - Support building both the static libraries and dynamic libraries >> using the same set of .o files, instead of separately compiled .o >> files. That helps improve build speed and reduce memory overhead for >> building JDK. Your current refactoring work aims to help that. >> - Clean up the usages of STATIC_BUILD macro. Most of the usages are in >> test code. >> - Other runtime fixes/enhancements in the leyden >> https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch >> >> I think most work mentioned in III has dependencies on II. We need a >> workable base to be able to build the "fully" statically linked >> launcher for building and testing the work mentioned in III, when >> integrating any of those to the JDK mainline. The makefile refactoring >> work can be done in parallel but does not need to be completed before >> we add the static linking step in JDK mainline. >> >>> 2. Additional runtime fixes/enhancements on top of the existing static support in JDK, e.g. support further lookup dynamic native library if the built-in native library cannot be found. >>> >>> 3. Some initial (prototype) work on supporting hermetic JDK resource files in the jimage (JDK modules image). >>> >>> To move forward, one of the earliest items needed is to add the capability of building the fully statically linked Java launcher in JDK mainline. The other static Java runtime changes can be followed up after the launcher linking part, so they can be built and tested as individual PRs created for the JDK mainline. Magnus, you have expressed interest in helping get the launcher linking part (refactor from https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) into JDK mainline. What's your thought on prioritizing the launcher static linking part before other makefile clean ups for static libraries? >>> >>> Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. >> Thanks! >> >>> I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? >> Please see my comments above. >> >>> The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. >>> >>> The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. >>> >>> My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). >> As of today, the leyden >> https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch >> can build a "fully" statically linked Java launcher. The issue of >> compiling the dynamic and static libraries .o files separately is not >> a blocker. It's good to have it resolved at some point of time. >> >>> This, in turn, require several changes: >>> >>> 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. >>> >>> 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. >>> >>> This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. >> Thumbs up! That seems to be a good direction. Currently in the leyden >> branch, it first looks up the unique >> JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in >> libraries, then search for the dynamic libraries using the >> conventional naming when necessary. e.g.: >> >> https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a >> https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 >> >> When spec supports JNI_OnLoad_ and etc. for dynamic >> libraries, we may still need to support the conventional naming >> without the <_lib_name> part for existing libraries out there. >> >>> And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. >>> >>> A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. >> Thank you for taking this on! Potentially we could consider taking the >> objcopy to localizing hotspot symbols on unix-like platforms, based on >> https://github.com/openjdk/jdk/pull/17456 discussions. Additional >> testing is still needed to verify the solution. >> >>> My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. >> Most of the JDK resources are now supported as hermetic jimage >> (lib/modules) bundled in the >> https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch. >> The remaining sound.properties, ct.sym and .jfc files can be handled >> later. Overally, that part of the work has confirmed the hermetic >> jimage bundled solution is robust and helps resolve some of the >> difficult start-up sequence issues observed when the hermetic resource >> was implemented using JAR file based solution. >> >> It might be a good idea to follow up on the static linking discussion >> in tomorrow's zoom meeting (hope you'll be able to join tomorrow). >> >> Thanks! >> >> Jiangli >>> /Magnus >>> >>> >>> >>> Thanks! >>> Jiangli >>> >>> On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou wrote: >>>> On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou wrote: >>>>> Hi Magnus, >>>>> >>>>> Thanks for looking into this from the build perspective. >>>>> >>>>> On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie >>>>> wrote: >>>>>> First some background for build-dev: I have spent some time looking at >>>>>> the build implications of the Hermetic Java effort, which is part of >>>>>> Project Leyden. A high-level overview is available here: >>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current source >>>>>> code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. >>>>> Some additional hermetic Java related references that are also useful: >>>>> >>>>> - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug that >>>>> links to the issues for resolving static linking issues so far >>>>> - https://github.com/openjdk/jdk21/pull/26 is the enhancement for >>>>> building the complete set of static libraries in JDK/VM, particularly >>>>> including libjvm.a >>>>> >>>>>> Hermetic Java faces several challenges, but the part that is relevant >>>>>> for the build system is the ability to create static libraries. We've >>>>>> had this functionality (in three different ways...) for some time, but >>>>>> it is rather badly implemented. >>>>>> >>>>>> As a result of my investigations, I have a bunch of questions. :-) I >>>>>> have gotten some answers in private discussion, but for the sake of >>>>>> transparency I will repeat them here, to foster an open dialogue. >>>>>> >>>>>> 1. Am I correct in understanding that the ultimate goal of this exercise >>>>>> is to be able to have jmods which include static libraries (*.a) of the >>>>>> native code which the module uses, and that the user can then run a >>>>>> special jlink command to have this linked into a single executable >>>>>> binary (which also bundles the *.class files and any additional >>>>>> resources needed)? >>>>>> >>>>>> 2. If so, is the idea to create special kinds of static jmods, like >>>>>> java.base-static.jmod, that contains *.a files instead of lib*.so files? >>>>>> Or is the idea that the normal jmod should contain both? >>>>>> >>>>>> 3. Linking .o and .a files into an executable is a formidable task. Is >>>>>> the intention to have jlink call a system-provided ld, or to bundle ld >>>>>> with jlink, or to reimplement this functionality in Java? >>>>> I have a similar view as Alan responded in your other email thread. >>>>> Things are still in the early stage for the general solution. >>>>> >>>>> In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime >>>>> branch, when configuring JDK with --with-static-java=yes, the JDK >>>>> binary contains the following extra artifacts: >>>>> >>>>> - static-libs/*.a: The complete set of JDK/VM static libraries >>>>> - jdk/bin/javastatic: A demo Java launcher fully statically linked >>>>> with the selected JDK .a libraries (e.g. it currently statically link >>>>> with the headless) and libjvm.a. It's the standard Java launcher >>>>> without additional work for hermetic Java. >>>>> >>>>> In our prototype for hermetic Java, we build the hermetic executable >>>>> image (a single image) from the following input (see description on >>>>> singlejar packaging tool in >>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf): >>>>> >>>>> - A customized launcher (with additional work for hermetic) executable >>>>> fully statically linked with JDK/VM static libraries (.a files), >>>>> application natives and dependencies (e.g. in .a static libraries) >>>>> - JDK lib/modules, JDK resource files >>>>> - Application classes and resource files >>>>> >>>>> Including a JDK library .a into the corresponding .jmod would require >>>>> extracting the .a for linking with the executable. In some systems >>>>> that may cause memory overhead due to the extracted copy of the .a >>>>> files. I think we should consider the memory overhead issue. >>>>> >>>>> One possibility (as Alan described in his response) is for jlink to >>>>> invoke the ld on the build system. jlink could pass the needed JDK >>>>> static libraries and libjvm.a (provided as part of the JDK binary) to >>>>> ld based on the modules required for the application. >>>>> >>>> I gave a bit more thoughts on this one. For jlink to trigger ld, it >>>> would need to know the complete linker options and inputs. Those >>>> include options and inputs related to the application part as well. In >>>> some usages, it might be easier to handle native linking separately >>>> and pass the linker output, the executable to jlink directly. Maybe we >>>> could consider supporting different modes for various usages >>>> requirements, from static libraries and native linking point of view: >>>> >>>> Mode #1 >>>> Support .jmod packaged natives static libraries, for both JDK/VM .a >>>> and application natives and dependencies. If the inputs to jlink >>>> include .jmods, jlink can extract the .a libraries and pass the >>>> information to ld to link the executable. >>>> >>>> Mode #2 >>>> Support separate .a as jlink input. Jlink could pass the path >>>> information to the .a libraries and other linker options to ld to >>>> create the executable. >>>> >>>> For both mode #1 and #2, jlink would then use the linker output >>>> executable to create the final hermetic image. >>>> >>>> Mode #3 >>>> Support a fully linked executable as a jlink input. When a linked >>>> executable is given to jlink, it can process it directly with other >>>> JDK data/files to create the final image, without native linking step. >>>> >>>> Any other thoughts and considerations? >>>> >>>> Best, >>>> Jiangli >>>> >>>>>> 4. Is the intention is to allow users to create their own jmods with >>>>>> static libraries, and have these linked in as well? This seems to be the >>>>>> case. >>>>> An alternative with less memory overhead could be using application >>>>> modular JAR and separate .a as the input for jlink. >>>>> >>>>>> If that is so, then there will always be the risk for name >>>>>> collisions, and we can only minimize the risk by making sure any global >>>>>> names are as unique as possible. >>>>> Part of the current effort includes resolving the discovered symbol >>>>> collision issues with static linking. Will respond to your other email >>>>> on the symbol issue separately later. >>>>> >>>>>> 5. The original implementation of static builds in the JDK, created for >>>>>> the Mobile project, used a configure flag, --enable-static-builds, to >>>>>> change the entire behavior of the build system to only produce *.a files >>>>>> instead of lib*.so. In contrast, the current system is using a special >>>>>> target instead. >>>>> I think we would need both configure flag and special target for the >>>>> static builds. >>>>> >>>>>> In my eyes, this is a much worse solution. Apart from >>>>>> the conceptual principle (if the build should generate static or dynamic >>>>>> libraries is definitely a property of what a "configuration" means), >>>>>> this makes it much harder to implement efficiently, since we cannot make >>>>>> changes in NativeCompilation.gmk, where they are needed. >>>>> For the potential objcopy work to resolve symbol issues, we can add >>>>> that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. We >>>>> have an internal prototype (not included in >>>>> https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) done >>>>> by one of colleagues for localizing symbols in libfreetype using >>>>> objcopy. >>>>> >>>>>> That was not as much a question as a statement. ? But here is the >>>>>> question: Do you think it would be reasonable to restore the old >>>>>> behavior but with the new methods, so that we don't use special targets, >>>>>> but instead tells configure to generate static libraries? I'm thinking >>>>>> we should have a flag like "--with-library-type=" that can have values >>>>>> "dynamic" (which is default), "static" or "both". >>>>> If we want to also build a fully statically linked launcher, maybe >>>>> --with-static-java? Being able to configure either dynamic, static or >>>>> both as you suggested also seems to be a good idea. >>>>> >>>>>> I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files >>>>>> into a single jmod file (see question 2 above), then it definitely is. >>>>>> In general, the cost of producing two kinds of libraries are quite >>>>>> small, compared to the cost of compiling the source code to object files. >>>>> Completely agree. It would be good to avoid recompiling the .o file >>>>> for static and dynamic builds. As proposed in >>>>> https://bugs.openjdk.org/browse/JDK-8303796: >>>>> >>>>> It's beneficial to be able to build both .so and .a from the same set >>>>> of .o files. That would involve some changes to handle the dynamic JDK >>>>> and static JDK difference at runtime, instead of relying on the >>>>> STATIC_BUILD macro. >>>>> >>>>>> Finally, I have looked at how to manipulate symbol visibility. There >>>>>> seems many ways forward, so I feel confident that we can find a good >>>>>> solution. >>>>>> >>>>>> One way forward is to use objcopy to manipulate symbol status >>>>>> (global/local). There is an option --localize-symbol in objcopy, that >>>>>> has been available in objcopy since at least 2.15, which was released >>>>>> 2004, so it should be safe to use. But ideally we should avoid using >>>>>> objcopy and do this as part of the linking process. This should be >>>>>> possible to do, given that we make changes in NativeCompilation.gmk -- >>>>>> see question 5 above. >>>>>> >>>>>> As a fallback, it is also possible to rename symbols, either piecewise >>>>>> or wholesale, using objcopy. There are many ways to do this, using >>>>>> --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this >>>>>> takes a file with a list of symbols). Thus we can always introduce a >>>>>> "post factum namespace" by renaming symbols. >>>>> Renaming or redefining the symbol at build time could cause confusions >>>>> with debugging. That's a concern raised in >>>>> https://github.com/openjdk/jdk/pull/17456 discussions. >>>>> >>>>> Additionally, redefining symbols using tools like objcopy may not >>>>> handle member names referenced in string literals. For example, in >>>>> https://github.com/openjdk/jdk/pull/17456 additional changes are >>>>> needed in assembling and SA to reflect the symbol change. >>>>> >>>>>> So in the end, I think it will be fully possible to produce .a files >>>>>> that only has global symbols for the functions that are part of the API >>>>>> exposed by that library, and have all other symbols local, and make this >>>>>> is in a way that is consistent with the rest of the build system. >>>>>> >>>>>> Finally, a note on Hotspot. Due to debugging reasons, we export >>>>>> basically all symbols in hotspot as global. This is not reasonable to do >>>>>> for a static build. The effect of not exporting those symbols will be >>>>>> that SA will not function to 100%. On the other hand, I have no idea if >>>>>> SA works at all with a static build. Have you tested this? Is this part >>>>>> of the plan to support, or will it be officially dropped for Hermetic Java? >>>>> We have done some testing with jtreg SA related tests for the fully >>>>> statically linked `javastatic`. >>>>> >>>>> If we use objcopy to localize symbols in hotspot, it's not yet clear >>>>> what's the impact on SA. We could do some tests. The other question >>>>> that I raised is the supported gcc versions (for partial linking) >>>>> related to the solution. >>>>> >>>>> Best, >>>>> Jiangli >>>>> >>>>>> /Magnus >>>>>> From jianglizhou at google.com Tue Apr 16 16:20:11 2024 From: jianglizhou at google.com (Jiangli Zhou) Date: Tue, 16 Apr 2024 09:20:11 -0700 Subject: Questions about the Hermetic Java project In-Reply-To: <8852ba23-2830-4d23-ae52-11c554d8d750@oracle.com> References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> <8852ba23-2830-4d23-ae52-11c554d8d750@oracle.com> Message-ID: On Tue, Apr 16, 2024 at 8:30?AM Magnus Ihse Bursie wrote: > > Jiangli, > > First of all: I tried building the leyden branch "hermetic-java-runtime" but failed. :-( I tried some various attempts to work around the failures, but the javastatic file I ended up with were unable to start. Magnus, I recall you mentioned that you had build issues during one of the zoom meetings earlier this year (didn't hear any follow-up on it). Let's get this resolved for you first. Can you please share the build failures/errors and runtime errors that you run into? Same questions about your build environment, as you asked below. > > What kind of environment are you using to build this branch? (OS, compiler, libc version, etc) Do you have any special configure flags that are required? For the leyden hermetic-java-runtime branch, I've only been building on Linux. I've used both gcc and clang. Yes, the branch requires `--with-static-java=yes`, e.g. bash configure --with-boot-jdk= --with-debug-level=slowdebug --with-static-java=yes I documented the build information in one of the comment messages. That's probably not the best place. I'll look for a better place. Let's discuss more on the rest of the topics below during our meeting today. We can update on the mailing list about the discussion, if others are interested. Best, Jiangli > > I would very much like to test it out myself to check this static java launcher that you are creating. > > On 2024-04-16 03:01, Jiangli Zhou wrote: > > The main purpose of StaticLink.gmk is to support the static-java-image > make target, which can be used to perform the actual static linking > step using libjvm.a and JDK static libraries. That currently doesn't > exist in the JDK mainline. Creating a "fully" statically linked Java > launcher is the first step (out of many) towards supporting > static/hermetic Java. > > I am still not sure it is the best design to have this in a separate file. You are in some way "just" creating a new launcher. > > As part of cleaning/refactoring/integrating for the static linking > step, we want to agree and decide/accept on the following: > > - Support the "fully" statically linked java launcher for testing and > demoing the capability of static JDK support, e.g. > - Support running jtreg testing using the "fully" statically linked > Java launcher > > So how do you want to test native jtreg tests? By compiling the jtreg tests to native .a files and statically link with those as well, or to make this a hybrid mode where a statically linked launcher loads the jtreg tests dynamically? > > - Set up tests in github workflow to help detect any breaking > changes for static support, e.g. new symbol issues introduced by any > changes. There were some earlier discussions on this with Ron and Alan > during the zoom meetings. > > If we do this correct, we should not have to worry about symbol conflicts. (Local symbols should all be hidden before creation of the static libraries.) That said, creating static libraries needs to be regularly tested or it will break. > > - Which JDK native libraries to be statically linked with the new > launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, > libawt_xawt.a, etc from statically linked with the launcher. > > That sounds rather arbitrarily. I would assume that a statically linked launcher should include native libraries from all included modules. If you are not interested in java.desktop native libraries, then you'd need to exclude java.desktop from the build. But my assumption here is that we need to build a solution that works with all modules included in the JDK, and that is what must be tested in GHA. > > - Do we want more than one statically linked launcher target, based on > the set of linked native libraries? > > Do you mean that you want like "static-java-java.base-only", "static-java-eveything-but-java.desktop" etc? That does not sound like something that belongs in the makefiles. If you want to build a JDK that can only support a certain subset of modules, you need to specify those module in your build scripts as input to configure. > > (II) What missing: > - Static linking step as mentioned above > > (III) What needs to be improved (require cleanups and refactoring, and > you mentioned some of those in your response as well): > - Support building both the static libraries and dynamic libraries > using the same set of .o files, instead of separately compiled .o > files. That helps improve build speed and reduce memory overhead for > building JDK. Your current refactoring work aims to help that. > - Clean up the usages of STATIC_BUILD macro. Most of the usages are in > test code. > - Other runtime fixes/enhancements in the leyden > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > > I think most work mentioned in III has dependencies on II. We need a > workable base to be able to build the "fully" statically linked > launcher for building and testing the work mentioned in III, when > integrating any of those to the JDK mainline. The makefile refactoring > work can be done in parallel but does not need to be completed before > we add the static linking step in JDK mainline. > > I'm not sure I understand why you consider (II) to be a prerequisite for the (III) issues. From my point of view, they are all mutually independent changes that needs to be done, and if anything, I think getting in code to create a statically linked launcher will probably be easier if we get further along on the other issues. > > In particular, I'd like to see changes to the STATIC_BUILD macro come in first. Ideally, we should not send in such a macro at all. The JNI_OnLoad stuff is bugging me mostly. > > As of today, the leyden > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > can build a "fully" statically linked Java launcher. The issue of > compiling the dynamic and static libraries .o files separately is not > a blocker. It's good to have it resolved at some point of time. > > Weeeell, yes and no... It is not *technically* a blocker, but unless we can reuse the .o files, including static builds in the standard testing on GHA will effectively double the build time, and that is definitely not acceptable. And the alternative is to bring in static builds without having it regularly tested, which I don't think is acceptable either. > > So the conclusion is that getting static and dynamic libraries built from the same .o files is in effect a blocker for Hermetic Java. > > This, in turn, require several changes: > > 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > > 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > > This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > > Thumbs up! That seems to be a good direction. Currently in the leyden > branch, it first looks up the unique > JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in > libraries, then search for the dynamic libraries using the > conventional naming when necessary. e.g.: > > https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a > https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 > > When spec supports JNI_OnLoad_ and etc. for dynamic > libraries, we may still need to support the conventional naming > without the <_lib_name> part for existing libraries out there. > > That looks interesting. Basically, this is very similar to what I imagine needs to be done in the mainline. However, I don't think we can just change this code like that without a CSR? Or are we allowed to treat JDK-internal libraries different than the specification states? > > I would say getting this part into mainline should be the main focus right now, not the StaticLink.gmk file. And that includes getting any CSR approvals etc. > > Thank you for taking this on! Potentially we could consider taking the > objcopy to localizing hotspot symbols on unix-like platforms, based on > https://github.com/openjdk/jdk/pull/17456 discussions. Additional > testing is still needed to verify the solution. > > We don't need any additional testing on that; proof of concept shows it works. It just needs to be implemented. With JDK-8330261 (pushed today), the ground is now prepared to get it done. It is next on my todo-list. > > It might be a good idea to follow up on the static linking discussion > in tomorrow's zoom meeting (hope you'll be able to join tomorrow). > > I'll join but might be somewhat late due to family commitments. > > /Magnus From sgehwolf at redhat.com Tue Apr 16 16:54:35 2024 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 16 Apr 2024 18:54:35 +0200 Subject: Questions about the Hermetic Java project In-Reply-To: <4bda696b-eb73-497c-ac9d-b128fc887514@oracle.com> References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> <4bda696b-eb73-497c-ac9d-b128fc887514@oracle.com> Message-ID: <2bcb09faef4c8e6bd1386c8a705ac243dbfe16ec.camel@redhat.com> Hi, On Tue, 2024-04-16 at 17:40 +0200, Magnus Ihse Bursie wrote: > On 2024-04-16 10:28, Volker Simonis wrote: > > > Hi, and sorry, I'm completely new to this discussion. > > > > I just wonder how `static-libs-image` is related to the > > `graal-builder-image` target which is available in mainline since JDK > > 11 and builds a static version of all the native libraries excluding > > the Hotspot ones? I cannot comment on the static-libs-image part, but graal-builder-image was my doing[1]. It's a vehicle to build an OpenJDK with static libs in lib/static which is sufficient to use as a from-source version build of OpenJDK to build Mandrel[2] or GraalVM, which is convenient. For graal-builder-image, the result must not include libjvm.a in lib/static of /graal-builder-jdk. So yes, the two are related. Thanks, Severin [1] https://bugs.openjdk.org/browse/JDK-8236921 [2] https://github.com/graalvm/mandrel > That's a very good question. You can also ask how it relates to > --enable-static-build from JDK-8136556 which has been in the JDK since > 9. :-) > > Or, to put in other words, we already have three different ways to > produce static libraries in the JDK, each of them created more or less > independently of the other. And now Hermetic Java wants to add a fourth. > > This is not a good situation. > > My intention here is to basically retire static-libs-image, > graal-builder-image and --enable-static-build, and replace them with a > more general solution that supports all these existing use cases, and > the new use case from the StaticLink.gmk file that the Hermetic Java > project wants to integrate. > > Unfortunately, this requires some delicate reverse-engineering of > specifications (i.e. "why the heck did they do it like this???") to > understand what the different use cases want to achieve. That is the > main driver behind my recent work in clearing out technical debt in the > native linking part of the build system. > > /Magnus > > > > > > @Magnus: will the new, static build you're working on also cover > > `graal-builder-image` or will it be independent of it? From a quick > > look it seems that currently both `static-libs-image` and > > `graal-builder-image` are handled in `make/StaticLibsImage.gmk` but > > I'm not sure if they use the same build functionality. I just wondered > > why `graal-builder-image` hasn't been mentioned in this thread but > > maybe it's something as obvious as that it already is the same > > functionality anway? > > > > Thank you and best regards, > > Volker > > > > On Tue, Apr 16, 2024 at 3:02?AM Jiangli Zhou wrote: > > > Magnus, thanks for the response. Please see comments inlined below. > > > > > > On Fri, Apr 12, 2024 at 4:52?AM Magnus Ihse Bursie > > > wrote: > > > > On 2024-04-02 21:16, Jiangli Zhou wrote: > > > > > > > > Hi Magnus, > > > > > > > > In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) discussed how to move forward contributing the static Java related changes (additional runtime fixes/enhancements on top of the existing static support in JDK) from https://github.com/openjdk/leyden/tree/hermetic-java-runtime?to JDK mainline. > > > > > > > > Just a bit more details/context below, which may be useful for others reading this thread. > > > > > > > > The https://github.com/openjdk/leyden/tree/hermetic-java-runtime?branch currently contains following for supporting hermetic Java (without the launcher work for runtime support): > > > > > > > > 1. Build change for linking the Java launcher (as bin/javastatic) with JDK/hotspot static libraries (.a), mainly in https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. The part for creating the complete sets of static libraries (including libjvm.a) has already been included in the mainline since last year. https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk?is in a very raw state and is intended to demonstrate the capability of building a static Java launcher. > > > > > > > > Indeed. It is nowhere near being able to be integrated. > > > > > > > The main purpose of StaticLink.gmk is to support the static-java-image > > > make target, which can be used to perform the actual static linking > > > step using libjvm.a and JDK static libraries. That currently doesn't > > > exist in the JDK mainline. Creating a "fully" statically linked Java > > > launcher is the first step (out of many) towards supporting > > > static/hermetic Java. > > > > > > As part of cleaning/refactoring/integrating for the static linking > > > step, we want to agree and decide/accept on the following: > > > > > > - Support the "fully" statically linked java launcher for testing and > > > demoing the capability of static JDK support, e.g. > > > ?? - Support running jtreg testing using the "fully" statically linked > > > Java launcher > > > ?? - Set up tests in github workflow to help detect any breaking > > > changes for static support, e.g. new symbol issues introduced by any > > > changes. There were some earlier discussions on this with Ron and Alan > > > during the zoom meetings. > > > - Which JDK native libraries to be statically linked with the new > > > launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, > > > libawt_xawt.a, etc from statically linked with the launcher. > > > - Do we want more than one statically linked launcher target, based on > > > the set of linked native libraries? > > > > > > Based on the decisions of the above, the launcher static linking part > > > would mostly be in a different shape when it's integrated into the > > > mainline. That's why I referred to StaticLink.gmk as in a "very raw" > > > state. > > > > > > Here is a high-level view of the state of things for static support: > > > > > > (I)? What we already have in the JDK mainline: > > > - Able to build a complete set of JDK/VM static libraries using > > > `static-libs-image` make target (necessary for supporting static JDK) > > > - Compilation for .o files are done separately for the static > > > libraries and dynamic library (ok for now) > > > > > > (II) What missing: > > > - Static linking step as mentioned above > > > > > > (III) What needs to be improved (require cleanups and refactoring, and > > > you mentioned some of those in your response as well): > > > - Support building both the static libraries and dynamic libraries > > > using the same set of .o files, instead of separately compiled .o > > > files. That helps improve build speed and reduce memory overhead for > > > building JDK. Your current refactoring work aims to help that. > > > - Clean up the usages of STATIC_BUILD macro. Most of the usages are in > > > test code. > > > - Other runtime fixes/enhancements in the leyden > > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime?branch > > > > > > I think most work mentioned in III has dependencies on II. We need a > > > workable base to be able to build the "fully" statically linked > > > launcher for building and testing the work mentioned in III, when > > > integrating any of those to the JDK mainline. The makefile refactoring > > > work can be done in parallel but does not need to be completed before > > > we add the static linking step in JDK mainline. > > > > > > > 2. Additional runtime fixes/enhancements on top of the existing static support in JDK, e.g. support further lookup dynamic native library if the built-in native library cannot be found. > > > > > > > > 3. Some initial (prototype) work on supporting hermetic JDK resource files in the jimage (JDK modules image). > > > > > > > > To move forward, one of the earliest items needed is to add the capability of building the fully statically linked Java launcher in JDK mainline. The other static Java runtime changes can be followed up after the launcher linking part, so they can be built and tested as individual PRs created for the JDK mainline. Magnus, you have expressed interest in helping get the launcher linking part (refactor from https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) into JDK mainline. What's your thought on prioritizing the launcher static linking part before other makefile clean ups for static libraries? > > > > > > > > Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. > > > Thanks! > > > > > > > I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? > > > Please see my comments above. > > > > > > > The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. > > > > > > > > The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. > > > > > > > > My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). > > > As of today, the leyden > > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime?branch > > > can build a "fully" statically linked Java launcher. The issue of > > > compiling the dynamic and static libraries .o files separately is not > > > a blocker. It's good to have it resolved at some point of time. > > > > > > > This, in turn, require several changes: > > > > > > > > 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > > > > > > > > 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > > > > > > > > This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > > > Thumbs up! That seems to be a good direction. Currently in the leyden > > > branch, it first looks up the unique > > > JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in > > > libraries, then search for the dynamic libraries using the > > > conventional naming when necessary. e.g.: > > > > > > https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a > > > https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 > > > > > > When spec supports JNI_OnLoad_ and etc. for dynamic > > > libraries, we may still need to support the conventional naming > > > without the <_lib_name> part for existing libraries out there. > > > > > > > And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. > > > > > > > > A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. > > > Thank you for taking this on! Potentially we could consider taking the > > > objcopy to localizing hotspot symbols on unix-like platforms, based on > > > https://github.com/openjdk/jdk/pull/17456?discussions. Additional > > > testing is still needed to verify the solution. > > > > > > > My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. > > > Most of the JDK resources are now supported as hermetic jimage > > > (lib/modules) bundled in the > > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime?branch. > > > The remaining sound.properties, ct.sym and .jfc files can be handled > > > later. Overally, that part of the work has confirmed the hermetic > > > jimage bundled solution is robust and helps resolve some of the > > > difficult start-up sequence issues observed when the hermetic resource > > > was implemented using JAR file based solution. > > > > > > It might be a good idea to follow up on the static linking discussion > > > in tomorrow's zoom meeting (hope you'll be able to join tomorrow). > > > > > > Thanks! > > > > > > Jiangli > > > > /Magnus > > > > > > > > > > > > > > > > Thanks! > > > > Jiangli > > > > > > > > On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou wrote: > > > > > On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou wrote: > > > > > > Hi Magnus, > > > > > > > > > > > > Thanks for looking into this from the build perspective. > > > > > > > > > > > > On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie > > > > > > wrote: > > > > > > > First some background for build-dev: I have spent some time looking at > > > > > > > the build implications of the Hermetic Java effort, which is part of > > > > > > > Project Leyden. A high-level overview is available here: > > > > > > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf?and the current source > > > > > > > code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. > > > > > > Some additional hermetic Java related references that are also useful: > > > > > > > > > > > > - https://bugs.openjdk.org/browse/JDK-8303796?is an umbrella bug that > > > > > > links to the issues for resolving static linking issues so far > > > > > > - https://github.com/openjdk/jdk21/pull/26?is the enhancement for > > > > > > building the complete set of static libraries in JDK/VM, particularly > > > > > > including libjvm.a > > > > > > > > > > > > > Hermetic Java faces several challenges, but the part that is relevant > > > > > > > for the build system is the ability to create static libraries. We've > > > > > > > had this functionality (in three different ways...) for some time, but > > > > > > > it is rather badly implemented. > > > > > > > > > > > > > > As a result of my investigations, I have a bunch of questions. :-) I > > > > > > > have gotten some answers in private discussion, but for the sake of > > > > > > > transparency I will repeat them here, to foster an open dialogue. > > > > > > > > > > > > > > 1. Am I correct in understanding that the ultimate goal of this exercise > > > > > > > is to be able to have jmods which include static libraries (*.a) of the > > > > > > > native code which the module uses, and that the user can then run a > > > > > > > special jlink command to have this linked into a single executable > > > > > > > binary (which also bundles the *.class files and any additional > > > > > > > resources needed)? > > > > > > > > > > > > > > 2. If so, is the idea to create special kinds of static jmods, like > > > > > > > java.base-static.jmod, that contains *.a files instead of lib*.so files? > > > > > > > Or is the idea that the normal jmod should contain both? > > > > > > > > > > > > > > 3. Linking .o and .a files into an executable is a formidable task. Is > > > > > > > the intention to have jlink call a system-provided ld, or to bundle ld > > > > > > > with jlink, or to reimplement this functionality in Java? > > > > > > I have a similar view as Alan responded in your other email thread. > > > > > > Things are still in the early stage for the general solution. > > > > > > > > > > > > In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime > > > > > > branch, when configuring JDK with --with-static-java=yes, the JDK > > > > > > binary contains the following extra artifacts: > > > > > > > > > > > > - static-libs/*.a: The complete set of JDK/VM static libraries > > > > > > - jdk/bin/javastatic: A demo Java launcher fully statically linked > > > > > > with the selected JDK .a libraries (e.g. it currently statically link > > > > > > with the headless) and libjvm.a. It's the standard Java launcher > > > > > > without additional work for hermetic Java. > > > > > > > > > > > > In our prototype for hermetic Java, we build the hermetic executable > > > > > > image (a single image) from the following input (see description on > > > > > > singlejar packaging tool in > > > > > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf): > > > > > > > > > > > > - A customized launcher (with additional work for hermetic) executable > > > > > > fully statically linked with JDK/VM static libraries (.a files), > > > > > > application natives and dependencies (e.g. in .a static libraries) > > > > > > - JDK lib/modules, JDK resource files > > > > > > - Application classes and resource files > > > > > > > > > > > > Including a JDK library .a into the corresponding .jmod would require > > > > > > extracting the .a for linking with the executable. In some systems > > > > > > that may cause memory overhead due to the extracted copy of the .a > > > > > > files. I think we should consider the memory overhead issue. > > > > > > > > > > > > One possibility (as Alan described in his response) is for jlink to > > > > > > invoke the ld on the build system. jlink could pass the needed JDK > > > > > > static libraries and libjvm.a (provided as part of the JDK binary) to > > > > > > ld based on the modules required for the application. > > > > > > > > > > > I gave a bit more thoughts on this one. For jlink to trigger ld, it > > > > > would need to know the complete linker options and inputs. Those > > > > > include options and inputs related to the application part as well. In > > > > > some usages, it might be easier to handle native linking separately > > > > > and pass the linker output, the executable to jlink directly. Maybe we > > > > > could consider supporting different modes for various usages > > > > > requirements, from static libraries and native linking point of view: > > > > > > > > > > Mode #1 > > > > > Support .jmod packaged natives static libraries, for both JDK/VM .a > > > > > and application natives and dependencies. If the inputs to jlink > > > > > include .jmods, jlink can extract the .a libraries and pass the > > > > > information to ld to link the executable. > > > > > > > > > > Mode #2 > > > > > Support separate .a as jlink input. Jlink could pass the path > > > > > information to the .a libraries and other linker options to ld to > > > > > create the executable. > > > > > > > > > > For both mode #1 and #2, jlink would then use the linker output > > > > > executable to create the final hermetic image. > > > > > > > > > > Mode #3 > > > > > Support a fully linked executable as a jlink input. When a linked > > > > > executable is given to jlink, it can process it directly with other > > > > > JDK data/files to create the final image, without native linking step. > > > > > > > > > > Any other thoughts and considerations? > > > > > > > > > > Best, > > > > > Jiangli > > > > > > > > > > > > 4. Is the intention is to allow users to create their own jmods with > > > > > > > static libraries, and have these linked in as well? This seems to be the > > > > > > > case. > > > > > > An alternative with less memory overhead could be using application > > > > > > modular JAR and separate .a as the input for jlink. > > > > > > > > > > > > > If that is so, then there will always be the risk for name > > > > > > > collisions, and we can only minimize the risk by making sure any global > > > > > > > names are as unique as possible. > > > > > > Part of the current effort includes resolving the discovered symbol > > > > > > collision issues with static linking. Will respond to your other email > > > > > > on the symbol issue separately later. > > > > > > > > > > > > > 5. The original implementation of static builds in the JDK, created for > > > > > > > the Mobile project, used a configure flag, --enable-static-builds, to > > > > > > > change the entire behavior of the build system to only produce *.a files > > > > > > > instead of lib*.so. In contrast, the current system is using a special > > > > > > > target instead. > > > > > > I think we would need both configure flag and special target for the > > > > > > static builds. > > > > > > > > > > > > > In my eyes, this is a much worse solution. Apart from > > > > > > > the conceptual principle (if the build should generate static or dynamic > > > > > > > libraries is definitely a property of what a "configuration" means), > > > > > > > this makes it much harder to implement efficiently, since we cannot make > > > > > > > changes in NativeCompilation.gmk, where they are needed. > > > > > > For the potential objcopy work to resolve symbol issues, we can add > > > > > > that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. We > > > > > > have an internal prototype (not included in > > > > > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime?yet) done > > > > > > by one of colleagues for localizing symbols in libfreetype using > > > > > > objcopy. > > > > > > > > > > > > > That was not as much a question as a statement. ? But here is the > > > > > > > question: Do you think it would be reasonable to restore the old > > > > > > > behavior but with the new methods, so that we don't use special targets, > > > > > > > but instead tells configure to generate static libraries? I'm thinking > > > > > > > we should have a flag like "--with-library-type=" that can have values > > > > > > > "dynamic" (which is default), "static" or "both". > > > > > > If we want to also build a fully statically linked launcher, maybe > > > > > > --with-static-java? Being able to configure either dynamic, static or > > > > > > both as you suggested also seems to be a good idea. > > > > > > > > > > > > > I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files > > > > > > > into a single jmod file (see question 2 above), then it definitely is. > > > > > > > In general, the cost of producing two kinds of libraries are quite > > > > > > > small, compared to the cost of compiling the source code to object files. > > > > > > Completely agree. It would be good to avoid recompiling the .o file > > > > > > for static and dynamic builds. As proposed in > > > > > > https://bugs.openjdk.org/browse/JDK-8303796: > > > > > > > > > > > > It's beneficial to be able to build both .so and .a from the same set > > > > > > of .o files. That would involve some changes to handle the dynamic JDK > > > > > > and static JDK difference at runtime, instead of relying on the > > > > > > STATIC_BUILD macro. > > > > > > > > > > > > > Finally, I have looked at how to manipulate symbol visibility. There > > > > > > > seems many ways forward, so I feel confident that we can find a good > > > > > > > solution. > > > > > > > > > > > > > > One way forward is to use objcopy to manipulate symbol status > > > > > > > (global/local). There is an option --localize-symbol in objcopy, that > > > > > > > has been available in objcopy since at least 2.15, which was released > > > > > > > 2004, so it should be safe to use. But ideally we should avoid using > > > > > > > objcopy and do this as part of the linking process. This should be > > > > > > > possible to do, given that we make changes in NativeCompilation.gmk -- > > > > > > > see question 5 above. > > > > > > > > > > > > > > As a fallback, it is also possible to rename symbols, either piecewise > > > > > > > or wholesale, using objcopy. There are many ways to do this, using > > > > > > > --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this > > > > > > > takes a file with a list of symbols). Thus we can always introduce a > > > > > > > "post factum namespace" by renaming symbols. > > > > > > Renaming or redefining the symbol at build time could cause confusions > > > > > > with debugging. That's a concern raised in > > > > > > https://github.com/openjdk/jdk/pull/17456?discussions. > > > > > > > > > > > > Additionally, redefining symbols using tools like objcopy may not > > > > > > handle member names referenced in string literals. For example, in > > > > > > https://github.com/openjdk/jdk/pull/17456?additional changes are > > > > > > needed in assembling and SA to reflect the symbol change. > > > > > > > > > > > > > So in the end, I think it will be fully possible to produce .a files > > > > > > > that only has global symbols for the functions that are part of the API > > > > > > > exposed by that library, and have all other symbols local, and make this > > > > > > > is in a way that is consistent with the rest of the build system. > > > > > > > > > > > > > > Finally, a note on Hotspot. Due to debugging reasons, we export > > > > > > > basically all symbols in hotspot as global. This is not reasonable to do > > > > > > > for a static build. The effect of not exporting those symbols will be > > > > > > > that SA will not function to 100%. On the other hand, I have no idea if > > > > > > > SA works at all with a static build. Have you tested this? Is this part > > > > > > > of the plan to support, or will it be officially dropped for Hermetic Java? > > > > > > We have done some testing with jtreg SA related tests for the fully > > > > > > statically linked `javastatic`. > > > > > > > > > > > > If we use objcopy to localize symbols in hotspot, it's not yet clear > > > > > > what's the impact on SA. We could do some tests. The other question > > > > > > that I raised is the supported gcc versions (for partial linking) > > > > > > related to the solution. > > > > > > > > > > > > Best, > > > > > > Jiangli > > > > > > > > > > > > > /Magnus > > > > > > > > From volker.simonis at gmail.com Tue Apr 16 17:03:35 2024 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 16 Apr 2024 19:03:35 +0200 Subject: Questions about the Hermetic Java project In-Reply-To: <4bda696b-eb73-497c-ac9d-b128fc887514@oracle.com> References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> <4bda696b-eb73-497c-ac9d-b128fc887514@oracle.com> Message-ID: On Tue, Apr 16, 2024 at 5:40?PM Magnus Ihse Bursie wrote: > > On 2024-04-16 10:28, Volker Simonis wrote: > > > Hi, and sorry, I'm completely new to this discussion. > > > > I just wonder how `static-libs-image` is related to the > > `graal-builder-image` target which is available in mainline since JDK > > 11 and builds a static version of all the native libraries excluding > > the Hotspot ones? > > That's a very good question. You can also ask how it relates to > --enable-static-build from JDK-8136556 which has been in the JDK since > 9. :-) > > Or, to put in other words, we already have three different ways to > produce static libraries in the JDK, each of them created more or less > independently of the other. And now Hermetic Java wants to add a fourth. > > This is not a good situation. > > My intention here is to basically retire static-libs-image, > graal-builder-image and --enable-static-build, and replace them with a > more general solution that supports all these existing use cases, and > the new use case from the StaticLink.gmk file that the Hermetic Java > project wants to integrate. > > Unfortunately, this requires some delicate reverse-engineering of > specifications (i.e. "why the heck did they do it like this???") to > understand what the different use cases want to achieve. That is the > main driver behind my recent work in clearing out technical debt in the > native linking part of the build system. Thanks for the more detailed background information Magnus. Consolidating the different possibilities for creating static libraries definitely looks like the right thing to do. I've added Doug from the GraalVM team on CC just in case he's not already aware of this refactoring. Regards, Volker > > /Magnus > > > > > > @Magnus: will the new, static build you're working on also cover > > `graal-builder-image` or will it be independent of it? From a quick > > look it seems that currently both `static-libs-image` and > > `graal-builder-image` are handled in `make/StaticLibsImage.gmk` but > > I'm not sure if they use the same build functionality. I just wondered > > why `graal-builder-image` hasn't been mentioned in this thread but > > maybe it's something as obvious as that it already is the same > > functionality anway? > > > > Thank you and best regards, > > Volker > > > > On Tue, Apr 16, 2024 at 3:02?AM Jiangli Zhou wrote: > >> Magnus, thanks for the response. Please see comments inlined below. > >> > >> On Fri, Apr 12, 2024 at 4:52?AM Magnus Ihse Bursie > >> wrote: > >>> On 2024-04-02 21:16, Jiangli Zhou wrote: > >>> > >>> Hi Magnus, > >>> > >>> In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) discussed how to move forward contributing the static Java related changes (additional runtime fixes/enhancements on top of the existing static support in JDK) from https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK mainline. > >>> > >>> Just a bit more details/context below, which may be useful for others reading this thread. > >>> > >>> The https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch currently contains following for supporting hermetic Java (without the launcher work for runtime support): > >>> > >>> 1. Build change for linking the Java launcher (as bin/javastatic) with JDK/hotspot static libraries (.a), mainly in https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. The part for creating the complete sets of static libraries (including libjvm.a) has already been included in the mainline since last year. https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk is in a very raw state and is intended to demonstrate the capability of building a static Java launcher. > >>> > >>> Indeed. It is nowhere near being able to be integrated. > >>> > >> The main purpose of StaticLink.gmk is to support the static-java-image > >> make target, which can be used to perform the actual static linking > >> step using libjvm.a and JDK static libraries. That currently doesn't > >> exist in the JDK mainline. Creating a "fully" statically linked Java > >> launcher is the first step (out of many) towards supporting > >> static/hermetic Java. > >> > >> As part of cleaning/refactoring/integrating for the static linking > >> step, we want to agree and decide/accept on the following: > >> > >> - Support the "fully" statically linked java launcher for testing and > >> demoing the capability of static JDK support, e.g. > >> - Support running jtreg testing using the "fully" statically linked > >> Java launcher > >> - Set up tests in github workflow to help detect any breaking > >> changes for static support, e.g. new symbol issues introduced by any > >> changes. There were some earlier discussions on this with Ron and Alan > >> during the zoom meetings. > >> - Which JDK native libraries to be statically linked with the new > >> launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, > >> libawt_xawt.a, etc from statically linked with the launcher. > >> - Do we want more than one statically linked launcher target, based on > >> the set of linked native libraries? > >> > >> Based on the decisions of the above, the launcher static linking part > >> would mostly be in a different shape when it's integrated into the > >> mainline. That's why I referred to StaticLink.gmk as in a "very raw" > >> state. > >> > >> Here is a high-level view of the state of things for static support: > >> > >> (I) What we already have in the JDK mainline: > >> - Able to build a complete set of JDK/VM static libraries using > >> `static-libs-image` make target (necessary for supporting static JDK) > >> - Compilation for .o files are done separately for the static > >> libraries and dynamic library (ok for now) > >> > >> (II) What missing: > >> - Static linking step as mentioned above > >> > >> (III) What needs to be improved (require cleanups and refactoring, and > >> you mentioned some of those in your response as well): > >> - Support building both the static libraries and dynamic libraries > >> using the same set of .o files, instead of separately compiled .o > >> files. That helps improve build speed and reduce memory overhead for > >> building JDK. Your current refactoring work aims to help that. > >> - Clean up the usages of STATIC_BUILD macro. Most of the usages are in > >> test code. > >> - Other runtime fixes/enhancements in the leyden > >> https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > >> > >> I think most work mentioned in III has dependencies on II. We need a > >> workable base to be able to build the "fully" statically linked > >> launcher for building and testing the work mentioned in III, when > >> integrating any of those to the JDK mainline. The makefile refactoring > >> work can be done in parallel but does not need to be completed before > >> we add the static linking step in JDK mainline. > >> > >>> 2. Additional runtime fixes/enhancements on top of the existing static support in JDK, e.g. support further lookup dynamic native library if the built-in native library cannot be found. > >>> > >>> 3. Some initial (prototype) work on supporting hermetic JDK resource files in the jimage (JDK modules image). > >>> > >>> To move forward, one of the earliest items needed is to add the capability of building the fully statically linked Java launcher in JDK mainline. The other static Java runtime changes can be followed up after the launcher linking part, so they can be built and tested as individual PRs created for the JDK mainline. Magnus, you have expressed interest in helping get the launcher linking part (refactor from https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) into JDK mainline. What's your thought on prioritizing the launcher static linking part before other makefile clean ups for static libraries? > >>> > >>> Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. > >> Thanks! > >> > >>> I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? > >> Please see my comments above. > >> > >>> The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. > >>> > >>> The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. > >>> > >>> My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). > >> As of today, the leyden > >> https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > >> can build a "fully" statically linked Java launcher. The issue of > >> compiling the dynamic and static libraries .o files separately is not > >> a blocker. It's good to have it resolved at some point of time. > >> > >>> This, in turn, require several changes: > >>> > >>> 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > >>> > >>> 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > >>> > >>> This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > >> Thumbs up! That seems to be a good direction. Currently in the leyden > >> branch, it first looks up the unique > >> JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in > >> libraries, then search for the dynamic libraries using the > >> conventional naming when necessary. e.g.: > >> > >> https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a > >> https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 > >> > >> When spec supports JNI_OnLoad_ and etc. for dynamic > >> libraries, we may still need to support the conventional naming > >> without the <_lib_name> part for existing libraries out there. > >> > >>> And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. > >>> > >>> A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. > >> Thank you for taking this on! Potentially we could consider taking the > >> objcopy to localizing hotspot symbols on unix-like platforms, based on > >> https://github.com/openjdk/jdk/pull/17456 discussions. Additional > >> testing is still needed to verify the solution. > >> > >>> My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. > >> Most of the JDK resources are now supported as hermetic jimage > >> (lib/modules) bundled in the > >> https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch. > >> The remaining sound.properties, ct.sym and .jfc files can be handled > >> later. Overally, that part of the work has confirmed the hermetic > >> jimage bundled solution is robust and helps resolve some of the > >> difficult start-up sequence issues observed when the hermetic resource > >> was implemented using JAR file based solution. > >> > >> It might be a good idea to follow up on the static linking discussion > >> in tomorrow's zoom meeting (hope you'll be able to join tomorrow). > >> > >> Thanks! > >> > >> Jiangli > >>> /Magnus > >>> > >>> > >>> > >>> Thanks! > >>> Jiangli > >>> > >>> On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou wrote: > >>>> On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou wrote: > >>>>> Hi Magnus, > >>>>> > >>>>> Thanks for looking into this from the build perspective. > >>>>> > >>>>> On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie > >>>>> wrote: > >>>>>> First some background for build-dev: I have spent some time looking at > >>>>>> the build implications of the Hermetic Java effort, which is part of > >>>>>> Project Leyden. A high-level overview is available here: > >>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current source > >>>>>> code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. > >>>>> Some additional hermetic Java related references that are also useful: > >>>>> > >>>>> - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug that > >>>>> links to the issues for resolving static linking issues so far > >>>>> - https://github.com/openjdk/jdk21/pull/26 is the enhancement for > >>>>> building the complete set of static libraries in JDK/VM, particularly > >>>>> including libjvm.a > >>>>> > >>>>>> Hermetic Java faces several challenges, but the part that is relevant > >>>>>> for the build system is the ability to create static libraries. We've > >>>>>> had this functionality (in three different ways...) for some time, but > >>>>>> it is rather badly implemented. > >>>>>> > >>>>>> As a result of my investigations, I have a bunch of questions. :-) I > >>>>>> have gotten some answers in private discussion, but for the sake of > >>>>>> transparency I will repeat them here, to foster an open dialogue. > >>>>>> > >>>>>> 1. Am I correct in understanding that the ultimate goal of this exercise > >>>>>> is to be able to have jmods which include static libraries (*.a) of the > >>>>>> native code which the module uses, and that the user can then run a > >>>>>> special jlink command to have this linked into a single executable > >>>>>> binary (which also bundles the *.class files and any additional > >>>>>> resources needed)? > >>>>>> > >>>>>> 2. If so, is the idea to create special kinds of static jmods, like > >>>>>> java.base-static.jmod, that contains *.a files instead of lib*.so files? > >>>>>> Or is the idea that the normal jmod should contain both? > >>>>>> > >>>>>> 3. Linking .o and .a files into an executable is a formidable task. Is > >>>>>> the intention to have jlink call a system-provided ld, or to bundle ld > >>>>>> with jlink, or to reimplement this functionality in Java? > >>>>> I have a similar view as Alan responded in your other email thread. > >>>>> Things are still in the early stage for the general solution. > >>>>> > >>>>> In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime > >>>>> branch, when configuring JDK with --with-static-java=yes, the JDK > >>>>> binary contains the following extra artifacts: > >>>>> > >>>>> - static-libs/*.a: The complete set of JDK/VM static libraries > >>>>> - jdk/bin/javastatic: A demo Java launcher fully statically linked > >>>>> with the selected JDK .a libraries (e.g. it currently statically link > >>>>> with the headless) and libjvm.a. It's the standard Java launcher > >>>>> without additional work for hermetic Java. > >>>>> > >>>>> In our prototype for hermetic Java, we build the hermetic executable > >>>>> image (a single image) from the following input (see description on > >>>>> singlejar packaging tool in > >>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf): > >>>>> > >>>>> - A customized launcher (with additional work for hermetic) executable > >>>>> fully statically linked with JDK/VM static libraries (.a files), > >>>>> application natives and dependencies (e.g. in .a static libraries) > >>>>> - JDK lib/modules, JDK resource files > >>>>> - Application classes and resource files > >>>>> > >>>>> Including a JDK library .a into the corresponding .jmod would require > >>>>> extracting the .a for linking with the executable. In some systems > >>>>> that may cause memory overhead due to the extracted copy of the .a > >>>>> files. I think we should consider the memory overhead issue. > >>>>> > >>>>> One possibility (as Alan described in his response) is for jlink to > >>>>> invoke the ld on the build system. jlink could pass the needed JDK > >>>>> static libraries and libjvm.a (provided as part of the JDK binary) to > >>>>> ld based on the modules required for the application. > >>>>> > >>>> I gave a bit more thoughts on this one. For jlink to trigger ld, it > >>>> would need to know the complete linker options and inputs. Those > >>>> include options and inputs related to the application part as well. In > >>>> some usages, it might be easier to handle native linking separately > >>>> and pass the linker output, the executable to jlink directly. Maybe we > >>>> could consider supporting different modes for various usages > >>>> requirements, from static libraries and native linking point of view: > >>>> > >>>> Mode #1 > >>>> Support .jmod packaged natives static libraries, for both JDK/VM .a > >>>> and application natives and dependencies. If the inputs to jlink > >>>> include .jmods, jlink can extract the .a libraries and pass the > >>>> information to ld to link the executable. > >>>> > >>>> Mode #2 > >>>> Support separate .a as jlink input. Jlink could pass the path > >>>> information to the .a libraries and other linker options to ld to > >>>> create the executable. > >>>> > >>>> For both mode #1 and #2, jlink would then use the linker output > >>>> executable to create the final hermetic image. > >>>> > >>>> Mode #3 > >>>> Support a fully linked executable as a jlink input. When a linked > >>>> executable is given to jlink, it can process it directly with other > >>>> JDK data/files to create the final image, without native linking step. > >>>> > >>>> Any other thoughts and considerations? > >>>> > >>>> Best, > >>>> Jiangli > >>>> > >>>>>> 4. Is the intention is to allow users to create their own jmods with > >>>>>> static libraries, and have these linked in as well? This seems to be the > >>>>>> case. > >>>>> An alternative with less memory overhead could be using application > >>>>> modular JAR and separate .a as the input for jlink. > >>>>> > >>>>>> If that is so, then there will always be the risk for name > >>>>>> collisions, and we can only minimize the risk by making sure any global > >>>>>> names are as unique as possible. > >>>>> Part of the current effort includes resolving the discovered symbol > >>>>> collision issues with static linking. Will respond to your other email > >>>>> on the symbol issue separately later. > >>>>> > >>>>>> 5. The original implementation of static builds in the JDK, created for > >>>>>> the Mobile project, used a configure flag, --enable-static-builds, to > >>>>>> change the entire behavior of the build system to only produce *.a files > >>>>>> instead of lib*.so. In contrast, the current system is using a special > >>>>>> target instead. > >>>>> I think we would need both configure flag and special target for the > >>>>> static builds. > >>>>> > >>>>>> In my eyes, this is a much worse solution. Apart from > >>>>>> the conceptual principle (if the build should generate static or dynamic > >>>>>> libraries is definitely a property of what a "configuration" means), > >>>>>> this makes it much harder to implement efficiently, since we cannot make > >>>>>> changes in NativeCompilation.gmk, where they are needed. > >>>>> For the potential objcopy work to resolve symbol issues, we can add > >>>>> that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. We > >>>>> have an internal prototype (not included in > >>>>> https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) done > >>>>> by one of colleagues for localizing symbols in libfreetype using > >>>>> objcopy. > >>>>> > >>>>>> That was not as much a question as a statement. ? But here is the > >>>>>> question: Do you think it would be reasonable to restore the old > >>>>>> behavior but with the new methods, so that we don't use special targets, > >>>>>> but instead tells configure to generate static libraries? I'm thinking > >>>>>> we should have a flag like "--with-library-type=" that can have values > >>>>>> "dynamic" (which is default), "static" or "both". > >>>>> If we want to also build a fully statically linked launcher, maybe > >>>>> --with-static-java? Being able to configure either dynamic, static or > >>>>> both as you suggested also seems to be a good idea. > >>>>> > >>>>>> I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files > >>>>>> into a single jmod file (see question 2 above), then it definitely is. > >>>>>> In general, the cost of producing two kinds of libraries are quite > >>>>>> small, compared to the cost of compiling the source code to object files. > >>>>> Completely agree. It would be good to avoid recompiling the .o file > >>>>> for static and dynamic builds. As proposed in > >>>>> https://bugs.openjdk.org/browse/JDK-8303796: > >>>>> > >>>>> It's beneficial to be able to build both .so and .a from the same set > >>>>> of .o files. That would involve some changes to handle the dynamic JDK > >>>>> and static JDK difference at runtime, instead of relying on the > >>>>> STATIC_BUILD macro. > >>>>> > >>>>>> Finally, I have looked at how to manipulate symbol visibility. There > >>>>>> seems many ways forward, so I feel confident that we can find a good > >>>>>> solution. > >>>>>> > >>>>>> One way forward is to use objcopy to manipulate symbol status > >>>>>> (global/local). There is an option --localize-symbol in objcopy, that > >>>>>> has been available in objcopy since at least 2.15, which was released > >>>>>> 2004, so it should be safe to use. But ideally we should avoid using > >>>>>> objcopy and do this as part of the linking process. This should be > >>>>>> possible to do, given that we make changes in NativeCompilation.gmk -- > >>>>>> see question 5 above. > >>>>>> > >>>>>> As a fallback, it is also possible to rename symbols, either piecewise > >>>>>> or wholesale, using objcopy. There are many ways to do this, using > >>>>>> --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this > >>>>>> takes a file with a list of symbols). Thus we can always introduce a > >>>>>> "post factum namespace" by renaming symbols. > >>>>> Renaming or redefining the symbol at build time could cause confusions > >>>>> with debugging. That's a concern raised in > >>>>> https://github.com/openjdk/jdk/pull/17456 discussions. > >>>>> > >>>>> Additionally, redefining symbols using tools like objcopy may not > >>>>> handle member names referenced in string literals. For example, in > >>>>> https://github.com/openjdk/jdk/pull/17456 additional changes are > >>>>> needed in assembling and SA to reflect the symbol change. > >>>>> > >>>>>> So in the end, I think it will be fully possible to produce .a files > >>>>>> that only has global symbols for the functions that are part of the API > >>>>>> exposed by that library, and have all other symbols local, and make this > >>>>>> is in a way that is consistent with the rest of the build system. > >>>>>> > >>>>>> Finally, a note on Hotspot. Due to debugging reasons, we export > >>>>>> basically all symbols in hotspot as global. This is not reasonable to do > >>>>>> for a static build. The effect of not exporting those symbols will be > >>>>>> that SA will not function to 100%. On the other hand, I have no idea if > >>>>>> SA works at all with a static build. Have you tested this? Is this part > >>>>>> of the plan to support, or will it be officially dropped for Hermetic Java? > >>>>> We have done some testing with jtreg SA related tests for the fully > >>>>> statically linked `javastatic`. > >>>>> > >>>>> If we use objcopy to localize symbols in hotspot, it's not yet clear > >>>>> what's the impact on SA. We could do some tests. The other question > >>>>> that I raised is the supported gcc versions (for partial linking) > >>>>> related to the solution. > >>>>> > >>>>> Best, > >>>>> Jiangli > >>>>> > >>>>>> /Magnus > >>>>>> From mchung at openjdk.org Tue Apr 16 17:48:06 2024 From: mchung at openjdk.org (Mandy Chung) Date: Tue, 16 Apr 2024 17:48:06 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: Message-ID: On Wed, 3 Apr 2024 22:31:39 GMT, Mandy Chung wrote: >> Severin Gehwolf has updated the pull request incrementally with one additional commit since the last revision: >> >> Move CreateLinkableRuntimePlugin to build folder >> >> Keep runtime link supporting classes in package >> jdk.tools.jlink.internal.runtimelink > > Thanks for the investigation w.r.t. extending jimage. It does not seem worth the effort in pursuing the support of adding resources to an existing jimage file. To me, putting the diff data under `lib` directory as a private file is a simpler and acceptable solution. It allows the second jlink invocation to avoid the plugin pipeline if the per-module metadata is generated in normal jlink invocation. > > (An observation: The current implementation requires the diffs to be generated as a plugin. For this approach, it may be possible to implement with a separate main class that calls JLink API and generates the diffs.) > What would you (and @mlchung) think of having jlink generate lib/runtime-image-link.delta as a step between the pipeline and image generation? I have similar thought before I went on vacation last week and also think that this would allow this be further simplified to one single jlink invocation to produce a linkable image. In `ImageFileCreator::generateJImage`, after the plugin pipeline, it already adds a step to record the per-module meta information as additional resources. The delta file can be generated after that comparing the original resource pool with the transformed resource pool. I believe `allContent` has the original content. If it is a single jlink step to produce a linkable image (generate the delta file after the plugin pipeline before generating the image), we might be able to get rid of `--ignore-modified-runtime` by storing a copy of conf files in the delta file and the original conf files will be used when the linkable image is used to create a new runtime image. [An observation from the above, it might be possible to generate the delta file and insert in the jimage :-) This might be a bonus. ] ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2059631414 From vromero at openjdk.org Tue Apr 16 18:05:06 2024 From: vromero at openjdk.org (Vicente Romero) Date: Tue, 16 Apr 2024 18:05:06 GMT Subject: RFR: 8317376: Minor improvements to the 'this' escape analyzer [v4] In-Reply-To: References: Message-ID: <5H9FriMwHXEi75SibY_zoPVu8LcLFmLgVrHKGL58Vh0=.cc3bbbc7-669b-471c-b4d2-ac54cf6df44c@github.com> On Tue, 9 Apr 2024 15:36:27 GMT, Archie Cobbs wrote: >> Please review several fixes and improvements to the `this-escape` lint warning analyzer. >> >> The goal here is to apply some relatively simple logical fixes that improve the precision and accuracy of the analyzer, and capture the remaining low-hanging fruit so we can consider the analyzer relatively complete with respect to what's feasible with its current design. >> >> Although the changes are small from a logical point of view, they generate a fairly large patch due to impact of refactoring (sorry!). Most of the patch derives from the first two changes listed below. >> >> The changes are summarized here: >> >> #### 1. Generalize how we categorize references >> >> The `Ref` class hierarchy models the various ways in which, at any point during the execution of a constructor or some other method/constructor that it invokes, there can be live references to the original object under construction lying around. We then look for places where one of these `Ref`'s might be passed to a subclass method. In other words, the analyzer keeps track of these references and watches what happens to them as the code executes so it can catch them trying to "escape". >> >> Previously the `Ref` categories were: >> * `ThisRef` - The current instance of the (non-static) method or constructor being analyzed >> * `OuterRef` - The current outer instance of the (non-static) method or constructor being analyzed >> * `VarRef` - A local variable or method parameter currently in scope >> * `ExprRef` - An object reference sitting on top of the Java execution stack >> * `YieldRef` - The current switch expression's yield value(s) >> * `ReturnRef` - The current method's return value(s) >> >> For each of those types, we further classified the "indirection" of the reference, i.e., whether the reference was direct (from the thing itself) or indirect (from something the thing referenced). >> >> The problem with that hierarchy is that we could only track outer instance references that happened to be associated with the current instance. So we might know that `this` had an outer instance reference, but if we said `var x = this` we wouldn't know that `x` had an outer instance reference. >> >> In other words, we should be treating "via an outer instance" as just another flavor of indirection along with "direct" and "indirect". >> >> As a result, with this patch the `OuterRef` class goes away and a new `Indirection` enum has been created, with values `DIRECT`, `INDIRECT`, and `OUTER`. >> >> #### 2. Track the types... > > Archie Cobbs has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Merge branch 'master' into JDK-8317376 > - Merge branch 'master' into JDK-8317376 > - Merge branch 'master' into JDK-8317376 > - Merge branch 'master' into JDK-8317376 > - Merge branch 'master' into JDK-8317376 > - Javadoc++ > - Merge branch 'master' into JDK-8317376 > - Several improvements to the 'this' escape analyzer. > > - Track direct, indirect, and outer references for all Ref types. > - Keep type information about all references to improve tracking precision. > - Track enhanced for() invocations of iterator(), hasNext(), and next(). > - Don't report an escape of a non-public outer instances as a leak. > - Fix omitted tracking of references from newly instantiated instances. > - Fix omitted tracking of leaks via lambda return values. > - Remove unneccesary suppressions of this-escape lint warning. src/jdk.compiler/share/classes/com/sun/tools/javac/comp/ThisEscapeAnalyzer.java line 1343: > 1341: * but also NOT an enclosing outer class of 'currentClass'. > 1342: */ > 1343: private boolean isExplicitThisReference(Types types, Type.ClassType currentClass, JCFieldAccess select) { suggestion: why not moving this method to TreeInfo already in preparation for the changes coming with flexible constructor bodies? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16208#discussion_r1567757547 From prr at openjdk.org Tue Apr 16 18:09:00 2024 From: prr at openjdk.org (Phil Race) Date: Tue, 16 Apr 2024 18:09:00 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk [v3] In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 10:03:27 GMT, Magnus Ihse Bursie wrote: >> The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. >> >> I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). > > Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: > > - Make the split based on the name "awt" instead, and document the reason > - Rename 2dLibraries.gmk to ClientLibraries.gmk I still find this artificial, arbitrary, inconsistent, and even wrong. The name "awt" in a library isn't a good way to divide it, but I don't have an alternative proposal because there is no good way to divide it. And I just don't agree with the premise. Or at least it is far outweighed by everything else. I see it just sowing confusion and having to change 2 files etc. So I don't support this change in any variation. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2059664922 From vromero at openjdk.org Tue Apr 16 18:17:42 2024 From: vromero at openjdk.org (Vicente Romero) Date: Tue, 16 Apr 2024 18:17:42 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 In-Reply-To: References: Message-ID: <8gjzk6jIhtLXgFgNO4udYRJaCCZkto2IE9ETZYZeSig=.04b09b83-0299-440f-bffa-b6da681fb462@github.com> On Mon, 15 Apr 2024 19:01:08 GMT, Joe Darcy wrote: > Get JDK 24 underway. lgtm ------------- Marked as reviewed by vromero (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18787#pullrequestreview-2004323374 From magnus.ihse.bursie at oracle.com Tue Apr 16 18:27:08 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 20:27:08 +0200 Subject: Hermetic Java project meeting Message-ID: This is my summary of what was said in today's meeting: * Jiangli reported that her team had done extensive testing and not seen any problems, both with just the static launcher as generated by the leyden branch, and with bundled user applications using the One Jar (?) tool. They tested JDK tier 1, and an suite of Google's internal tests. When testing JTReg tests with native libraries, these were dynamically loaded. * Alan asked about Hotspot JTReg tests that launched "java". Jiangli reported that they had not seen any problems, but my understanding was that there was some confusion if any such tests were actually run. I think this is something that will need further attention, but if someone said they would look into it, I missed it. ?* Jiangli will get numbers on how much time is added to the GHA testing if we add building and linking of static libraries, without fixing so we can compile to a single set of object files. * We did not fully come to a conclusion if compiling on a single set of object files for static and dynamic linking was a hard requirement or not, but at a minimum it is a desirable goal. (My personal opinion is that is a hard requirement if the GHA build times are seriously affected otherwise.) There are basically two problems prohibiting single object file compilation: 1) using dynamic checks instead of #ifdefs for code that differs between static and dynamic. 2) Handling the difference between JNI_OnLoad (as required for dynamic libraries) and JNI_OnLoad_ (as required for static libraries). * The leyden branch has basically solved both these problems. The first one could more or less be integrated already (given perhaps some discussion on exactly *how* the JDK should discover in runtime if it is static or dynamic), but the latter requires a spec change to be integrated. * I think everyone agreed that moving on with a spec change was a good idea, regardless of if this is blocker or not, but I don't recall that there were any concrete next steps decided. Ron and Alan said that we do spec changes all the time so it will not add as much bureaucracy as one might fear. * Regarding which native libraries to include, I think we agreed on the following: ? - Static linking will only support headless-only builds (in which the build system excludes the AWT library that does "headful" stuff -- otherwise there would be duplicate symbols) ? - As a first delivery, the build system will just create a static version of the "java" launcher (not jar, javac, etc). This will include all native libraries from all modules that are included in the build. ? - Going forward, the correct solution is to make jlink create a launcher that includes just the native libraries from the modules that is included in the jlink command. This will require jlink to understand how to call the native linker. ? - Somewhere in there we probably also needs to have jlink know about headless-only vs normal (headless or "headful" determined on runtime), so it can create a java.desktop output that includes only the headless library. * Magnus reported that the refactoring and fixing of technical debt that was required for doing static builds properly has just been finished, and that his attention is now turning into creating a properly integrated system for generating static builds alongside dynamic builds. * Jiangli and Magnus will work outside the meeting to resolve the build issues Magnus faced with the hermetic java branch in the Leyden repo. * Just before the meeting unfortunately had to be aborted, Jiangli mentioned that they had discovered issues with some JDK native libraries when using objcopy to localize all non-visible symbols. It was at the time of writing not clear what those issues were. Jiangli will report back with what they found. (And while I had not time to mention it on the meeting, I will also look into this.) /Magnus From ihse at openjdk.org Tue Apr 16 18:37:59 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 18:37:59 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk [v3] In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 10:03:27 GMT, Magnus Ihse Bursie wrote: >> The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. >> >> I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). > > Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: > > - Make the split based on the name "awt" instead, and document the reason > - Rename 2dLibraries.gmk to ClientLibraries.gmk **I** am the one working with this file all the time, not you. There is no good IDE support for Makefiles, especially not with the level of customization that we have done to make. I am positively struggling with navigating this file, time and time again. Can that not be enough reason for you to accept that I want to split this file into more manageable chunks? Nothing here will affect how the actual client libraries are built. It is *purely* about making this part of the JDK build less of a complete PITA for me. Why are you opposing this? I tried two different approaches, and suggested a third (alphabetic) and you just say "no". I don't find this very constructive or helpful. This is not the first time we split a makefile into "arbitrary" chunks. Since make is not a proper programming language, there is no way to express abstraction, objects or other clear delimiters. The most recently file that was split this way was JdkNativeCompilation.gmk. One of the first was configure.ac. All those split were "arbitrary" and "artificial", but they are still good enough to provide help to the poor sods that need to work with those files. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2059708051 From ihse at openjdk.org Tue Apr 16 18:46:00 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 16 Apr 2024 18:46:00 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk [v3] In-Reply-To: References: Message-ID: <7WhYZucmWOpro6BaMe5yniONsmhZrKNsMII_d7mtZv8=.5bcd2da8-4726-4b6f-85b9-21084048f1d3@github.com> On Tue, 16 Apr 2024 10:03:27 GMT, Magnus Ihse Bursie wrote: >> The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. >> >> I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). > > Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: > > - Make the split based on the name "awt" instead, and document the reason > - Rename 2dLibraries.gmk to ClientLibraries.gmk And just to put some perspective here: The .gmk files for all other modules are at an average of 2 kB. The Awt2dLibraries.gmk file is 32 kB. Even with the split the two remaining parts will be 16 kB, which is still larger than the next biggest file (java.base/gensrc/GensrcBuffer.gmk) at 14 kB. So this is really an outlier. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2059720604 From prr at openjdk.org Tue Apr 16 18:53:42 2024 From: prr at openjdk.org (Phil Race) Date: Tue, 16 Apr 2024 18:53:42 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk [v3] In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 10:03:27 GMT, Magnus Ihse Bursie wrote: >> The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. >> >> I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). > > Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: > > - Make the split based on the name "awt" instead, and document the reason > - Rename 2dLibraries.gmk to ClientLibraries.gmk I appreciate that the build group does a lot of work maintaining this file. I'd like to think that even though it is a build file, it is build file for the desktop module, so I have skin in this game. I am coming at it from the point of view of what makes sense (at least to me). Even if I was the maintainer of this file, working on it every day, I wouldn't think splitting it any kind of priority. So I need to be clear, if you REALLY want to do this, then I won't stop you, I just think it is the wrong thing to do. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2059731122 From acobbs at openjdk.org Tue Apr 16 19:15:14 2024 From: acobbs at openjdk.org (Archie Cobbs) Date: Tue, 16 Apr 2024 19:15:14 GMT Subject: RFR: 8317376: Minor improvements to the 'this' escape analyzer [v5] In-Reply-To: References: Message-ID: > Please review several fixes and improvements to the `this-escape` lint warning analyzer. > > The goal here is to apply some relatively simple logical fixes that improve the precision and accuracy of the analyzer, and capture the remaining low-hanging fruit so we can consider the analyzer relatively complete with respect to what's feasible with its current design. > > Although the changes are small from a logical point of view, they generate a fairly large patch due to impact of refactoring (sorry!). Most of the patch derives from the first two changes listed below. > > The changes are summarized here: > > #### 1. Generalize how we categorize references > > The `Ref` class hierarchy models the various ways in which, at any point during the execution of a constructor or some other method/constructor that it invokes, there can be live references to the original object under construction lying around. We then look for places where one of these `Ref`'s might be passed to a subclass method. In other words, the analyzer keeps track of these references and watches what happens to them as the code executes so it can catch them trying to "escape". > > Previously the `Ref` categories were: > * `ThisRef` - The current instance of the (non-static) method or constructor being analyzed > * `OuterRef` - The current outer instance of the (non-static) method or constructor being analyzed > * `VarRef` - A local variable or method parameter currently in scope > * `ExprRef` - An object reference sitting on top of the Java execution stack > * `YieldRef` - The current switch expression's yield value(s) > * `ReturnRef` - The current method's return value(s) > > For each of those types, we further classified the "indirection" of the reference, i.e., whether the reference was direct (from the thing itself) or indirect (from something the thing referenced). > > The problem with that hierarchy is that we could only track outer instance references that happened to be associated with the current instance. So we might know that `this` had an outer instance reference, but if we said `var x = this` we wouldn't know that `x` had an outer instance reference. > > In other words, we should be treating "via an outer instance" as just another flavor of indirection along with "direct" and "indirect". > > As a result, with this patch the `OuterRef` class goes away and a new `Indirection` enum has been created, with values `DIRECT`, `INDIRECT`, and `OUTER`. > > #### 2. Track the types of all references > > By keeping track of the actual type of... Archie Cobbs has updated the pull request incrementally with one additional commit since the last revision: Synchronize TreeInfo.isExplicitThisReference() with PR #18088. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16208/files - new: https://git.openjdk.org/jdk/pull/16208/files/e09e4229..304a92b6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16208&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16208&range=03-04 Stats: 57 lines in 2 files changed: 35 ins; 21 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16208.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16208/head:pull/16208 PR: https://git.openjdk.org/jdk/pull/16208 From acobbs at openjdk.org Tue Apr 16 19:15:18 2024 From: acobbs at openjdk.org (Archie Cobbs) Date: Tue, 16 Apr 2024 19:15:18 GMT Subject: RFR: 8317376: Minor improvements to the 'this' escape analyzer [v4] In-Reply-To: <5H9FriMwHXEi75SibY_zoPVu8LcLFmLgVrHKGL58Vh0=.cc3bbbc7-669b-471c-b4d2-ac54cf6df44c@github.com> References: <5H9FriMwHXEi75SibY_zoPVu8LcLFmLgVrHKGL58Vh0=.cc3bbbc7-669b-471c-b4d2-ac54cf6df44c@github.com> Message-ID: <2Uim0unYk6q6KQnNsOM68Ki8vWom0I4nnIcxGonJP9U=.69837268-3b52-45fc-a6f9-71086608142a@github.com> On Tue, 16 Apr 2024 18:00:50 GMT, Vicente Romero wrote: >> Archie Cobbs has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: >> >> - Merge branch 'master' into JDK-8317376 >> - Merge branch 'master' into JDK-8317376 >> - Merge branch 'master' into JDK-8317376 >> - Merge branch 'master' into JDK-8317376 >> - Merge branch 'master' into JDK-8317376 >> - Javadoc++ >> - Merge branch 'master' into JDK-8317376 >> - Several improvements to the 'this' escape analyzer. >> >> - Track direct, indirect, and outer references for all Ref types. >> - Keep type information about all references to improve tracking precision. >> - Track enhanced for() invocations of iterator(), hasNext(), and next(). >> - Don't report an escape of a non-public outer instances as a leak. >> - Fix omitted tracking of references from newly instantiated instances. >> - Fix omitted tracking of leaks via lambda return values. >> - Remove unneccesary suppressions of this-escape lint warning. > > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/ThisEscapeAnalyzer.java line 1343: > >> 1341: * but also NOT an enclosing outer class of 'currentClass'. >> 1342: */ >> 1343: private boolean isExplicitThisReference(Types types, Type.ClassType currentClass, JCFieldAccess select) { > > suggestion: why not moving this method to TreeInfo already in preparation for the changes coming with flexible constructor bodies? Good idea - thanks. Done in 304a92b619e. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16208#discussion_r1567832080 From jianglizhou at google.com Tue Apr 16 19:39:28 2024 From: jianglizhou at google.com (Jiangli Zhou) Date: Tue, 16 Apr 2024 12:39:28 -0700 Subject: Hermetic Java project meeting In-Reply-To: References: Message-ID: Magnus, thanks for the meeting summary! Please see clarifications below, to avoid any confusion. On Tue, Apr 16, 2024 at 11:27?AM Magnus Ihse Bursie wrote: > > This is my summary of what was said in today's meeting: > > * Jiangli reported that her team had done extensive testing and not seen > any problems, both with just the static launcher as generated by the > leyden branch, and with bundled user applications using the One Jar (?) > tool. They tested JDK tier 1, and an suite of Google's internal tests. > When testing JTReg tests with native libraries, these were dynamically > loaded. Clarifications, as discussed in the meeting: We have done multiple levels of testing for static/hermetic Java prototype with our internal codebase on JDK 11, JDK 21 and JDK head (mainline based). We run into bugs/failures and have addressed the issues found along the way. The layden branch contains most of the static/hermetic Java work from the prototype. In general static/hermetic Java support looks robust in current state on JDK 21 in our prototype based on testing (already communicated some of the the testing outcome on JDK 11 in last years JVMLS), with some of the remaining areas (e.g. how to handle user code accessing java.home, and https://github.com/openjdk/leyden/blob/hermetic-java-runtime/src/java.base/share/classes/jdk/internal/misc/JavaHome.java is still a internal class in the prototpye) that require broad discussions involving potential spec updates, and a small number of remaining failures to be looked into. Testing we have done includes: - jtreg with tier1 using `javastatic` ("fully" statically linked Java launcher) on JDK 11 and JDK 21 - Explicit functional and integration tests (most of them are developed base on existing jtreg tests) to test the final hermetic Java image. Image is built using singlejar, and the test and JDK are in a single image. - Scattered hermetic Java testing using our internal tests. The scattered hermetic image is a special mode for testing to emulate hermetic Java image execution without building the final hermetic Java. This requires some additional launcher changes that are not in the lenden branch currently. - Some production testing on JDK 11 and JDK 21 > > * Alan asked about Hotspot JTReg tests that launched "java". Jiangli > reported that they had not seen any problems, Also clarification: As mentioned during the meeting, most of the issues that we found with jtreg tier1 testing were due to the assumption of using dynamic libraries in the tests. We have addressed those. Alan had some questions about launching sub-processes. In the prototype, we had done work to support POSIX_SPAWN launch mechanism for ProcessBuilder.start() on hermetic Java, e.g. https://github.com/openjdk/leyden/commit/931b71b0845d24b1949a23333ef1cdb3d6622596. We need to look into tier1 testing to verify if they cover sub-processing testing (Alan mentioned there are some in tier1). > but my understanding was > that there was some confusion if any such tests were actually run. I > think this is something that will need further attention, but if someone > said they would look into it, I missed it. Megnas, can you please elaborate the above about "if any such tests were actually run"? > > * Jiangli will get numbers on how much time is added to the GHA > testing if we add building and linking of static libraries, without > fixing so we can compile to a single set of object files. Will follow up on this. > > * We did not fully come to a conclusion if compiling on a single set of > object files for static and dynamic linking was a hard requirement or > not, but at a minimum it is a desirable goal. (My personal opinion is > that is a hard requirement if the GHA build times are seriously affected > otherwise.) > > There are basically two problems prohibiting single object file > compilation: > > 1) using dynamic checks instead of #ifdefs for code that differs between > static and dynamic. > > 2) Handling the difference between JNI_OnLoad (as required for dynamic > libraries) and JNI_OnLoad_ (as required for static libraries). > > * The leyden branch has basically solved both these problems. The first > one could more or less be integrated already (given perhaps some > discussion on exactly *how* the JDK should discover in runtime if it is > static or dynamic), but the latter requires a spec change to be integrated. > > * I think everyone agreed that moving on with a spec change was a good > idea, regardless of if this is blocker or not, but I don't recall that > there were any concrete next steps decided. Ron and Alan said that we do > spec changes all the time so it will not add as much bureaucracy as one > might fear. There was also a question raised by Dave during the meeting on the timeline for the spec/JSR related work. > > * Regarding which native libraries to include, I think we agreed on the > following: > - Static linking will only support headless-only builds (in which the > build system excludes the AWT library that does "headful" stuff -- > otherwise there would be duplicate symbols) Yes. > - As a first delivery, the build system will just create a static > version of the "java" launcher (not jar, javac, etc). This will include > all native libraries from all modules that are included in the build. Yes. It would be headless based. > - Going forward, the correct solution is to make jlink create a > launcher that includes just the native libraries from the modules that > is included in the jlink command. This will require jlink to understand > how to call the native linker. Yes. That would be one of the N-step for supporting hermetic Java. > - Somewhere in there we probably also needs to have jlink know about > headless-only vs normal (headless or "headful" determined on runtime), > so it can create a java.desktop output that includes only the headless > library. Alan has described an idea of dealing with java.desktop using jlink. > > * Magnus reported that the refactoring and fixing of technical debt that > was required for doing static builds properly has just been finished, > and that his attention is now turning into creating a properly > integrated system for generating static builds alongside dynamic builds. Thank you, Magnus! > > * Jiangli and Magnus will work outside the meeting to resolve the build > issues Magnus faced with the hermetic java branch in the Leyden repo. Yes. > > * Just before the meeting unfortunately had to be aborted, Jiangli > mentioned that they had discovered issues with some JDK native libraries > when using objcopy to localize all non-visible symbols. It was at the > time of writing not clear what those issues were. Jiangli will report > back with what they found. (And while I had not time to mention it on > the meeting, I will also look into this.) Best, Jiangli > > /Magnus > From liach at openjdk.org Tue Apr 16 21:24:59 2024 From: liach at openjdk.org (Chen Liang) Date: Tue, 16 Apr 2024 21:24:59 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 In-Reply-To: References: Message-ID: On Mon, 15 Apr 2024 19:01:08 GMT, Joe Darcy wrote: > Get JDK 24 underway. src/java.base/share/classes/java/lang/classfile/ClassFile.java line 1481: > 1479: int JAVA_23_VERSION = 67; > 1480: > 1481: /** 68 */ We need `@since 24` here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18787#discussion_r1567955182 From jianglizhou at google.com Tue Apr 16 21:28:58 2024 From: jianglizhou at google.com (Jiangli Zhou) Date: Tue, 16 Apr 2024 14:28:58 -0700 Subject: Questions about the Hermetic Java project In-Reply-To: References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> <8852ba23-2830-4d23-ae52-11c554d8d750@oracle.com> Message-ID: On Tue, Apr 16, 2024 at 9:20?AM Jiangli Zhou wrote: > > On Tue, Apr 16, 2024 at 8:30?AM Magnus Ihse Bursie > wrote: > > > > Jiangli, > > > > First of all: I tried building the leyden branch "hermetic-java-runtime" but failed. :-( I tried some various attempts to work around the failures, but the javastatic file I ended up with were unable to start. > > Magnus, I recall you mentioned that you had build issues during one of > the zoom meetings earlier this year (didn't hear any follow-up on it). > Let's get this resolved for you first. Can you please share the build > failures/errors and runtime errors that you run into? > > Same questions about your build environment, as you asked below. > > > > > What kind of environment are you using to build this branch? (OS, compiler, libc version, etc) Do you have any special configure flags that are required? > > For the leyden hermetic-java-runtime branch, I've only been building > on Linux. I've used both gcc and clang. > > Yes, the branch requires `--with-static-java=yes`, e.g. > > bash configure --with-boot-jdk= > --with-debug-level=slowdebug --with-static-java=yes > > I documented the build information in one of the comment messages. > That's probably not the best place. I'll look for a better place. > > Let's discuss more on the rest of the topics below during our meeting > today. We can update on the mailing list about the discussion, if > others are interested. For anyone who's interested in the topics, https://mail.openjdk.org/pipermail/leyden-dev/2024-April/000643.html is a separate email thread with today's meeting summary from @Magnus Ihse Bursie and responses from me. Best, Jiangli > > Best, > Jiangli > > > > > I would very much like to test it out myself to check this static java launcher that you are creating. > > > > On 2024-04-16 03:01, Jiangli Zhou wrote: > > > > The main purpose of StaticLink.gmk is to support the static-java-image > > make target, which can be used to perform the actual static linking > > step using libjvm.a and JDK static libraries. That currently doesn't > > exist in the JDK mainline. Creating a "fully" statically linked Java > > launcher is the first step (out of many) towards supporting > > static/hermetic Java. > > > > I am still not sure it is the best design to have this in a separate file. You are in some way "just" creating a new launcher. > > > > As part of cleaning/refactoring/integrating for the static linking > > step, we want to agree and decide/accept on the following: > > > > - Support the "fully" statically linked java launcher for testing and > > demoing the capability of static JDK support, e.g. > > - Support running jtreg testing using the "fully" statically linked > > Java launcher > > > > So how do you want to test native jtreg tests? By compiling the jtreg tests to native .a files and statically link with those as well, or to make this a hybrid mode where a statically linked launcher loads the jtreg tests dynamically? > > > > - Set up tests in github workflow to help detect any breaking > > changes for static support, e.g. new symbol issues introduced by any > > changes. There were some earlier discussions on this with Ron and Alan > > during the zoom meetings. > > > > If we do this correct, we should not have to worry about symbol conflicts. (Local symbols should all be hidden before creation of the static libraries.) That said, creating static libraries needs to be regularly tested or it will break. > > > > - Which JDK native libraries to be statically linked with the new > > launcher target? E.g. StaticLink.gmk currently excludes libjsound.a, > > libawt_xawt.a, etc from statically linked with the launcher. > > > > That sounds rather arbitrarily. I would assume that a statically linked launcher should include native libraries from all included modules. If you are not interested in java.desktop native libraries, then you'd need to exclude java.desktop from the build. But my assumption here is that we need to build a solution that works with all modules included in the JDK, and that is what must be tested in GHA. > > > > - Do we want more than one statically linked launcher target, based on > > the set of linked native libraries? > > > > Do you mean that you want like "static-java-java.base-only", "static-java-eveything-but-java.desktop" etc? That does not sound like something that belongs in the makefiles. If you want to build a JDK that can only support a certain subset of modules, you need to specify those module in your build scripts as input to configure. > > > > (II) What missing: > > - Static linking step as mentioned above > > > > (III) What needs to be improved (require cleanups and refactoring, and > > you mentioned some of those in your response as well): > > - Support building both the static libraries and dynamic libraries > > using the same set of .o files, instead of separately compiled .o > > files. That helps improve build speed and reduce memory overhead for > > building JDK. Your current refactoring work aims to help that. > > - Clean up the usages of STATIC_BUILD macro. Most of the usages are in > > test code. > > - Other runtime fixes/enhancements in the leyden > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > > > > I think most work mentioned in III has dependencies on II. We need a > > workable base to be able to build the "fully" statically linked > > launcher for building and testing the work mentioned in III, when > > integrating any of those to the JDK mainline. The makefile refactoring > > work can be done in parallel but does not need to be completed before > > we add the static linking step in JDK mainline. > > > > I'm not sure I understand why you consider (II) to be a prerequisite for the (III) issues. From my point of view, they are all mutually independent changes that needs to be done, and if anything, I think getting in code to create a statically linked launcher will probably be easier if we get further along on the other issues. > > > > In particular, I'd like to see changes to the STATIC_BUILD macro come in first. Ideally, we should not send in such a macro at all. The JNI_OnLoad stuff is bugging me mostly. > > > > As of today, the leyden > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch > > can build a "fully" statically linked Java launcher. The issue of > > compiling the dynamic and static libraries .o files separately is not > > a blocker. It's good to have it resolved at some point of time. > > > > Weeeell, yes and no... It is not *technically* a blocker, but unless we can reuse the .o files, including static builds in the standard testing on GHA will effectively double the build time, and that is definitely not acceptable. And the alternative is to bring in static builds without having it regularly tested, which I don't think is acceptable either. > > > > So the conclusion is that getting static and dynamic libraries built from the same .o files is in effect a blocker for Hermetic Java. > > > > This, in turn, require several changes: > > > > 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > > > > 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > > > > This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > > > > Thumbs up! That seems to be a good direction. Currently in the leyden > > branch, it first looks up the unique > > JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in > > libraries, then search for the dynamic libraries using the > > conventional naming when necessary. e.g.: > > > > https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a > > https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135 > > > > When spec supports JNI_OnLoad_ and etc. for dynamic > > libraries, we may still need to support the conventional naming > > without the <_lib_name> part for existing libraries out there. > > > > That looks interesting. Basically, this is very similar to what I imagine needs to be done in the mainline. However, I don't think we can just change this code like that without a CSR? Or are we allowed to treat JDK-internal libraries different than the specification states? > > > > I would say getting this part into mainline should be the main focus right now, not the StaticLink.gmk file. And that includes getting any CSR approvals etc. > > > > Thank you for taking this on! Potentially we could consider taking the > > objcopy to localizing hotspot symbols on unix-like platforms, based on > > https://github.com/openjdk/jdk/pull/17456 discussions. Additional > > testing is still needed to verify the solution. > > > > We don't need any additional testing on that; proof of concept shows it works. It just needs to be implemented. With JDK-8330261 (pushed today), the ground is now prepared to get it done. It is next on my todo-list. > > > > It might be a good idea to follow up on the static linking discussion > > in tomorrow's zoom meeting (hope you'll be able to join tomorrow). > > > > I'll join but might be somewhat late due to family commitments. > > > > /Magnus From jianglizhou at google.com Tue Apr 16 22:44:17 2024 From: jianglizhou at google.com (Jiangli Zhou) Date: Tue, 16 Apr 2024 15:44:17 -0700 Subject: Hermetic Java project meeting In-Reply-To: References: Message-ID: On Tue, Apr 16, 2024 at 12:39?PM Jiangli Zhou wrote: > > Magnus, thanks for the meeting summary! Please see clarifications > below, to avoid any confusion. > > On Tue, Apr 16, 2024 at 11:27?AM Magnus Ihse Bursie > wrote: > > > > This is my summary of what was said in today's meeting: > > > > * Jiangli reported that her team had done extensive testing and not seen > > any problems, both with just the static launcher as generated by the > > leyden branch, and with bundled user applications using the One Jar (?) > > tool. They tested JDK tier 1, and an suite of Google's internal tests. > > When testing JTReg tests with native libraries, these were dynamically > > loaded. > > Clarifications, as discussed in the meeting: > > We have done multiple levels of testing for static/hermetic Java > prototype with our internal codebase on JDK 11, JDK 21 and JDK head > (mainline based). We run into bugs/failures and have addressed the > issues found along the way. The layden branch contains most of the > static/hermetic Java work from the prototype. In general > static/hermetic Java support looks robust in current state on JDK 21 > in our prototype based on testing (already communicated some of the > the testing outcome on JDK 11 in last years JVMLS), with some of the > remaining areas (e.g. how to handle user code accessing java.home, and > https://github.com/openjdk/leyden/blob/hermetic-java-runtime/src/java.base/share/classes/jdk/internal/misc/JavaHome.java > is still a internal class in the prototpye) that require broad > discussions involving potential spec updates, and a small number of > remaining failures to be looked into. > > Testing we have done includes: > - jtreg with tier1 using `javastatic` ("fully" statically linked Java > launcher) on JDK 11 and JDK 21 > - Explicit functional and integration tests (most of them are > developed base on existing jtreg tests) to test the final hermetic > Java image. Image is built using singlejar, and the test and JDK are > in a single image. > - Scattered hermetic Java testing using our internal tests. The > scattered hermetic image is a special mode for testing to emulate > hermetic Java image execution without building the final hermetic > Java. This requires some additional launcher changes that are not in > the lenden branch currently. > - Some production testing on JDK 11 and JDK 21 > > > > > * Alan asked about Hotspot JTReg tests that launched "java". Jiangli > > reported that they had not seen any problems, > > Also clarification: > > As mentioned during the meeting, most of the issues that we found with > jtreg tier1 testing were due to the assumption of using dynamic > libraries in the tests. We have addressed those. > > Alan had some questions about launching sub-processes. In the > prototype, we had done work to support POSIX_SPAWN launch mechanism > for ProcessBuilder.start() on hermetic Java, e.g. > https://github.com/openjdk/leyden/commit/931b71b0845d24b1949a23333ef1cdb3d6622596. > We need to look into tier1 testing to verify if they cover > sub-processing testing (Alan mentioned there are some in tier1). > > > but my understanding was > > that there was some confusion if any such tests were actually run. I > > think this is something that will need further attention, but if someone > > said they would look into it, I missed it. > > Megnas, can you please elaborate the above about "if any such tests > were actually run"? > > > > > * Jiangli will get numbers on how much time is added to the GHA > > testing if we add building and linking of static libraries, without > > fixing so we can compile to a single set of object files. > > Will follow up on this. I did some measurements using the https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch on my Linux desktop today. Here's the detail: - Build machine: Debian 6.6.* - gcc version 13.2.0 - `Time` command is used to do the measurements - Same configuration is used: --with-boot-jdk= --with-debug-level=slowdebug --with-static-java=yes - `make clean` is done before both of the measurements # # Building JDK image with just dynamic libraries # Build command: make images # real 5m27.582s user 75m11.272s sys 8m2.783s # # Building JDK image with both dynamic libraries and static libraries, also link bin/javastatic # Build command: make static-java-image # real 6m5.958s user 115m59.353s sys 15m11.664s There is some overhead with the overall build time, but it's not as significant as doubling the time. I think we should consider prioritizing the launcher static linking work for now. I also want to reiterate the importance of compiling just the .o files only once and use them for both the dynamic libraries and static libraries. In our prototype on JDK 11, we did that. When moving the work to JDK head for the leyden branch and mainline, we adopted to use the existing `static-libs-image` for building the full set of static libraries with separately compiled .o files (https://bugs.openjdk.org/browse/JDK-8307858). As mentioned in some of the earlier meetings with Alan and Ron last year, we ran into memory issues during JDK build time on our system due to the duplication of the .o files. I did some workaround to mitigate the memory issue. So we should also prioritize the work although it is a non-blocker for integrating some of the leyden branch work into the mainline at this point. I'd recommend addressing that immediately after the launcher static linking work. As you are already aware of, it would require some of the runtime work to clean up the STATIC_BUILD macro usages. Best, Jiangli > > > > > * We did not fully come to a conclusion if compiling on a single set of > > object files for static and dynamic linking was a hard requirement or > > not, but at a minimum it is a desirable goal. (My personal opinion is > > that is a hard requirement if the GHA build times are seriously affected > > otherwise.) > > > > There are basically two problems prohibiting single object file > > compilation: > > > > 1) using dynamic checks instead of #ifdefs for code that differs between > > static and dynamic. > > > > 2) Handling the difference between JNI_OnLoad (as required for dynamic > > libraries) and JNI_OnLoad_ (as required for static libraries). > > > > * The leyden branch has basically solved both these problems. The first > > one could more or less be integrated already (given perhaps some > > discussion on exactly *how* the JDK should discover in runtime if it is > > static or dynamic), but the latter requires a spec change to be integrated. > > > > * I think everyone agreed that moving on with a spec change was a good > > idea, regardless of if this is blocker or not, but I don't recall that > > there were any concrete next steps decided. Ron and Alan said that we do > > spec changes all the time so it will not add as much bureaucracy as one > > might fear. > > There was also a question raised by Dave during the meeting on the > timeline for the spec/JSR related work. > > > > > * Regarding which native libraries to include, I think we agreed on the > > following: > > - Static linking will only support headless-only builds (in which the > > build system excludes the AWT library that does "headful" stuff -- > > otherwise there would be duplicate symbols) > > Yes. > > > - As a first delivery, the build system will just create a static > > version of the "java" launcher (not jar, javac, etc). This will include > > all native libraries from all modules that are included in the build. > > Yes. It would be headless based. > > > - Going forward, the correct solution is to make jlink create a > > launcher that includes just the native libraries from the modules that > > is included in the jlink command. This will require jlink to understand > > how to call the native linker. > > Yes. That would be one of the N-step for supporting hermetic Java. > > > - Somewhere in there we probably also needs to have jlink know about > > headless-only vs normal (headless or "headful" determined on runtime), > > so it can create a java.desktop output that includes only the headless > > library. > > Alan has described an idea of dealing with java.desktop using jlink. > > > > > * Magnus reported that the refactoring and fixing of technical debt that > > was required for doing static builds properly has just been finished, > > and that his attention is now turning into creating a properly > > integrated system for generating static builds alongside dynamic builds. > > Thank you, Magnus! > > > > > * Jiangli and Magnus will work outside the meeting to resolve the build > > issues Magnus faced with the hermetic java branch in the Leyden repo. > > Yes. > > > > > * Just before the meeting unfortunately had to be aborted, Jiangli > > mentioned that they had discovered issues with some JDK native libraries > > when using objcopy to localize all non-visible symbols. It was at the > > time of writing not clear what those issues were. Jiangli will report > > back with what they found. (And while I had not time to mention it on > > the meeting, I will also look into this.) > > Best, > Jiangli > > > > > /Magnus > > From erikj at openjdk.org Wed Apr 17 00:53:45 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 17 Apr 2024 00:53:45 GMT Subject: Integrated: 8329970: Update autoconf build-aux files with latest from 2024-01-01 In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 19:43:27 GMT, Erik Joelsson wrote: > Since we are now able to update the autoconf build-aux files, I think it's prudent to semi regularly do just that. I'm not aware of any particular platform that has been improved that would affect OpenJDK and I don't think we can further trim down our wrappers this time. This is more about staying with the latest. As of the filing of this bug, that is 2024-01-01, found here: > > https://git.savannah.gnu.org/cgit/config.git/ > > I have verified this with the platforms we build for at Oracle. I would encourage other OpenJDK distributors to verify on any platforms not covered by us. I will leave this open for a few days. This pull request has now been integrated. Changeset: e57a322d Author: Erik Joelsson URL: https://git.openjdk.org/jdk/commit/e57a322d7076474806458cc4b796bdb874e8e92e Stats: 224 lines in 2 files changed: 126 ins; 24 del; 74 mod 8329970: Update autoconf build-aux files with latest from 2024-01-01 Reviewed-by: ihse, clanger ------------- PR: https://git.openjdk.org/jdk/pull/18704 From darcy at openjdk.org Wed Apr 17 05:27:59 2024 From: darcy at openjdk.org (Joe Darcy) Date: Wed, 17 Apr 2024 05:27:59 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 21:21:43 GMT, Chen Liang wrote: >> Get JDK 24 underway. > > src/java.base/share/classes/java/lang/classfile/ClassFile.java line 1481: > >> 1479: int JAVA_23_VERSION = 67; >> 1480: >> 1481: /** 68 */ > > We need `@since 24` here. Ah, good catch; looks like I was treating that as an internal API element when it is indeed a public API element. I've filed JDK-8330458 to fix-up the Java 23 constant as well. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18787#discussion_r1568231344 From darcy at openjdk.org Wed Apr 17 05:43:12 2024 From: darcy at openjdk.org (Joe Darcy) Date: Wed, 17 Apr 2024 05:43:12 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 [v2] In-Reply-To: References: Message-ID: > Get JDK 24 underway. Joe Darcy has updated the pull request incrementally with two additional commits since the last revision: - Correct release date as observed in review feedback. - Improve javadoc of class file update. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18787/files - new: https://git.openjdk.org/jdk/pull/18787/files/dcec9e96..dc488f21 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18787&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18787&range=00-01 Stats: 5 lines in 2 files changed: 3 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/18787.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18787/head:pull/18787 PR: https://git.openjdk.org/jdk/pull/18787 From sgehwolf at openjdk.org Wed Apr 17 08:29:09 2024 From: sgehwolf at openjdk.org (Severin Gehwolf) Date: Wed, 17 Apr 2024 08:29:09 GMT Subject: RFR: 8311302: Allow for jlinking a custom runtime without packaged modules being present [v23] In-Reply-To: References: <_sbNFJXG7ghwsfNfxaGvvoDQ7-UtWpnhGnygKsFTsgQ=.623700db-7306-4b34-9e05-ca9059797e39@github.com> Message-ID: On Tue, 16 Apr 2024 13:54:53 GMT, Alan Bateman wrote: > I think it continues to build time, it's just using some hidden jlink option. So yes, it similar to a previous iteration except that it doesn't run as a plugin the pipeline and the delta goes to the lib directory. OK. If a hidden jlink option is good enough, then this works for me. I hope to have time to update the patch in the coming days. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14787#issuecomment-2060691122 From asotona at openjdk.org Wed Apr 17 09:23:42 2024 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 17 Apr 2024 09:23:42 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 [v2] In-Reply-To: References: Message-ID: On Wed, 17 Apr 2024 05:43:12 GMT, Joe Darcy wrote: >> Get JDK 24 underway. > > Joe Darcy has updated the pull request incrementally with two additional commits since the last revision: > > - Correct release date as observed in review feedback. > - Improve javadoc of class file update. ClassFile changes are OK. ------------- Marked as reviewed by asotona (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18787#pullrequestreview-2005504192 From jkern at openjdk.org Wed Apr 17 10:56:49 2024 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 17 Apr 2024 10:56:49 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> Message-ID: <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> On Tue, 16 Apr 2024 09:15:19 GMT, Magnus Ihse Bursie wrote: >> That was kind of where the discussion started, and which Kim did not like. If I read him correctly, his suggestion was instead to place: >> >> #if defined(_AIX) >> #include >> #endif >> >> in the files where `alloca` is needed on AIX. > > (If some of these files happen to be files which are not compiled on Windows, I assume it will not hurt to drop the ifdef guard, but then again, it can certainly be kept as well for consistency.) @magicus @TheShermanTanker @TheRealMDoerr @kimbarrett Let me summarize the choices we have and ask for your vote. Julian dislikes the `-Dalloca'(size)'=__builtin_alloca'(size)'` in `flags-cflags.m4` I introduced to get rid of #if defined(_AIX) #include #endif in `globalDefinitions_gcc.hpp`. We have three possible solutions 1. Reintroduce #if defined(_AIX) #include #endif in `globalDefinitions_gcc.hpp`. 2. Unconditionally introduce only `#include ` in `globalDefinitions_gcc.hpp`. This should work for all platforms using this header including the unofficial Windows/gcc Port, although only AIX needs it. 3. Add #if defined(_AIX) #include #endif to the sources using alloca(). These are /hotspot/share/runtime/os.cpp /hotspot/share/runtime/javaThread.cpp /hotspot/share/utilities/vmError.cpp Here we need the AIX condition, because otherwise the classic Windows Build (NTAMD64) fails. I will implement the solution with the most likes and having no dislike. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1568650313 From jwaters at openjdk.org Wed Apr 17 11:55:48 2024 From: jwaters at openjdk.org (Julian Waters) Date: Wed, 17 Apr 2024 11:55:48 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> Message-ID: On Wed, 17 Apr 2024 10:54:25 GMT, Joachim Kern wrote: >> (If some of these files happen to be files which are not compiled on Windows, I assume it will not hurt to drop the ifdef guard, but then again, it can certainly be kept as well for consistency.) > > @magicus @TheShermanTanker @TheRealMDoerr @kimbarrett > Let me summarize the choices we have and ask for your vote. > Julian dislikes the `-Dalloca'(size)'=__builtin_alloca'(size)'` in `flags-cflags.m4` I introduced to get rid of > > #if defined(_AIX) > #include > #endif > > in `globalDefinitions_gcc.hpp`. > > We have three possible solutions > > 1. Reintroduce > > #if defined(_AIX) > #include > #endif > > in `globalDefinitions_gcc.hpp`. > > 2. Unconditionally introduce only `#include ` in `globalDefinitions_gcc.hpp`. This should work for all platforms using this header including the unofficial Windows/gcc Port, although only AIX needs it. > > 3. Add > > #if defined(_AIX) > #include > #endif > > to the sources using alloca(). These are > /hotspot/share/runtime/os.cpp > /hotspot/share/runtime/javaThread.cpp > /hotspot/share/utilities/vmError.cpp > Here we need the AIX condition, because otherwise the classic Windows Build (NTAMD64) fails. > > I will implement the solution with the most likes and having no dislike. I don't mind all 3, though I certainly prefer 1 and 3 over 2 (The way I see it, the AIX macro check is more of a message to the programmer than it is important to the compiler, so I prefer the options that have it. However, I also don't mind if we were to go the way of option 2, this is more of a preference thing). The fact that only 3 files need it is also surprising to me, and makes option 3 seem like a good fit (Again, personal preference) Magnus and Kim, what do you guys think? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1568718288 From ihse at openjdk.org Wed Apr 17 12:25:04 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 17 Apr 2024 12:25:04 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> Message-ID: <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> On Wed, 17 Apr 2024 11:52:42 GMT, Julian Waters wrote: >> @magicus @TheShermanTanker @TheRealMDoerr @kimbarrett >> Let me summarize the choices we have and ask for your vote. >> Julian dislikes the `-Dalloca'(size)'=__builtin_alloca'(size)'` in `flags-cflags.m4` I introduced to get rid of >> >> #if defined(_AIX) >> #include >> #endif >> >> in `globalDefinitions_gcc.hpp`. >> >> We have three possible solutions >> >> 1. Reintroduce >> >> #if defined(_AIX) >> #include >> #endif >> >> in `globalDefinitions_gcc.hpp`. >> >> 2. Unconditionally introduce only `#include ` in `globalDefinitions_gcc.hpp`. This should work for all platforms using this header including the unofficial Windows/gcc Port, although only AIX needs it. >> >> 3. Add >> >> #if defined(_AIX) >> #include >> #endif >> >> to the sources using alloca(). These are >> /hotspot/share/runtime/os.cpp >> /hotspot/share/runtime/javaThread.cpp >> /hotspot/share/utilities/vmError.cpp >> Here we need the AIX condition, because otherwise the classic Windows Build (NTAMD64) fails. >> >> I will implement the solution with the most likes and having no dislike. > > I don't mind all 3, though I certainly prefer 1 and 3 over 2 (The way I see it, the AIX macro check is more of a message to the programmer than it is important to the compiler, so I prefer the options that have it. However, I also don't mind if we were to go the way of option 2, this is more of a preference thing). The fact that only 3 files need it is also surprising to me, and makes option 3 seem like a good fit (Again, personal preference) > > Magnus and Kim, what do you guys think? If there are just 3 files using alloca, I strongly prefer solution 3. I think solution 1 has already been rejected by Kim. (Also, for the record, it was me, not Julian, who expressed dislike about the `-Dalloca'(size)'=__builtin_alloca'(size)'` change) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1568754458 From erikj at openjdk.org Wed Apr 17 12:32:43 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Wed, 17 Apr 2024 12:32:43 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk [v3] In-Reply-To: References: Message-ID: <79xD6pvDwtSM5MLRjDzjPzFABNCLGJM3b34gcBKOTCw=.fa94699c-5541-44b5-b305-61921deba475@github.com> On Tue, 16 Apr 2024 10:03:27 GMT, Magnus Ihse Bursie wrote: >> The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. >> >> I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). > > Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: > > - Make the split based on the name "awt" instead, and document the reason > - Rename 2dLibraries.gmk to ClientLibraries.gmk This looks good to me. The split may be arbitrary from a functional point of view, but will make navigating the makefiles easier. ------------- Marked as reviewed by erikj (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18743#pullrequestreview-2005913700 From ihse at openjdk.org Wed Apr 17 12:42:03 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 17 Apr 2024 12:42:03 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk [v3] In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 10:03:27 GMT, Magnus Ihse Bursie wrote: >> The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. >> >> I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). > > Magnus Ihse Bursie has updated the pull request incrementally with two additional commits since the last revision: > > - Make the split based on the name "awt" instead, and document the reason > - Rename 2dLibraries.gmk to ClientLibraries.gmk Yes, I really want to do this. I am sorry that we could not reach an agreement, but I need to be able to work with these files in a better way than is currently possible. Thanks Erik and Phil. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2061165564 From ihse at openjdk.org Wed Apr 17 12:42:04 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 17 Apr 2024 12:42:04 GMT Subject: Integrated: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk In-Reply-To: References: Message-ID: On Thu, 11 Apr 2024 13:53:23 GMT, Magnus Ihse Bursie wrote: > The file to build most of the java.desktop native libraries, Awt2dLibraries.gmk, is large and unstructured, making it hard to navigate. > > I want to split it into two parts, one for the AWT libraries, and one for the 2D libraries. I also used this opportunity to change the order to be more logical (e.g. grouping "image" libraries and "font" libraries in 2D). This pull request has now been integrated. Changeset: 5841cb3b Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/5841cb3b51e45e7c3aaa086e179815fa8184f22d Stats: 1724 lines in 4 files changed: 880 ins; 843 del; 1 mod 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk Reviewed-by: erikj ------------- PR: https://git.openjdk.org/jdk/pull/18743 From mdoerr at openjdk.org Wed Apr 17 13:05:02 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 17 Apr 2024 13:05:02 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> Message-ID: On Wed, 17 Apr 2024 12:22:10 GMT, Magnus Ihse Bursie wrote: >> I don't mind all 3, though I certainly prefer 1 and 3 over 2 (The way I see it, the AIX macro check is more of a message to the programmer than it is important to the compiler, so I prefer the options that have it. However, I also don't mind if we were to go the way of option 2, this is more of a preference thing). The fact that only 3 files need it is also surprising to me, and makes option 3 seem like a good fit (Again, personal preference) >> >> Magnus and Kim, what do you guys think? > > If there are just 3 files using alloca, I strongly prefer solution 3. I think solution 1 has already been rejected by Kim. > > (Also, for the record, it was me, not Julian, who expressed dislike about the `-Dalloca'(size)'=__builtin_alloca'(size)'` change) https://man7.org/linux/man-pages/man3/alloca.3.html sounds like solution 2 is the cleanest one ("standards conformance"). It is also the version with minimal code and which will even work with future alloca usages :-) If solution 2 has any disadvantage, I'd prefer solution 3. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1568809674 From mbaesken at openjdk.org Wed Apr 17 13:16:09 2024 From: mbaesken at openjdk.org (Matthias Baesken) Date: Wed, 17 Apr 2024 13:16:09 GMT Subject: RFR: 8314488: Compile the JDK as C++17 [v7] In-Reply-To: References: Message-ID: On Wed, 20 Mar 2024 05:44:48 GMT, Julian Waters wrote: >> Compile the JDK as C++17, enabling the use of all C++17 language features > > Julian Waters has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: > > - Merge branch 'master' into patch-7 > - Require clang 13 in toolchain.m4 > - Remove unnecessary -std=c++17 option in Lib.gmk > - Merge branch 'openjdk:master' into patch-7 > - Compiler versions in toolchain.m4 > - Merge branch 'openjdk:master' into patch-7 > - Merge branch 'openjdk:master' into patch-7 > - Revert vm_version_linux_riscv.cpp > - vm_version_linux_riscv.cpp > - allocation.cpp > - ... and 1 more: https://git.openjdk.org/jdk/compare/269163d5...9286a964 Seems we use already a little bit of C++17 coding in the Linux codebase . Just came across this little error when trying to build with clang on Linux jdk/src/hotspot/os/linux/os_linux.cpp:2975:65: error: 'static_assert' with no message is a C++17 extension [-Werror,-Wc++17-extensions] static_assert(MADV_POPULATE_WRITE == MADV_POPULATE_WRITE_value); So switching to C++17 would make our codebase compile :-) (at least in this case) ! (to be more serious, I guess I better file a JBS bug and post a PR/fix for this static_assert) ------------- PR Comment: https://git.openjdk.org/jdk/pull/14988#issuecomment-2061233028 From kbarrett at openjdk.org Wed Apr 17 15:13:04 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 17 Apr 2024 15:13:04 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> Message-ID: <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@github.com> On Wed, 17 Apr 2024 13:02:33 GMT, Martin Doerr wrote: >> If there are just 3 files using alloca, I strongly prefer solution 3. I think solution 1 has already been rejected by Kim. >> >> (Also, for the record, it was me, not Julian, who expressed dislike about the `-Dalloca'(size)'=__builtin_alloca'(size)'` change) > > https://man7.org/linux/man-pages/man3/alloca.3.html sounds like solution 2 is the cleanest one ("standards conformance"). It is also the version with minimal code and which will even work with future alloca usages :-) > If solution 2 has any disadvantage, I'd prefer solution 3. I'm aware of this discussion and looking into the issues, but a personal matter has intervened and it will take me a while to respond properly. Maybe next week. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1569012102 From vromero at openjdk.org Wed Apr 17 15:28:03 2024 From: vromero at openjdk.org (Vicente Romero) Date: Wed, 17 Apr 2024 15:28:03 GMT Subject: RFR: 8317376: Minor improvements to the 'this' escape analyzer [v5] In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 19:15:14 GMT, Archie Cobbs wrote: >> Please review several fixes and improvements to the `this-escape` lint warning analyzer. >> >> The goal here is to apply some relatively simple logical fixes that improve the precision and accuracy of the analyzer, and capture the remaining low-hanging fruit so we can consider the analyzer relatively complete with respect to what's feasible with its current design. >> >> Although the changes are small from a logical point of view, they generate a fairly large patch due to impact of refactoring (sorry!). Most of the patch derives from the first two changes listed below. >> >> The changes are summarized here: >> >> #### 1. Generalize how we categorize references >> >> The `Ref` class hierarchy models the various ways in which, at any point during the execution of a constructor or some other method/constructor that it invokes, there can be live references to the original object under construction lying around. We then look for places where one of these `Ref`'s might be passed to a subclass method. In other words, the analyzer keeps track of these references and watches what happens to them as the code executes so it can catch them trying to "escape". >> >> Previously the `Ref` categories were: >> * `ThisRef` - The current instance of the (non-static) method or constructor being analyzed >> * `OuterRef` - The current outer instance of the (non-static) method or constructor being analyzed >> * `VarRef` - A local variable or method parameter currently in scope >> * `ExprRef` - An object reference sitting on top of the Java execution stack >> * `YieldRef` - The current switch expression's yield value(s) >> * `ReturnRef` - The current method's return value(s) >> >> For each of those types, we further classified the "indirection" of the reference, i.e., whether the reference was direct (from the thing itself) or indirect (from something the thing referenced). >> >> The problem with that hierarchy is that we could only track outer instance references that happened to be associated with the current instance. So we might know that `this` had an outer instance reference, but if we said `var x = this` we wouldn't know that `x` had an outer instance reference. >> >> In other words, we should be treating "via an outer instance" as just another flavor of indirection along with "direct" and "indirect". >> >> As a result, with this patch the `OuterRef` class goes away and a new `Indirection` enum has been created, with values `DIRECT`, `INDIRECT`, and `OUTER`. >> >> #### 2. Track the types... > > Archie Cobbs has updated the pull request incrementally with one additional commit since the last revision: > > Synchronize TreeInfo.isExplicitThisReference() with PR #18088. looks sensible to me ------------- Marked as reviewed by vromero (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16208#pullrequestreview-2006363434 From iris at openjdk.org Wed Apr 17 16:51:43 2024 From: iris at openjdk.org (Iris Clark) Date: Wed, 17 Apr 2024 16:51:43 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 [v2] In-Reply-To: References: Message-ID: On Wed, 17 Apr 2024 05:43:12 GMT, Joe Darcy wrote: >> Get JDK 24 underway. > > Joe Darcy has updated the pull request incrementally with two additional commits since the last revision: > > - Correct release date as observed in review feedback. > - Improve javadoc of class file update. Marked as reviewed by iris (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18787#pullrequestreview-2006604933 From acobbs at openjdk.org Wed Apr 17 17:38:02 2024 From: acobbs at openjdk.org (Archie Cobbs) Date: Wed, 17 Apr 2024 17:38:02 GMT Subject: Integrated: 8317376: Minor improvements to the 'this' escape analyzer In-Reply-To: References: Message-ID: On Mon, 16 Oct 2023 22:08:53 GMT, Archie Cobbs wrote: > Please review several fixes and improvements to the `this-escape` lint warning analyzer. > > The goal here is to apply some relatively simple logical fixes that improve the precision and accuracy of the analyzer, and capture the remaining low-hanging fruit so we can consider the analyzer relatively complete with respect to what's feasible with its current design. > > Although the changes are small from a logical point of view, they generate a fairly large patch due to impact of refactoring (sorry!). Most of the patch derives from the first two changes listed below. > > The changes are summarized here: > > #### 1. Generalize how we categorize references > > The `Ref` class hierarchy models the various ways in which, at any point during the execution of a constructor or some other method/constructor that it invokes, there can be live references to the original object under construction lying around. We then look for places where one of these `Ref`'s might be passed to a subclass method. In other words, the analyzer keeps track of these references and watches what happens to them as the code executes so it can catch them trying to "escape". > > Previously the `Ref` categories were: > * `ThisRef` - The current instance of the (non-static) method or constructor being analyzed > * `OuterRef` - The current outer instance of the (non-static) method or constructor being analyzed > * `VarRef` - A local variable or method parameter currently in scope > * `ExprRef` - An object reference sitting on top of the Java execution stack > * `YieldRef` - The current switch expression's yield value(s) > * `ReturnRef` - The current method's return value(s) > > For each of those types, we further classified the "indirection" of the reference, i.e., whether the reference was direct (from the thing itself) or indirect (from something the thing referenced). > > The problem with that hierarchy is that we could only track outer instance references that happened to be associated with the current instance. So we might know that `this` had an outer instance reference, but if we said `var x = this` we wouldn't know that `x` had an outer instance reference. > > In other words, we should be treating "via an outer instance" as just another flavor of indirection along with "direct" and "indirect". > > As a result, with this patch the `OuterRef` class goes away and a new `Indirection` enum has been created, with values `DIRECT`, `INDIRECT`, and `OUTER`. > > #### 2. Track the types of all references > > By keeping track of the actual type of... This pull request has now been integrated. Changeset: 06462847 Author: Archie Cobbs Committer: Vicente Romero URL: https://git.openjdk.org/jdk/commit/064628471b83616b4463baa78618d1b7a66d0c7c Stats: 915 lines in 20 files changed: 552 ins; 171 del; 192 mod 8317376: Minor improvements to the 'this' escape analyzer Reviewed-by: vromero ------------- PR: https://git.openjdk.org/jdk/pull/16208 From prr at openjdk.org Wed Apr 17 20:13:01 2024 From: prr at openjdk.org (Phil Race) Date: Wed, 17 Apr 2024 20:13:01 GMT Subject: RFR: 8315113: Print request Chromaticity.MONOCHROME attribute does not work on macOS In-Reply-To: References: Message-ID: On Mon, 11 Mar 2024 13:54:02 GMT, Alexander Scherbatiy wrote: > The fix provides ability to print Black & White pages on macOS. > > Cocoa API has [PMSetColorMode](https://developer.apple.com/documentation/applicationservices/core_printing/1805783-pmsetcolormode) function but it is marked as deprecated and really does nothing. > > There is no replacement; this function was included to facilitate porting legacy applications to macOS, > but it serves no useful purpose. > > Dumping `NSPrintInfo` print settings which were set by the native print dialog on macOS shows that the keys and values used for Black & White printing depend on the used printer type. > For example, the tested > `HP Color LaserJet MFP M180n` printer uses `ColorModel` key and`Gray` value, and > `HP Ink Tank 110 series` uses `HPColorMode` key and `grayscale` value. > > Printing all `NSPrintInfo` presets shows that they do not contain key/value pairs for Black&White settings. > This is the code snippet used to print `NSPrintInfo` presets: > > PMPrinter pr; > PMPrintSession printSession = (PMPrintSession)[printInfo PMPrintSession]; > OSStatus status = PMSessionGetCurrentPrinter(printSession, &pr); > CFArrayRef presetsList = nil; > status = PMPrinterCopyPresets(pr, &presetsList); > CFIndex arrayCount = CFArrayGetCount(presetsList); > > for (CFIndex index = 0; index < arrayCount; index++) { > PMPreset preset = (PMPreset)CFArrayGetValueAtIndex(presetsList, index); > CFStringRef presetName = nil; > if (PMPresetCopyName(preset, &presetName) == noErr && CFStringGetLength(presetName) > 0) { > NSLog(@" presetName: '%@'", presetName); > NSDictionary* dict = nil; > if (PMPresetGetAttributes(preset, (CFDictionaryRef*)(&dict)) == noErr) { > // print preset dict > > > The idea of the proposed fix is to store printer dependent key/value pairs in the `/conf/printer.properties` property file. > > The property file has the format: > > print-attribute.print-attribute-value.key=value > > where `print-attribute` is the java print attribute, `print-attribute-value` is the corresponding attribute value, and `key` and `value` is the key/value pair used by a specific printer. > > For example, for `Chromaticity` attribute the property file could look like: > > Chromaticity.MONOCHROME.ColorModel=Gray > Chromaticity.COLOR.ColorModel=CMYK > Chromaticity.MONOCHROME.HPColorMode=grayscale > Chromaticity.COLOR.HPColorMode=color > > where `Chromaticity.MONOCHROME` key prefix corresponds to `Chromaticity.MONOCHOROME` print attribute constant with (... src/java.desktop/macosx/native/libawt_lwawt/awt/CPrinterJob.m line 551: > 549: > 550: if (chromaticityKeyValues == nil) { > 551: I'm not trying to revive this PR, but I was looking at it and I got curious about a couple of things. What is going on here ? First the Java code will never (I think) return NULL, so the "if" will never be used. Second, why in the case that it does, did you decide to set ColorModel=Gray or ColorModel=Color ? Are these a best guess default ? In which case why don't you always specify them anyway ? You are already throwing the entire set of ones you read from the conf file into the settings. Also there's no way I can see for you to KNOW that the settings will work. This could be confusing for apps wanting monochrome/gray Since you don't report back the supported status then they can't tell that when monochrome would work. and (btw) you don't even filter this by whether the printer supports color when setting Color. And on a broader note, the entire PPD API and PPD printer drivers are deprecated by CUPS. So this seems like an approach on which the clock is ticking before it is even deployed. test/jdk/javax/print/attribute/ChromaticityAttributeTest.java line 168: > 166: } > 167: > 168: private static Set getSupportedChromaticityAttributes() { The presence of this method is misleading in the test. You don't use it and just hard code Color + Monochrome in what you test. And in fact I don't see anywhere you've updated the implementation to report supported values. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18195#discussion_r1569463240 PR Review Comment: https://git.openjdk.org/jdk/pull/18195#discussion_r1569440468 From prr at openjdk.org Wed Apr 17 20:44:01 2024 From: prr at openjdk.org (Phil Race) Date: Wed, 17 Apr 2024 20:44:01 GMT Subject: RFR: 8315113: Print request Chromaticity.MONOCHROME attribute does not work on macOS In-Reply-To: <9NaQ7MzCph2B2CYYqk_o_yMqLtOz45sWsq8fbCk67Es=.d278e2e4-2dfe-499f-913f-b7ec47e577b9@github.com> References: <9NaQ7MzCph2B2CYYqk_o_yMqLtOz45sWsq8fbCk67Es=.d278e2e4-2dfe-499f-913f-b7ec47e577b9@github.com> Message-ID: On Mon, 8 Apr 2024 23:19:13 GMT, Phil Race wrote: >> The fix provides ability to print Black & White pages on macOS. >> >> Cocoa API has [PMSetColorMode](https://developer.apple.com/documentation/applicationservices/core_printing/1805783-pmsetcolormode) function but it is marked as deprecated and really does nothing. >> >> There is no replacement; this function was included to facilitate porting legacy applications to macOS, >> but it serves no useful purpose. >> >> Dumping `NSPrintInfo` print settings which were set by the native print dialog on macOS shows that the keys and values used for Black & White printing depend on the used printer type. >> For example, the tested >> `HP Color LaserJet MFP M180n` printer uses `ColorModel` key and`Gray` value, and >> `HP Ink Tank 110 series` uses `HPColorMode` key and `grayscale` value. >> >> Printing all `NSPrintInfo` presets shows that they do not contain key/value pairs for Black&White settings. >> This is the code snippet used to print `NSPrintInfo` presets: >> >> PMPrinter pr; >> PMPrintSession printSession = (PMPrintSession)[printInfo PMPrintSession]; >> OSStatus status = PMSessionGetCurrentPrinter(printSession, &pr); >> CFArrayRef presetsList = nil; >> status = PMPrinterCopyPresets(pr, &presetsList); >> CFIndex arrayCount = CFArrayGetCount(presetsList); >> >> for (CFIndex index = 0; index < arrayCount; index++) { >> PMPreset preset = (PMPreset)CFArrayGetValueAtIndex(presetsList, index); >> CFStringRef presetName = nil; >> if (PMPresetCopyName(preset, &presetName) == noErr && CFStringGetLength(presetName) > 0) { >> NSLog(@" presetName: '%@'", presetName); >> NSDictionary* dict = nil; >> if (PMPresetGetAttributes(preset, (CFDictionaryRef*)(&dict)) == noErr) { >> // print preset dict >> >> >> The idea of the proposed fix is to store printer dependent key/value pairs in the `/conf/printer.properties` property file. >> >> The property file has the format: >> >> print-attribute.print-attribute-value.key=value >> >> where `print-attribute` is the java print attribute, `print-attribute-value` is the corresponding attribute value, and `key` and `value` is the key/value pair used by a specific printer. >> >> For example, for `Chromaticity` attribute the property file could look like: >> >> Chromaticity.MONOCHROME.ColorModel=Gray >> Chromaticity.COLOR.ColorModel=CMYK >> Chromaticity.MONOCHROME.HPColorMode=grayscale >> Chromaticity.COLOR.HPColorMode=color >> >> where `Chromaticity.MO... > > Since the user can always select greyscale in the user dialog, I presume you have some requirement > to do this printing without user intervention. Perhaps there's no user there (server-side printing), or > the app would like to make sure the user always defaults to B&W. > > As far as I can tell, this isn't supported via any kind of API on macOS, so it is not a Java problem. > You could be printing from an Objective-C app written directly to Cocoa APIs and still have the same problem. > So a Java-specific workaround like this seems inappropriate. > > I've poked around to help me understand what is going on. > > When I do "Print to File" on macOS - meaning use the native printer dialog's "Save as PDF" option, > then the generated PDF is always colour even if I select the printer dialog's "Grayscale Printing" option. > This isn't just true for Java apps, the same happens if I print a web page from Safari. > > And when I query the Chromaticity support for a couple of colour printers - one Canon, one HP, > both report they support it but only the value COLOR. > > So I conclude the grayscale printing option is something handled by the printer driver at the time it is converted to > the printer-specific format and sent to the physical printer and it isn't changing the rendering. > > So in general without an API, to get greyscale it requires the end-user to set an option in a print dialog > and as mentioned above, what you are trying o enable isn't possible even if you wrote the app in > Objective-C or Swift directly as a macOS native app. > > But what works for me to get greyscale by default is to set that up as the default for the print queue. > It is easy to add a printer twice - with two queues for it. > One has the default set to greyscale, the other to colour. Then the app just needs to select the required queue. > This is clearly more site-specific than having the app just specify MONOCHROME but is a lot lower cost all round. > > I don't understand why PMSetColorMode is deprecated and non-functional since it seems like it would > be the right option here, but then internally macOS would need to be able to map it to the right option. > Doubtless that would be easier if MONOCHROME were supported by all these drivers and I don't understand > that omission either. In fact if it were we probably could just specify that directly without macOS needing > to understand anything - like I expect it doesn't understand the key/value pairs your code is sending it. > I note that you send ALL known key/value pairs and hope one ... > @prrace thanks for the detailed thoughts on this problem. As the stakeholder in this issue, I wanted to offer some feedback to your thoughts. I understand that the desire for OpenJDK is to NOT maintain a list of proprietary printer settings, but since many of the thoughts above get into use-case scenarios, I feel compelled to share my perspective. I hope these are well received. > > > When I do "Print to File" on macOS - meaning use the native printer dialog's "Save as PDF" option, > > then the generated PDF is always colour even if I select the printer dialog's "Grayscale Printing" option. > > This isn't just true for Java apps, the same happens if I print a web page from Safari. > > Several PDF printers (on several OSs) ignore grayscale/b&w settings. I'm not sure why, but I've observed the same. > > > As far as I can tell, this isn't supported via any kind of API on macOS, so it is not a Java problem. > > You could be printing from an Objective-C app written directly to Cocoa APIs and still have the same problem. > > Agreed. > > > So a Java-specific workaround like this seems inappropriate. > > As the stakeholder (biased) it's hard for me to agree, however allowing the implementing application to maintain (and thus offer) this workaround would certainly suffice. Since AWT (quite graciously) abstracts these attributes away from the user, "wontfix" status naturally causes a feature gap, albeit not the fault of OpenJDK, but a parity that -- at a bare minimum -- could live on as -- for example -- a workaround on Stackoverflow, etc. > > > But what works for me to get greyscale by default is to set that up as the default for the print queue. > > It is easy to add a printer twice - with two queues for it. > > Easy is subjective (and prescriptive). Again, biased, but setting up additional printer queues is a viable workaround, but comes at a factor of 2 queues for each color printer added to a system. This also offers support redundancies when removing and re-adding a printer (e.g. IP, port or driver change). > > > I don't understand why `PMSetColorMode` is deprecated and non-functional since it seems like it would > > be the right option here, but then internally macOS would need to be able to map it to the right option. > > Agreed. > > > [...] all it takes is some vendor API change or a different printer and it just won't work again. > > These points and the implied maintenance cost are some of my bigger concerns with this. > > Understood. > > > The other idea that crossed my mind is that instead of relying on a printing solution, we could > > explicitly render in greyscale. Then we could always declare MONOCHROME support and just emulate it. > > But I am not sure if this is really possible - it would require changes to the rendering code and > > I'm not sure if or how if we'd be able to do that properly and safely. And the risks of that clearly extend beyond printing. > > Since printers can emulate grayscale using color cartridges, I do not believe this will yield the same results as expected on all printers, but I do believe this to be a creative stop-gap/workaround which may be suitable for some use-cases. Naturally, I would expect some performance implications as a result. As the stakeholder, I can say that this will be trickier with prints using vector graphics than those already rastered/pixels. Speculating, vector graphics will likely be less performance-affecting, but require diving into the document modifications (e.g. PDFBOX, JavaFX) albeit more performant, will require some creative code to desaturate all color data in all objects prior to sending (or requests to add APIs thereo to each specific project). > > > Perhaps there's no user there (server-side printing), or the app would like to make sure the user always defaults to B&W. > > I'm not sure how much interest there is in understanding the use-case that discovered this issue, but it's part of a (pretty popular) Java app that helps print documents. It does so through a WebSocket interface... so in that sense it is server-like (effectively headless), although it's predominantly used by ordinary, non-technical end-users and runs as a sort of background service. Much like AWT offers a (rather) standardized API for printing, so does the Websocket interface, so "Black & White" or "Monochrome" are offered as common, selectable options. "Always default to B&W" would be a case-by-case situation, depending on how the websocket API is offered up (e.g. web app) to the end-user. Use cases are always interesting. The idea about changing our rendering to be greyscale would be easier if the printing API explicitly supported it. It could be quite challenging and risky to actually do it otherwise, so it is just an idea. Windows/GDI printing lets you set whether you want color or not with a standard field in the struct which controls printing https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-devmodew SFAIK that isn't using some driver-specific way of rendering and it seems to work fine. Although I've yet to find any evidence of it being available, it is possible that printers using IPP on macOS will support print-color-mode as documented here : https://www.pwg.org/ipp/ippguide.html If that becomes standard and widely available it would be a better way to do this. But I need to understand it better, which would start with finding *anything* that supports it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18195#issuecomment-2062236965 From ihse at openjdk.org Wed Apr 17 20:53:03 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 17 Apr 2024 20:53:03 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: On Wed, 17 Apr 2024 15:09:50 GMT, Kim Barrett wrote: >> https://man7.org/linux/man-pages/man3/alloca.3.html sounds like solution 2 is the cleanest one ("standards conformance"). It is also the version with minimal code and which will even work with future alloca usages :-) >> If solution 2 has any disadvantage, I'd prefer solution 3. > > I'm aware of this discussion and looking into the issues, but a personal matter has intervened and it will take > me a while to respond properly. Maybe next week. I opened https://bugs.openjdk.org/browse/JDK-8330539 so we don't lose track of this, but we can keep the discussion/voting here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1569528717 From kbarrett at openjdk.org Thu Apr 18 04:29:02 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Thu, 18 Apr 2024 04:29:02 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: On Wed, 17 Apr 2024 20:49:37 GMT, Magnus Ihse Bursie wrote: >> I'm aware of this discussion and looking into the issues, but a personal matter has intervened and it will take >> me a while to respond properly. Maybe next week. > > I opened https://bugs.openjdk.org/browse/JDK-8330539 so we don't lose track of this, but we can keep the discussion/voting here. For the impatient, I suggest adopting mechanism 2, i.e. unconditionally include in globalDefinitions_gcc.hpp. We can't include in shared code, and there is a use in shared code (in the relatively recently added JavaThread::pretouch_stack). When I questioned whether we needed to include at all, I referred to a Linux man page I'd found on the internet (the same page mdoerr linked to), which says (in part) "By default, modern compilers automatically translate all uses of alloca() into the built-in ..." Apparently I should have kept digging, because it seems that page is old/incorrect. A seemingly more recent Linux man page describes a different way of handling it that is closer to what we're seeing, but still not quite correct. glibc's includes if __USE_MISC is defined. One of the ways __USE_MISC can become defined is if _GNU_SOURCE is defined, and we define that for both gcc and clang toolchains. We include in globalDefinitions_gcc.hpp. So when building with gcc, globalDefinitions.hpp implicitly includes . The glibc definition of alloca is #ifdef __GNUC__ # define alloca(size) __builtin_alloca (size) #endif /* GCC. */ So that explains why we don't need any explicit include of when building with gcc. I expect there's something similar going on with Visual Studio and Xcode/clang. But apparently not with Open XLC clang. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1569930781 From jwaters at openjdk.org Thu Apr 18 05:33:05 2024 From: jwaters at openjdk.org (Julian Waters) Date: Thu, 18 Apr 2024 05:33:05 GMT Subject: RFR: 8330107: Separate out "awt" libraries from Awt2dLibraries.gmk [v3] In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 18:34:49 GMT, Magnus Ihse Bursie wrote: > There is no good IDE support for Makefiles, especially not with the level of customization that we have done to make. I am positively struggling with navigating this file, time and time again. I find Eclipse CDT to be rather helpful with Makefile syntax highlighting, if that helps. Just have to set the .gmk extension to be recognized as a Makefile, and then you should be good to go ------------- PR Comment: https://git.openjdk.org/jdk/pull/18743#issuecomment-2063032309 From tanksherman27 at gmail.com Thu Apr 18 12:27:18 2024 From: tanksherman27 at gmail.com (Julian Waters) Date: Thu, 18 Apr 2024 20:27:18 +0800 Subject: Questions about the Hermetic Java project In-Reply-To: <624a831e-6f5d-47dc-a2c9-3040dabb8fd4@oracle.com> References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> <5b35878b-6413-4e3f-b9e3-a9b761516c99@oracle.com> <624a831e-6f5d-47dc-a2c9-3040dabb8fd4@oracle.com> Message-ID: I have good (and bad news) for you! The good news is that from memory, ld and gcc (but I assume you are only concerned about ld) can work entirely on cl compiled object files with no hitch whatsoever, and partial linking is fully functional with ld on Windows! Symbol hiding also works, but with a gotcha. Now, the bad news. --localize-hidden, to my knowledge, only works on ELF visibilities. This means that for the following file: exports.c __declspec(dllexport) void exportedMethod() () void unexportedMethod() {} When objcopy is run over exports.o with --localize-hidden, it will work on the cl compiled object file just fine, but it will not recognise unexportedMethod() as a hidden method, because dllexport and hidden visibility are entirely separate concepts to gcc. The resulting exports-objcopy.o file will still have unexportedMethod as public. However, not all is lost just yet. --localize-symbol= still works perfectly fine, and can be used as a workaround if we can find a way to check for non-dllexport methods in cl object files and then feed that through the command line. I should note that clang also suffers from the same issue, as --localize-hidden on clang's objcopy also leaves unexportedMethod as public. Happy to help! best regards, Julian On Thu, Apr 18, 2024 at 6:28?PM Magnus Ihse Bursie wrote: > > On 2024-04-17 14:06, Julian Waters wrote: > > > Hi Magnus, > > > > Yes, I'm talking about the MSYS2 objcopy, but to a lesser extent also > > standalone Windows objcopy builds too. objcopy should be able to > > handle .o files from cl.exe (I assume that's what everyone here is > > after considering all the talk about .o files?), but to answer that > > question properly, I'd need a bit more detail. What kind of usage of > > objcopy do you have in mind? A general-ish command line example could > > be helpful > > To make a static library behave somewhat like a dynamic library, and not > expose all its internal symbols to the rest of the world, there are > basically two operations that needs to be done: > > 1) Combine all .o files into a single .o file, to make it look like it > was compiled by a single source code. That way, symbols that were > created in one source file and referenced in another will now behave as > if they are internal to the "compilation unit", i.e. like they were > declared static. > > 2) Modify the symbol status so that symbols that are not exported are > changed so they look like they were actually declared "static" in the > source code. > > These operations are based on concepts that exists in the gcc and clang > toolchain, about symbol visibility. I am not sure how well they > translate to a Microsoft setting. But, if the dll_export marker > corresponds to visible symbols, then I guess we should be able to > achieve something similar. > > What needs to be done then is: > > 1) Combine multiple .obj (COFF object files) into one. > > 2) Change the visibility of symbols that are not marked as dll_export:ed > to they appear like they were declared static. > > In the clang/gcc world, the first step is done by "partial linking" by > ld. That is our first blocker -- link.exe cannot do that. So the first > question is really, is there a Windows build of ld that can work on COFF > files to achieve this? > > The second step is done by objcopy using the "--localize-hidden" > argument. The second question is, could this work on a COFF object file? > > /Magnus > > > > > > best regards, > > Julian > > > > On Wed, Apr 17, 2024 at 5:55?PM Magnus Ihse Bursie > > wrote: > >> On 2024-04-16 07:23, Julian Waters wrote: > >> > >>>> And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. > >>> objcopy is also available on Windows, if the question about > >>> alternative tooling is still unanswered :) > >> At this point, I think support for static build on Windows is to either > >> require additional tooling on top of the Microsoft Visual Studio > >> toolchain, or to drop it completely, so I am definitely interested in > >> researching alternatives. > >> > >> Can objcopy (I assume this is from msys?) deal with COFF files generated > >> by cl? > >> > >> Switching the entire toolchain is not relevant at this point (but if a > >> non-Microsoft toolchain build for Windows is ever integrated, it might > >> get static builds with no extra work as a bonus), but I could certainly > >> accept the idea of having one or a few additional tools required to get > >> the normal Microsoft toolchain to produce static builds. > >> > >> /Magnus > >> > >>> best regards, > >>> Julian > >>> > >>> On Fri, Apr 12, 2024 at 7:52?PM Magnus Ihse Bursie > >>> wrote: > >>>> On 2024-04-02 21:16, Jiangli Zhou wrote: > >>>> > >>>> Hi Magnus, > >>>> > >>>> In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) discussed how to move forward contributing the static Java related changes (additional runtime fixes/enhancements on top of the existing static support in JDK) from https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK mainline. > >>>> > >>>> Just a bit more details/context below, which may be useful for others reading this thread. > >>>> > >>>> The https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch currently contains following for supporting hermetic Java (without the launcher work for runtime support): > >>>> > >>>> 1. Build change for linking the Java launcher (as bin/javastatic) with JDK/hotspot static libraries (.a), mainly in https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. The part for creating the complete sets of static libraries (including libjvm.a) has already been included in the mainline since last year. https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk is in a very raw state and is intended to demonstrate the capability of building a static Java launcher. > >>>> > >>>> Indeed. It is nowhere near being able to be integrated. > >>>> > >>>> > >>>> 2. Additional runtime fixes/enhancements on top of the existing static support in JDK, e.g. support further lookup dynamic native library if the built-in native library cannot be found. > >>>> > >>>> 3. Some initial (prototype) work on supporting hermetic JDK resource files in the jimage (JDK modules image). > >>>> > >>>> To move forward, one of the earliest items needed is to add the capability of building the fully statically linked Java launcher in JDK mainline. The other static Java runtime changes can be followed up after the launcher linking part, so they can be built and tested as individual PRs created for the JDK mainline. Magnus, you have expressed interest in helping get the launcher linking part (refactor from https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) into JDK mainline. What's your thought on prioritizing the launcher static linking part before other makefile clean ups for static libraries? > >>>> > >>>> Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. > >>>> > >>>> I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? > >>>> > >>>> The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. > >>>> > >>>> The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. > >>>> > >>>> My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). > >>>> > >>>> This, in turn, require several changes: > >>>> > >>>> 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > >>>> > >>>> 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > >>>> > >>>> This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > >>>> > >>>> And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. > >>>> > >>>> A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. > >>>> > >>>> My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. > >>>> > >>>> /Magnus > >>>> > >>>> > >>>> > >>>> Thanks! > >>>> Jiangli > >>>> > >>>> On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou wrote: > >>>>> On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou wrote: > >>>>>> Hi Magnus, > >>>>>> > >>>>>> Thanks for looking into this from the build perspective. > >>>>>> > >>>>>> On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie > >>>>>> wrote: > >>>>>>> First some background for build-dev: I have spent some time looking at > >>>>>>> the build implications of the Hermetic Java effort, which is part of > >>>>>>> Project Leyden. A high-level overview is available here: > >>>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current source > >>>>>>> code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. > >>>>>> Some additional hermetic Java related references that are also useful: > >>>>>> > >>>>>> - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug that > >>>>>> links to the issues for resolving static linking issues so far > >>>>>> - https://github.com/openjdk/jdk21/pull/26 is the enhancement for > >>>>>> building the complete set of static libraries in JDK/VM, particularly > >>>>>> including libjvm.a > >>>>>> > >>>>>>> Hermetic Java faces several challenges, but the part that is relevant > >>>>>>> for the build system is the ability to create static libraries. We've > >>>>>>> had this functionality (in three different ways...) for some time, but > >>>>>>> it is rather badly implemented. > >>>>>>> > >>>>>>> As a result of my investigations, I have a bunch of questions. :-) I > >>>>>>> have gotten some answers in private discussion, but for the sake of > >>>>>>> transparency I will repeat them here, to foster an open dialogue. > >>>>>>> > >>>>>>> 1. Am I correct in understanding that the ultimate goal of this exercise > >>>>>>> is to be able to have jmods which include static libraries (*.a) of the > >>>>>>> native code which the module uses, and that the user can then run a > >>>>>>> special jlink command to have this linked into a single executable > >>>>>>> binary (which also bundles the *.class files and any additional > >>>>>>> resources needed)? > >>>>>>> > >>>>>>> 2. If so, is the idea to create special kinds of static jmods, like > >>>>>>> java.base-static.jmod, that contains *.a files instead of lib*.so files? > >>>>>>> Or is the idea that the normal jmod should contain both? > >>>>>>> > >>>>>>> 3. Linking .o and .a files into an executable is a formidable task. Is > >>>>>>> the intention to have jlink call a system-provided ld, or to bundle ld > >>>>>>> with jlink, or to reimplement this functionality in Java? > >>>>>> I have a similar view as Alan responded in your other email thread. > >>>>>> Things are still in the early stage for the general solution. > >>>>>> > >>>>>> In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime > >>>>>> branch, when configuring JDK with --with-static-java=yes, the JDK > >>>>>> binary contains the following extra artifacts: > >>>>>> > >>>>>> - static-libs/*.a: The complete set of JDK/VM static libraries > >>>>>> - jdk/bin/javastatic: A demo Java launcher fully statically linked > >>>>>> with the selected JDK .a libraries (e.g. it currently statically link > >>>>>> with the headless) and libjvm.a. It's the standard Java launcher > >>>>>> without additional work for hermetic Java. > >>>>>> > >>>>>> In our prototype for hermetic Java, we build the hermetic executable > >>>>>> image (a single image) from the following input (see description on > >>>>>> singlejar packaging tool in > >>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf): > >>>>>> > >>>>>> - A customized launcher (with additional work for hermetic) executable > >>>>>> fully statically linked with JDK/VM static libraries (.a files), > >>>>>> application natives and dependencies (e.g. in .a static libraries) > >>>>>> - JDK lib/modules, JDK resource files > >>>>>> - Application classes and resource files > >>>>>> > >>>>>> Including a JDK library .a into the corresponding .jmod would require > >>>>>> extracting the .a for linking with the executable. In some systems > >>>>>> that may cause memory overhead due to the extracted copy of the .a > >>>>>> files. I think we should consider the memory overhead issue. > >>>>>> > >>>>>> One possibility (as Alan described in his response) is for jlink to > >>>>>> invoke the ld on the build system. jlink could pass the needed JDK > >>>>>> static libraries and libjvm.a (provided as part of the JDK binary) to > >>>>>> ld based on the modules required for the application. > >>>>>> > >>>>> I gave a bit more thoughts on this one. For jlink to trigger ld, it > >>>>> would need to know the complete linker options and inputs. Those > >>>>> include options and inputs related to the application part as well. In > >>>>> some usages, it might be easier to handle native linking separately > >>>>> and pass the linker output, the executable to jlink directly. Maybe we > >>>>> could consider supporting different modes for various usages > >>>>> requirements, from static libraries and native linking point of view: > >>>>> > >>>>> Mode #1 > >>>>> Support .jmod packaged natives static libraries, for both JDK/VM .a > >>>>> and application natives and dependencies. If the inputs to jlink > >>>>> include .jmods, jlink can extract the .a libraries and pass the > >>>>> information to ld to link the executable. > >>>>> > >>>>> Mode #2 > >>>>> Support separate .a as jlink input. Jlink could pass the path > >>>>> information to the .a libraries and other linker options to ld to > >>>>> create the executable. > >>>>> > >>>>> For both mode #1 and #2, jlink would then use the linker output > >>>>> executable to create the final hermetic image. > >>>>> > >>>>> Mode #3 > >>>>> Support a fully linked executable as a jlink input. When a linked > >>>>> executable is given to jlink, it can process it directly with other > >>>>> JDK data/files to create the final image, without native linking step. > >>>>> > >>>>> Any other thoughts and considerations? > >>>>> > >>>>> Best, > >>>>> Jiangli > >>>>> > >>>>>>> 4. Is the intention is to allow users to create their own jmods with > >>>>>>> static libraries, and have these linked in as well? This seems to be the > >>>>>>> case. > >>>>>> An alternative with less memory overhead could be using application > >>>>>> modular JAR and separate .a as the input for jlink. > >>>>>> > >>>>>>> If that is so, then there will always be the risk for name > >>>>>>> collisions, and we can only minimize the risk by making sure any global > >>>>>>> names are as unique as possible. > >>>>>> Part of the current effort includes resolving the discovered symbol > >>>>>> collision issues with static linking. Will respond to your other email > >>>>>> on the symbol issue separately later. > >>>>>> > >>>>>>> 5. The original implementation of static builds in the JDK, created for > >>>>>>> the Mobile project, used a configure flag, --enable-static-builds, to > >>>>>>> change the entire behavior of the build system to only produce *.a files > >>>>>>> instead of lib*.so. In contrast, the current system is using a special > >>>>>>> target instead. > >>>>>> I think we would need both configure flag and special target for the > >>>>>> static builds. > >>>>>> > >>>>>>> In my eyes, this is a much worse solution. Apart from > >>>>>>> the conceptual principle (if the build should generate static or dynamic > >>>>>>> libraries is definitely a property of what a "configuration" means), > >>>>>>> this makes it much harder to implement efficiently, since we cannot make > >>>>>>> changes in NativeCompilation.gmk, where they are needed. > >>>>>> For the potential objcopy work to resolve symbol issues, we can add > >>>>>> that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. We > >>>>>> have an internal prototype (not included in > >>>>>> https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) done > >>>>>> by one of colleagues for localizing symbols in libfreetype using > >>>>>> objcopy. > >>>>>> > >>>>>>> That was not as much a question as a statement. ? But here is the > >>>>>>> question: Do you think it would be reasonable to restore the old > >>>>>>> behavior but with the new methods, so that we don't use special targets, > >>>>>>> but instead tells configure to generate static libraries? I'm thinking > >>>>>>> we should have a flag like "--with-library-type=" that can have values > >>>>>>> "dynamic" (which is default), "static" or "both". > >>>>>> If we want to also build a fully statically linked launcher, maybe > >>>>>> --with-static-java? Being able to configure either dynamic, static or > >>>>>> both as you suggested also seems to be a good idea. > >>>>>> > >>>>>>> I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files > >>>>>>> into a single jmod file (see question 2 above), then it definitely is. > >>>>>>> In general, the cost of producing two kinds of libraries are quite > >>>>>>> small, compared to the cost of compiling the source code to object files. > >>>>>> Completely agree. It would be good to avoid recompiling the .o file > >>>>>> for static and dynamic builds. As proposed in > >>>>>> https://bugs.openjdk.org/browse/JDK-8303796: > >>>>>> > >>>>>> It's beneficial to be able to build both .so and .a from the same set > >>>>>> of .o files. That would involve some changes to handle the dynamic JDK > >>>>>> and static JDK difference at runtime, instead of relying on the > >>>>>> STATIC_BUILD macro. > >>>>>> > >>>>>>> Finally, I have looked at how to manipulate symbol visibility. There > >>>>>>> seems many ways forward, so I feel confident that we can find a good > >>>>>>> solution. > >>>>>>> > >>>>>>> One way forward is to use objcopy to manipulate symbol status > >>>>>>> (global/local). There is an option --localize-symbol in objcopy, that > >>>>>>> has been available in objcopy since at least 2.15, which was released > >>>>>>> 2004, so it should be safe to use. But ideally we should avoid using > >>>>>>> objcopy and do this as part of the linking process. This should be > >>>>>>> possible to do, given that we make changes in NativeCompilation.gmk -- > >>>>>>> see question 5 above. > >>>>>>> > >>>>>>> As a fallback, it is also possible to rename symbols, either piecewise > >>>>>>> or wholesale, using objcopy. There are many ways to do this, using > >>>>>>> --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this > >>>>>>> takes a file with a list of symbols). Thus we can always introduce a > >>>>>>> "post factum namespace" by renaming symbols. > >>>>>> Renaming or redefining the symbol at build time could cause confusions > >>>>>> with debugging. That's a concern raised in > >>>>>> https://github.com/openjdk/jdk/pull/17456 discussions. > >>>>>> > >>>>>> Additionally, redefining symbols using tools like objcopy may not > >>>>>> handle member names referenced in string literals. For example, in > >>>>>> https://github.com/openjdk/jdk/pull/17456 additional changes are > >>>>>> needed in assembling and SA to reflect the symbol change. > >>>>>> > >>>>>>> So in the end, I think it will be fully possible to produce .a files > >>>>>>> that only has global symbols for the functions that are part of the API > >>>>>>> exposed by that library, and have all other symbols local, and make this > >>>>>>> is in a way that is consistent with the rest of the build system. > >>>>>>> > >>>>>>> Finally, a note on Hotspot. Due to debugging reasons, we export > >>>>>>> basically all symbols in hotspot as global. This is not reasonable to do > >>>>>>> for a static build. The effect of not exporting those symbols will be > >>>>>>> that SA will not function to 100%. On the other hand, I have no idea if > >>>>>>> SA works at all with a static build. Have you tested this? Is this part > >>>>>>> of the plan to support, or will it be officially dropped for Hermetic Java? > >>>>>> We have done some testing with jtreg SA related tests for the fully > >>>>>> statically linked `javastatic`. > >>>>>> > >>>>>> If we use objcopy to localize symbols in hotspot, it's not yet clear > >>>>>> what's the impact on SA. We could do some tests. The other question > >>>>>> that I raised is the supported gcc versions (for partial linking) > >>>>>> related to the solution. > >>>>>> > >>>>>> Best, > >>>>>> Jiangli > >>>>>> > >>>>>>> /Magnus > >>>>>>> From tanksherman27 at gmail.com Thu Apr 18 12:31:25 2024 From: tanksherman27 at gmail.com (Julian Waters) Date: Thu, 18 Apr 2024 20:31:25 +0800 Subject: Questions about the Hermetic Java project In-Reply-To: References: <6a9afe3f-e232-4636-8a2e-6112a6e68cce@oracle.com> <5b35878b-6413-4e3f-b9e3-a9b761516c99@oracle.com> <624a831e-6f5d-47dc-a2c9-3040dabb8fd4@oracle.com> Message-ID: Oops, I meant __declspec(dllexport) void exportedMethod() {} with braces, not brackets On Thu, Apr 18, 2024 at 8:27?PM Julian Waters wrote: > > I have good (and bad news) for you! > > The good news is that from memory, ld and gcc (but I assume you are > only concerned about ld) can work entirely on cl compiled object files > with no hitch whatsoever, and partial linking is fully functional with > ld on Windows! Symbol hiding also works, but with a gotcha. > > Now, the bad news. --localize-hidden, to my knowledge, only works on > ELF visibilities. This means that for the following file: > > exports.c > __declspec(dllexport) void exportedMethod() () > void unexportedMethod() {} > > When objcopy is run over exports.o with --localize-hidden, it will > work on the cl compiled object file just fine, but it will not > recognise unexportedMethod() as a hidden method, because dllexport and > hidden visibility are entirely separate concepts to gcc. The resulting > exports-objcopy.o file will still have unexportedMethod as public. > However, not all is lost just yet. --localize-symbol= still > works perfectly fine, and can be used as a workaround if we can find a > way to check for non-dllexport methods in cl object files and then > feed that through the command line. I should note that clang also > suffers from the same issue, as --localize-hidden on clang's objcopy > also leaves unexportedMethod as public. > > Happy to help! > > best regards, > Julian > > On Thu, Apr 18, 2024 at 6:28?PM Magnus Ihse Bursie > wrote: > > > > On 2024-04-17 14:06, Julian Waters wrote: > > > > > Hi Magnus, > > > > > > Yes, I'm talking about the MSYS2 objcopy, but to a lesser extent also > > > standalone Windows objcopy builds too. objcopy should be able to > > > handle .o files from cl.exe (I assume that's what everyone here is > > > after considering all the talk about .o files?), but to answer that > > > question properly, I'd need a bit more detail. What kind of usage of > > > objcopy do you have in mind? A general-ish command line example could > > > be helpful > > > > To make a static library behave somewhat like a dynamic library, and not > > expose all its internal symbols to the rest of the world, there are > > basically two operations that needs to be done: > > > > 1) Combine all .o files into a single .o file, to make it look like it > > was compiled by a single source code. That way, symbols that were > > created in one source file and referenced in another will now behave as > > if they are internal to the "compilation unit", i.e. like they were > > declared static. > > > > 2) Modify the symbol status so that symbols that are not exported are > > changed so they look like they were actually declared "static" in the > > source code. > > > > These operations are based on concepts that exists in the gcc and clang > > toolchain, about symbol visibility. I am not sure how well they > > translate to a Microsoft setting. But, if the dll_export marker > > corresponds to visible symbols, then I guess we should be able to > > achieve something similar. > > > > What needs to be done then is: > > > > 1) Combine multiple .obj (COFF object files) into one. > > > > 2) Change the visibility of symbols that are not marked as dll_export:ed > > to they appear like they were declared static. > > > > In the clang/gcc world, the first step is done by "partial linking" by > > ld. That is our first blocker -- link.exe cannot do that. So the first > > question is really, is there a Windows build of ld that can work on COFF > > files to achieve this? > > > > The second step is done by objcopy using the "--localize-hidden" > > argument. The second question is, could this work on a COFF object file? > > > > /Magnus > > > > > > > > > > best regards, > > > Julian > > > > > > On Wed, Apr 17, 2024 at 5:55?PM Magnus Ihse Bursie > > > wrote: > > >> On 2024-04-16 07:23, Julian Waters wrote: > > >> > > >>>> And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. > > >>> objcopy is also available on Windows, if the question about > > >>> alternative tooling is still unanswered :) > > >> At this point, I think support for static build on Windows is to either > > >> require additional tooling on top of the Microsoft Visual Studio > > >> toolchain, or to drop it completely, so I am definitely interested in > > >> researching alternatives. > > >> > > >> Can objcopy (I assume this is from msys?) deal with COFF files generated > > >> by cl? > > >> > > >> Switching the entire toolchain is not relevant at this point (but if a > > >> non-Microsoft toolchain build for Windows is ever integrated, it might > > >> get static builds with no extra work as a bonus), but I could certainly > > >> accept the idea of having one or a few additional tools required to get > > >> the normal Microsoft toolchain to produce static builds. > > >> > > >> /Magnus > > >> > > >>> best regards, > > >>> Julian > > >>> > > >>> On Fri, Apr 12, 2024 at 7:52?PM Magnus Ihse Bursie > > >>> wrote: > > >>>> On 2024-04-02 21:16, Jiangli Zhou wrote: > > >>>> > > >>>> Hi Magnus, > > >>>> > > >>>> In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) discussed how to move forward contributing the static Java related changes (additional runtime fixes/enhancements on top of the existing static support in JDK) from https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK mainline. > > >>>> > > >>>> Just a bit more details/context below, which may be useful for others reading this thread. > > >>>> > > >>>> The https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch currently contains following for supporting hermetic Java (without the launcher work for runtime support): > > >>>> > > >>>> 1. Build change for linking the Java launcher (as bin/javastatic) with JDK/hotspot static libraries (.a), mainly in https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. The part for creating the complete sets of static libraries (including libjvm.a) has already been included in the mainline since last year. https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk is in a very raw state and is intended to demonstrate the capability of building a static Java launcher. > > >>>> > > >>>> Indeed. It is nowhere near being able to be integrated. > > >>>> > > >>>> > > >>>> 2. Additional runtime fixes/enhancements on top of the existing static support in JDK, e.g. support further lookup dynamic native library if the built-in native library cannot be found. > > >>>> > > >>>> 3. Some initial (prototype) work on supporting hermetic JDK resource files in the jimage (JDK modules image). > > >>>> > > >>>> To move forward, one of the earliest items needed is to add the capability of building the fully statically linked Java launcher in JDK mainline. The other static Java runtime changes can be followed up after the launcher linking part, so they can be built and tested as individual PRs created for the JDK mainline. Magnus, you have expressed interest in helping get the launcher linking part (refactor from https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) into JDK mainline. What's your thought on prioritizing the launcher static linking part before other makefile clean ups for static libraries? > > >>>> > > >>>> Trust me, my absolute top priority now is working on getting the proper build support needed for Hermetic Java. I can't prioritize it any higher. > > >>>> > > >>>> I am not sure what you are asking for. We can't just merge StaticLink.gmk from your prototype. And even if we did, what good will it do you? > > >>>> > > >>>> The problem you are running into is that the build system has not been designed to properly support static linking. There are already 3-4 hacks in place to get something sort-of useful out, but they are prone to breaking. I assume that we agree that for Hermetic Java to become a success, we need to have a stable foundation for static builds. > > >>>> > > >>>> The core problem of all static linking hacks is that they are not integrated in the right place. They need to be a core part of what NativeCompilation delivers, not something done in a separate file. To put it in other words, StaticLink.gmk from your branch do not need cleanup -- it needs to go away, and the functionality moved to the proper place. > > >>>> > > >>>> My approach is that NativeCompilation should support doing either only dynamic linking (as today), or static linking (as today with STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the latter will be default, or at least should be tested by default in GHA. For this to work, we need to compile the source code to .o files only once, and then link these .o files either into a dynamic or a static library (or both). > > >>>> > > >>>> This, in turn, require several changes: > > >>>> > > >>>> 1) The linking code needs to be cleaned up, and all technical debt needs to be resolved. This is what I have been doing since I started working on static builds for Hermetic Java. JDK-8329704 (which was integrated yesterday) was the first major milestone of this cleanup. Now, the path were to find a library created by the JDK (static or dynamic) is encapsulated in ResolveLibPath. This is currently a monster, but at least all knowledge is collected in a single location, instead of spread over the code base. Getting this simplified is the next step. > > >>>> > > >>>> 2) We need to stop passing the STATIC_BUILD define when compiling. This is partially addressed in your PR, where you have replaced #ifdef STATIC_BUILD with a dynamic lookup. But there is also the problem of JNI/JVMTI entry points. I have been pondering how we can compile the code in a way so we support both dynamic and static name resolution, and I think I have a solution. > > >>>> > > >>>> This is unfortunately quite complex, and I have started a discussion with Alan if it is possible to update the JNI spec so that both static and dynamic entry points can have the form "JNI_OnLoad_". Ideally, I'd like to see us push for this with as much effort as possible. If we got this in place, static builds would be much easier, and the changes required for Hermetic Java even smaller. > > >>>> > > >>>> And finally, on top of all of this, is the question of widening the platform support. To support linux/gcc with objcopy is trivial, but the question about Windows still remain. I have two possible ways forward, one is to check if there is alternative tooling to use (the prime candidate is the clang-ldd), and the other is to try to "fake" a partial linking by concatenating all source code before compiling. This is not ideal, though, for many reasons, and I am not keen on implementing it, not even for testing. And at this point, I have not had time to investigate any of these options much further, since I have been focusing on 1) above. > > >>>> > > >>>> A third option is of course to just say that due to toolchain limitations, static linking is not available on Windows. > > >>>> > > >>>> My recommendation is that you keep on working to resolve the (much more thorny) issues of resource access in Hermetic Java in your branch, where you have a prototype static build that works for you. In the meantime, I will make sure that there will be a functioning, stable and robust way of creating static builds in the mainline, that can be regularly tested and not bit-rot, like the static build hacks that has gone in before. > > >>>> > > >>>> /Magnus > > >>>> > > >>>> > > >>>> > > >>>> Thanks! > > >>>> Jiangli > > >>>> > > >>>> On Thu, Feb 15, 2024 at 12:01?PM Jiangli Zhou wrote: > > >>>>> On Wed, Feb 14, 2024 at 5:07?PM Jiangli Zhou wrote: > > >>>>>> Hi Magnus, > > >>>>>> > > >>>>>> Thanks for looking into this from the build perspective. > > >>>>>> > > >>>>>> On Wed, Feb 14, 2024 at 1:00?AM Magnus Ihse Bursie > > >>>>>> wrote: > > >>>>>>> First some background for build-dev: I have spent some time looking at > > >>>>>>> the build implications of the Hermetic Java effort, which is part of > > >>>>>>> Project Leyden. A high-level overview is available here: > > >>>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current source > > >>>>>>> code is here: https://github.com/openjdk/leyden/tree/hermetic-java-runtime. > > >>>>>> Some additional hermetic Java related references that are also useful: > > >>>>>> > > >>>>>> - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug that > > >>>>>> links to the issues for resolving static linking issues so far > > >>>>>> - https://github.com/openjdk/jdk21/pull/26 is the enhancement for > > >>>>>> building the complete set of static libraries in JDK/VM, particularly > > >>>>>> including libjvm.a > > >>>>>> > > >>>>>>> Hermetic Java faces several challenges, but the part that is relevant > > >>>>>>> for the build system is the ability to create static libraries. We've > > >>>>>>> had this functionality (in three different ways...) for some time, but > > >>>>>>> it is rather badly implemented. > > >>>>>>> > > >>>>>>> As a result of my investigations, I have a bunch of questions. :-) I > > >>>>>>> have gotten some answers in private discussion, but for the sake of > > >>>>>>> transparency I will repeat them here, to foster an open dialogue. > > >>>>>>> > > >>>>>>> 1. Am I correct in understanding that the ultimate goal of this exercise > > >>>>>>> is to be able to have jmods which include static libraries (*.a) of the > > >>>>>>> native code which the module uses, and that the user can then run a > > >>>>>>> special jlink command to have this linked into a single executable > > >>>>>>> binary (which also bundles the *.class files and any additional > > >>>>>>> resources needed)? > > >>>>>>> > > >>>>>>> 2. If so, is the idea to create special kinds of static jmods, like > > >>>>>>> java.base-static.jmod, that contains *.a files instead of lib*.so files? > > >>>>>>> Or is the idea that the normal jmod should contain both? > > >>>>>>> > > >>>>>>> 3. Linking .o and .a files into an executable is a formidable task. Is > > >>>>>>> the intention to have jlink call a system-provided ld, or to bundle ld > > >>>>>>> with jlink, or to reimplement this functionality in Java? > > >>>>>> I have a similar view as Alan responded in your other email thread. > > >>>>>> Things are still in the early stage for the general solution. > > >>>>>> > > >>>>>> In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime > > >>>>>> branch, when configuring JDK with --with-static-java=yes, the JDK > > >>>>>> binary contains the following extra artifacts: > > >>>>>> > > >>>>>> - static-libs/*.a: The complete set of JDK/VM static libraries > > >>>>>> - jdk/bin/javastatic: A demo Java launcher fully statically linked > > >>>>>> with the selected JDK .a libraries (e.g. it currently statically link > > >>>>>> with the headless) and libjvm.a. It's the standard Java launcher > > >>>>>> without additional work for hermetic Java. > > >>>>>> > > >>>>>> In our prototype for hermetic Java, we build the hermetic executable > > >>>>>> image (a single image) from the following input (see description on > > >>>>>> singlejar packaging tool in > > >>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf): > > >>>>>> > > >>>>>> - A customized launcher (with additional work for hermetic) executable > > >>>>>> fully statically linked with JDK/VM static libraries (.a files), > > >>>>>> application natives and dependencies (e.g. in .a static libraries) > > >>>>>> - JDK lib/modules, JDK resource files > > >>>>>> - Application classes and resource files > > >>>>>> > > >>>>>> Including a JDK library .a into the corresponding .jmod would require > > >>>>>> extracting the .a for linking with the executable. In some systems > > >>>>>> that may cause memory overhead due to the extracted copy of the .a > > >>>>>> files. I think we should consider the memory overhead issue. > > >>>>>> > > >>>>>> One possibility (as Alan described in his response) is for jlink to > > >>>>>> invoke the ld on the build system. jlink could pass the needed JDK > > >>>>>> static libraries and libjvm.a (provided as part of the JDK binary) to > > >>>>>> ld based on the modules required for the application. > > >>>>>> > > >>>>> I gave a bit more thoughts on this one. For jlink to trigger ld, it > > >>>>> would need to know the complete linker options and inputs. Those > > >>>>> include options and inputs related to the application part as well. In > > >>>>> some usages, it might be easier to handle native linking separately > > >>>>> and pass the linker output, the executable to jlink directly. Maybe we > > >>>>> could consider supporting different modes for various usages > > >>>>> requirements, from static libraries and native linking point of view: > > >>>>> > > >>>>> Mode #1 > > >>>>> Support .jmod packaged natives static libraries, for both JDK/VM .a > > >>>>> and application natives and dependencies. If the inputs to jlink > > >>>>> include .jmods, jlink can extract the .a libraries and pass the > > >>>>> information to ld to link the executable. > > >>>>> > > >>>>> Mode #2 > > >>>>> Support separate .a as jlink input. Jlink could pass the path > > >>>>> information to the .a libraries and other linker options to ld to > > >>>>> create the executable. > > >>>>> > > >>>>> For both mode #1 and #2, jlink would then use the linker output > > >>>>> executable to create the final hermetic image. > > >>>>> > > >>>>> Mode #3 > > >>>>> Support a fully linked executable as a jlink input. When a linked > > >>>>> executable is given to jlink, it can process it directly with other > > >>>>> JDK data/files to create the final image, without native linking step. > > >>>>> > > >>>>> Any other thoughts and considerations? > > >>>>> > > >>>>> Best, > > >>>>> Jiangli > > >>>>> > > >>>>>>> 4. Is the intention is to allow users to create their own jmods with > > >>>>>>> static libraries, and have these linked in as well? This seems to be the > > >>>>>>> case. > > >>>>>> An alternative with less memory overhead could be using application > > >>>>>> modular JAR and separate .a as the input for jlink. > > >>>>>> > > >>>>>>> If that is so, then there will always be the risk for name > > >>>>>>> collisions, and we can only minimize the risk by making sure any global > > >>>>>>> names are as unique as possible. > > >>>>>> Part of the current effort includes resolving the discovered symbol > > >>>>>> collision issues with static linking. Will respond to your other email > > >>>>>> on the symbol issue separately later. > > >>>>>> > > >>>>>>> 5. The original implementation of static builds in the JDK, created for > > >>>>>>> the Mobile project, used a configure flag, --enable-static-builds, to > > >>>>>>> change the entire behavior of the build system to only produce *.a files > > >>>>>>> instead of lib*.so. In contrast, the current system is using a special > > >>>>>>> target instead. > > >>>>>> I think we would need both configure flag and special target for the > > >>>>>> static builds. > > >>>>>> > > >>>>>>> In my eyes, this is a much worse solution. Apart from > > >>>>>>> the conceptual principle (if the build should generate static or dynamic > > >>>>>>> libraries is definitely a property of what a "configuration" means), > > >>>>>>> this makes it much harder to implement efficiently, since we cannot make > > >>>>>>> changes in NativeCompilation.gmk, where they are needed. > > >>>>>> For the potential objcopy work to resolve symbol issues, we can add > > >>>>>> that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. We > > >>>>>> have an internal prototype (not included in > > >>>>>> https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) done > > >>>>>> by one of colleagues for localizing symbols in libfreetype using > > >>>>>> objcopy. > > >>>>>> > > >>>>>>> That was not as much a question as a statement. ? But here is the > > >>>>>>> question: Do you think it would be reasonable to restore the old > > >>>>>>> behavior but with the new methods, so that we don't use special targets, > > >>>>>>> but instead tells configure to generate static libraries? I'm thinking > > >>>>>>> we should have a flag like "--with-library-type=" that can have values > > >>>>>>> "dynamic" (which is default), "static" or "both". > > >>>>>> If we want to also build a fully statically linked launcher, maybe > > >>>>>> --with-static-java? Being able to configure either dynamic, static or > > >>>>>> both as you suggested also seems to be a good idea. > > >>>>>> > > >>>>>>> I am not sure if "both" are needed, but if we want to bundle both lib*.so and *.a files > > >>>>>>> into a single jmod file (see question 2 above), then it definitely is. > > >>>>>>> In general, the cost of producing two kinds of libraries are quite > > >>>>>>> small, compared to the cost of compiling the source code to object files. > > >>>>>> Completely agree. It would be good to avoid recompiling the .o file > > >>>>>> for static and dynamic builds. As proposed in > > >>>>>> https://bugs.openjdk.org/browse/JDK-8303796: > > >>>>>> > > >>>>>> It's beneficial to be able to build both .so and .a from the same set > > >>>>>> of .o files. That would involve some changes to handle the dynamic JDK > > >>>>>> and static JDK difference at runtime, instead of relying on the > > >>>>>> STATIC_BUILD macro. > > >>>>>> > > >>>>>>> Finally, I have looked at how to manipulate symbol visibility. There > > >>>>>>> seems many ways forward, so I feel confident that we can find a good > > >>>>>>> solution. > > >>>>>>> > > >>>>>>> One way forward is to use objcopy to manipulate symbol status > > >>>>>>> (global/local). There is an option --localize-symbol in objcopy, that > > >>>>>>> has been available in objcopy since at least 2.15, which was released > > >>>>>>> 2004, so it should be safe to use. But ideally we should avoid using > > >>>>>>> objcopy and do this as part of the linking process. This should be > > >>>>>>> possible to do, given that we make changes in NativeCompilation.gmk -- > > >>>>>>> see question 5 above. > > >>>>>>> > > >>>>>>> As a fallback, it is also possible to rename symbols, either piecewise > > >>>>>>> or wholesale, using objcopy. There are many ways to do this, using > > >>>>>>> --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, this > > >>>>>>> takes a file with a list of symbols). Thus we can always introduce a > > >>>>>>> "post factum namespace" by renaming symbols. > > >>>>>> Renaming or redefining the symbol at build time could cause confusions > > >>>>>> with debugging. That's a concern raised in > > >>>>>> https://github.com/openjdk/jdk/pull/17456 discussions. > > >>>>>> > > >>>>>> Additionally, redefining symbols using tools like objcopy may not > > >>>>>> handle member names referenced in string literals. For example, in > > >>>>>> https://github.com/openjdk/jdk/pull/17456 additional changes are > > >>>>>> needed in assembling and SA to reflect the symbol change. > > >>>>>> > > >>>>>>> So in the end, I think it will be fully possible to produce .a files > > >>>>>>> that only has global symbols for the functions that are part of the API > > >>>>>>> exposed by that library, and have all other symbols local, and make this > > >>>>>>> is in a way that is consistent with the rest of the build system. > > >>>>>>> > > >>>>>>> Finally, a note on Hotspot. Due to debugging reasons, we export > > >>>>>>> basically all symbols in hotspot as global. This is not reasonable to do > > >>>>>>> for a static build. The effect of not exporting those symbols will be > > >>>>>>> that SA will not function to 100%. On the other hand, I have no idea if > > >>>>>>> SA works at all with a static build. Have you tested this? Is this part > > >>>>>>> of the plan to support, or will it be officially dropped for Hermetic Java? > > >>>>>> We have done some testing with jtreg SA related tests for the fully > > >>>>>> statically linked `javastatic`. > > >>>>>> > > >>>>>> If we use objcopy to localize symbols in hotspot, it's not yet clear > > >>>>>> what's the impact on SA. We could do some tests. The other question > > >>>>>> that I raised is the supported gcc versions (for partial linking) > > >>>>>> related to the solution. > > >>>>>> > > >>>>>> Best, > > >>>>>> Jiangli > > >>>>>> > > >>>>>>> /Magnus > > >>>>>>> From jjg at openjdk.org Thu Apr 18 20:52:29 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 18 Apr 2024 20:52:29 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v4] In-Reply-To: References: Message-ID: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - improve handling of ignorable doc comments suppress warning when building test code - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - call `saveDanglingDocComments` for local variable declarations - adjust call for `saveDanglingDocComments` for enum members - JDK-8303689: javac -Xlint could/should report on "dangling" doc comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18527/files - new: https://git.openjdk.org/jdk/pull/18527/files/3f745431..f3670e7a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=02-03 Stats: 42074 lines in 1058 files changed: 18282 ins; 15937 del; 7855 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From jjg at openjdk.org Thu Apr 18 21:34:13 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 18 Apr 2024 21:34:13 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v5] In-Reply-To: References: Message-ID: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: update test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18527/files - new: https://git.openjdk.org/jdk/pull/18527/files/f3670e7a..8ad8b818 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=03-04 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From jjg at openjdk.org Thu Apr 18 21:56:16 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 18 Apr 2024 21:56:16 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v6] In-Reply-To: References: Message-ID: <_EMQ8vgc0hQdgeWdnqirLLAN8Cj6jcxP0belwJffD8A=.2c536c2b-f0e1-49b4-8fc7-e3a9252e20e2@github.com> > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: - Merge with upstream/master - update test - improve handling of ignorable doc comments suppress warning when building test code - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - call `saveDanglingDocComments` for local variable declarations - adjust call for `saveDanglingDocComments` for enum members - JDK-8303689: javac -Xlint could/should report on "dangling" doc comments ------------- Changes: https://git.openjdk.org/jdk/pull/18527/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=05 Stats: 488 lines in 61 files changed: 389 ins; 3 del; 96 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From dnsimon at openjdk.org Sun Apr 21 22:13:39 2024 From: dnsimon at openjdk.org (Doug Simon) Date: Sun, 21 Apr 2024 22:13:39 GMT Subject: RFR: 8330755: ProblemList files have entries referring to non-existent tests Message-ID: <5vZvc83Zn4IhI5s_IdYqRqw4zjWF93TcQUzl2cD5JLU=.12464c13-9ccc-47d8-851e-883f3fea4a04@github.com> This PR adds a check for the format of ProblemList files and ensures they only have entries referring to existing tests. The cleanups in the second commit of this PR were done based on the output of `CheckProblemLists`: > make test TEST=build/problemLists/CheckProblemLists.java ... STDOUT: Checking /Users/dnsimon/dev/jdk-jdk/open/test/hotspot/jtreg/ProblemList-Virtual.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/hotspot/jtreg/ProblemList-Xcomp.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/hotspot/jtreg/ProblemList-generational-zgc.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/hotspot/jtreg/ProblemList-zgc.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/hotspot/jtreg/ProblemList.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/jaxp/ProblemList.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList-Virtual.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList-Xcomp.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList-generational-zgc.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList-zgc.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/langtools/ProblemList.txt Checking /Users/dnsimon/dev/jdk-jdk/open/test/lib-test/ProblemList.txt Checked 13 problem list files Test roots: /Users/dnsimon/dev/jdk-jdk/open/test/jdk /Users/dnsimon/dev/jdk-jdk/open/test/lib-test /Users/dnsimon/dev/jdk-jdk/open/test/failure_handler/test /Users/dnsimon/dev/jdk-jdk/open/test/jaxp /Users/dnsimon/dev/jdk-jdk/open/test/langtools /Users/dnsimon/dev/jdk-jdk/open/test/hotspot/jtreg Following errors found: /Users/dnsimon/dev/jdk-jdk/open/test/hotspot/jtreg/ProblemList.txt:174: vmTestbase/gc/lock/jni/jnilock002/TestDescription.java does not exist under any test root vmTestbase/gc/lock/jni/jnilock002/TestDescription.java 8192647 generic-all /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList-Virtual.txt:77: TestAndIssue[test=java/util/Properties/StoreReproducibilityTest.java, issueId=0000000] duplicates /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList-Virtual.txt:76 java/util/Properties/StoreReproducibilityTest.java 0000000 generic-all /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList.txt:516: java/lang/management/MemoryMXBean/PendingAllGC.sh does not exist under any test root java/lang/management/MemoryMXBean/PendingAllGC.sh 8158837 generic-all /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList.txt:667: javax/swing/JFileChooser/6798062/bug6798062.java does not exist under any test root javax/swing/JFileChooser/6798062/bug6798062.java 8146446 windows-all /Users/dnsimon/dev/jdk-jdk/open/test/jdk/ProblemList.txt:775: javax/swing/JTabbedPane/4666224/bug4666224.java does not exist under any test root javax/swing/JTabbedPane/4666224/bug4666224.java 8144124 macosx-all STDERR: java.lang.AssertionError: 5 errors found while checking 13 problem list files at CheckProblemLists.main(CheckProblemLists.java:96) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333) at java.base/java.lang.Thread.run(Thread.java:1575) ------------- Commit messages: - removed problem list entries referring to non-existent tests - added CheckProblemLists Changes: https://git.openjdk.org/jdk/pull/18879/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18879&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330755 Stats: 217 lines in 4 files changed: 211 ins; 6 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/18879.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18879/head:pull/18879 PR: https://git.openjdk.org/jdk/pull/18879 From ihse at openjdk.org Mon Apr 22 13:17:36 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 22 Apr 2024 13:17:36 GMT Subject: RFR: 8330820: Remove remnants of operator_new.cpp in build system Message-ID: In [JDK-8282469](https://bugs.openjdk.org/browse/JDK-8282469), the file operator_new.cpp was removed from Hotspot. Unfortunately, some ugly hacks related to this file remained in the build system. They should be removed. ------------- Commit messages: - 8330820: Remove remnants of operator_new.cpp in build system Changes: https://git.openjdk.org/jdk/pull/18886/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18886&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330820 Stats: 10 lines in 2 files changed: 0 ins; 9 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18886.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18886/head:pull/18886 PR: https://git.openjdk.org/jdk/pull/18886 From tbell at openjdk.org Mon Apr 22 13:34:31 2024 From: tbell at openjdk.org (Tim Bell) Date: Mon, 22 Apr 2024 13:34:31 GMT Subject: RFR: 8330820: Remove remnants of operator_new.cpp in build system In-Reply-To: References: Message-ID: On Mon, 22 Apr 2024 13:11:02 GMT, Magnus Ihse Bursie wrote: > In [JDK-8282469](https://bugs.openjdk.org/browse/JDK-8282469), the file operator_new.cpp was removed from Hotspot. Unfortunately, some ugly hacks related to this file remained in the build system. They should be removed. Approved ------------- Marked as reviewed by tbell (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18886#pullrequestreview-2014770529 From ihse at openjdk.org Mon Apr 22 13:34:32 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Mon, 22 Apr 2024 13:34:32 GMT Subject: Integrated: 8330820: Remove remnants of operator_new.cpp in build system In-Reply-To: References: Message-ID: On Mon, 22 Apr 2024 13:11:02 GMT, Magnus Ihse Bursie wrote: > In [JDK-8282469](https://bugs.openjdk.org/browse/JDK-8282469), the file operator_new.cpp was removed from Hotspot. Unfortunately, some ugly hacks related to this file remained in the build system. They should be removed. This pull request has now been integrated. Changeset: 3e65d90b Author: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/3e65d90b4ddb52878ebdc2150790c0333b9c0920 Stats: 10 lines in 2 files changed: 0 ins; 9 del; 1 mod 8330820: Remove remnants of operator_new.cpp in build system Reviewed-by: tbell ------------- PR: https://git.openjdk.org/jdk/pull/18886 From dholmes at openjdk.org Tue Apr 23 07:07:31 2024 From: dholmes at openjdk.org (David Holmes) Date: Tue, 23 Apr 2024 07:07:31 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 [v2] In-Reply-To: References: Message-ID: On Wed, 17 Apr 2024 05:43:12 GMT, Joe Darcy wrote: >> Get JDK 24 underway. > > Joe Darcy has updated the pull request incrementally with two additional commits since the last revision: > > - Correct release date as observed in review feedback. > - Improve javadoc of class file update. LGTM Thanks test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java line 1: > 1: /* There are further updates to this test in the pipeline (new deprecated flags in 23) so you will need to keep updating to reflect that. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18787#pullrequestreview-2016422206 PR Review Comment: https://git.openjdk.org/jdk/pull/18787#discussion_r1575750313 From magnus.ihse.bursie at oracle.com Tue Apr 23 10:41:53 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 23 Apr 2024 12:41:53 +0200 Subject: Hermetic Java project meeting In-Reply-To: References: Message-ID: I will not be able to participate in the meeting today. Let me report a bit on my work this week. I have made a proof of concept branch which can properly compile static java on macos with clang and linux with gcc. (With "properly" I mean hiding non-exported symbols). I still have problems replicating the build with clang on linux. I suspect my linux workstation has a broken installation of clang. That machine has grown more erratic over time, and I need to reinstall it, but I'm procrastinating spending the time doing that... So for now I'm just skipping the clang on linux part. I have also made great progress on windows. Julian's hint about objcopy working fine on COFF, and being present in cygwin, got me to realize that this was in effect a possible way forward. My PoC currently manages to extract a list of exported symbols from the static libs using dumpbin. This is necessary since --localize-hidden does not work on COFF (the concept of "hidden" symbols is an ELF-only concept), and instead we need to use the --keep-global-symbols option, which (despite the name) converts all symbols in the given list to global and all other to local. I am currently working actively with getting the next steps done in this PoC. I have initiated talks with SAP, and they are interested in helping out getting static linking working on AIX (given that it is possible to do with not too much effort.) I have also tried to extract all the changes (and only the changes) related to static build from the hermetic-java-runtime branch (ignoring the JavaHome/resource loading changes), to see if I could get something like StaticLink.gmk in mainline. I thought I was doing quite fine, but after a while I realized my testing was botched since the launcher had actually loaded the libraries dynamically instead, even though they were statically linked. :-( I am currently trying to bisect my way thought my repo to understand where things went wrong. This problem was exaggerated by the fact that we cannot build *only* static libs, so the dynamic ones where there alongside to be (improperly) picked up. I might want to spend some time on fixing this first. That will help both with speeding up the build/test cycle for static builds, and help avoid that kind of issue repeating. That will require some more refactoring in the core build/link code though. Experimenting with the static launcher on linux/gcc and macos made me realize that we will need to know the set of external libraries needed by each individual JDK library. When building a dynamic library, this knowledge (e.g. -liconv -framework Application) is encoded into the lib*.so file by the linker. But not so for a static library. Instead, we need to store this information for each JDK library, and then in the end, when we want to pick up all static libraries and link them together to form the "javastatic" executable, we need to pass this set of external libraries to the linker. This was done haphazardly in StaticLink.gmk in the hermetic-java-runtime branch, where an arbitrary subset of external libraries were hard-coded. Before integration in mainline can be possible, this information needs to be collected correctly and automatically for all included JDK libraries. Fortunately, it is not likely to be too hard. I basically just need to store the information from the LIBS provided to the NativeCompilation, and pick that up for all static libraries we include in the static launcher. (A complication is that we need to de-duplicate the list, and that some libraries are specified using two words, like "-framework Application" on macos, so it will take some care getting it right.) I have also been thinking about possible ways that we can share compiled obj files between static and dynamic libraries, even if we cannot do it fully. Most files do not need the STATIC_BUILD define and will thus be identical for both static and dynamic builds. It might be possible to just hard-code the exact files that needs to be different. It's ugly, and I still would like to make sure we press forward with the spec changes to JNI/JVMTI, but it would work as a stop-gap measure. /Magnus From jwaters at openjdk.org Tue Apr 23 14:00:38 2024 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 23 Apr 2024 14:00:38 GMT Subject: RFR: 8330988: Implementation of 8288293: Windows/gcc Port for hsdis Message-ID: WIP This changeset contains hsdis for Windows/gcc Port. It supports both the binutils and capstone backends, though the LLVM backend is left out due to compatibility issues encountered during the build. Currently, which gcc distributions are supported is still to be clarified, as several, ranging from Cygwin, to MinGW64, to the gcc subsystems from MSYS2. For this reason, the build system hack in place at the moment to compile the binutils backend on Windows is still left in place, for now. ------------- Commit messages: - Implementation of hsdis for 8288293: Windows/gcc Port Changes: https://git.openjdk.org/jdk/pull/18915/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18915&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8330988 Stats: 347 lines in 28 files changed: 161 ins; 22 del; 164 mod Patch: https://git.openjdk.org/jdk/pull/18915.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18915/head:pull/18915 PR: https://git.openjdk.org/jdk/pull/18915 From ihse at openjdk.org Tue Apr 23 14:28:31 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 23 Apr 2024 14:28:31 GMT Subject: RFR: 8330988: Implementation of 8288293: Windows/gcc Port for hsdis In-Reply-To: References: Message-ID: On Tue, 23 Apr 2024 13:56:32 GMT, Julian Waters wrote: > WIP > > This changeset contains hsdis for Windows/gcc Port. It supports both the binutils and capstone backends, though the LLVM backend is left out due to compatibility issues encountered during the build. Currently, which gcc distributions are supported is still to be clarified, as several, ranging from Cygwin, to MinGW64, to the gcc subsystems from MSYS2. For this reason, the build system hack in place at the moment to compile the binutils backend on Windows is still left in place, for now. There's a huge amount of changes for just hsdis... You might have to separate out the infrastructure changes that seem to amount to most of the changes here. This is going to take me a while to get through. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18915#issuecomment-2072458273 From jwaters at openjdk.org Tue Apr 23 14:34:27 2024 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 23 Apr 2024 14:34:27 GMT Subject: RFR: 8330988: Implementation of 8288293: Windows/gcc Port for hsdis In-Reply-To: References: Message-ID: On Tue, 23 Apr 2024 14:25:22 GMT, Magnus Ihse Bursie wrote: > There's a huge amount of changes for just hsdis... You might have to separate out the infrastructure changes that seem to amount to most of the changes here. > > This is going to take me a while to get through. Sorry, it's still a WIP, and I believe something broke and scattered changes from one of my other local branches into this current changeset ------------- PR Comment: https://git.openjdk.org/jdk/pull/18915#issuecomment-2072483937 From ascarpino at openjdk.org Tue Apr 23 20:09:30 2024 From: ascarpino at openjdk.org (Anthony Scarpino) Date: Tue, 23 Apr 2024 20:09:30 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v3] In-Reply-To: <-64Xlhk6ln43-xTmlv_cvloS-gzDrKMyiPUdPbMNlIM=.2b524654-ca5b-4a7a-a7da-316e99cfea35@github.com> References: <-64Xlhk6ln43-xTmlv_cvloS-gzDrKMyiPUdPbMNlIM=.2b524654-ca5b-4a7a-a7da-316e99cfea35@github.com> Message-ID: On Mon, 15 Apr 2024 22:12:30 GMT, Volodymyr Paprotski wrote: >> Performance. Before: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s >> >> Performance, no intrinsic: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply true thrpt 3 1919.57... > > Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: > > Comments from Jatin and Tony src/java.base/share/classes/sun/security/ec/ECOperations.java line 204: > 202: * @return the product > 203: */ > 204: public MutablePoint multiply(AffinePoint affineP, byte[] s) { It seems like there could be some combining of both `multiply()`. If `multiply(AffinePoint, ...)` is called, it can call `DefaultMultiplier` with the `affineP`, but internally call the other `multiply(ECPoint, ...)` for the other situations. I'd rather not have two methods doing most of the same code, but different methods. src/java.base/share/classes/sun/security/ec/ECOperations.java line 467: > 465: sealed static abstract class SmallWindowMultiplier implements PointMultiplier > 466: permits DefaultMultiplier, DefaultMontgomeryMultiplier { > 467: private final AffinePoint affineP; I don't think `affineP` needs to be a class variable anymore. It's only used in the constructor src/java.base/share/classes/sun/security/ec/ECOperations.java line 592: > 590: } > 591: > 592: private final ProjectivePoint.Immutable[][] points; Can you define this at the top please. src/java.base/share/classes/sun/security/ec/ECOperations.java line 668: > 666: } > 667: > 668: private final BigInteger[] base; Can you define this at the top. You use it in the constructor but it's defined later on. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1576821201 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1575499019 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1575495263 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1575491814 From ascarpino at openjdk.org Tue Apr 23 20:09:32 2024 From: ascarpino at openjdk.org (Anthony Scarpino) Date: Tue, 23 Apr 2024 20:09:32 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 2 Apr 2024 19:19:59 GMT, Volodymyr Paprotski wrote: >> Performance. Before: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s >> >> Performance, no intrinsic: >> >> Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units >> SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s >> SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s >> SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s >> Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units >> o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s >> o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s >> Benchmark (isMontBench) Mode Cnt Score Error Units >> PolynomialP256Bench.benchMultiply true thrpt 3 1919.57... > > Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: > > remove use of jdk.crypto.ec src/java.base/share/classes/sun/security/ec/ECOperations.java line 308: > 306: > 307: /* > 308: * public Point addition. Used by ECDSAOperations Was the old description not applicable anymore? It would be nice to improve on the existing description that shortening it. src/java.base/share/classes/sun/security/ec/ECOperations.java line 321: > 319: ECOperations ops = this; > 320: if (this.montgomeryOps != null) { > 321: assert p.getField() instanceof IntegerMontgomeryFieldModuloP; This should throw a ProviderException, I believe this would throw an AssertionException ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1556740469 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1558824543 From jjg at openjdk.org Tue Apr 23 20:26:57 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Tue, 23 Apr 2024 20:26:57 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v7] In-Reply-To: References: Message-ID: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits: - Merge with upstream/master - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - Merge with upstream/master - update test - improve handling of ignorable doc comments suppress warning when building test code - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - call `saveDanglingDocComments` for local variable declarations - adjust call for `saveDanglingDocComments` for enum members - JDK-8303689: javac -Xlint could/should report on "dangling" doc comments ------------- Changes: https://git.openjdk.org/jdk/pull/18527/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=06 Stats: 485 lines in 59 files changed: 387 ins; 3 del; 95 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From ihse at openjdk.org Wed Apr 24 09:17:28 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 24 Apr 2024 09:17:28 GMT Subject: RFR: 8330988: Implementation of 8288293: Windows/gcc Port for hsdis In-Reply-To: References: Message-ID: On Tue, 23 Apr 2024 13:56:32 GMT, Julian Waters wrote: > WIP > > This changeset contains hsdis for Windows/gcc Port. It supports both the binutils and capstone backends, though the LLVM backend is left out due to compatibility issues encountered during the build. Currently, which gcc distributions are supported is still to be clarified, as several, ranging from Cygwin, to MinGW64, to the gcc subsystems from MSYS2. For this reason, the build system hack in place at the moment to compile the binutils backend on Windows is still left in place, for now. Please mark the PR as draft it is not intended for review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18915#issuecomment-2074481887 From ihse at openjdk.org Wed Apr 24 09:20:34 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Wed, 24 Apr 2024 09:20:34 GMT Subject: RFR: 8314488: Compile the JDK as C++17 [v6] In-Reply-To: References: Message-ID: On Fri, 19 Jan 2024 12:08:21 GMT, Julian Waters wrote: >> Julian Waters has updated the pull request incrementally with one additional commit since the last revision: >> >> Require clang 13 in toolchain.m4 > > Should I split the compiler upgrades into a different change and integrate that first? Going off the conversation in this thread it would seem like the compiler upgrade would benefit us a lot more than just having C++17 (The noreturn attribute is one big motivating factor for instance) and it might help if the compiler upgrades were not delayed by the discussion of when to jump to C++17 @TheShermanTanker I suggest you close this PR. If we are going to switch to C++17, it should start by a discussion in the mailing list, not with a PR (the change itself is trivial). ------------- PR Comment: https://git.openjdk.org/jdk/pull/14988#issuecomment-2074487573 From magnus.ihse.bursie at oracle.com Wed Apr 24 11:24:32 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 24 Apr 2024 13:24:32 +0200 Subject: Usage of iconv() In-Reply-To: <20240423121114.EE241662C87@dd33810.kasserver.com> References: <20240423121114.EE241662C87@dd33810.kasserver.com> Message-ID: That is a good question. libiconv is used only on macOS and AIX, for a few libraries, as you say. I just tried removing -liconv from the macOS dependencies and recompiled, just to see what would happen. There were three instances for macOS: libsplashscreen, libjdwp and libinstrument. Out of these, libinstrument compiled and linked fine without the -liconv argument. It looks like iconv is referenced in unix/.../EncodingSupport_md.c, but otoh it looks like it is as much (or as little) referenced on macOS as on linux (where we never have linked with -liconv) so it is perhaps just dead code. I did not study it in detail. The code looks very much duplicated from libjdwp. The other two actually failed linking, so they do use libiconv. libsplashscreen uses it in splashscreen_sys.m, where it is used to convert the jar file name. libjdwp uses it in utf_util.c, where it is used to convert file name and command lines, iiuc. It is likely that we have similar (but better) ways to convert charsets elsewhere in our libraries that can be used instead of libiconv. I guess these are not the only two places where a filename must be converted from the platform charset to UTF8. /Magnus On 2024-04-23 14:11, Bernd Eckenfels wrote: > Du to a glibc security alert about a charset in iconv() I checked OpenJDK (since I was quite sure encoding is handled in JCL), however there are a few utilities (related to libinstrument and splash Screens) which use iconv. > > If I see it correctly it?s mostly used for utf8 so it should not expose this particular globe weakness, but I still wonder if that dependency is needed. For some platforms like AIX it even drags on an additional library dependency. (Not to mention different charger tables and especially ugly locale dependencies for containers). > > Gru? > Bernd > ? > https://bernd.eckenfels.net From duke at openjdk.org Wed Apr 24 17:01:59 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Wed, 24 Apr 2024 17:01:59 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v4] In-Reply-To: References: Message-ID: > Performance. Before: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s > > Performance, no intrinsic: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply true thrpt 3 1919.574 ? 10.591 ops/s > > Performance, **with intrinsics*... Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: Comments from Tony and Jatin ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18583/files - new: https://git.openjdk.org/jdk/pull/18583/files/6f9ac046..c93a71f0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=02-03 Stats: 48 lines in 2 files changed: 20 ins; 20 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/18583.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18583/head:pull/18583 PR: https://git.openjdk.org/jdk/pull/18583 From duke at openjdk.org Wed Apr 24 17:01:59 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Wed, 24 Apr 2024 17:01:59 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 16 Apr 2024 02:26:57 GMT, Jatin Bhateja wrote: >> Per-above, this is a switch statement (`UNLIKELY`) fallback. I can still add alignment and loop rotation, but being a fallback figured its more important to keep it small&readable... > > It's all part of intrinsic, no harm in polishing it. Done (normalized loop/backedge). There was actually a problem in the loop counter.. (`i-=1` instead of `i-=16`). Can't include a test since classes are sealed, but verified manually. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1578172873 From duke at openjdk.org Wed Apr 24 17:02:00 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Wed, 24 Apr 2024 17:02:00 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v3] In-Reply-To: References: <-64Xlhk6ln43-xTmlv_cvloS-gzDrKMyiPUdPbMNlIM=.2b524654-ca5b-4a7a-a7da-316e99cfea35@github.com> Message-ID: <6lemy0F_PaECRhIOAlCkUCSvGeE8kaAZ7RpqoB1nJeQ=.721e6902-76b3-4112-923f-4dcbeeebb94f@github.com> On Tue, 23 Apr 2024 19:55:57 GMT, Anthony Scarpino wrote: >> Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: >> >> Comments from Jatin and Tony > > src/java.base/share/classes/sun/security/ec/ECOperations.java line 204: > >> 202: * @return the product >> 203: */ >> 204: public MutablePoint multiply(AffinePoint affineP, byte[] s) { > > It seems like there could be some combining of both `multiply()`. If `multiply(AffinePoint, ...)` is called, it can call `DefaultMultiplier` with the `affineP`, but internally call the other `multiply(ECPoint, ...)` for the other situations. I'd rather not have two methods doing most of the same code, but different methods. Thanks, they indeed look identical, didnt notice. Fixed. (repeated the same hashmap refactoring and didnt notice I produced identical code twice) > src/java.base/share/classes/sun/security/ec/ECOperations.java line 467: > >> 465: sealed static abstract class SmallWindowMultiplier implements PointMultiplier >> 466: permits DefaultMultiplier, DefaultMontgomeryMultiplier { >> 467: private final AffinePoint affineP; > > I don't think `affineP` needs to be a class variable anymore. It's only used in the constructor Didn't notice, thanks, fixed. > src/java.base/share/classes/sun/security/ec/ECOperations.java line 592: > >> 590: } >> 591: >> 592: private final ProjectivePoint.Immutable[][] points; > > Can you define this at the top please. Done > src/java.base/share/classes/sun/security/ec/ECOperations.java line 668: > >> 666: } >> 667: >> 668: private final BigInteger[] base; > > Can you define this at the top. You use it in the constructor but it's defined later on. Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1578117929 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1578147190 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1578148562 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1578150303 From duke at openjdk.org Wed Apr 24 17:02:00 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Wed, 24 Apr 2024 17:02:00 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v2] In-Reply-To: References: Message-ID: On Tue, 9 Apr 2024 02:01:36 GMT, Anthony Scarpino wrote: >> Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: >> >> remove use of jdk.crypto.ec > > src/java.base/share/classes/sun/security/ec/ECOperations.java line 308: > >> 306: >> 307: /* >> 308: * public Point addition. Used by ECDSAOperations > > Was the old description not applicable anymore? It would be nice to improve on the existing description that shortening it. Forgot to go back and fix the comment. Fixed.. As for the 'meaning'. Notice the signature of the function changed (i.e. no longer a 'mixed point', but two ProjectivePoints. This is a good idea regardless of Montgomery, but it affects montgomery particularly badly (need to compute zInv for 'no reason'. ) For sake of completeness. Apart from constructor, the 'API' for ECOperations (i.e. as used by ECDHE, ECDSAOperations and KeyGeneration) are these three functions (everything else is used internally by this class) public void setSum(MutablePoint p, MutablePoint p2) public MutablePoint multiply(AffinePoint affineP, byte[] s) public MutablePoint multiply(ECPoint ecPoint, byte[] s) > src/java.base/share/classes/sun/security/ec/ECOperations.java line 321: > >> 319: ECOperations ops = this; >> 320: if (this.montgomeryOps != null) { >> 321: assert p.getField() instanceof IntegerMontgomeryFieldModuloP; > > This should throw a ProviderException, I believe this would throw an AssertionException Missed this comment. No longer applicable (this.montgomeryOps got refactored away) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1578144125 PR Review Comment: https://git.openjdk.org/jdk/pull/18583#discussion_r1578161140 From jianglizhou at google.com Wed Apr 24 17:57:17 2024 From: jianglizhou at google.com (Jiangli Zhou) Date: Wed, 24 Apr 2024 10:57:17 -0700 Subject: Hermetic Java project meeting In-Reply-To: References: Message-ID: Magnus, thanks for the update! Looks like you've made some good progress. Please see my comments below. On Tue, Apr 23, 2024 at 3:42?AM Magnus Ihse Bursie < magnus.ihse.bursie at oracle.com> wrote: > I will not be able to participate in the meeting today. > > Let me report a bit on my work this week. > > I have made a proof of concept branch which can properly compile static > java on macos with clang and linux with gcc. (With "properly" I mean > hiding non-exported symbols). I still have problems replicating the > build with clang on linux. I suspect my linux workstation has a broken > installation of clang. That machine has grown more erratic over time, > and I need to reinstall it, but I'm procrastinating spending the time > doing that... So for now I'm just skipping the clang on linux part. > Just to be more clear, that's with using `objcopy` to localize non-exported symbols for all JDK static libraries and libjvm.a, not just libjvm.a right? Can you please include the compiler or linker errors on linux/clang? > > I have also made great progress on windows. Julian's hint about objcopy > working fine on COFF, and being present in cygwin, got me to realize > that this was in effect a possible way forward. My PoC currently manages > to extract a list of exported symbols from the static libs using > dumpbin. This is necessary since --localize-hidden does not work on COFF > (the concept of "hidden" symbols is an ELF-only concept), and instead we > need to use the --keep-global-symbols option, which (despite the name) > converts all symbols in the given list to global and all other to local. > I am currently working actively with getting the next steps done in this > PoC. > > I have initiated talks with SAP, and they are interested in helping out > getting static linking working on AIX (given that it is possible to do > with not too much effort.) > > I have also tried to extract all the changes (and only the changes) > related to static build from the hermetic-java-runtime branch (ignoring > the JavaHome/resource loading changes), to see if I could get something > like StaticLink.gmk in mainline. I thought I was doing quite fine, but > after a while I realized my testing was botched since the launcher had > actually loaded the libraries dynamically instead, even though they were > statically linked. :-( I am currently trying to bisect my way thought my > repo to understand where things went wrong. > Did you run with `bin/javastatic`? The system automatically detects if the binary contains statically linked native libraries and avoids loading the dynamic libraries. Can you please share which test(s) ran into the library loading issue? I'll see if I can reproduce the problem that you are running into. For your work to get StaticLink.gmk into the mainline, I suggest taking all changes in the hermetic-java-runtime branch for testing to simplify things on your side. Then you can just extract the makefile part to integrate with the mainline. As we've discussed in the zoom meeting, I'll work on getting the runtime changes for static support into the mainline incrementally. > This problem was exaggerated by the fact that we cannot build *only* > static libs, so the dynamic ones where there alongside to be > (improperly) picked up. I might want to spend some time on fixing this > first. That will help both with speeding up the build/test cycle for > static builds, and help avoid that kind of issue repeating. That will > require some more refactoring in the core build/link code though. > > Experimenting with the static launcher on linux/gcc and macos made me > realize that we will need to know the set of external libraries needed > by each individual JDK library. When building a dynamic library, this > knowledge (e.g. -liconv -framework Application) is encoded into the > lib*.so file by the linker. But not so for a static library. Instead, we > need to store this information for each JDK library, and then in the > end, when we want to pick up all static libraries and link them together > to form the "javastatic" executable, we need to pass this set of > external libraries to the linker. > > This was done haphazardly in StaticLink.gmk in the hermetic-java-runtime > branch, where an arbitrary subset of external libraries were hard-coded. > Before integration in mainline can be possible, this information needs > to be collected correctly and automatically for all included JDK > libraries. Fortunately, it is not likely to be too hard. I basically > just need to store the information from the LIBS provided to the > NativeCompilation, and pick that up for all static libraries we include > in the static launcher. (A complication is that we need to de-duplicate > the list, and that some libraries are specified using two words, like > "-framework Application" on macos, so it will take some care getting it > right.) > Right, currently the hermetic-java-runtime branch specifies a list of hard-coded dependency libraries for linking. One of the goals of the hermetic prototype was avoiding/reducing refactoring/restructuring the existing code whenever possible. The reason is to reduce merge overhead when integrating with new changes from the mainline. We can do the proper refactoring and cleanups when getting the changes into the mainline. > > I have also been thinking about possible ways that we can share compiled > obj files between static and dynamic libraries, even if we cannot do it > fully. Most files do not need the STATIC_BUILD define and will thus be > identical for both static and dynamic builds. It might be possible to > just hard-code the exact files that needs to be different. It's ugly, > and I still would like to make sure we press forward with the spec > changes to JNI/JVMTI, but it would work as a stop-gap measure. > We have only briefly touched on the spec change topic (for the naming of native libraries) during the zoom meetings. I also agree that we should get that part started now. It's unclear to me if there's any existing blocker for that. Best, Jiangli > > /Magnus > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From syan at openjdk.org Thu Apr 25 09:52:51 2024 From: syan at openjdk.org (SendaoYan) Date: Thu, 25 Apr 2024 09:52:51 GMT Subject: RFR: 8331113: createJMHBundle.sh support configurable maven repo mirror Message-ID: The script make/devkit/createJMHBundle.sh use fixed maven repo server: https://repo.maven.apache.org/maven2. It's maybe useful to make the maven repo mirror configurable. Only change devkit shell script, no risk. ------------- Commit messages: - change copyright 2023 to 2024 - 8331113: createJMHBundle.sh support configurable maven repo mirror Changes: https://git.openjdk.org/jdk/pull/18946/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18946&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8331113 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/18946.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18946/head:pull/18946 PR: https://git.openjdk.org/jdk/pull/18946 From redestad at openjdk.org Thu Apr 25 11:07:27 2024 From: redestad at openjdk.org (Claes Redestad) Date: Thu, 25 Apr 2024 11:07:27 GMT Subject: RFR: 8331113: createJMHBundle.sh support configurable maven repo mirror In-Reply-To: References: Message-ID: On Thu, 25 Apr 2024 09:47:11 GMT, SendaoYan wrote: > The script make/devkit/createJMHBundle.sh use fixed maven repo server: https://repo.maven.apache.org/maven2. It's maybe useful to make the maven repo mirror configurable. > > Only change devkit shell script, no risk. Looks reasonable. ------------- Marked as reviewed by redestad (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18946#pullrequestreview-2022194650 From syan at openjdk.org Thu Apr 25 11:49:28 2024 From: syan at openjdk.org (SendaoYan) Date: Thu, 25 Apr 2024 11:49:28 GMT Subject: RFR: 8331113: createJMHBundle.sh support configurable maven repo mirror In-Reply-To: References: Message-ID: On Thu, 25 Apr 2024 11:04:31 GMT, Claes Redestad wrote: > Looks reasonable. Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18946#issuecomment-2076994082 From erikj at openjdk.org Thu Apr 25 12:21:33 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Thu, 25 Apr 2024 12:21:33 GMT Subject: RFR: 8331113: createJMHBundle.sh support configurable maven repo mirror In-Reply-To: References: Message-ID: On Thu, 25 Apr 2024 09:47:11 GMT, SendaoYan wrote: > The script make/devkit/createJMHBundle.sh use fixed maven repo server: https://repo.maven.apache.org/maven2. It's maybe useful to make the maven repo mirror configurable. > > Only change devkit shell script, no risk. Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18946#pullrequestreview-2022334899 From syan at openjdk.org Thu Apr 25 12:40:35 2024 From: syan at openjdk.org (SendaoYan) Date: Thu, 25 Apr 2024 12:40:35 GMT Subject: RFR: 8331113: createJMHBundle.sh support configurable maven repo mirror In-Reply-To: References: Message-ID: On Thu, 25 Apr 2024 11:04:31 GMT, Claes Redestad wrote: >> The script make/devkit/createJMHBundle.sh use fixed maven repo server: https://repo.maven.apache.org/maven2. It's maybe useful to make the maven repo mirror configurable. >> >> Only change devkit shell script, no risk. > > Looks reasonable. @cl4es @erikj79 Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18946#issuecomment-2077083272 From duke at openjdk.org Thu Apr 25 15:05:18 2024 From: duke at openjdk.org (Volodymyr Paprotski) Date: Thu, 25 Apr 2024 15:05:18 GMT Subject: RFR: 8329538: Accelerate P256 on x86_64 using Montgomery intrinsic [v5] In-Reply-To: References: Message-ID: <5HhjM9q2E4xtZVQitu5UGkNgdyBbqZbQOwnvJIUIr2U=.778a6d07-fe5d-4c6b-8bdf-353342df5904@github.com> > Performance. Before: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6443.934 ? 6.491 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6152.979 ? 4.954 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1895.410 ? 36.979 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1878.955 ? 45.487 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1357.810 ? 26.584 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1352.119 ? 23.547 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply false thrpt 3 1746.126 ? 10.970 ops/s > > Performance, no intrinsic: > > Benchmark (algorithm) (dataSize) (keyLength) (provider) Mode Cnt Score Error Units > SignatureBench.ECDSA.sign SHA256withECDSA 1024 256 thrpt 3 6529.839 ? 42.420 ops/s > SignatureBench.ECDSA.sign SHA256withECDSA 16384 256 thrpt 3 6199.747 ? 133.566 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 1024 256 thrpt 3 1973.676 ? 54.071 ops/s > SignatureBench.ECDSA.verify SHA256withECDSA 16384 256 thrpt 3 1932.127 ? 35.920 ops/s > Benchmark (algorithm) (keyLength) (kpgAlgorithm) (provider) Mode Cnt Score Error Units > o.o.b.j.c.full.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1355.788 ? 29.858 ops/s > o.o.b.j.c.small.KeyAgreementBench.EC.generateSecret ECDH 256 EC thrpt 3 1346.523 ? 28.722 ops/s > Benchmark (isMontBench) Mode Cnt Score Error Units > PolynomialP256Bench.benchMultiply true thrpt 3 1919.574 ? 10.591 ops/s > > Performance, **with intrinsics*... Volodymyr Paprotski has updated the pull request incrementally with one additional commit since the last revision: whitespace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18583/files - new: https://git.openjdk.org/jdk/pull/18583/files/c93a71f0..a1984501 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18583&range=03-04 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/18583.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18583/head:pull/18583 PR: https://git.openjdk.org/jdk/pull/18583 From syan at openjdk.org Thu Apr 25 15:56:40 2024 From: syan at openjdk.org (SendaoYan) Date: Thu, 25 Apr 2024 15:56:40 GMT Subject: Integrated: 8331113: createJMHBundle.sh support configurable maven repo mirror In-Reply-To: References: Message-ID: On Thu, 25 Apr 2024 09:47:11 GMT, SendaoYan wrote: > The script make/devkit/createJMHBundle.sh use fixed maven repo server: https://repo.maven.apache.org/maven2. It's maybe useful to make the maven repo mirror configurable. > > Only change devkit shell script, no risk. This pull request has now been integrated. Changeset: ce9eac38 Author: SendaoYan Committer: Magnus Ihse Bursie URL: https://git.openjdk.org/jdk/commit/ce9eac38191fa700afa3ac06b2b202576a11dd71 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod 8331113: createJMHBundle.sh support configurable maven repo mirror Reviewed-by: redestad, erikj ------------- PR: https://git.openjdk.org/jdk/pull/18946 From magnus.ihse.bursie at oracle.com Thu Apr 25 16:28:16 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 25 Apr 2024 18:28:16 +0200 Subject: Hermetic Java project meeting In-Reply-To: References: Message-ID: <65fdda77-1291-437e-b000-ca0307cf8084@oracle.com> > Just to be more clear, that's with using `objcopy` to localize > non-exported symbols for all JDK static libraries and libjvm.a, not > just libjvm.a right? Yes. > > Can you please include the compiler or linker errors on linux/clang? It is a bit tricky. The problem arises at the partial linking step. The problem seem to arise out of how clang converts a request to link into an actual call to ld. I enabled debug code (printing the command line, and running clang with `-v` to get it to print the actual command line used to run ld) and ran it on GHA, where it worked fine. This is how it looks there: WILL_RUN: /usr/bin/clang -v -m64 -r -o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/librmi_relocatable.o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/GC.o Ubuntu clang version 14.0.0-1ubuntu1.1 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/10 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13 Candidate multilib: .;@m64 Selected multilib: .;@m64 "/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/librmi_relocatable.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/lib/llvm-14/bin/../lib -L/lib -L/usr/lib -r /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/GC.o In contrast, on my machine it looks like this: WILL_RUN: /usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/clang -v -m64 -r -o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/librmi_relocatable.o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/GC.o clang version 13.0.1 Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64 "/usr/bin/ld" --hash-style=both --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/librmi_relocatable.o /lib/x86_64-linux-gnu/crt1.o /lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/../lib -L/lib -L/usr/lib -r /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/GC.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/9/crtend.o /lib/x86_64-linux-gnu/crtn.o /usr/bin/ld: cannot find -lgcc_s /usr/bin/ld: cannot find -lgcc_s clang-13: error: linker command failed with exit code 1 (use -v to see invocation) I don't understand what makes clang think it should include "-lgcc --as-needed -lgcc_s" and the crt*.o files when doing a partial link. In fact, the entire process on how clang (and gcc) builds up the linker command line is bordering on black magic to me. I think it can be affected by variables set at compile time (at least this was the case for gcc, last I checked), or maybe it picks up some kind of script from the environment. That's why I believe my machine could just be messed up. I could get a bit further by passing "-nodefaultlibs" (or whatever it was), but then the generated .o file were messed up wrt to library symbols and it failed dramatically when trying to do the final link of the static java launcher. > > I have also tried to extract all the changes (and only the changes) > related to static build from the hermetic-java-runtime branch > (ignoring > the JavaHome/resource loading changes), to see if I could get > something > like StaticLink.gmk in mainline. I thought I was doing quite fine, > but > after a while I realized my testing was botched since the launcher > had > actually loaded the libraries dynamically instead, even though > they were > statically linked. :-( I am currently trying to bisect my way > thought my > repo to understand where things went wrong. > > > Did you run with `bin/javastatic`? The system automatically detects if > the binary contains statically linked native libraries and avoids > loading the dynamic libraries. Can you please share which test(s) ran > into the library loading issue? I'll see if I can reproduce the > problem that you are?running into. It was in fact not a problem. I was fooled by an error message. To be sure I was not loading any dynamically linked libraries, I removed the jdk/lib directory. Now the launcher failed, saying something like: "Error: Cannot locate dynamic library libjava.dylib". which was a bit of a jump scare. However, it turned out that it actually tried to load lib/jvm.cfg, and failed in loading this (since I had removed the entire lib directory), and this failure caused the above error message to be printed. When I restored lib/jvm.cfg (but not any dynamic libraries), the launcher worked. There are several bugs lurking here. For once, the error message is incorrect and should be corrected. Secondly, a statically linked launcher has just a single JVM included and should not have to look for the lib/jvm.cfg file at all. After looking around a bit in the launcher/jli code, my conclusion is that this code will need some additional care and loving attention to make it properly adjusted to function as a static launcher. We can't have a static launcher that tries to load a jvm.cfg file it does not need, and when it fails, complains that it is missing a dynamic library that it should not load. I'll try to get this fixed as part of my efforts to get the static launcher into mainline. > This was done haphazardly in StaticLink.gmk in the > hermetic-java-runtime > branch, where an arbitrary subset of external libraries were > hard-coded. > Before integration in mainline can be possible, this information > needs > to be collected correctly and automatically for all included JDK > libraries. Fortunately, it is not likely to be too hard. I basically > just need to store the information from the LIBS provided to the > NativeCompilation, and pick that up for all static libraries we > include > in the static launcher. (A complication is that we need to > de-duplicate > the list, and that some libraries are specified using two words, like > "-framework Application" on macos, so it will take some care > getting it > right.) > > > Right, currently the hermetic-java-runtime branch specifies a list of > hard-coded dependency libraries for linking. One of the goals of the > hermetic prototype was?avoiding/reducing refactoring/restructuring the > existing code whenever possible. The reason is to reduce merge > overhead when integrating with new changes from the mainline. We can > do the proper refactoring and cleanups when getting the changes into > the mainline. That is basically what I am doing right now. I am looking at your prototype and trying to reimplement this functionality properly so that it can be merged into mainline. The first step on that way was to actually get your prototype running. Now I have managed to get a version of your prototype that only includes the minimal set of changes needed to support the static launcher, and that works on mac and linux/gcc. Since your prototype is based on 586396cbb55a31 from March, I am trying to merge the patch with the latest master. This worked fine for macOS, but I hit some unexpected snag on Linux which I'm currently investigating. > We have only briefly touched on the spec change topic (for the naming > of native libraries) during the zoom meetings. I also agree that we > should get that part started now. It's unclear to me if there's any > existing blocker for that. > I don't think there is. It's just that someone needs to step up and do it. /Magnus -------------- next part -------------- An HTML attachment was scrubbed... URL: From philip.race at oracle.com Thu Apr 25 18:30:43 2024 From: philip.race at oracle.com (Philip Race) Date: Thu, 25 Apr 2024 11:30:43 -0700 Subject: Usage of iconv() In-Reply-To: References: <20240423121114.EE241662C87@dd33810.kasserver.com> Message-ID: <91874482-df68-4e76-99e9-4fb90ef2c7a7@oracle.com> On 4/24/24 4:24 AM, Magnus Ihse Bursie wrote: > That is a good question. libiconv is used only on macOS and AIX, for a > few libraries, as you say. I just tried removing -liconv from the > macOS dependencies and recompiled, just to see what would happen. > There were three instances for macOS: libsplashscreen, libjdwp and > libinstrument. > > > > libsplashscreen uses it in splashscreen_sys.m, where it is used to > convert the jar file name. This is called from the launcher, in java.base/share/native/libjli/java.c > > libjdwp uses it in utf_util.c, where it is used to convert file name > and command lines, iiuc. > > It is likely that we have similar (but better) ways to convert > charsets elsewhere in our libraries that can be used instead of > libiconv. I guess these are not the only two places where a filename > must be converted from the platform charset to UTF8. So whatever replacement there might be, it needs to be something that is available very early in the life of the VM, in fact before there is a VM running. -phil. From jjg at openjdk.org Thu Apr 25 22:53:04 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 25 Apr 2024 22:53:04 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v8] In-Reply-To: References: Message-ID: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit. - Merge with upstream/master - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - Merge with upstream/master - update test - improve handling of ignorable doc comments suppress warning when building test code - Merge remote-tracking branch 'upstream/master' into 8303689.dangling-comments - call `saveDanglingDocComments` for local variable declarations - adjust call for `saveDanglingDocComments` for enum members - ... and 1 more: https://git.openjdk.org/jdk/compare/1c238d43...16a265a2 ------------- Changes: https://git.openjdk.org/jdk/pull/18527/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=07 Stats: 485 lines in 59 files changed: 387 ins; 3 del; 95 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From jjg at openjdk.org Thu Apr 25 23:24:07 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Thu, 25 Apr 2024 23:24:07 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v9] In-Reply-To: References: Message-ID: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: revert need to disable warning ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18527/files - new: https://git.openjdk.org/jdk/pull/18527/files/16a265a2..39689a52 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=07-08 Stats: 3 lines in 1 file changed: 0 ins; 2 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From vromero at openjdk.org Fri Apr 26 00:23:33 2024 From: vromero at openjdk.org (Vicente Romero) Date: Fri, 26 Apr 2024 00:23:33 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v9] In-Reply-To: References: Message-ID: On Thu, 25 Apr 2024 23:24:07 GMT, Jonathan Gibbons wrote: >> Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. >> >> The work can be thought of as in 3 parts: >> >> 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. >> >> 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. >> >> * Dangling documentation comments are handled as follows. >> * 1. {@code Scanner} adds all doc comments to a queue of >> * recent doc comments. The queue is flushed whenever >> * it is known that the recent doc comments should be >> * ignored and should not cause any warnings. >> * 2. The primary documentation comment is the one obtained >> * from the first token of any declaration. >> * (using {@code token.getDocComment()}. >> * 3. At the end of the "signature" of the declaration >> * (that is, before any initialization or body for the >> * declaration) any other "recent" comments are saved >> * in a map using the primary comment as a key, >> * using this method, {@code saveDanglingComments}. >> * 4. When the tree node for the declaration is finally >> * available, and the primary comment, if any, >> * is "attached", (in {@link #attach}) any related >> * dangling comments are also attached to the tree node >> * by registering them using the {@link #deferredLintHandler}. >> * 5. (Later) Warnings may be genereated for the dangling >> * comments, subject to the {@code -Xlint} and >> * {@code @SuppressWarnings}. >> >> >> 3. Updates to the make files to disable the warnings in modules for which the >> warning is generated. This is often because of the confusing use of `/**` to >> create box or other standout comments. > > Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: > > revert need to disable warning looks good src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java line 583: > 581: * dangling comments are also attached to the tree node > 582: * by registering them using the {@link #deferredLintHandler}. > 583: * 5. (Later) Warnings may be genereated for the dangling typo: generated ------------- Marked as reviewed by vromero (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18527#pullrequestreview-2023792395 PR Review Comment: https://git.openjdk.org/jdk/pull/18527#discussion_r1580263826 From jianglizhou at google.com Fri Apr 26 01:15:26 2024 From: jianglizhou at google.com (Jiangli Zhou) Date: Thu, 25 Apr 2024 18:15:26 -0700 Subject: Hermetic Java project meeting In-Reply-To: <65fdda77-1291-437e-b000-ca0307cf8084@oracle.com> References: <65fdda77-1291-437e-b000-ca0307cf8084@oracle.com> Message-ID: On Thu, Apr 25, 2024 at 9:28?AM Magnus Ihse Bursie wrote: > > > Just to be more clear, that's with using `objcopy` to localize non-exported symbols for all JDK static libraries and libjvm.a, not just libjvm.a right? > > Yes. > > > Can you please include the compiler or linker errors on linux/clang? > > It is a bit tricky. The problem arises at the partial linking step. The problem seem to arise out of how clang converts a request to link into an actual call to ld. I enabled debug code (printing the command line, and running clang with `-v` to get it to print the actual command line used to run ld) and ran it on GHA, where it worked fine. This is how it looks there: > > WILL_RUN: /usr/bin/clang -v -m64 -r -o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/librmi_relocatable.o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/GC.o > Ubuntu clang version 14.0.0-1ubuntu1.1 > Target: x86_64-pc-linux-gnu > Thread model: posix > InstalledDir: /usr/bin > Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/10 > Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11 > Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12 > Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13 > Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 > Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13 > Candidate multilib: .;@m64 > Selected multilib: .;@m64 > "/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/librmi_relocatable.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/lib/llvm-14/bin/../lib -L/lib -L/usr/lib -r /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/GC.o > > In contrast, on my machine it looks like this: > > WILL_RUN: /usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/clang -v -m64 -r -o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/librmi_relocatable.o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/GC.o > clang version 13.0.1 > Target: x86_64-unknown-linux-gnu > Thread model: posix > InstalledDir: /usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin > Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 > Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 > Candidate multilib: .;@m64 > Candidate multilib: 32;@m32 > Candidate multilib: x32;@mx32 > Selected multilib: .;@m64 > "/usr/bin/ld" --hash-style=both --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/librmi_relocatable.o /lib/x86_64-linux-gnu/crt1.o /lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/../lib -L/lib -L/usr/lib -r /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/GC.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/9/crtend.o /lib/x86_64-linux-gnu/crtn.o > /usr/bin/ld: cannot find -lgcc_s > /usr/bin/ld: cannot find -lgcc_s > clang-13: error: linker command failed with exit code 1 (use -v to see invocation) > > I don't understand what makes clang think it should include "-lgcc --as-needed -lgcc_s" and the crt*.o files when doing a partial link. In fact, the entire process on how clang (and gcc) builds up the linker command line is bordering on black magic to me. I think it can be affected by variables set at compile time (at least this was the case for gcc, last I checked), or maybe it picks up some kind of script from the environment. That's why I believe my machine could just be messed up. > > I could get a bit further by passing "-nodefaultlibs" (or whatever it was), but then the generated .o file were messed up wrt to library symbols and it failed dramatically when trying to do the final link of the static java launcher. > > Looks like you are using /usr/bin/ld and not lld. I haven't run into this type of issue. Have you tried -fuse-ld=lld? >> >> >> I have also tried to extract all the changes (and only the changes) >> related to static build from the hermetic-java-runtime branch (ignoring >> the JavaHome/resource loading changes), to see if I could get something >> like StaticLink.gmk in mainline. I thought I was doing quite fine, but >> after a while I realized my testing was botched since the launcher had >> actually loaded the libraries dynamically instead, even though they were >> statically linked. :-( I am currently trying to bisect my way thought my >> repo to understand where things went wrong. > > > Did you run with `bin/javastatic`? The system automatically detects if the binary contains statically linked native libraries and avoids loading the dynamic libraries. Can you please share which test(s) ran into the library loading issue? I'll see if I can reproduce the problem that you are running into. > > It was in fact not a problem. I was fooled by an error message. To be sure I was not loading any dynamically linked libraries, I removed the jdk/lib directory. Now the launcher failed, saying something like: > > "Error: Cannot locate dynamic library libjava.dylib". > > which was a bit of a jump scare. > > However, it turned out that it actually tried to load lib/jvm.cfg, and failed in loading this (since I had removed the entire lib directory), and this failure caused the above error message to be printed. When I restored lib/jvm.cfg (but not any dynamic libraries), the launcher worked. > Sounds like you are running into problems immediately during startup. Does the problem occur with just running bin/javastatic using a simple HelloWorld? Can you please send me your command line for reproducing? For the static Java support, I changed CreateExecutionEnvironment to return immediately if it executes statically. jvm.cfg is not loaded. Please see https://github.com/openjdk/leyden/blob/c1c5fc686c1452550e1b3663a320fba652248505/src/java.base/unix/native/libjli/java_md.c#L296. Sounds like the JLI_IsStaticJDK() check is not working properly in your case. Best, Jiangli > There are several bugs lurking here. For once, the error message is incorrect and should be corrected. Secondly, a statically linked launcher has just a single JVM included and should not have to look for the lib/jvm.cfg file at all. > > After looking around a bit in the launcher/jli code, my conclusion is that this code will need some additional care and loving attention to make it properly adjusted to function as a static launcher. We can't have a static launcher that tries to load a jvm.cfg file it does not need, and when it fails, complains that it is missing a dynamic library that it should not load. > > I'll try to get this fixed as part of my efforts to get the static launcher into mainline. >> >> This was done haphazardly in StaticLink.gmk in the hermetic-java-runtime >> branch, where an arbitrary subset of external libraries were hard-coded. >> Before integration in mainline can be possible, this information needs >> to be collected correctly and automatically for all included JDK >> libraries. Fortunately, it is not likely to be too hard. I basically >> just need to store the information from the LIBS provided to the >> NativeCompilation, and pick that up for all static libraries we include >> in the static launcher. (A complication is that we need to de-duplicate >> the list, and that some libraries are specified using two words, like >> "-framework Application" on macos, so it will take some care getting it >> right.) > > > Right, currently the hermetic-java-runtime branch specifies a list of hard-coded dependency libraries for linking. One of the goals of the hermetic prototype was avoiding/reducing refactoring/restructuring the existing code whenever possible. The reason is to reduce merge overhead when integrating with new changes from the mainline. We can do the proper refactoring and cleanups when getting the changes into the mainline. > > That is basically what I am doing right now. I am looking at your prototype and trying to reimplement this functionality properly so that it can be merged into mainline. The first step on that way was to actually get your prototype running. > > Now I have managed to get a version of your prototype that only includes the minimal set of changes needed to support the static launcher, and that works on mac and linux/gcc. Since your prototype is based on 586396cbb55a31 from March, I am trying to merge the patch with the latest master. This worked fine for macOS, but I hit some unexpected snag on Linux which I'm currently investigating. > > We have only briefly touched on the spec change topic (for the naming of native libraries) during the zoom meetings. I also agree that we should get that part started now. It's unclear to me if there's any existing blocker for that. > > I don't think there is. It's just that someone needs to step up and do it. > > /Magnus From syan at openjdk.org Fri Apr 26 01:29:50 2024 From: syan at openjdk.org (SendaoYan) Date: Fri, 26 Apr 2024 01:29:50 GMT Subject: RFR: 8331113: createJMHBundle.sh support configurable maven repo mirror In-Reply-To: References: Message-ID: On Thu, 25 Apr 2024 09:47:11 GMT, SendaoYan wrote: > The script make/devkit/createJMHBundle.sh use fixed maven repo server: https://repo.maven.apache.org/maven2. It's maybe useful to make the maven repo mirror configurable. > > Only change devkit shell script, no risk. > /sponsor Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18946#issuecomment-2078469303 From syan at openjdk.org Fri Apr 26 03:37:59 2024 From: syan at openjdk.org (SendaoYan) Date: Fri, 26 Apr 2024 03:37:59 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected Message-ID: Hi, The curl command lack of "-L" option, cause download file fail, the size of download file is empty. ![image](https://github.com/openjdk/jdk/assets/24123821/7b5471ae-8e00-4a06-a327-7186c42bd6a6) Only change devkit shell script, the risk is low. ------------- Commit messages: - 8331164: createJMHBundle.sh download jars fail when url needed to be redirected Changes: https://git.openjdk.org/jdk/pull/18965/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18965&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8331164 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18965.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18965/head:pull/18965 PR: https://git.openjdk.org/jdk/pull/18965 From jpai at openjdk.org Fri Apr 26 08:57:34 2024 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 26 Apr 2024 08:57:34 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 03:32:25 GMT, SendaoYan wrote: > Hi, > > The curl command lack of "-L" option, cause download file fail, the size of download file is empty. > > ![image](https://github.com/openjdk/jdk/assets/24123821/7b5471ae-8e00-4a06-a327-7186c42bd6a6) > > Only change devkit shell script, the risk is low. Hello @sendaoYan, who is adding the 302 redirect to that jar location and to which location is the URL being redirected to? What does `curl -v -O ` show in the logs (please paste the textual logs instead of an image). ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2078940654 From syan at openjdk.org Fri Apr 26 09:51:33 2024 From: syan at openjdk.org (SendaoYan) Date: Fri, 26 Apr 2024 09:51:33 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 08:53:21 GMT, Jaikiran Pai wrote: > Hello @sendaoYan, who is adding the 302 redirect to that jar location and to which location is the URL being redirected to? What does `curl -v -O ` show in the logs (please paste the textual logs instead of an image). Hello @jaikiran - who is adding the 302 redirect to that jar location Is `server: Tengine` adding the 302 redirect? I am not sure. - which location is the URL being redirected to It's redirected to oss URL: `https://archiva-maven-storage-prod.oss-cn-beijing.aliyuncs.com/repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714128070&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=RulXVJOdnC09nHSkGmZmN5NIS98%3D` This is the full testual logs [curl.log](https://github.com/openjdk/jdk/files/15128464/curl.log). ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2079033234 From jpai at openjdk.org Fri Apr 26 11:33:35 2024 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 26 Apr 2024 11:33:35 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 09:48:51 GMT, SendaoYan wrote: >> Hello @sendaoYan, who is adding the 302 redirect to that jar location and to which location is the URL being redirected to? What does `curl -v -O ` show in the logs (please paste the textual logs instead of an image). > >> Hello @sendaoYan, who is adding the 302 redirect to that jar location and to which location is the URL being redirected to? What does `curl -v -O ` show in the logs (please paste the textual logs instead of an image). > > Hello @jaikiran > > - who is adding the 302 redirect to that jar location > > Is `server: Tengine` adding the 302 redirect? I am not sure. > - which location is the URL being redirected to > It's redirected to oss URL: `https://archiva-maven-storage-prod.oss-cn-beijing.aliyuncs.com/repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714128070&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=RulXVJOdnC09nHSkGmZmN5NIS98%3D` > > > This is the full testual logs [curl.log](https://github.com/openjdk/jdk/files/15128464/curl.log). Thank you for those details @sendaoYan. Adding `-L` (follow redirects) to unconditionally follow redirects doesn't look right to me. I think, one would want to know, during the build process, if any URLs that are in use (like this one) have changed their location and then decide if the build script should be updated to point to the new URL. I'll let the build team decide if this is OK to change. I don't know anything about the server (Maven mirror?) you are using that's generating this redirect, to suggest a workaround. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2079206615 From erikj at openjdk.org Fri Apr 26 12:45:34 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 26 Apr 2024 12:45:34 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: <69nQxUpQUaabsDMvZPBeHfOlsGM0QOdp2dkfcLQ503I=.340b6d3c-86d4-4747-8c04-dfe82dda4a8d@github.com> On Fri, 26 Apr 2024 03:32:25 GMT, SendaoYan wrote: > Hi, > > The curl command lack of "-L" option, cause download file fail, the size of download file is empty. > > curl download fail without `-L`: > ```log >> rm -rf jmh-core-1.37.jar ; curl -O --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > -rw-rw-r-- 1 yansendao yansendao 0 Apr 26 17:37 jmh-core-1.37.jar > 0 jmh-core-1.37.jar > > > curl download success with `-L`: > >> rm -rf jmh-core-1.37.jar ; curl -OL --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > 100 540k 100 540k 0 0 1097k 0 --:--:-- --:--:-- --:--:-- 1097k > -rw-rw-r-- 1 yansendao yansendao 541K Apr 26 17:38 jmh-core-1.37.jar > 544K jmh-core-1.37.jar > > > > Only change devkit shell script, the risk is low. Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18965#pullrequestreview-2024954628 From erikj at openjdk.org Fri Apr 26 12:45:36 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Fri, 26 Apr 2024 12:45:36 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 11:30:24 GMT, Jaikiran Pai wrote: > Adding `-L` (follow redirects) to unconditionally follow redirects doesn't look right to me. I think, one would want to know, during the build process, if any URLs that are in use (like this one) have changed their location and then decide if the build script should be updated to point to the new URL. I'll let the build team decide if this is OK to change. I don't know anything about the server (Maven mirror?) you are using that's generating this redirect, to suggest a workaround. The script already falls back on wget if curl isn't found and that will AFAIK follow redirects by default. If we want to secure the download, we should add checksums in the script for each jar being downloaded. I don't think inconveniencing the download is the right approach for improving security. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2079313636 From mli at openjdk.org Fri Apr 26 14:25:11 2024 From: mli at openjdk.org (Hamlin Li) Date: Fri, 26 Apr 2024 14:25:11 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v4] In-Reply-To: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: > Hi, > Can you help to review the patch? > This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). > > Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. > Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. > > Besides of the code changes, one important task is to handle the legal process. > > Thanks! > > ## Performance > NOTE: > * `Src` means implementation in this pr, i.e. without depenency on external sleef. > * `Disabled` means disable intrinsics by `-XX:-UseVectorStubs` > * `system_sleef` means implementation in [previous pr 18294](https://github.com/openjdk/jdk/pull/18294), i.e. build and run jdk with depenency on external sleef. > > Basically, the perf data below shows that > * this implementation has better performance than previous version in [pr 18294](https://github.com/openjdk/jdk/pull/18294), > * and both sleef versions has much better performance compared with non-sleef version. > > |Benchmark |(size)|Src |Units|system_sleef|(system_sleef-Src)/Src|Diabled |(Disable-Src)/Src| > |------------------------------|------|---------|-----|------------|----------------------|---------|-----------------| > |3472:Double128Vector.ACOS |1024 |8546.842 |ns/op|8516.007 |-0.004 |16799.273|0.966 | > |3473:Double128Vector.ASIN |1024 |6864.656 |ns/op|6987.328 |0.018 |16602.442|1.419 | > |3474:Double128Vector.ATAN |1024 |11489.255|ns/op|12261.800 |0.067 |26329.320|1.292 | > |3475:Double128Vector.ATAN2 |1024 |16661.170|ns/op|17234.472 |0.034 |42084.100|1.526 | > |3476:Double128Vector.CBRT |1024 |18999.387|ns/op|20298.458 |0.068 |35998.688|0.895 | > |3477:Double128Vector.COS |1024 |14081.857|ns/op|14846.117 |0.054 |24420.692|0.734 | > |3478:Double128Vector.COSH |1024 |12202.306|ns/op|12237.772 |0.003 |21343.863|0.749 | > |3479:Double128Vector.EXP |1024 |4553.108 |ns/op|4777.638 |0.049 |20155.903|3.427 | > |3480:D... Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: remove notes about sleef changes ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18605/files - new: https://git.openjdk.org/jdk/pull/18605/files/cd70f5a9..cbcd4634 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18605&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18605&range=02-03 Stats: 4 lines in 1 file changed: 0 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/18605.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18605/head:pull/18605 PR: https://git.openjdk.org/jdk/pull/18605 From mli at openjdk.org Fri Apr 26 14:25:12 2024 From: mli at openjdk.org (Hamlin Li) Date: Fri, 26 Apr 2024 14:25:12 GMT Subject: RFR: 8312425: [vectorapi] AArch64: Optimize vector math operations with SLEEF [v3] In-Reply-To: References: <0cUurmXlMJ_B66Wy1umd2n4r9ve7_Q4WOU0ffMd8s5Y=.bbc93b65-382c-4139-aaec-cb835d94a06e@github.com> Message-ID: On Thu, 11 Apr 2024 10:36:03 GMT, Hamlin Li wrote: >> Hi, >> Can you help to review the patch? >> This pr is based on previous work and discussion in [pr 16234](https://github.com/openjdk/jdk/pull/16234), [pr 18294](https://github.com/openjdk/jdk/pull/18294). >> >> Compared with previous prs, the major change in this pr is to integrate the source of sleef (for the steps, please check `src/jdk.incubator.vector/linux/native/libvectormath/README`), rather than depends on external sleef things (header or lib) at build or run time. >> Besides of this change, also modify the previous changes accordingly, e.g. remove some uncessary files or changes especially in make dir of jdk. >> >> Besides of the code changes, one important task is to handle the legal process. >> >> Thanks! >> >> ## Performance >> NOTE: >> * `Src` means implementation in this pr, i.e. without depenency on external sleef. >> * `Disabled` means disable intrinsics by `-XX:-UseVectorStubs` >> * `system_sleef` means implementation in [previous pr 18294](https://github.com/openjdk/jdk/pull/18294), i.e. build and run jdk with depenency on external sleef. >> >> Basically, the perf data below shows that >> * this implementation has better performance than previous version in [pr 18294](https://github.com/openjdk/jdk/pull/18294), >> * and both sleef versions has much better performance compared with non-sleef version. >> >> |Benchmark |(size)|Src |Units|system_sleef|(system_sleef-Src)/Src|Diabled |(Disable-Src)/Src| >> |------------------------------|------|---------|-----|------------|----------------------|---------|-----------------| >> |3472:Double128Vector.ACOS |1024 |8546.842 |ns/op|8516.007 |-0.004 |16799.273|0.966 | >> |3473:Double128Vector.ASIN |1024 |6864.656 |ns/op|6987.328 |0.018 |16602.442|1.419 | >> |3474:Double128Vector.ATAN |1024 |11489.255|ns/op|12261.800 |0.067 |26329.320|1.292 | >> |3475:Double128Vector.ATAN2 |1024 |16661.170|ns/op|17234.472 |0.034 |42084.100|1.526 | >> |3476:Double128Vector.CBRT |1024 |18999.387|ns/op|20298.458 |0.068 |35998.688|0.895 | >> |3477:Double128Vector.COS |1024 |14081.857|ns/op|14846.117 |0.054 |24420.692|0.734 | >> |3478:Double128Vector.COSH |1024 |12202.306|ns/op|12237.772 |0.003 |21343.863|0.749 | >> |3479:Double128Vector.EXP |1024 |4553.108 |ns/op|4777.638 ... > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > fix performance issue Based on these 2 pr (https://github.com/shibatch/sleef/pull/537, https://github.com/shibatch/sleef/pull/536), there is no necessary code change in sleef files anymore. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18605#issuecomment-2079495537 From jjg at openjdk.org Fri Apr 26 16:04:09 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Fri, 26 Apr 2024 16:04:09 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v10] In-Reply-To: References: Message-ID: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: fix typo ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18527/files - new: https://git.openjdk.org/jdk/pull/18527/files/39689a52..48e8b0a2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18527&range=08-09 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18527.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18527/head:pull/18527 PR: https://git.openjdk.org/jdk/pull/18527 From syan at openjdk.org Fri Apr 26 16:25:52 2024 From: syan at openjdk.org (SendaoYan) Date: Fri, 26 Apr 2024 16:25:52 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 09:48:51 GMT, SendaoYan wrote: >> Hello @sendaoYan, who is adding the 302 redirect to that jar location and to which location is the URL being redirected to? What does `curl -v -O ` show in the logs (please paste the textual logs instead of an image). > >> Hello @sendaoYan, who is adding the 302 redirect to that jar location and to which location is the URL being redirected to? What does `curl -v -O ` show in the logs (please paste the textual logs instead of an image). > > Hello @jaikiran > > - who is adding the 302 redirect to that jar location > > Is `server: Tengine` adding the 302 redirect? I am not sure. > - which location is the URL being redirected to > It's redirected to oss URL: `https://archiva-maven-storage-prod.oss-cn-beijing.aliyuncs.com/repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714128070&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=RulXVJOdnC09nHSkGmZmN5NIS98%3D` > > > This is the full testual logs [curl.log](https://github.com/openjdk/jdk/files/15128464/curl.log). > Thank you for those details @sendaoYan. > > Adding `-L` (follow redirects) to unconditionally follow redirects doesn't look right to me. I think, one would want to know, during the build process, if any URLs that are in use (like this one) have changed their location and then decide if the build script should be updated to point to the new URL. I'll let the build team decide if this is OK to change. I don't know anything about the server (Maven mirror?) you are using that's generating this redirect, to suggest a workaround. @jaikiran Hello The `maven.aliyun.com` always redirects to new OSS URL, and the OSS URL will expired in 3600 secnods, as show below: yansendao at j66e07344:[tmp]> date +%s 1714148237 yansendao at j66e07344:[tmp]> curl -OL -v --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar 2>&1 | grep '> GET' > GET /repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar HTTP/2 > GET /repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714151841&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=%2BU29N2ems10WCjO%2FhhLJm6tT6tU%3D HTTP/1.1 yansendao at j66e07344:[tmp]> curl -OL -v --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar 2>&1 | grep '> GET' > GET /repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar HTTP/2 > GET /repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714151845&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=iu2IfGjRq2p2PZJ6zMg8DtqWg%2Fg%3D HTTP/1.1 So we can't get the final URL, because it will expired in short time. On the other hand, the `make/devkit/createGraphvizBundle.sh` file already use `curl -L` > grep curl make/devkit/createGraphvizBundle.sh curl -L -o "${file}" "${url}" ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2079706237 From prr at openjdk.org Fri Apr 26 19:34:00 2024 From: prr at openjdk.org (Phil Race) Date: Fri, 26 Apr 2024 19:34:00 GMT Subject: RFR: 8303689: javac -Xlint could/should report on "dangling" doc comments [v10] In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 16:04:09 GMT, Jonathan Gibbons wrote: >> Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. >> >> The work can be thought of as in 3 parts: >> >> 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. >> >> 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. >> >> * Dangling documentation comments are handled as follows. >> * 1. {@code Scanner} adds all doc comments to a queue of >> * recent doc comments. The queue is flushed whenever >> * it is known that the recent doc comments should be >> * ignored and should not cause any warnings. >> * 2. The primary documentation comment is the one obtained >> * from the first token of any declaration. >> * (using {@code token.getDocComment()}. >> * 3. At the end of the "signature" of the declaration >> * (that is, before any initialization or body for the >> * declaration) any other "recent" comments are saved >> * in a map using the primary comment as a key, >> * using this method, {@code saveDanglingComments}. >> * 4. When the tree node for the declaration is finally >> * available, and the primary comment, if any, >> * is "attached", (in {@link #attach}) any related >> * dangling comments are also attached to the tree node >> * by registering them using the {@link #deferredLintHandler}. >> * 5. (Later) Warnings may be genereated for the dangling >> * comments, subject to the {@code -Xlint} and >> * {@code @SuppressWarnings}. >> >> >> 3. Updates to the make files to disable the warnings in modules for which the >> warning is generated. This is often because of the confusing use of `/**` to >> create box or other standout comments. > > Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: > > fix typo Marked as reviewed by prr (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18527#pullrequestreview-2025744069 From jjg at openjdk.org Fri Apr 26 19:49:56 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Fri, 26 Apr 2024 19:49:56 GMT Subject: Integrated: 8303689: javac -Xlint could/should report on "dangling" doc comments In-Reply-To: References: Message-ID: On Wed, 27 Mar 2024 22:04:30 GMT, Jonathan Gibbons wrote: > Please review the updates to support a proposed new `-Xlint:dangling-doc-comments` option. > > The work can be thought of as in 3 parts: > > 1. An update to the `javac` internal class `DeferredLintHandler` so that it is possible to specify the appropriately configured `Lint` object when it is time to consider whether to generate the diagnostic or not. > > 2. Updates to the `javac` front end to record "dangling docs comments" found near the beginning of a declaration, and to report them using an instance of `DeferredLintHandler`. This allows the warnings to be enabled or disabled using the standard mechanisms for `-Xlint` and `@SuppressWarnings`. The procedure for handling dangling doc comments is described in this comment in `JavacParser`. > > * Dangling documentation comments are handled as follows. > * 1. {@code Scanner} adds all doc comments to a queue of > * recent doc comments. The queue is flushed whenever > * it is known that the recent doc comments should be > * ignored and should not cause any warnings. > * 2. The primary documentation comment is the one obtained > * from the first token of any declaration. > * (using {@code token.getDocComment()}. > * 3. At the end of the "signature" of the declaration > * (that is, before any initialization or body for the > * declaration) any other "recent" comments are saved > * in a map using the primary comment as a key, > * using this method, {@code saveDanglingComments}. > * 4. When the tree node for the declaration is finally > * available, and the primary comment, if any, > * is "attached", (in {@link #attach}) any related > * dangling comments are also attached to the tree node > * by registering them using the {@link #deferredLintHandler}. > * 5. (Later) Warnings may be genereated for the dangling > * comments, subject to the {@code -Xlint} and > * {@code @SuppressWarnings}. > > > 3. Updates to the make files to disable the warnings in modules for which the > warning is generated. This is often because of the confusing use of `/**` to > create box or other standout comments. This pull request has now been integrated. Changeset: a920af23 Author: Jonathan Gibbons URL: https://git.openjdk.org/jdk/commit/a920af233a11592af113f456f7608cb59dd1617e Stats: 482 lines in 58 files changed: 385 ins; 3 del; 94 mod 8303689: javac -Xlint could/should report on "dangling" doc comments Reviewed-by: vromero, ihse, prr ------------- PR: https://git.openjdk.org/jdk/pull/18527 From jjg at openjdk.org Fri Apr 26 20:34:32 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Fri, 26 Apr 2024 20:34:32 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v57] In-Reply-To: References: Message-ID: <-gBUJIN77XgSqa6gYERcGf2AeCmTWxNzRDnwmzrpS2U=.bac755e7-ba7f-41b5-9d1b-2b6f56c02f3e@github.com> > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 81 commits: - Merge with upstream/master - address review feedback for updates to Elements and friends - address review feedback for updates to Elements and friends - add support for JDK-8329296: Update Elements for '///' documentation comments - add support for `--disable-line-doc-comments` - Merge branch '8298405.doclet-markdown-v3' of https://github.com/jonathan-gibbons/jdk into 8298405.doclet-markdown-v3 - Merge pull request #3 from lahodaj/fix-positions-replacement Fixing positions when transforming javadoc text with replacement characters. - Merge branch '8298405.doclet-markdown-v3' into fix-positions-replacement - Fixing positions when transforming javadoc text with replacement characters. - Merge remote-tracking branch 'upstream/master' into 8298405.doclet-markdown-v3 - ... and 71 more: https://git.openjdk.org/jdk/compare/a920af23...c4709cf5 ------------- Changes: https://git.openjdk.org/jdk/pull/16388/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=56 Stats: 24098 lines in 228 files changed: 23453 ins; 260 del; 385 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388 From jjg at openjdk.org Fri Apr 26 21:26:19 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Fri, 26 Apr 2024 21:26:19 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v58] In-Reply-To: References: Message-ID: > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: Suppress warnings building tests ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16388/files - new: https://git.openjdk.org/jdk/pull/16388/files/c4709cf5..a884ae36 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=57 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=56-57 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388 From jpai at openjdk.org Sat Apr 27 01:18:05 2024 From: jpai at openjdk.org (Jaikiran Pai) Date: Sat, 27 Apr 2024 01:18:05 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: <-Yt6973XjXEuMJVNVZk4z6kr7GfV6cRtErgiWbbbk-Y=.8cd63f43-1e13-4c6a-ba96-7ad0068c8184@github.com> On Fri, 26 Apr 2024 16:23:28 GMT, SendaoYan wrote: >>> Hello @sendaoYan, who is adding the 302 redirect to that jar location and to which location is the URL being redirected to? What does `curl -v -O ` show in the logs (please paste the textual logs instead of an image). >> >> Hello @jaikiran >> >> - who is adding the 302 redirect to that jar location >> >> Is `server: Tengine` adding the 302 redirect? I am not sure. >> - which location is the URL being redirected to >> It's redirected to oss URL: `https://archiva-maven-storage-prod.oss-cn-beijing.aliyuncs.com/repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714128070&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=RulXVJOdnC09nHSkGmZmN5NIS98%3D` >> >> >> This is the full testual logs [curl.log](https://github.com/openjdk/jdk/files/15128464/curl.log). > >> Thank you for those details @sendaoYan. >> >> Adding `-L` (follow redirects) to unconditionally follow redirects doesn't look right to me. I think, one would want to know, during the build process, if any URLs that are in use (like this one) have changed their location and then decide if the build script should be updated to point to the new URL. I'll let the build team decide if this is OK to change. I don't know anything about the server (Maven mirror?) you are using that's generating this redirect, to suggest a workaround. > > @jaikiran Hello > The `maven.aliyun.com` always redirects to a new OSS URL everytime, and the OSS URL will expired in 3600 secnods, as show below: > > yansendao at j66e07344:[tmp]> date +%s > 1714148237 > yansendao at j66e07344:[tmp]> curl -OL -v --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar 2>&1 | grep '> GET' >> GET /repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar HTTP/2 >> GET /repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714151841&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=%2BU29N2ems10WCjO%2FhhLJm6tT6tU%3D HTTP/1.1 > yansendao at j66e07344:[tmp]> curl -OL -v --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar 2>&1 | grep '> GET' >> GET /repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar HTTP/2 >> GET /repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714151845&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=iu2IfGjRq2p2PZJ6zMg8DtqWg%2Fg%3D HTTP/1.1 > > So we can't use the final URL in script, because it will expired in short time. > By the way, the [make/devkit/createGraphvizBundle.sh](https://github.com/openjdk/jdk/blob/master/make/devkit/createGraphvizBundle.sh#L93) file already use `curl -L` > >> grep curl make/devkit/createGraphvizBundle.sh > curl -L -o "${file}" "${url}" Hello @sendaoYan, I hadn't checked whether we already have such usage in our build scripts. Thank you for pointing me to it. Erik's review is what matters here and he's explained that this change is OK and has approved the PR. So you can go ahead with this proposed change. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2080295263 From syan at openjdk.org Sat Apr 27 13:20:04 2024 From: syan at openjdk.org (SendaoYan) Date: Sat, 27 Apr 2024 13:20:04 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: <-Yt6973XjXEuMJVNVZk4z6kr7GfV6cRtErgiWbbbk-Y=.8cd63f43-1e13-4c6a-ba96-7ad0068c8184@github.com> References: <-Yt6973XjXEuMJVNVZk4z6kr7GfV6cRtErgiWbbbk-Y=.8cd63f43-1e13-4c6a-ba96-7ad0068c8184@github.com> Message-ID: <1mURp1CywNkmmLyiQqO_8alV-eHg1WM4KUxG_Ij7XkU=.76fa3df0-3be5-463a-b00b-cbe64b4df94a@github.com> On Sat, 27 Apr 2024 01:15:45 GMT, Jaikiran Pai wrote: >>> Thank you for those details @sendaoYan. >>> >>> Adding `-L` (follow redirects) to unconditionally follow redirects doesn't look right to me. I think, one would want to know, during the build process, if any URLs that are in use (like this one) have changed their location and then decide if the build script should be updated to point to the new URL. I'll let the build team decide if this is OK to change. I don't know anything about the server (Maven mirror?) you are using that's generating this redirect, to suggest a workaround. >> >> @jaikiran Hello >> The `maven.aliyun.com` always redirects to a new OSS URL everytime, and the OSS URL will expired in 3600 secnods, as show below: >> >> yansendao at j66e07344:[tmp]> date +%s >> 1714148237 >> yansendao at j66e07344:[tmp]> curl -OL -v --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar 2>&1 | grep '> GET' >>> GET /repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar HTTP/2 >>> GET /repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714151841&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=%2BU29N2ems10WCjO%2FhhLJm6tT6tU%3D HTTP/1.1 >> yansendao at j66e07344:[tmp]> curl -OL -v --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar 2>&1 | grep '> GET' >>> GET /repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar HTTP/2 >>> GET /repository/central/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar?Expires=1714151845&OSSAccessKeyId=LTAIfU51SusnnfCC&Signature=iu2IfGjRq2p2PZJ6zMg8DtqWg%2Fg%3D HTTP/1.1 >> >> So we can't use the final URL in script, because it will expired in short time. >> By the way, the [make/devkit/createGraphvizBundle.sh](https://github.com/openjdk/jdk/blob/master/make/devkit/createGraphvizBundle.sh#L93) file already use `curl -L` >> >>> grep curl make/devkit/createGraphvizBundle.sh >> curl -L -o "${file}" "${url}" > > Hello @sendaoYan, I hadn't checked whether we already have such usage in our build scripts. Thank you for pointing me to it. > > Erik's review is what matters here and he's explained that this change is OK and has approved the PR. So you can go ahead with this proposed change. @jaikiran @erikj79 Thanks for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2080638052 From mbaesken at openjdk.org Mon Apr 29 12:20:33 2024 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 29 Apr 2024 12:20:33 GMT Subject: RFR: 8331298: avoid alignment checks in UBSAN enabled build Message-ID: Currently we run into some alignment related issues when building with '--enable-ubsan' . Those errors already occur in the build. Fixing them might take some time and maybe also some discussion if it is worth the effort , So for now the alignment related checks should be disabled to get the UBSAN build working. Example : /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128:13: runtime error: store to misaligned address 0x15099c3cf4ce for type 'int', which requires 4 byte alignment 0x15099c3cf4ce: note: pointer points here 00 80 0f 86 00 00 00 00 3d 06 00 00 80 76 60 3d 07 00 00 80 76 40 3d 08 00 00 80 76 20 3d 1e 00 ^ #0 0x1509b3b04f10 in MacroAssembler::pd_patch_instruction(unsigned char*, unsigned char*, char const*, int) /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128 #1 0x1509b3b04f10 in Label::patch_instructions(MacroAssembler*) /jdk/src/hotspot/share/asm/assembler.cpp:201 #2 0x1509b940b6d8 in VM_Version_StubGenerator::generate_get_cpu_info() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:381 #3 0x1509b94059bd in VM_Version::initialize() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:2138 #4 0x1509b93fb56e in VM_Version_init() /jdk/src/hotspot/share/runtime/vm_version.cpp:32 #5 0x1509b61ef947 in init_globals() /jdk/src/hotspot/share/runtime/init.cpp:126 #6 0x1509b8fb0e29 in Threads::create_vm(JavaVMInitArgs*, bool*) /jdk/src/hotspot/share/runtime/threads.cpp:553 #7 0x1509b67da3d7 in JNI_CreateJavaVM_inner /jdk/src/hotspot/share/prims/jni.cpp:3581 #8 0x1509b67da3d7 in JNI_CreateJavaVM /jdk/src/hotspot/share/prims/jni.cpp:3672 #9 0x1509c0eed957 in InitializeJVM /jdk/src/java.base/share/native/libjli/java.c:1550 #10 0x1509c0eed957 in JavaMain /jdk/src/java.base/share/native/libjli/java.c:491 ... (rest of output omitted) ------------- Commit messages: - JDK-8331298 Changes: https://git.openjdk.org/jdk/pull/18998/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18998&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8331298 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/18998.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18998/head:pull/18998 PR: https://git.openjdk.org/jdk/pull/18998 From erikj at openjdk.org Mon Apr 29 12:41:04 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 29 Apr 2024 12:41:04 GMT Subject: RFR: 8331298: avoid alignment checks in UBSAN enabled build In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 12:15:55 GMT, Matthias Baesken wrote: > Currently we run into some alignment related issues when building with '--enable-ubsan' . Those errors already occur in the build. Fixing them might take some time and maybe also some discussion if it is worth the effort , > So for now the alignment related checks should be disabled to get the UBSAN build working. > Example : > > /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128:13: runtime error: store to misaligned address 0x15099c3cf4ce for type 'int', which requires 4 byte alignment > 0x15099c3cf4ce: note: pointer points here > 00 80 0f 86 00 00 00 00 3d 06 00 00 80 76 60 3d 07 00 00 80 76 40 3d 08 00 00 80 76 20 3d 1e 00 > ^ > #0 0x1509b3b04f10 in MacroAssembler::pd_patch_instruction(unsigned char*, unsigned char*, char const*, int) /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128 > #1 0x1509b3b04f10 in Label::patch_instructions(MacroAssembler*) /jdk/src/hotspot/share/asm/assembler.cpp:201 > #2 0x1509b940b6d8 in VM_Version_StubGenerator::generate_get_cpu_info() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:381 > #3 0x1509b94059bd in VM_Version::initialize() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:2138 > #4 0x1509b93fb56e in VM_Version_init() /jdk/src/hotspot/share/runtime/vm_version.cpp:32 > #5 0x1509b61ef947 in init_globals() /jdk/src/hotspot/share/runtime/init.cpp:126 > #6 0x1509b8fb0e29 in Threads::create_vm(JavaVMInitArgs*, bool*) /jdk/src/hotspot/share/runtime/threads.cpp:553 > #7 0x1509b67da3d7 in JNI_CreateJavaVM_inner /jdk/src/hotspot/share/prims/jni.cpp:3581 > #8 0x1509b67da3d7 in JNI_CreateJavaVM /jdk/src/hotspot/share/prims/jni.cpp:3672 > #9 0x1509c0eed957 in InitializeJVM /jdk/src/java.base/share/native/libjli/java.c:1550 > #10 0x1509c0eed957 in JavaMain /jdk/src/java.base/share/native/libjli/java.c:491 > ... (rest of output omitted) Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18998#pullrequestreview-2028396835 From mdoerr at openjdk.org Mon Apr 29 12:44:04 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 29 Apr 2024 12:44:04 GMT Subject: RFR: 8331298: avoid alignment checks in UBSAN enabled build In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 12:15:55 GMT, Matthias Baesken wrote: > Currently we run into some alignment related issues when building with '--enable-ubsan' . Those errors already occur in the build. Fixing them might take some time and maybe also some discussion if it is worth the effort , > So for now the alignment related checks should be disabled to get the UBSAN build working. > Example : > > /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128:13: runtime error: store to misaligned address 0x15099c3cf4ce for type 'int', which requires 4 byte alignment > 0x15099c3cf4ce: note: pointer points here > 00 80 0f 86 00 00 00 00 3d 06 00 00 80 76 60 3d 07 00 00 80 76 40 3d 08 00 00 80 76 20 3d 1e 00 > ^ > #0 0x1509b3b04f10 in MacroAssembler::pd_patch_instruction(unsigned char*, unsigned char*, char const*, int) /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128 > #1 0x1509b3b04f10 in Label::patch_instructions(MacroAssembler*) /jdk/src/hotspot/share/asm/assembler.cpp:201 > #2 0x1509b940b6d8 in VM_Version_StubGenerator::generate_get_cpu_info() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:381 > #3 0x1509b94059bd in VM_Version::initialize() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:2138 > #4 0x1509b93fb56e in VM_Version_init() /jdk/src/hotspot/share/runtime/vm_version.cpp:32 > #5 0x1509b61ef947 in init_globals() /jdk/src/hotspot/share/runtime/init.cpp:126 > #6 0x1509b8fb0e29 in Threads::create_vm(JavaVMInitArgs*, bool*) /jdk/src/hotspot/share/runtime/threads.cpp:553 > #7 0x1509b67da3d7 in JNI_CreateJavaVM_inner /jdk/src/hotspot/share/prims/jni.cpp:3581 > #8 0x1509b67da3d7 in JNI_CreateJavaVM /jdk/src/hotspot/share/prims/jni.cpp:3672 > #9 0x1509c0eed957 in InitializeJVM /jdk/src/java.base/share/native/libjli/java.c:1550 > #10 0x1509c0eed957 in JavaMain /jdk/src/java.base/share/native/libjli/java.c:491 > ... (rest of output omitted) I wish we could fix all UB, but that seems unrealistic for the time being. LGTM. ------------- Marked as reviewed by mdoerr (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/18998#pullrequestreview-2028402095 From syan at openjdk.org Mon Apr 29 16:19:12 2024 From: syan at openjdk.org (SendaoYan) Date: Mon, 29 Apr 2024 16:19:12 GMT Subject: RFR: 8331331: :tier1 target explanation in doc/testing.md is incorrect Message-ID: Hi, In doc/testing.md file, it says: As an example, :tier1 will expand to jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1. The actual situation is :tier1 doesn't contains test/nashorn:tier1, and the document missed test/lib-test:tier1 $ make -n test-tier1 &> test-tier1.log $ grep "Running test " test-tier1.log Running test 'jtreg:test/hotspot/jtreg:tier1' Running test 'jtreg:test/jdk:tier1' Running test 'jtreg:test/langtools:tier1' Running test 'jtreg:test/jaxp:tier1' Running test 'jtreg:test/lib-test:tier1' Only change the document, no risk. ------------- Commit messages: - 8331331 :tier1 target explanation in doc/testing.md is incorrect Changes: https://git.openjdk.org/jdk/pull/19002/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=19002&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8331331 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/19002.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/19002/head:pull/19002 PR: https://git.openjdk.org/jdk/pull/19002 From jkern at openjdk.org Mon Apr 29 16:20:14 2024 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 29 Apr 2024 16:20:14 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: On Thu, 18 Apr 2024 04:26:21 GMT, Kim Barrett wrote: >> I opened https://bugs.openjdk.org/browse/JDK-8330539 so we don't lose track of this, but we can keep the discussion/voting here. > > For the impatient, I suggest adopting mechanism 2, i.e. unconditionally > include in globalDefinitions_gcc.hpp. > > We can't include in shared code, and there is a use in shared code > (in the relatively recently added JavaThread::pretouch_stack). > > When I questioned whether we needed to include at all, I referred > to a Linux man page I'd found on the internet (the same page mdoerr linked > to), which says (in part) > > "By default, modern compilers automatically translate all uses of alloca() > into the built-in ..." > > Apparently I should have kept digging, because it seems that page is > old/incorrect. A seemingly more recent Linux man page describes a different > way of handling it that is closer to what we're seeing, but still not quite > correct. > > glibc's includes if __USE_MISC is defined. > One of the ways __USE_MISC can become defined is if _GNU_SOURCE is defined, > and we define that for both gcc and clang toolchains. > > We include in globalDefinitions_gcc.hpp. So when building with gcc, > globalDefinitions.hpp implicitly includes . > > The glibc definition of alloca is > > #ifdef __GNUC__ > # define alloca(size) __builtin_alloca (size) > #endif /* GCC. */ > > So that explains why we don't need any explicit include of when > building with gcc. I expect there's something similar going on with Visual > Studio and Xcode/clang. But apparently not with Open XLC clang. On AIX `stdlib.h` also would define `alloca`, if `__STRICT_ANSI__` wouldn't be set. 780 #if !defined(__xlC__) || defined(__ibmxl__) || defined(__cplusplus) 781 #if defined(__IBMCPP__) && !defined(__ibmxl__) 782 extern "builtin" char *__alloca (size_t); 783 # define alloca __alloca 784 #elif defined(__GNUC__) && !defined(__STRICT_ANSI__) 785 #undef alloca 786 #define alloca(size) __builtin_alloca (size) 787 #endif A small plain Testprogramm not using all of the flags we used in jdk build, does not set `__STRICT_ANSI__` and then `alloca` is defined correct. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1583360569 From darcy at openjdk.org Mon Apr 29 17:49:13 2024 From: darcy at openjdk.org (Joe Darcy) Date: Mon, 29 Apr 2024 17:49:13 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 [v2] In-Reply-To: References: Message-ID: On Tue, 23 Apr 2024 07:03:45 GMT, David Holmes wrote: > There are further updates to this test in the pipeline (new deprecated flags in 23) so you will need to keep updating to reflect that. Thanks @dholmes-ora ; acknowledged. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18787#discussion_r1583479535 From shade at openjdk.org Mon Apr 29 17:53:09 2024 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 29 Apr 2024 17:53:09 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 03:32:25 GMT, SendaoYan wrote: > Hi, > > The curl command lack of "-L" option, cause download file fail, the size of download file is empty. > > curl download fail without `-L`: > ```log >> rm -rf jmh-core-1.37.jar ; curl -O --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > -rw-rw-r-- 1 yansendao yansendao 0 Apr 26 17:37 jmh-core-1.37.jar > 0 jmh-core-1.37.jar > > > curl download success with `-L`: > >> rm -rf jmh-core-1.37.jar ; curl -OL --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > 100 540k 100 540k 0 0 1097k 0 --:--:-- --:--:-- --:--:-- 1097k > -rw-rw-r-- 1 yansendao yansendao 541K Apr 26 17:38 jmh-core-1.37.jar > 544K jmh-core-1.37.jar > > > > Only change devkit shell script, the risk is low. Marked as reviewed by shade (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18965#pullrequestreview-2029168618 From syan at openjdk.org Mon Apr 29 17:53:09 2024 From: syan at openjdk.org (SendaoYan) Date: Mon, 29 Apr 2024 17:53:09 GMT Subject: Integrated: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 03:32:25 GMT, SendaoYan wrote: > Hi, > > The curl command lack of "-L" option, cause download file fail, the size of download file is empty. > > curl download fail without `-L`: > ```log >> rm -rf jmh-core-1.37.jar ; curl -O --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > -rw-rw-r-- 1 yansendao yansendao 0 Apr 26 17:37 jmh-core-1.37.jar > 0 jmh-core-1.37.jar > > > curl download success with `-L`: > >> rm -rf jmh-core-1.37.jar ; curl -OL --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > 100 540k 100 540k 0 0 1097k 0 --:--:-- --:--:-- --:--:-- 1097k > -rw-rw-r-- 1 yansendao yansendao 541K Apr 26 17:38 jmh-core-1.37.jar > 544K jmh-core-1.37.jar > > > > Only change devkit shell script, the risk is low. This pull request has now been integrated. Changeset: eb88343f Author: SendaoYan Committer: Aleksey Shipilev URL: https://git.openjdk.org/jdk/commit/eb88343fb763ee89010b7a9879638152decc6892 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8331164: createJMHBundle.sh download jars fail when url needed to be redirected Reviewed-by: erikj, shade ------------- PR: https://git.openjdk.org/jdk/pull/18965 From syan at openjdk.org Mon Apr 29 17:56:11 2024 From: syan at openjdk.org (SendaoYan) Date: Mon, 29 Apr 2024 17:56:11 GMT Subject: RFR: 8331164: createJMHBundle.sh download jars fail when url needed to be redirected In-Reply-To: References: Message-ID: On Fri, 26 Apr 2024 03:32:25 GMT, SendaoYan wrote: > Hi, > > The curl command lack of "-L" option, cause download file fail, the size of download file is empty. > > curl download fail without `-L`: > ```log >> rm -rf jmh-core-1.37.jar ; curl -O --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > -rw-rw-r-- 1 yansendao yansendao 0 Apr 26 17:37 jmh-core-1.37.jar > 0 jmh-core-1.37.jar > > > curl download success with `-L`: > >> rm -rf jmh-core-1.37.jar ; curl -OL --fail https://maven.aliyun.com/repository/public/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar ; ls -lh jmh-core-1.37.jar ; du -sh jmh-core-1.37.jar > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 > 100 540k 100 540k 0 0 1097k 0 --:--:-- --:--:-- --:--:-- 1097k > -rw-rw-r-- 1 yansendao yansendao 541K Apr 26 17:38 jmh-core-1.37.jar > 544K jmh-core-1.37.jar > > > > Only change devkit shell script, the risk is low. > /sponsor @shipilev Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18965#issuecomment-2083317843 From jjg at openjdk.org Mon Apr 29 18:15:52 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Mon, 29 Apr 2024 18:15:52 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v59] In-Reply-To: References: Message-ID: > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: Remove links to `jdk.javadoc` module from `java.compiler` module` ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16388/files - new: https://git.openjdk.org/jdk/pull/16388/files/a884ae36..fadc130b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=58 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=57-58 Stats: 9 lines in 1 file changed: 0 ins; 5 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388 From erikj at openjdk.org Mon Apr 29 19:25:04 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Mon, 29 Apr 2024 19:25:04 GMT Subject: RFR: 8331331: :tier1 target explanation in doc/testing.md is incorrect In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 16:14:45 GMT, SendaoYan wrote: > Hi, > > In doc/testing.md file, it says: > > As an example, :tier1 will expand to jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1. > > The actual situation is :tier1 doesn't contains test/nashorn:tier1, and the document missed test/lib-test:tier1 > > $ make -n test-tier1 &> test-tier1.log > $ grep "Running test " test-tier1.log > Running test 'jtreg:test/hotspot/jtreg:tier1' > Running test 'jtreg:test/jdk:tier1' > Running test 'jtreg:test/langtools:tier1' > Running test 'jtreg:test/jaxp:tier1' > Running test 'jtreg:test/lib-test:tier1' > > Only change the document, no risk. Marked as reviewed by erikj (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/19002#pullrequestreview-2029396718 From darcy at openjdk.org Mon Apr 29 22:50:26 2024 From: darcy at openjdk.org (Joe Darcy) Date: Mon, 29 Apr 2024 22:50:26 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 [v3] In-Reply-To: References: Message-ID: > Get JDK 24 underway. Joe Darcy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: - Update deprecated options test. - Merge branch 'master' into JDK-8330188 - Merge branch 'master' into JDK-8330188 - Merge branch 'master' into JDK-8330188 - Correct release date as observed in review feedback. - Improve javadoc of class file update. - JDK-8330182: Start of release updates for JDK 24 JDK-8330183: Add SourceVersion.RELEASE_24 JDK-8330184: Add source 24 and target 24 to javac ------------- Changes: - all: https://git.openjdk.org/jdk/pull/18787/files - new: https://git.openjdk.org/jdk/pull/18787/files/dc488f21..47100a28 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=18787&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=18787&range=01-02 Stats: 69237 lines in 1984 files changed: 31363 ins; 31713 del; 6161 mod Patch: https://git.openjdk.org/jdk/pull/18787.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18787/head:pull/18787 PR: https://git.openjdk.org/jdk/pull/18787 From iris at openjdk.org Tue Apr 30 06:25:08 2024 From: iris at openjdk.org (Iris Clark) Date: Tue, 30 Apr 2024 06:25:08 GMT Subject: RFR: 8330182: Start of release updates for JDK 24 [v3] In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 22:50:26 GMT, Joe Darcy wrote: >> Get JDK 24 underway. > > Joe Darcy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: > > - Update deprecated options test. > - Merge branch 'master' into JDK-8330188 > - Merge branch 'master' into JDK-8330188 > - Merge branch 'master' into JDK-8330188 > - Correct release date as observed in review feedback. > - Improve javadoc of class file update. > - JDK-8330182: Start of release updates for JDK 24 > JDK-8330183: Add SourceVersion.RELEASE_24 > JDK-8330184: Add source 24 and target 24 to javac Marked as reviewed by iris (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/18787#pullrequestreview-2030298396 From dholmes at openjdk.org Tue Apr 30 06:28:04 2024 From: dholmes at openjdk.org (David Holmes) Date: Tue, 30 Apr 2024 06:28:04 GMT Subject: RFR: 8331331: :tier1 target explanation in doc/testing.md is incorrect In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 16:14:45 GMT, SendaoYan wrote: > Hi, > > In doc/testing.md file, it says: > > As an example, :tier1 will expand to jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1. > > The actual situation is :tier1 doesn't contains test/nashorn:tier1, and the document missed test/lib-test:tier1 > > $ make -n test-tier1 &> test-tier1.log > $ grep "Running test " test-tier1.log > Running test 'jtreg:test/hotspot/jtreg:tier1' > Running test 'jtreg:test/jdk:tier1' > Running test 'jtreg:test/langtools:tier1' > Running test 'jtreg:test/jaxp:tier1' > Running test 'jtreg:test/lib-test:tier1' > > Only change the document, no risk. Perhaps we should rephrase this so it is not an exhaustive list that has to be updated if the test groupings change. Suggestion: As an example, :tier1 will expand to include all subcomponent test directories that define `tier1`, for example: jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 .... ------------- PR Review: https://git.openjdk.org/jdk/pull/19002#pullrequestreview-2030302932 From mbaesken at openjdk.org Tue Apr 30 07:34:12 2024 From: mbaesken at openjdk.org (Matthias Baesken) Date: Tue, 30 Apr 2024 07:34:12 GMT Subject: RFR: 8331298: avoid alignment checks in UBSAN enabled build In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 12:15:55 GMT, Matthias Baesken wrote: > Currently we run into some alignment related issues when building with '--enable-ubsan' . Those errors already occur in the build. Fixing them might take some time and maybe also some discussion if it is worth the effort , > So for now the alignment related checks should be disabled to get the UBSAN build working. > Example : > > /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128:13: runtime error: store to misaligned address 0x15099c3cf4ce for type 'int', which requires 4 byte alignment > 0x15099c3cf4ce: note: pointer points here > 00 80 0f 86 00 00 00 00 3d 06 00 00 80 76 60 3d 07 00 00 80 76 40 3d 08 00 00 80 76 20 3d 1e 00 > ^ > #0 0x1509b3b04f10 in MacroAssembler::pd_patch_instruction(unsigned char*, unsigned char*, char const*, int) /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128 > #1 0x1509b3b04f10 in Label::patch_instructions(MacroAssembler*) /jdk/src/hotspot/share/asm/assembler.cpp:201 > #2 0x1509b940b6d8 in VM_Version_StubGenerator::generate_get_cpu_info() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:381 > #3 0x1509b94059bd in VM_Version::initialize() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:2138 > #4 0x1509b93fb56e in VM_Version_init() /jdk/src/hotspot/share/runtime/vm_version.cpp:32 > #5 0x1509b61ef947 in init_globals() /jdk/src/hotspot/share/runtime/init.cpp:126 > #6 0x1509b8fb0e29 in Threads::create_vm(JavaVMInitArgs*, bool*) /jdk/src/hotspot/share/runtime/threads.cpp:553 > #7 0x1509b67da3d7 in JNI_CreateJavaVM_inner /jdk/src/hotspot/share/prims/jni.cpp:3581 > #8 0x1509b67da3d7 in JNI_CreateJavaVM /jdk/src/hotspot/share/prims/jni.cpp:3672 > #9 0x1509c0eed957 in InitializeJVM /jdk/src/java.base/share/native/libjli/java.c:1550 > #10 0x1509c0eed957 in JavaMain /jdk/src/java.base/share/native/libjli/java.c:491 > ... (rest of output omitted) Thanks for the reviews ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/18998#issuecomment-2084588114 From mbaesken at openjdk.org Tue Apr 30 07:34:13 2024 From: mbaesken at openjdk.org (Matthias Baesken) Date: Tue, 30 Apr 2024 07:34:13 GMT Subject: Integrated: 8331298: avoid alignment checks in UBSAN enabled build In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 12:15:55 GMT, Matthias Baesken wrote: > Currently we run into some alignment related issues when building with '--enable-ubsan' . Those errors already occur in the build. Fixing them might take some time and maybe also some discussion if it is worth the effort , > So for now the alignment related checks should be disabled to get the UBSAN build working. > Example : > > /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128:13: runtime error: store to misaligned address 0x15099c3cf4ce for type 'int', which requires 4 byte alignment > 0x15099c3cf4ce: note: pointer points here > 00 80 0f 86 00 00 00 00 3d 06 00 00 80 76 60 3d 07 00 00 80 76 40 3d 08 00 00 80 76 20 3d 1e 00 > ^ > #0 0x1509b3b04f10 in MacroAssembler::pd_patch_instruction(unsigned char*, unsigned char*, char const*, int) /jdk/src/hotspot/cpu/x86/macroAssembler_x86.hpp:128 > #1 0x1509b3b04f10 in Label::patch_instructions(MacroAssembler*) /jdk/src/hotspot/share/asm/assembler.cpp:201 > #2 0x1509b940b6d8 in VM_Version_StubGenerator::generate_get_cpu_info() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:381 > #3 0x1509b94059bd in VM_Version::initialize() /jdk/src/hotspot/cpu/x86/vm_version_x86.cpp:2138 > #4 0x1509b93fb56e in VM_Version_init() /jdk/src/hotspot/share/runtime/vm_version.cpp:32 > #5 0x1509b61ef947 in init_globals() /jdk/src/hotspot/share/runtime/init.cpp:126 > #6 0x1509b8fb0e29 in Threads::create_vm(JavaVMInitArgs*, bool*) /jdk/src/hotspot/share/runtime/threads.cpp:553 > #7 0x1509b67da3d7 in JNI_CreateJavaVM_inner /jdk/src/hotspot/share/prims/jni.cpp:3581 > #8 0x1509b67da3d7 in JNI_CreateJavaVM /jdk/src/hotspot/share/prims/jni.cpp:3672 > #9 0x1509c0eed957 in InitializeJVM /jdk/src/java.base/share/native/libjli/java.c:1550 > #10 0x1509c0eed957 in JavaMain /jdk/src/java.base/share/native/libjli/java.c:491 > ... (rest of output omitted) This pull request has now been integrated. Changeset: 60b61e58 Author: Matthias Baesken URL: https://git.openjdk.org/jdk/commit/60b61e588c1252b4b1fbc64d0f818a85670f7146 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8331298: avoid alignment checks in UBSAN enabled build Reviewed-by: erikj, mdoerr ------------- PR: https://git.openjdk.org/jdk/pull/18998 From jkern at openjdk.org Tue Apr 30 09:39:09 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 30 Apr 2024 09:39:09 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: On Mon, 29 Apr 2024 16:17:13 GMT, Joachim Kern wrote: >> For the impatient, I suggest adopting mechanism 2, i.e. unconditionally >> include in globalDefinitions_gcc.hpp. >> >> We can't include in shared code, and there is a use in shared code >> (in the relatively recently added JavaThread::pretouch_stack). >> >> When I questioned whether we needed to include at all, I referred >> to a Linux man page I'd found on the internet (the same page mdoerr linked >> to), which says (in part) >> >> "By default, modern compilers automatically translate all uses of alloca() >> into the built-in ..." >> >> Apparently I should have kept digging, because it seems that page is >> old/incorrect. A seemingly more recent Linux man page describes a different >> way of handling it that is closer to what we're seeing, but still not quite >> correct. >> >> glibc's includes if __USE_MISC is defined. >> One of the ways __USE_MISC can become defined is if _GNU_SOURCE is defined, >> and we define that for both gcc and clang toolchains. >> >> We include in globalDefinitions_gcc.hpp. So when building with gcc, >> globalDefinitions.hpp implicitly includes . >> >> The glibc definition of alloca is >> >> #ifdef __GNUC__ >> # define alloca(size) __builtin_alloca (size) >> #endif /* GCC. */ >> >> So that explains why we don't need any explicit include of when >> building with gcc. I expect there's something similar going on with Visual >> Studio and Xcode/clang. But apparently not with Open XLC clang. > > On AIX `stdlib.h` also would define `alloca`, if `__STRICT_ANSI__` wouldn't be set. > > > 780 #if !defined(__xlC__) || defined(__ibmxl__) || defined(__cplusplus) > 781 #if defined(__IBMCPP__) && !defined(__ibmxl__) > 782 extern "builtin" char *__alloca (size_t); > 783 # define alloca __alloca > 784 #elif defined(__GNUC__) && !defined(__STRICT_ANSI__) > 785 #undef alloca > 786 #define alloca(size) __builtin_alloca (size) > 787 #endif > > > A small plain Testprogramm not using all of the flags we used in jdk build, does not set `__STRICT_ANSI__` and then `alloca` is defined correct. The compiler flag introducing __STRICT_ANSI__ is -std=c++14. If I omit this explicit compiler flag the default is used, which is also c++14. But the default does not set __STRICT_ANSI__ but 2 other defines. I will try a build without -std=c++14 and if this works, we have a solution. Nevertheless i will interrogate IBM what the hell this behavior should be. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1584461665 From jwaters at openjdk.org Tue Apr 30 09:46:28 2024 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 30 Apr 2024 09:46:28 GMT Subject: RFR: 8331020: Assign LANG to C++ for Windows code that uses C++ Message-ID: Currently, on Windows LANG is not assigned to C++ for some code that does use C++. This just works because link.exe does not bother about what kind of code it is currently linking. gcc however, does. It doesn't hurt to assign LANG to C++ as a formality in such cases, which this changeset does. This also renames LINK_TYPE to LANG, which the original change to remove the TOOLCHAIN parameter used to do ------------- Commit messages: - Assign LANG to C++ in Lib.gmk - Assign LANG to C++ in Lib.gmk - Assign LANG to C++ in Launcher.gmk - Assign LANG to C++ in Lib.gmk - Assign LANG to C++ in AwtLibraries.gmk - Rename import in Lib.gmk - Rename ClientLibraries.gmk to 2dLibraries.gmk - LINK_TYPE to LANG in ClientLibraries.gmk - LINK_TYPE to LANG in JdkNativeCompilation.gmk - Add C++ to Windows in Lib.gmk - ... and 11 more: https://git.openjdk.org/jdk/compare/7a895552...8d0701bd Changes: https://git.openjdk.org/jdk/pull/18927/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=18927&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8331020 Stats: 52 lines in 19 files changed: 15 ins; 3 del; 34 mod Patch: https://git.openjdk.org/jdk/pull/18927.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/18927/head:pull/18927 PR: https://git.openjdk.org/jdk/pull/18927 From ihse at openjdk.org Tue Apr 30 10:22:12 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 10:22:12 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: <9YS0_M4NEVmNW42dPYHTfuYarIWGCxKdXQqFeWTt4hI=.a427f219-a7db-472f-8e20-722d3bd516d6@github.com> On Tue, 30 Apr 2024 09:36:52 GMT, Joachim Kern wrote: >> On AIX `stdlib.h` also would define `alloca`, if `__STRICT_ANSI__` wouldn't be set. >> >> >> 780 #if !defined(__xlC__) || defined(__ibmxl__) || defined(__cplusplus) >> 781 #if defined(__IBMCPP__) && !defined(__ibmxl__) >> 782 extern "builtin" char *__alloca (size_t); >> 783 # define alloca __alloca >> 784 #elif defined(__GNUC__) && !defined(__STRICT_ANSI__) >> 785 #undef alloca >> 786 #define alloca(size) __builtin_alloca (size) >> 787 #endif >> >> >> A small plain Testprogramm not using all of the flags we used in jdk build, does not set `__STRICT_ANSI__` and then `alloca` is defined correct. > > The compiler flag introducing __STRICT_ANSI__ is -std=c++14. If I omit this explicit compiler flag the default is used, which is also c++14. But the default does not set __STRICT_ANSI__ but 2 other defines. I will try a build without -std=c++14 and if this works, we have a solution. Nevertheless i will interrogate IBM what the hell this behavior should be. I don't think leaving out `-std=c++14` for AIX is a good solution. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1584529538 From magnus.ihse.bursie at oracle.com Tue Apr 30 12:16:14 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 14:16:14 +0200 Subject: Usage of iconv() In-Reply-To: <91874482-df68-4e76-99e9-4fb90ef2c7a7@oracle.com> References: <20240423121114.EE241662C87@dd33810.kasserver.com> <91874482-df68-4e76-99e9-4fb90ef2c7a7@oracle.com> Message-ID: <77bc954e-e0bb-40f3-a602-cb58cb7f74a3@oracle.com> On 2024-04-25 20:30, Philip Race wrote: > > > On 4/24/24 4:24 AM, Magnus Ihse Bursie wrote: >> That is a good question. libiconv is used only on macOS and AIX, for >> a few libraries, as you say. I just tried removing -liconv from the >> macOS dependencies and recompiled, just to see what would happen. >> There were three instances for macOS: libsplashscreen, libjdwp and >> libinstrument. >> >> >> >> libsplashscreen uses it in splashscreen_sys.m, where it is used to >> convert the jar file name. > > This is called from the launcher, in java.base/share/native/libjli/java.c > > >> >> libjdwp uses it in utf_util.c, where it is used to convert file name >> and command lines, iiuc. >> >> It is likely that we have similar (but better) ways to convert >> charsets elsewhere in our libraries that can be used instead of >> libiconv. I guess these are not the only two places where a filename >> must be converted from the platform charset to UTF8. > > So whatever replacement there might be, it needs to be something that > is available very early in the life of the VM, in fact before there is > a VM running. Agreed. But it seems to be that this is something that needs to be handled by libjli, to properly deal with paths and command lines. I'm wondering if the places which to *not* use iconv (or similar) is actually incorrect. /Magnus > > -phil. From ihse at openjdk.org Tue Apr 30 12:27:05 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 12:27:05 GMT Subject: RFR: 8331331: :tier1 target explanation in doc/testing.md is incorrect In-Reply-To: References: Message-ID: On Mon, 29 Apr 2024 16:14:45 GMT, SendaoYan wrote: > Hi, > > In doc/testing.md file, it says: > > As an example, :tier1 will expand to jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1. > > The actual situation is :tier1 doesn't contains test/nashorn:tier1, and the document missed test/lib-test:tier1 > > $ make -n test-tier1 &> test-tier1.log > $ grep "Running test " test-tier1.log > Running test 'jtreg:test/hotspot/jtreg:tier1' > Running test 'jtreg:test/jdk:tier1' > Running test 'jtreg:test/langtools:tier1' > Running test 'jtreg:test/jaxp:tier1' > Running test 'jtreg:test/lib-test:tier1' > > Only change the document, no risk. This fix works, but I like David's idea better. It was meant as an example, not necessarily kept up to date, so by deliberately making it shorter and ading `...` we show that this is not am exhaustive list. ------------- PR Comment: https://git.openjdk.org/jdk/pull/19002#issuecomment-2085193115 From ihse at openjdk.org Tue Apr 30 12:35:07 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 12:35:07 GMT Subject: RFR: 8331020: Assign LANG to C++ for Windows code that uses C++ In-Reply-To: References: Message-ID: <_4VibJgEPJGgaA7bMYFP5wCBFgpgdntmlSdCMePNaOE=.b8222916-bf5c-4771-ac0b-afaa1d061cad@github.com> On Wed, 24 Apr 2024 01:02:36 GMT, Julian Waters wrote: > Currently, on Windows LANG is not assigned to C++ for some code that does use C++. This just works because link.exe does not bother about what kind of code it is currently linking. gcc however, does. It doesn't hurt to assign LANG to C++ as a formality in such cases, which this changeset does. This also renames LINK_TYPE to LANG, which the original change to remove the TOOLCHAIN parameter used to do Please revert back to the LINK_TYPE name. As long as it is not used for anything else, this is a better description. If we start to use it to have a broader meaning, we can rename it then. make/hotspot/gensrc/GensrcAdlc.gmk line 45: > 43: endif > 44: else ifeq ($(call isBuildOs, windows), true) > 45: ifeq ($(TOOLCHAIN_TYPE), microsoft) You're kind of sneaking in some of your "support other toolchain than ms on windows" changes here. While it does not matter that much, right now we have the assumption that platform=windows <=> toolchain=microsoft in the entire code base. With that assumption, this change looks strange. So I'd rather not take this in now, but instead do a complete integration of the changes needed to support multiple toolchains on Windows. make/hotspot/lib/CompileGtest.gmk line 98: > 96: -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \ > 97: $(addprefix -I,$(GTEST_TEST_SRC)), \ > 98: CFLAGS_windows := -EHsc, \ Just to clarify: these kind of changes are okay, since for mainline it is equivalent if you do `CFLAGS_windows` or `CFLAGS_microsoft`, so if it helps you I am completely okay with converting one kind of check to another. (It was the double-checking that I objected to.) make/modules/java.desktop/lib/AwtLibraries.gmk line 102: > 100: $(eval $(call SetupJdkLibrary, BUILD_LIBAWT, \ > 101: NAME := awt, \ > 102: LANG := $(if $(filter $(OPENJDK_TARGET_OS), windows), C++, C), \ No, this is not okay. You need to do this as for the LIBJSOUND_LINK_TYPE above. The same goes for the one below, too. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18927#issuecomment-2085199170 PR Review Comment: https://git.openjdk.org/jdk/pull/18927#discussion_r1584720911 PR Review Comment: https://git.openjdk.org/jdk/pull/18927#discussion_r1584722581 PR Review Comment: https://git.openjdk.org/jdk/pull/18927#discussion_r1584723694 From jkern at openjdk.org Tue Apr 30 12:36:16 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 30 Apr 2024 12:36:16 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <9YS0_M4NEVmNW42dPYHTfuYarIWGCxKdXQqFeWTt4hI=.a427f219-a7db-472f-8e20-722d3bd516d6@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> <9YS0_M4NEVmNW42dPYHTfuYarIWGCxKdXQqFeWTt4hI=.a427f219-a7db-472f-8e20-722d3bd516d6@github.com> Message-ID: On Tue, 30 Apr 2024 10:19:30 GMT, Magnus Ihse Bursie wrote: >> The compiler flag introducing __STRICT_ANSI__ is -std=c++14. If I omit this explicit compiler flag the default is used, which is also c++14. But the default does not set __STRICT_ANSI__ but 2 other defines. I will try a build without -std=c++14 and if this works, we have a solution. Nevertheless i will interrogate IBM what the hell this behavior should be. > > I don't think leaving out `-std=c++14` for AIX is a good solution. I got it. And what about simply disabling the `__STRICT_ANSI__` with `CFLAGS_OS_DEF_JVM="-DAIX -D_LARGE_FILES -U__STRICT_ANSI__"` in flags-cflags.m4 for AIX. This worked too. The build is fine. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1584725053 From magnus.ihse.bursie at oracle.com Tue Apr 30 12:42:07 2024 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 14:42:07 +0200 Subject: Hermetic Java project meeting In-Reply-To: References: <65fdda77-1291-437e-b000-ca0307cf8084@oracle.com> Message-ID: <779749fa-8427-4b48-b897-5210b36d5be8@oracle.com> On 2024-04-26 03:15, Jiangli Zhou wrote: > On Thu, Apr 25, 2024 at 9:28?AM Magnus Ihse Bursie > wrote: >> >> Just to be more clear, that's with using `objcopy` to localize non-exported symbols for all JDK static libraries and libjvm.a, not just libjvm.a right? >> >> Yes. >> >> >> Can you please include the compiler or linker errors on linux/clang? >> >> It is a bit tricky. The problem arises at the partial linking step. The problem seem to arise out of how clang converts a request to link into an actual call to ld. I enabled debug code (printing the command line, and running clang with `-v` to get it to print the actual command line used to run ld) and ran it on GHA, where it worked fine. This is how it looks there: >> >> WILL_RUN: /usr/bin/clang -v -m64 -r -o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/librmi_relocatable.o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/GC.o >> Ubuntu clang version 14.0.0-1ubuntu1.1 >> Target: x86_64-pc-linux-gnu >> Thread model: posix >> InstalledDir: /usr/bin >> Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/10 >> Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11 >> Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12 >> Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13 >> Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 >> Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13 >> Candidate multilib: .;@m64 >> Selected multilib: .;@m64 >> "/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/librmi_relocatable.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/lib/llvm-14/bin/../lib -L/lib -L/usr/lib -r /home/runner/work/jdk/jdk/build/linux-x64/support/native/java.rmi/librmi/static/GC.o >> >> In contrast, on my machine it looks like this: >> >> WILL_RUN: /usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/clang -v -m64 -r -o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/librmi_relocatable.o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/GC.o >> clang version 13.0.1 >> Target: x86_64-unknown-linux-gnu >> Thread model: posix >> InstalledDir: /usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin >> Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 >> Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 >> Candidate multilib: .;@m64 >> Candidate multilib: 32;@m32 >> Candidate multilib: x32;@mx32 >> Selected multilib: .;@m64 >> "/usr/bin/ld" --hash-style=both --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/librmi_relocatable.o /lib/x86_64-linux-gnu/crt1.o /lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/local/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/../lib -L/lib -L/usr/lib -r /localhome/git/jdk-ALT/build/clangherm/support/native/java.rmi/librmi/static/GC.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/9/crtend.o /lib/x86_64-linux-gnu/crtn.o >> /usr/bin/ld: cannot find -lgcc_s >> /usr/bin/ld: cannot find -lgcc_s >> clang-13: error: linker command failed with exit code 1 (use -v to see invocation) >> >> I don't understand what makes clang think it should include "-lgcc --as-needed -lgcc_s" and the crt*.o files when doing a partial link. In fact, the entire process on how clang (and gcc) builds up the linker command line is bordering on black magic to me. I think it can be affected by variables set at compile time (at least this was the case for gcc, last I checked), or maybe it picks up some kind of script from the environment. That's why I believe my machine could just be messed up. >> >> I could get a bit further by passing "-nodefaultlibs" (or whatever it was), but then the generated .o file were messed up wrt to library symbols and it failed dramatically when trying to do the final link of the static java launcher. >> >> > Looks like you are using /usr/bin/ld and not lld. I haven't run into > this type of issue. Have you tried -fuse-ld=lld? I am not sure why clang insisted on picking up ld and not lld. I remeber trying with -fuse-ld=lld, and that it did not work either. Unfortunately, I don't remember exactly what the problems were. I started reinstalling my Linux workstation yesterday, but something went wrong, and it failed so hard that it got semi-bricked by the new installation, so I need to redo everything from scratch. :-( After that is done, I'll re-test. Hopefully this was just my old installation that was too broken. > >>> >>> I have also tried to extract all the changes (and only the changes) >>> related to static build from the hermetic-java-runtime branch (ignoring >>> the JavaHome/resource loading changes), to see if I could get something >>> like StaticLink.gmk in mainline. I thought I was doing quite fine, but >>> after a while I realized my testing was botched since the launcher had >>> actually loaded the libraries dynamically instead, even though they were >>> statically linked. :-( I am currently trying to bisect my way thought my >>> repo to understand where things went wrong. >> >> Did you run with `bin/javastatic`? The system automatically detects if the binary contains statically linked native libraries and avoids loading the dynamic libraries. Can you please share which test(s) ran into the library loading issue? I'll see if I can reproduce the problem that you are running into. >> >> It was in fact not a problem. I was fooled by an error message. To be sure I was not loading any dynamically linked libraries, I removed the jdk/lib directory. Now the launcher failed, saying something like: >> >> "Error: Cannot locate dynamic library libjava.dylib". >> >> which was a bit of a jump scare. >> >> However, it turned out that it actually tried to load lib/jvm.cfg, and failed in loading this (since I had removed the entire lib directory), and this failure caused the above error message to be printed. When I restored lib/jvm.cfg (but not any dynamic libraries), the launcher worked. >> > Sounds like you are running into problems immediately during startup. > Does the problem occur with just running bin/javastatic using a simple > HelloWorld? Can you please send me your command line for reproducing? Maybe I was not clear enough: I did resolve the problem. > For the static Java support, I changed CreateExecutionEnvironment to > return immediately if it executes statically. jvm.cfg is not loaded. > Please see https://github.com/openjdk/leyden/blob/c1c5fc686c1452550e1b3663a320fba652248505/src/java.base/unix/native/libjli/java_md.c#L296. > Sounds like the JLI_IsStaticJDK() check is not working properly in > your case. I've been trying to extract from your port a minimal set of patches that is needed to get static build to work. In that process, JavaHome and JLI_IsStaticJDK have been removed. It might be that this issue arised only in my slimmed-down branch, and not on your leyden branch (at this point I don't recall exactly). But, we need to fix this separately, since we must be able to build a static launcher without the hermetic changes. In my branch, I am only using compile-time #ifdef checks for static vs dynamic. In the long run, the runtime checks that you have done are a good thing, but at the moment they are just adding intrusive changes without providing any benefit -- if we can't reuse .o files between dynamic and static compilation, there is no point in introducing a runtime check when we already have a working compile-time check. I did think I correctly changed every dynamic check that you had added to a compile-time check, so it bewilders me somewhat when you say that jvm.cfg is not needed in your branch. Can you verify and confirm that the static launcher actually works in your branch, if there is no "lib/jvm.cfg" present? /Magnus > > Best, > Jiangli > >> There are several bugs lurking here. For once, the error message is incorrect and should be corrected. Secondly, a statically linked launcher has just a single JVM included and should not have to look for the lib/jvm.cfg file at all. >> >> After looking around a bit in the launcher/jli code, my conclusion is that this code will need some additional care and loving attention to make it properly adjusted to function as a static launcher. We can't have a static launcher that tries to load a jvm.cfg file it does not need, and when it fails, complains that it is missing a dynamic library that it should not load. >> >> I'll try to get this fixed as part of my efforts to get the static launcher into mainline. >>> This was done haphazardly in StaticLink.gmk in the hermetic-java-runtime >>> branch, where an arbitrary subset of external libraries were hard-coded. >>> Before integration in mainline can be possible, this information needs >>> to be collected correctly and automatically for all included JDK >>> libraries. Fortunately, it is not likely to be too hard. I basically >>> just need to store the information from the LIBS provided to the >>> NativeCompilation, and pick that up for all static libraries we include >>> in the static launcher. (A complication is that we need to de-duplicate >>> the list, and that some libraries are specified using two words, like >>> "-framework Application" on macos, so it will take some care getting it >>> right.) >> >> Right, currently the hermetic-java-runtime branch specifies a list of hard-coded dependency libraries for linking. One of the goals of the hermetic prototype was avoiding/reducing refactoring/restructuring the existing code whenever possible. The reason is to reduce merge overhead when integrating with new changes from the mainline. We can do the proper refactoring and cleanups when getting the changes into the mainline. >> >> That is basically what I am doing right now. I am looking at your prototype and trying to reimplement this functionality properly so that it can be merged into mainline. The first step on that way was to actually get your prototype running. >> >> Now I have managed to get a version of your prototype that only includes the minimal set of changes needed to support the static launcher, and that works on mac and linux/gcc. Since your prototype is based on 586396cbb55a31 from March, I am trying to merge the patch with the latest master. This worked fine for macOS, but I hit some unexpected snag on Linux which I'm currently investigating. >> >> We have only briefly touched on the spec change topic (for the naming of native libraries) during the zoom meetings. I also agree that we should get that part started now. It's unclear to me if there's any existing blocker for that. >> >> I don't think there is. It's just that someone needs to step up and do it. >> >> /Magnus From erikj at openjdk.org Tue Apr 30 12:42:04 2024 From: erikj at openjdk.org (Erik Joelsson) Date: Tue, 30 Apr 2024 12:42:04 GMT Subject: RFR: 8331331: :tier1 target explanation in doc/testing.md is incorrect In-Reply-To: References: Message-ID: <37oVJIUspydhX6AwxF2dJ4PfBbHZBdeU2yT_r68_6f8=.70b4a76e-bca2-4a0f-b019-ff8cebd17b6e@github.com> On Mon, 29 Apr 2024 16:14:45 GMT, SendaoYan wrote: > Hi, > > In doc/testing.md file, it says: > > As an example, :tier1 will expand to jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1. > > The actual situation is :tier1 doesn't contains test/nashorn:tier1, and the document missed test/lib-test:tier1 > > $ make -n test-tier1 &> test-tier1.log > $ grep "Running test " test-tier1.log > Running test 'jtreg:test/hotspot/jtreg:tier1' > Running test 'jtreg:test/jdk:tier1' > Running test 'jtreg:test/langtools:tier1' > Running test 'jtreg:test/jaxp:tier1' > Running test 'jtreg:test/lib-test:tier1' > > Only change the document, no risk. I agree with Magnus and David, that is a better approach. ------------- PR Comment: https://git.openjdk.org/jdk/pull/19002#issuecomment-2085224067 From ihse at openjdk.org Tue Apr 30 12:49:11 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 12:49:11 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> <9YS0_M4NEVmNW42dPYHTfuYarIWGCxKdXQqFeWTt4hI=.a427f219-a7db-472f-8e20-722d3bd516d6@github.com> Message-ID: <6HAzyjAGGGHoyHiS_34GvALrggfbGsbAF6IRJlx5WTI=.8f36bfdd-3708-4101-8539-ccfe53cfb6e9@github.com> On Tue, 30 Apr 2024 12:33:19 GMT, Joachim Kern wrote: >> I don't think leaving out `-std=c++14` for AIX is a good solution. > > I got it. And what about simply disabling the `__STRICT_ANSI__` with > `CFLAGS_OS_DEF_JVM="-DAIX -D_LARGE_FILES -U__STRICT_ANSI__"` in flags-cflags.m4 for AIX. This worked too. The build is fine. So what you are saing is basically replacing CFLAGS_OS_DEF_JVM="-DAIX -Dalloca'(size)'=__builtin_alloca'(size)' -D_LARGE_FILES" ``` with CFLAGS_OS_DEF_JVM="-DAIX -D_LARGE_FILES -U__STRICT_ANSI__" ``` ? Yeah, that'll work, I guess. "strict ansi" sounds like a problematic thing to have enabled, and that it is added by `-std=c++14` sounds close to a bug in my ears. So a "workaround" where this is disabled seem reasonable. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1584754261 From jkern at openjdk.org Tue Apr 30 13:45:13 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 30 Apr 2024 13:45:13 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: <6HAzyjAGGGHoyHiS_34GvALrggfbGsbAF6IRJlx5WTI=.8f36bfdd-3708-4101-8539-ccfe53cfb6e9@github.com> References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> <9YS0_M4NEVmNW42dPYHTfuYarIWGCxKdXQqFeWTt4hI=.a427f219-a7db-472f-8e20-722d3bd516d6@github.com> <6HAzyjAGGGHoyHiS_34GvALrggfbGsbAF6IRJlx5WTI=.8f36bfdd-3708-4101-8539-ccfe53cfb6e9@github.com> Message-ID: On Tue, 30 Apr 2024 12:46:31 GMT, Magnus Ihse Bursie wrote: >> I got it. And what about simply disabling the `__STRICT_ANSI__` with >> `CFLAGS_OS_DEF_JVM="-DAIX -D_LARGE_FILES -U__STRICT_ANSI__"` in flags-cflags.m4 for AIX. This worked too. The build is fine. > > So what you are saing is basically replacing > > CFLAGS_OS_DEF_JVM="-DAIX -Dalloca'(size)'=__builtin_alloca'(size)' -D_LARGE_FILES" > ``` > with > > CFLAGS_OS_DEF_JVM="-DAIX -D_LARGE_FILES -U__STRICT_ANSI__" > ``` > ? > > Yeah, that'll work, I guess. "strict ansi" sounds like a problematic thing to have enabled, and that it is added by `-std=c++14` sounds close to a bug in my ears. So a "workaround" where this is disabled seem reasonable. Yes this would be the replacement. This is our 4th way to fix the issue. Anyone else who would prefer this too? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1584847372 From jwaters at openjdk.org Tue Apr 30 13:53:08 2024 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 30 Apr 2024 13:53:08 GMT Subject: RFR: 8331020: Assign LANG to C++ for Windows code that uses C++ In-Reply-To: <_4VibJgEPJGgaA7bMYFP5wCBFgpgdntmlSdCMePNaOE=.b8222916-bf5c-4771-ac0b-afaa1d061cad@github.com> References: <_4VibJgEPJGgaA7bMYFP5wCBFgpgdntmlSdCMePNaOE=.b8222916-bf5c-4771-ac0b-afaa1d061cad@github.com> Message-ID: On Tue, 30 Apr 2024 12:32:09 GMT, Magnus Ihse Bursie wrote: >> Currently, on Windows LANG is not assigned to C++ for some code that does use C++. This just works because link.exe does not bother about what kind of code it is currently linking. gcc however, does. It doesn't hurt to assign LANG to C++ as a formality in such cases, which this changeset does. This also renames LINK_TYPE to LANG, which the original change to remove the TOOLCHAIN parameter used to do > > make/modules/java.desktop/lib/AwtLibraries.gmk line 102: > >> 100: $(eval $(call SetupJdkLibrary, BUILD_LIBAWT, \ >> 101: NAME := awt, \ >> 102: LANG := $(if $(filter $(OPENJDK_TARGET_OS), windows), C++, C), \ > > No, this is not okay. You need to do this as for the LIBJSOUND_LINK_TYPE above. The same goes for the one below, too. Oh, I'm surprised! I thought that you'd prefer the more lambda-like approach. I guess the other way of LIBAWT_LINK_TYPE works too in that case ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18927#discussion_r1584867458 From jwaters at openjdk.org Tue Apr 30 13:56:05 2024 From: jwaters at openjdk.org (Julian Waters) Date: Tue, 30 Apr 2024 13:56:05 GMT Subject: RFR: 8331020: Assign LANG to C++ for Windows code that uses C++ In-Reply-To: <_4VibJgEPJGgaA7bMYFP5wCBFgpgdntmlSdCMePNaOE=.b8222916-bf5c-4771-ac0b-afaa1d061cad@github.com> References: <_4VibJgEPJGgaA7bMYFP5wCBFgpgdntmlSdCMePNaOE=.b8222916-bf5c-4771-ac0b-afaa1d061cad@github.com> Message-ID: On Tue, 30 Apr 2024 12:27:42 GMT, Magnus Ihse Bursie wrote: > Please revert back to the LINK_TYPE name. As long as it is not used for anything else, this is a better description. If we start to use it to have a broader meaning, we can rename it then. I switched it back to LANG since in the original change you switched it to LINK_TYPE from LANG after one of my objections. I had since retracted that objection and have been feeling bad about it. Have you since changed your mind about LANG vs LINK_TYPE in that time? It might take me a bit of time to address these reviews, sorry about that ------------- PR Comment: https://git.openjdk.org/jdk/pull/18927#issuecomment-2085401560 From mdoerr at openjdk.org Tue Apr 30 14:03:13 2024 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 30 Apr 2024 14:03:13 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: On Thu, 18 Apr 2024 04:26:21 GMT, Kim Barrett wrote: >> I opened https://bugs.openjdk.org/browse/JDK-8330539 so we don't lose track of this, but we can keep the discussion/voting here. > > For the impatient, I suggest adopting mechanism 2, i.e. unconditionally > include in globalDefinitions_gcc.hpp. > > We can't include in shared code, and there is a use in shared code > (in the relatively recently added JavaThread::pretouch_stack). > > When I questioned whether we needed to include at all, I referred > to a Linux man page I'd found on the internet (the same page mdoerr linked > to), which says (in part) > > "By default, modern compilers automatically translate all uses of alloca() > into the built-in ..." > > Apparently I should have kept digging, because it seems that page is > old/incorrect. A seemingly more recent Linux man page describes a different > way of handling it that is closer to what we're seeing, but still not quite > correct. > > glibc's includes if __USE_MISC is defined. > One of the ways __USE_MISC can become defined is if _GNU_SOURCE is defined, > and we define that for both gcc and clang toolchains. > > We include in globalDefinitions_gcc.hpp. So when building with gcc, > globalDefinitions.hpp implicitly includes . > > The glibc definition of alloca is > > #ifdef __GNUC__ > # define alloca(size) __builtin_alloca (size) > #endif /* GCC. */ > > So that explains why we don't need any explicit include of when > building with gcc. I expect there's something similar going on with Visual > Studio and Xcode/clang. But apparently not with Open XLC clang. Ok for me. Let's hear what @kimbarrett thinks. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1584884372 From ihse at openjdk.org Tue Apr 30 14:10:06 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 14:10:06 GMT Subject: RFR: 8331020: Assign LANG to C++ for Windows code that uses C++ In-Reply-To: References: <_4VibJgEPJGgaA7bMYFP5wCBFgpgdntmlSdCMePNaOE=.b8222916-bf5c-4771-ac0b-afaa1d061cad@github.com> Message-ID: On Tue, 30 Apr 2024 13:53:11 GMT, Julian Waters wrote: > I switched it back to LANG since in the original change you switched it to LINK_TYPE from LANG after one of my objections. I had since retracted that objection and have been feeling bad about it. Have you since changed your mind about LANG vs LINK_TYPE in that time? Yes, I have changed my mind. I think your objection back then was valid; I created an argument which implied a wider scope than it really delivered, with some vague hand-waving about future extensions. It is better to be more concrete here and now, and rename the parameter if we ever add more meaning to it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/18927#issuecomment-2085433698 From ihse at openjdk.org Tue Apr 30 14:15:05 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 14:15:05 GMT Subject: RFR: 8331020: Assign LANG to C++ for Windows code that uses C++ In-Reply-To: References: <_4VibJgEPJGgaA7bMYFP5wCBFgpgdntmlSdCMePNaOE=.b8222916-bf5c-4771-ac0b-afaa1d061cad@github.com> Message-ID: <2SOmwkgvwie1C-BepIrzYFhy4DL4kUxmmUJrsH1Rhkk=.a8ccba8b-d216-4679-9796-ec0c0e478d78@github.com> On Tue, 30 Apr 2024 13:50:22 GMT, Julian Waters wrote: >> make/modules/java.desktop/lib/AwtLibraries.gmk line 102: >> >>> 100: $(eval $(call SetupJdkLibrary, BUILD_LIBAWT, \ >>> 101: NAME := awt, \ >>> 102: LANG := $(if $(filter $(OPENJDK_TARGET_OS), windows), C++, C), \ >> >> No, this is not okay. You need to do this as for the LIBJSOUND_LINK_TYPE above. The same goes for the one below, too. > > Oh, I'm surprised! I thought that you'd prefer the more lambda-like approach. I guess the other way of LIBAWT_LINK_TYPE works too in that case There are two reasons: 1) To keep up with the style elsewhere, were we use "ifeq (... platform)" to setup platform-specific arguments. Even if this was not an ideal style, it would still make sense to keep to one way of doing it. 2) I actually think that is better. We have gravitated towards that solution over the years. The make syntax is hard to read and easy to get wrong. We try to make the arguments in the Setup calls trivial, and if we can't do that in place, then we create a "local" variable (by prefixing it with the name of the lib) outside the Setup call, were we can use more space to clearly show what is going on. In fact, I really dislike the `$(if...)` syntax, and use it only if I must. It is hopeless to see what is the if-clause and the else-clause, and it is way too easy to get a "false positive" since you do not compare the variable with another value, but checks if it evaluates to non-empty. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18927#discussion_r1584903238 From ihse at openjdk.org Tue Apr 30 14:42:13 2024 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Tue, 30 Apr 2024 14:42:13 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: On Tue, 30 Apr 2024 14:00:25 GMT, Martin Doerr wrote: >> For the impatient, I suggest adopting mechanism 2, i.e. unconditionally >> include in globalDefinitions_gcc.hpp. >> >> We can't include in shared code, and there is a use in shared code >> (in the relatively recently added JavaThread::pretouch_stack). >> >> When I questioned whether we needed to include at all, I referred >> to a Linux man page I'd found on the internet (the same page mdoerr linked >> to), which says (in part) >> >> "By default, modern compilers automatically translate all uses of alloca() >> into the built-in ..." >> >> Apparently I should have kept digging, because it seems that page is >> old/incorrect. A seemingly more recent Linux man page describes a different >> way of handling it that is closer to what we're seeing, but still not quite >> correct. >> >> glibc's includes if __USE_MISC is defined. >> One of the ways __USE_MISC can become defined is if _GNU_SOURCE is defined, >> and we define that for both gcc and clang toolchains. >> >> We include in globalDefinitions_gcc.hpp. So when building with gcc, >> globalDefinitions.hpp implicitly includes . >> >> The glibc definition of alloca is >> >> #ifdef __GNUC__ >> # define alloca(size) __builtin_alloca (size) >> #endif /* GCC. */ >> >> So that explains why we don't need any explicit include of when >> building with gcc. I expect there's something similar going on with Visual >> Studio and Xcode/clang. But apparently not with Open XLC clang. > > Ok for me. Let's hear what @kimbarrett thinks. It might be easier to get input if you create a new PR with the change. This discussion is hidden deep down in a closed PR. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1584947979 From jkern at openjdk.org Tue Apr 30 15:22:19 2024 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 30 Apr 2024 15:22:19 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: On Tue, 30 Apr 2024 14:39:29 GMT, Magnus Ihse Bursie wrote: >> Ok for me. Let's hear what @kimbarrett thinks. > > It might be easier to get input if you create a new PR with the change. This discussion is hidden deep down in a closed PR. I will do after labor day and create a PR with this suggested solution in your JDK-8330539. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1585020690 From kbarrett at openjdk.org Tue Apr 30 16:39:13 2024 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 30 Apr 2024 16:39:13 GMT Subject: RFR: 8329257: AIX: Switch HOTSPOT_TOOLCHAIN_TYPE from xlc to gcc [v3] In-Reply-To: References: <-XeYeJ0OEmauTYsEoSXxzRmQXSKMOLw87GSpqDnEmug=.5cb7e71f-fea6-4a84-8260-5f515d3d3810@github.com> <18WjPZeDIWkxGIB0BJgyDg5VipCtY4EOlWmIGPWZGCw=.b50cf4a9-61a4-421e-97eb-3dbac94c14f9@github.com> <_xcaF7UUDHA11loD89Dz871vAQgRqMzCdPkahFDfKv8=.a2c6dcbe-5942-4fb7-9d8b-4239ea048e56@github.com> <76P7uKTuqo7IKYr5yBP4Vx1SS0AcEXC_6vDAU6LfIzo=.d939556f-6fab-4009-820b-821376bfdb7c@github.com> <6aR5nvKhz28A1CkxtaAD9CwTjILBjwZrrRwP3988oEc=.72203104-2ae5-40ff-bd87-168b684446e6@ github.com> Message-ID: <1EgO9Z2UdqtUYN2oNClYl_evpBDw1asCxQRWPk0w_6E=.db209d00-845d-44bc-9ca1-e5c533087638@github.com> On Tue, 30 Apr 2024 15:19:47 GMT, Joachim Kern wrote: >> It might be easier to get input if you create a new PR with the change. This discussion is hidden deep down in a closed PR. > > I will do after labor day and create a PR with this suggested solution in your JDK-8330539. I think I still prefer just unconditionally including in globalDefinitions_gcc.hpp. For gcc/clang we are using `-std=c++14` + `-D_GNU_SOURCE` instead of `-std=gnu++14`. I forget exactly why. I don't really want to be messing with `__STRICT_ANSI__`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/18536#discussion_r1585181094 From jjg at openjdk.org Tue Apr 30 20:33:50 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Tue, 30 Apr 2024 20:33:50 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v60] In-Reply-To: References: Message-ID: > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: update commonmark-java from 0.21.0 to 0.22.0 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16388/files - new: https://git.openjdk.org/jdk/pull/16388/files/fadc130b..74c86f51 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=59 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=58-59 Stats: 2382 lines in 53 files changed: 2020 ins; 258 del; 104 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388 From jjg at openjdk.org Tue Apr 30 20:56:21 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Tue, 30 Apr 2024 20:56:21 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v61] In-Reply-To: References: Message-ID: > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 85 commits: - Merge remote-tracking branch 'upstream/master' into 8298405.doclet-markdown-v3 - update commonmark-java from 0.21.0 to 0.22.0 - Remove links to `jdk.javadoc` module from `java.compiler` module` - Suppress warnings building tests - Merge with upstream/master - address review feedback for updates to Elements and friends - address review feedback for updates to Elements and friends - add support for JDK-8329296: Update Elements for '///' documentation comments - add support for `--disable-line-doc-comments` - Merge branch '8298405.doclet-markdown-v3' of https://github.com/jonathan-gibbons/jdk into 8298405.doclet-markdown-v3 - ... and 75 more: https://git.openjdk.org/jdk/compare/b96b38c2...20a384b6 ------------- Changes: https://git.openjdk.org/jdk/pull/16388/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=60 Stats: 26326 lines in 243 files changed: 25679 ins; 260 del; 387 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388 From jjg at openjdk.org Tue Apr 30 21:32:13 2024 From: jjg at openjdk.org (Jonathan Gibbons) Date: Tue, 30 Apr 2024 21:32:13 GMT Subject: RFR: 8298405: Implement JEP 467: Markdown Documentation Comments [v62] In-Reply-To: References: Message-ID: > Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP. > > Notable features: > > * support for `///` documentation comments in `JavaTokenizer` > * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library > * updates to `DocCommentParser` to treat `///` comments as Markdown > * updates to the standard doclet to render Markdown comments in HTML Jonathan Gibbons has updated the pull request incrementally with one additional commit since the last revision: update copyright years ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16388/files - new: https://git.openjdk.org/jdk/pull/16388/files/20a384b6..6576d024 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=61 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16388&range=60-61 Stats: 68 lines in 62 files changed: 0 ins; 7 del; 61 mod Patch: https://git.openjdk.org/jdk/pull/16388.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16388/head:pull/16388 PR: https://git.openjdk.org/jdk/pull/16388