From serguei.spitsyn at oracle.com Fri May 1 06:22:09 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 30 Apr 2020 23:22:09 -0700 Subject: RFR(S/M) : 8243436 : use reproducible random in :vmTestbase_nsk_monitoring In-Reply-To: References: Message-ID: <4ed57228-efa0-593d-eedd-c06deaf41749@oracle.com> Hi Igor, It looks good. Thanks, Serguei On 4/30/20 16:30, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8243436/webrev.00 >> 305 lines changed: 76 ins; 11 del; 218 mod; > Hi all, > > could you please review this patch? > from JBS: >> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_monitoring test group and marking the tests which make use of "randomness" with a proper k/w. > testing: : vmTestbase_nsk_monitoring test group > JBS: https://bugs.openjdk.java.net/browse/JDK-8243436 > webrevs: > - code changes: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.code >> 19 lines changed: 5 ins; 11 del; 3 mod; > - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.kw >> 141 lines changed: 71 ins; 0 del; 70 mod; > - full: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00 >> 305 lines changed: 76 ins; 11 del; 218 mod; > Thanks, > -- Igor From serguei.spitsyn at oracle.com Fri May 1 06:24:02 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 30 Apr 2020 23:24:02 -0700 Subject: RFR(S) : 8243435 : use reproducible random in :vmTestbase_nsk_jvmti In-Reply-To: References: Message-ID: This one looks good too. Thanks, Serguei On 4/30/20 16:46, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8243435/webrev.00 >> 49 lines changed: 19 ins; 5 del; 25 mod; > Hi all, > > could you please review this patch? > from JBS: >> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jvmti test group and marking the tests which make use of "randomness" with a proper k/w. > testing: : vmTestbase_nsk_jvmti test group > JBS: https://bugs.openjdk.java.net/browse/JDK-8243435 > webrevs: > - code changes: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.code >> 7 lines changed: 1 ins; 5 del; 1 mod; > - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.kw >> 18 lines changed: 18 ins; 0 del; 0 mod; > - full: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00 >> 49 lines changed: 19 ins; 5 del; 25 mod; > Thanks, > -- Igor From serguei.spitsyn at oracle.com Fri May 1 06:27:11 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 30 Apr 2020 23:27:11 -0700 Subject: RFR(S) : 8243437 : use reproducible random in :vmTestbase_nsk_jdi In-Reply-To: <32CF125B-6481-48EA-873B-194537179776@oracle.com> References: <32CF125B-6481-48EA-873B-194537179776@oracle.com> Message-ID: LGTM Thanks, Serguei On 4/30/20 16:52, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 >> 62 lines changed: 26 ins; 0 del; 36 mod; > Hi all, > > could you please review this small patch? > from JBS: >> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jdi test group and marking the tests which make use of "randomness" with a proper k/w. > testing: : vmTestbase_nsk_jdi test group > JBS: https://bugs.openjdk.java.net/browse/JDK-8243437 > webrev: http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 > (all code changes have already happened in 8242314 'use reproducible random in vmTestbase shared code', so just k/w and copyright years this time) > > Thanks, > -- Igor From igor.ignatyev at oracle.com Fri May 1 16:33:16 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 1 May 2020 09:33:16 -0700 Subject: RFR(S) : 8243435 : use reproducible random in :vmTestbase_nsk_jvmti In-Reply-To: References: Message-ID: Hi Serguei, thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? -- Igor > On Apr 30, 2020, at 11:24 PM, serguei.spitsyn at oracle.com wrote: > > This one looks good too. > > Thanks, > Serguei > > > On 4/30/20 16:46, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8243435/webrev.00 >>> 49 lines changed: 19 ins; 5 del; 25 mod; >> Hi all, >> >> could you please review this patch? >> from JBS: >>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jvmti test group and marking the tests which make use of "randomness" with a proper k/w. >> testing: : vmTestbase_nsk_jvmti test group >> JBS: https://bugs.openjdk.java.net/browse/JDK-8243435 >> webrevs: >> - code changes: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.code >>> 7 lines changed: 1 ins; 5 del; 1 mod; >> - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.kw >>> 18 lines changed: 18 ins; 0 del; 0 mod; >> - full: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00 >>> 49 lines changed: 19 ins; 5 del; 25 mod; >> Thanks, >> -- Igor > From igor.ignatyev at oracle.com Fri May 1 16:32:57 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 1 May 2020 09:32:57 -0700 Subject: RFR(S/M) : 8243436 : use reproducible random in :vmTestbase_nsk_monitoring In-Reply-To: <4ed57228-efa0-593d-eedd-c06deaf41749@oracle.com> References: <4ed57228-efa0-593d-eedd-c06deaf41749@oracle.com> Message-ID: <0772099A-CBF7-468C-9FB6-4715B2DF2C9B@oracle.com> Hi Serguei, thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? -- Igor > On Apr 30, 2020, at 11:22 PM, serguei.spitsyn at oracle.com wrote: > > Hi Igor, > > It looks good. > > Thanks, > Serguei > > > On 4/30/20 16:30, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8243436/webrev.00 >>> 305 lines changed: 76 ins; 11 del; 218 mod; >> Hi all, >> >> could you please review this patch? >> from JBS: >>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_monitoring test group and marking the tests which make use of "randomness" with a proper k/w. >> testing: : vmTestbase_nsk_monitoring test group >> JBS: https://bugs.openjdk.java.net/browse/JDK-8243436 >> webrevs: >> - code changes: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.code >>> 19 lines changed: 5 ins; 11 del; 3 mod; >> - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.kw >>> 141 lines changed: 71 ins; 0 del; 70 mod; >> - full: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00 >>> 305 lines changed: 76 ins; 11 del; 218 mod; >> Thanks, >> -- Igor > From igor.ignatyev at oracle.com Fri May 1 16:33:21 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 1 May 2020 09:33:21 -0700 Subject: RFR(S) : 8243437 : use reproducible random in :vmTestbase_nsk_jdi In-Reply-To: References: <32CF125B-6481-48EA-873B-194537179776@oracle.com> Message-ID: <7E157D44-29B0-41B2-95B5-CA24F3F3B60C@oracle.com> Hi Serguei, thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? -- Igor > On Apr 30, 2020, at 11:27 PM, serguei.spitsyn at oracle.com wrote: > > LGTM > > Thanks, > Serguei > > > On 4/30/20 16:52, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 >>> 62 lines changed: 26 ins; 0 del; 36 mod; >> Hi all, >> >> could you please review this small patch? >> from JBS: >>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jdi test group and marking the tests which make use of "randomness" with a proper k/w. >> testing: : vmTestbase_nsk_jdi test group >> JBS: https://bugs.openjdk.java.net/browse/JDK-8243437 >> webrev: http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 >> (all code changes have already happened in 8242314 'use reproducible random in vmTestbase shared code', so just k/w and copyright years this time) >> >> Thanks, >> -- Igor > From christoph.langer at sap.com Fri May 1 16:46:10 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 1 May 2020 16:46:10 +0000 Subject: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump In-Reply-To: <2363c58d-38c1-ae19-ed34-c82af6304780@oracle.com> References: <0343dfac-61f7-1b1c-ee96-bdee130578ad@oracle.com> <2363c58d-38c1-ae19-ed34-c82af6304780@oracle.com> Message-ID: Hi Ralf, while I'm reviewing your change I think extracting the compression coding to an own file would be a good idea. Maybe you could name it heapDumpCompression.cpp? When looking at the webrev I also figured that there are some very long lines (beyond 90 chars or so). Maybe you could have a look if you could shorten some of them and break a few of these long lines? More detailed review to follow. Best regards Christoph > -----Original Message----- > From: coleen.phillimore at oracle.com > Sent: Montag, 20. April 2020 14:13 > To: Reingruber, Richard ; Schmelter, Ralf > ; Ioi Lam ; Langer, Christoph > ; Yasumasa Suenaga > ; serguei.spitsyn at oracle.com; hotspot- > runtime-dev at openjdk.java.net runtime dev at openjdk.java.net> > Cc: serviceability-dev at openjdk.java.net > Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped heap > dump > > Hi, I don't want to review this but could you put this new code in its > own file?? heapDumper only needs CompressionBackend to be exported, > from > what I can tell. > > Thanks, > Coleen > > On 4/20/20 6:12 AM, Reingruber, Richard wrote: > > Hi Ralf, > > > >>> 767: I think _current->in_used doesn't take the final 9 bytes into account > that are written in > >>> DumperSupport::end_of_dump() after the last dump segment has been > finished. > >>> You could call get_new_buffer() instead of the if clause. > >> Wow, how did you found this? I've fixed it by making sure we flush the > DumpWriter before calling the deactivate method. > > Spending long hours on the review ;) > > Ok with the fix. > > > >>> ### src/java.base/share/native/libzip/zip_util.c > >>> 1610: Will be hard to beat zlib_block_alloc() and zlib_block_free() > performance wise. But have you > >>> measured the performance gain? In other words: is it worth it? :) > >> This is not done for performance, but to make sure the allocation will not > fail midway during writing the dump. Maybe it is not worth it, though. > > Understood. The heap dump will succeed if you can allocate at least one > WriteWork instance. Without > > that you could get out of memory errors in the zlib which would make the > dump fail. Ok! > > > >> http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.2/ > > Thanks for the clarifications and the changes in the new webrev. > > Webrev.2 looks good to me. > > > > Cheers, Richard. > > > > -----Original Message----- > > From: Schmelter, Ralf > > Sent: Montag, 20. April 2020 10:14 > > To: Reingruber, Richard ; Ioi Lam > ; Langer, Christoph ; > Yasumasa Suenaga ; > serguei.spitsyn at oracle.com; hotspot-runtime-dev at openjdk.java.net > runtime > > Cc: serviceability-dev at openjdk.java.net > > Subject: RE: RFR(L) 8237354: Add option to jcmd to write a gzipped heap > dump > > > > Hi Richard, > > > > thanks for the review. I have incorporated your remarks into a new > webrev: > > http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.2/ > > > > Some remarks to specific points: > > > >> ### src/hotspot/share/services/heapDumper.cpp > >> 762: assert(_active, "Must be active"); > >> > >> It appears to me that the assertion would fail, if an error occurred creating > the CompressionBackend. > > You are supposed to check for errors after creating the DumpWriter (which > creates the CompressionBackend). And in case of an error, you directly > destruct the object. I've added a comment to make that clear. > > > >> 767: I think _current->in_used doesn't take the final 9 bytes into account > that are written in > >> DumperSupport::end_of_dump() after the last dump segment has been > finished. > >> You could call get_new_buffer() instead of the if clause. > > Wow, how did you found this? I've fixed it by making sure we flush the > DumpWriter before calling the deactivate method. > > > >> 1064: DumpWriter::DumpWriter() > >> > >> There doesn't seem to be enough error handling if _buffer cannot be > allocated. > >> E.g. DumpWriter::write_raw() at line 1091 will enter an endless loop. > > As described above, this will not happen if we check for error after > constructing the DumpWriter. > > > >> ### src/java.base/share/native/libzip/zip_util.c > >> 1610: Will be hard to beat zlib_block_alloc() and zlib_block_free() > performance wise. But have you > >> measured the performance gain? In other words: is it worth it? :) > > This is not done for performance, but to make sure the allocation will not > fail midway during writing the dump. Maybe it is not worth it, though. > > > >> 1655: The result of deflateBound() seems to depend on the header > comment, which is not given > >> here. Could this be an issue, because ZIP_GZip_Fully() can take a > comment? > > I've added a 1024 byte additional bytes to avoid the problem. > > > >> ### test/lib/jdk/test/lib/hprof/parser/Reader.java > >> > >> 93: is the created GzipRandomAccess instance closed somewhere? > > The object is not closed since it is still used by the Snapshot returned. > > > > Best regard, > > Ralf > > > > > > -----Original Message----- > > From: Reingruber, Richard > > Sent: Tuesday, 14 April 2020 10:30 > > To: Schmelter, Ralf ; Ioi Lam > ; Langer, Christoph ; > Yasumasa Suenaga ; > serguei.spitsyn at oracle.com; hotspot-runtime-dev at openjdk.java.net > runtime > > Cc: serviceability-dev at openjdk.java.net > > Subject: RE: RFR(L) 8237354: Add option to jcmd to write a gzipped heap > dump > > > > Hi Ralf, > > > > thanks for providing this enhancement to parallel gzip-compress heap > dumps! > > > > I reckon it's safe to say that the coding is sophisticated. It would be > awesome if you could sketch > > the idea of how HeapDumper, DumpWriter and CompressionBackend work > together to produce the gzipped > > dump in a source code comment. Just enough to get started if somebody > should ever have to track down > > a bug -- an unlikely event, I know ;) > > > > Please find the details of my review below. > > > > Thanks, Richard. > > // Not Reviewer > > > > -- > > > > ### src/hotspot/share/services/diagnosticCommand.cpp > > > > 510 _gzip_level("-gz-level", "The compression level from 0 (store) to 9 > (best) when writing in gzipped format.", > > 511 "INT", "FALSE", "1") { > > > > "FALSE" should be probably false. > > > > ### src/hotspot/share/services/diagnosticCommand.hpp > > Ok. > > > > ### src/hotspot/share/services/heapDumper.cpp > > > > 390: Typo: initized > > > > 415: Typo: GZipComressor > > > > 477: Could you please add a comment, how the "HPROF BLOCKSIZE" > comment is helpful? > > > > 539: Member variables of WriteWork are missing the '_' prefix. > > > > 546: Just a comment: WriteWork::in_max is actually a compile time > constant. Would be nice if it could be > > declared so. One could use templates for this, but then my favourite ide > (eclipse cdt) doesn't > > show me references and call hierarchies anymore. So I don't think it is > worth it. > > > > 591: Typo: Removes the first element. Returns NULL is empty. > > > > 663: _writer, _compressor, _lock could be const. > > > > 762: assert(_active, "Must be active"); > > > > It appears to me that the assertion would fail, if an error occurred > creating the CompressionBackend. > > > > 767: I think _current->in_used doesn't take the final 9 bytes into account > that are written in > > DumperSupport::end_of_dump() after the last dump segment has > been finished. > > You could call get_new_buffer() instead of the if clause. > > > > 903: Typo: Check if we don not waste more than _max_waste > > > > 1064: DumpWriter::DumpWriter() > > > > There doesn't seem to be enough error handling if _buffer cannot be > allocated. > > E.g. DumpWriter::write_raw() at line 1091 will enter an endless loop. > > > > 2409: A comment, why Shenandoah is not supported, would be good. > > In general I'd say it is good and natural to use the GC work threads. > > > > ### src/hotspot/share/services/heapDumper.hpp > > Ok. > > > > ### src/java.base/share/native/libzip/zip_util.c > > > > I'm not familiar with zlib, but here are my .02? :) > > > > 1610: Will be hard to beat zlib_block_alloc() and zlib_block_free() > performance wise. But have you > > measured the performance gain? In other words: is it worth it? :) > > > > 1655: The result of deflateBound() seems to depend on the header > comment, which is not given > > here. Could this be an issue, because ZIP_GZip_Fully() can take a > comment? > > > > 1658: deflateEnd() should not be called if deflateInit2Wrapper() failed. I > think this can lead > > otherwise to a double free() call. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTest.java > > > > 66: Maybe additionally check the exit value? > > > > 73: It's unclear to me, why this fails. Because the dump already exists? > Because the level is > > invalid? Reading the comment I'd expect success, not failure. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTestEpsilo > n.java > > Ok. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTestShen > andoah.java > > Ok. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTestZ.jav > a > > Ok. > > > > ### test/lib/jdk/test/lib/hprof/parser/GzipRandomAccess.java > > Ok. > > > > ### test/lib/jdk/test/lib/hprof/parser/HprofReader.java > > Ok. > > > > ### test/lib/jdk/test/lib/hprof/parser/Reader.java > > > > 93: is the created GzipRandomAccess instance closed somewhere? From yumin.qi at oracle.com Fri May 1 17:24:55 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 1 May 2020 10:24:55 -0700 Subject: (S) 8237750: Load libzip.so only if necessary Message-ID: Hi, please review: bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ Summary: ? zip library is loaded unconditionally at start up but it is only needed when ? 1) bootclasspath contains zip or ? 2) UseAOT enabled or ? 3) VerifySharedArchive turned on or ? 4) CDS archives custom loaded classes ?? If none of above in java application, it is not necessary to have zip library loaded. ? Solution by loading zip library on demand when needed. Performance for java -Xint version: Results of " perf stat -r 50 bin/java -Xshare:on -XX:SharedArchiveFile=jdk2.jsa -Xint -version " ?? 1:???? 59611556??? 59450206 (-161350)????? -----???? 39.799 40.274 (? 0.475)??? ++ ?? 2:???? 59602708??? 59425234 (-177474)????? -----???? 40.591 41.183 (? 0.592)??? ++ ?? 3:???? 59579718??? 59441272 (-138446)????? ----????? 40.777 40.471 ( -0.306)????? - ?? 4:???? 59584882??? 59410155 (-174727)????? -----???? 40.824 40.233 ( -0.591)????? -- ?? 5:???? 59590998??? 59447252 (-143746)????? ----????? 40.400 40.493 (? 0.093) ?? 6:???? 59589523??? 59441934 (-147589)????? ----????? 40.475 40.064 ( -0.411)????? -- ?? 7:???? 59581820??? 59413612 (-168208)????? -----???? 39.763 40.077 (? 0.314)???? + ?? 8:???? 59593678??? 59418738 (-174940)????? -----???? 40.912 39.724 ( -1.188)????? ----- ?? 9:???? 59573058??? 59412554 (-160504)????? -----???? 40.126 40.033 ( -0.093) ? 10:???? 59591469??? 59419291 (-172178)????? -----???? 40.731 40.689 ( -0.042) ============================================================ ????????? 59589940??? 59428022 (-161917)????? -----???? 40.438 40.322 ( -0.116) instr delta =????? -161917??? -0.2717% time? delta =?????? -0.116 ms -0.2859% Tests: hs-tier1-4. Due to zip library not loaded at default, I removed 'zip' from pmap list in test case: *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java ** * Thanks Yumin From chris.plummer at oracle.com Fri May 1 18:20:31 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 May 2020 11:20:31 -0700 Subject: RFR(S/M) : 8243436 : use reproducible random in :vmTestbase_nsk_monitoring In-Reply-To: <0772099A-CBF7-468C-9FB6-4715B2DF2C9B@oracle.com> References: <4ed57228-efa0-593d-eedd-c06deaf41749@oracle.com> <0772099A-CBF7-468C-9FB6-4715B2DF2C9B@oracle.com> Message-ID: Looks good. Chris On 5/1/20 9:32 AM, Igor Ignatyev wrote: > Hi Serguei, > > thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? > > -- Igor > >> On Apr 30, 2020, at 11:22 PM, serguei.spitsyn at oracle.com wrote: >> >> Hi Igor, >> >> It looks good. >> >> Thanks, >> Serguei >> >> >> On 4/30/20 16:30, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8243436/webrev.00 >>>> 305 lines changed: 76 ins; 11 del; 218 mod; >>> Hi all, >>> >>> could you please review this patch? >>> from JBS: >>>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_monitoring test group and marking the tests which make use of "randomness" with a proper k/w. >>> testing: : vmTestbase_nsk_monitoring test group >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243436 >>> webrevs: >>> - code changes: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.code >>>> 19 lines changed: 5 ins; 11 del; 3 mod; >>> - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.kw >>>> 141 lines changed: 71 ins; 0 del; 70 mod; >>> - full: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00 >>>> 305 lines changed: 76 ins; 11 del; 218 mod; >>> Thanks, >>> -- Igor From chris.plummer at oracle.com Fri May 1 18:52:49 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 May 2020 11:52:49 -0700 Subject: RFR(S) : 8243435 : use reproducible random in :vmTestbase_nsk_jvmti In-Reply-To: References: Message-ID: <90346959-fa85-7121-bcc5-688933dd069a@oracle.com> Looks good. Chris On 5/1/20 9:33 AM, Igor Ignatyev wrote: > Hi Serguei, > > thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? > > -- Igor > >> On Apr 30, 2020, at 11:24 PM, serguei.spitsyn at oracle.com wrote: >> >> This one looks good too. >> >> Thanks, >> Serguei >> >> >> On 4/30/20 16:46, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8243435/webrev.00 >>>> 49 lines changed: 19 ins; 5 del; 25 mod; >>> Hi all, >>> >>> could you please review this patch? >>> from JBS: >>>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jvmti test group and marking the tests which make use of "randomness" with a proper k/w. >>> testing: : vmTestbase_nsk_jvmti test group >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243435 >>> webrevs: >>> - code changes: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.code >>>> 7 lines changed: 1 ins; 5 del; 1 mod; >>> - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.kw >>>> 18 lines changed: 18 ins; 0 del; 0 mod; >>> - full: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00 >>>> 49 lines changed: 19 ins; 5 del; 25 mod; >>> Thanks, >>> -- Igor From chris.plummer at oracle.com Fri May 1 19:04:42 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 May 2020 12:04:42 -0700 Subject: RFR(S) : 8243437 : use reproducible random in :vmTestbase_nsk_jdi In-Reply-To: <7E157D44-29B0-41B2-95B5-CA24F3F3B60C@oracle.com> References: <32CF125B-6481-48EA-873B-194537179776@oracle.com> <7E157D44-29B0-41B2-95B5-CA24F3F3B60C@oracle.com> Message-ID: Looks good. Chris On 5/1/20 9:33 AM, Igor Ignatyev wrote: > Hi Serguei, > > thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? > > -- Igor > >> On Apr 30, 2020, at 11:27 PM, serguei.spitsyn at oracle.com wrote: >> >> LGTM >> >> Thanks, >> Serguei >> >> >> On 4/30/20 16:52, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 >>>> 62 lines changed: 26 ins; 0 del; 36 mod; >>> Hi all, >>> >>> could you please review this small patch? >>> from JBS: >>>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jdi test group and marking the tests which make use of "randomness" with a proper k/w. >>> testing: : vmTestbase_nsk_jdi test group >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243437 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 >>> (all code changes have already happened in 8242314 'use reproducible random in vmTestbase shared code', so just k/w and copyright years this time) >>> >>> Thanks, >>> -- Igor From chris.plummer at oracle.com Fri May 1 19:34:25 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 May 2020 12:34:25 -0700 Subject: RFR: 8244078: ProcessTools executeTestJvm and createJavaProcessBuilder have inconsistent handling of test.*.opts In-Reply-To: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com> References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com> Message-ID: On 4/30/20 2:07 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to make it less likely that we accidentally > add or fail to add test.java.opts and test.vm.opts to our spawned test > JVMs. > > https://cr.openjdk.java.net/~stefank/8244078/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8244078 > > ProcessTools.createJavaProcessBuilder(cmd) creates a ProcessBuilder > *without* test.java.opts and test.vm.opts. There is a > (addTestVmAndJavaOptions, cmd) overload that allows the caller to > opt-in to the addition of these flags. The created ProcessBuilder is > then used to start the JVM, and almost always fed into an OutputAnalyzer. > > There's another function executeTestJvm, that both creates a > ProcessBuilder and then feeds it into an OutputAnalyzer (plus a bit > more). This function uses createJavaProcessBuilder(true, cmd), and > thereby adds the test.java.opts and test.vm.opts flags. > > This means that one has to know about this difference when reading > code using createJavaProcessBuilder(cmd) and executeTestJvm(cmd), and > when creating (copying) code using these functions. > > It has been suggested that createJavaProcessBuilder is intended to be > a lower-level, building block API and that it's not confusing that > they have different behavior. I don't really agree, but I'm buying > into the notion of lower-level vs higher-level APIs here. So, my > proposal is to remove the addTestVmAndJavaOptions feature from > createJavaProcessBuilder, and instead create a new function called > createTestJvm that adds test.java.opts and test.vm.opts. The name is > intentionally similar to executeTestJvm, and in fact, the > executeTestJvm implementation will use createTestJvm: > > ???? public static OutputAnalyzer executeTestJvm(String... cmds) > throws Exception { > - ProcessBuilder pb = createJavaProcessBuilder(true, cmds); > + ProcessBuilder pb = createTestJvm(cmds); > ???????? return executeProcess(pb); > ???? } > > The rest of the patch is mainly: > - leaving createJavaProcessBuilder(cmd) as is > - replacing createJavaProcessBuilder(false, cmd) with > createJavaProcessBuilder(cmd) > - replacing createJavaProcessBuilder(true, cmd) with createTestJvm(cmd) > > There was one odd thing in jdi that requires extra scrutiny: > https://cr.openjdk.java.net/~stefank/8244078/webrev.01/test/jdk/com/sun/jdi/lib/jdb/Debuggee.java.udiff.html > Yes, that did look a odd at first glance, but I think that fact that your removed addTestVmAndJavaOptions() and everything still built is proof enough that it was just bit rotted code and not needed. Chris > > I've run this through tier1-3, and are currently running this through > higher tiers. > > Thanks, > StefanK From chris.plummer at oracle.com Fri May 1 20:12:13 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 May 2020 13:12:13 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From dean.long at oracle.com Fri May 1 20:46:08 2020 From: dean.long at oracle.com (Dean Long) Date: Fri, 1 May 2020 13:46:08 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: References: Message-ID: <88b1fd25-46ff-8f1f-c6b0-2d5fdbc29b27@oracle.com> Does UseAOT really need libzip? dl On 5/1/20 10:24 AM, Yumin Qi wrote: > Hi, please review: > bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 > webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ > Summary: > ? zip library is loaded unconditionally at start up but it is only > needed when > ? 1) bootclasspath contains zip or > ? 2) UseAOT enabled or > ? 3) VerifySharedArchive turned on or > ? 4) CDS archives custom loaded classes > ?? If none of above in java application, it is not necessary to have > zip library loaded. > > ? Solution by loading zip library on demand when needed. > > Performance for java -Xint version: > > Results of " perf stat -r 50 bin/java -Xshare:on > -XX:SharedArchiveFile=jdk2.jsa -Xint -version " > ?? 1:???? 59611556??? 59450206 (-161350)????? -----???? 39.799 40.274 > (? 0.475)??? ++ > ?? 2:???? 59602708??? 59425234 (-177474)????? -----???? 40.591 41.183 > (? 0.592)??? ++ > ?? 3:???? 59579718??? 59441272 (-138446)????? ----????? 40.777 40.471 > ( -0.306)????? - > ?? 4:???? 59584882??? 59410155 (-174727)????? -----???? 40.824 40.233 > ( -0.591)????? -- > ?? 5:???? 59590998??? 59447252 (-143746)????? ----????? 40.400 40.493 > (? 0.093) > ?? 6:???? 59589523??? 59441934 (-147589)????? ----????? 40.475 40.064 > ( -0.411)????? -- > ?? 7:???? 59581820??? 59413612 (-168208)????? -----???? 39.763 40.077 > (? 0.314)???? + > ?? 8:???? 59593678??? 59418738 (-174940)????? -----???? 40.912 39.724 > ( -1.188)????? ----- > ?? 9:???? 59573058??? 59412554 (-160504)????? -----???? 40.126 40.033 > ( -0.093) > ? 10:???? 59591469??? 59419291 (-172178)????? -----???? 40.731 40.689 > ( -0.042) > ============================================================ > ????????? 59589940??? 59428022 (-161917)????? -----???? 40.438 40.322 > ( -0.116) > instr delta =????? -161917??? -0.2717% > time? delta =?????? -0.116 ms -0.2859% > > Tests: hs-tier1-4. > Due to zip library not loaded at default, I removed 'zip' from pmap > list in test case: *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java > > ** > * Thanks > Yumin From dean.long at oracle.com Fri May 1 21:11:08 2020 From: dean.long at oracle.com (Dean Long) Date: Fri, 1 May 2020 14:11:08 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: References: Message-ID: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> It looks like Zip_lock could be a MutexLocker instead of a MonitorLocker. dl On 5/1/20 10:24 AM, Yumin Qi wrote: > Hi, please review: > bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 > webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ > Summary: > ? zip library is loaded unconditionally at start up but it is only > needed when > ? 1) bootclasspath contains zip or > ? 2) UseAOT enabled or > ? 3) VerifySharedArchive turned on or > ? 4) CDS archives custom loaded classes > ?? If none of above in java application, it is not necessary to have > zip library loaded. > > ? Solution by loading zip library on demand when needed. > > Performance for java -Xint version: > > Results of " perf stat -r 50 bin/java -Xshare:on > -XX:SharedArchiveFile=jdk2.jsa -Xint -version " > ?? 1:???? 59611556??? 59450206 (-161350)????? -----???? 39.799 40.274 > (? 0.475)??? ++ > ?? 2:???? 59602708??? 59425234 (-177474)????? -----???? 40.591 41.183 > (? 0.592)??? ++ > ?? 3:???? 59579718??? 59441272 (-138446)????? ----????? 40.777 40.471 > ( -0.306)????? - > ?? 4:???? 59584882??? 59410155 (-174727)????? -----???? 40.824 40.233 > ( -0.591)????? -- > ?? 5:???? 59590998??? 59447252 (-143746)????? ----????? 40.400 40.493 > (? 0.093) > ?? 6:???? 59589523??? 59441934 (-147589)????? ----????? 40.475 40.064 > ( -0.411)????? -- > ?? 7:???? 59581820??? 59413612 (-168208)????? -----???? 39.763 40.077 > (? 0.314)???? + > ?? 8:???? 59593678??? 59418738 (-174940)????? -----???? 40.912 39.724 > ( -1.188)????? ----- > ?? 9:???? 59573058??? 59412554 (-160504)????? -----???? 40.126 40.033 > ( -0.093) > ? 10:???? 59591469??? 59419291 (-172178)????? -----???? 40.731 40.689 > ( -0.042) > ============================================================ > ????????? 59589940??? 59428022 (-161917)????? -----???? 40.438 40.322 > ( -0.116) > instr delta =????? -161917??? -0.2717% > time? delta =?????? -0.116 ms -0.2859% > > Tests: hs-tier1-4. > Due to zip library not loaded at default, I removed 'zip' from pmap > list in test case: *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java > > ** > * Thanks > Yumin From yumin.qi at oracle.com Fri May 1 21:42:17 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 1 May 2020 14:42:17 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> Message-ID: <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> Hi, Dean ? Thanks for the review. I will try MutextLocker, for AOT, I think currently the tests are not using CDS then it will load classes from stream, that will use libzip's Crc32. ? I will check for AOT to see if it really loads libzip with the patch. For test compiler/aot/DeoptimizationTest.java: #0? ClassLoader::load_zip_library () at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint (this=0x7ffff0245640) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., __the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream (stream=0x7ffff0245640, name=0x7ffff40550f0, loader_data=0x7ffff022dbc0, cl_info=..., __the_thread__=0x7ffff0033000) ??? at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 #5? 0x00007ffff57d53e4 in ClassLoader::load_class (name=0x7ffff40550f0, search_append_only=false, __the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class (class_name=0x7ffff40550f0, class_loader=..., __the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 #7? 0x00007ffff6232d0a in SystemDictionary::resolve_instance_class_or_null (name=0x7ffff40550f0, class_loader=..., protection_domain=..., __the_thread__=0x7ffff0033000) ??? at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 #8? 0x00007ffff623137e in SystemDictionary::resolve_instance_class_or_null_helper (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., __the_thread__=0x7ffff0033000) ??? at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., __the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., throw_error=true, __the_thread__=0x7ffff0033000) ??? at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail (class_name=0x7ffff40550f0, throw_error=true, __the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass (id=SystemDictionary::Object_klass_knum, __the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 #13 0x00007ffff623681e in SystemDictionary::resolve_wk_klasses_until (limit_id=SystemDictionary::Cloneable_klass_knum, start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, __the_thread__=0x7ffff0033000) ??? at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 #14 0x00007ffff623ac01 in SystemDictionary::resolve_wk_klasses_through (end_id=SystemDictionary::Class_klass_knum, start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, __the_thread__=0x7ffff0033000) ??? at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 #15 0x00007ffff62369b0 in SystemDictionary::resolve_well_known_classes (__the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 #16 0x00007ffff6236517 in SystemDictionary::initialize (__the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 #17 0x00007ffff62afe15 in Universe::genesis (__the_thread__=0x7ffff0033000) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 #18 0x00007ffff62b1e51 in universe2_init () at /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 #19 0x00007ffff5af21ed in init_globals () at /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, canTryAgain=0x7ffff7fc7d5b) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at pthread_create.c:333 #27 0x00007ffff790041d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 This is not with CDS. Thanks Yumin On 5/1/20 2:11 PM, Dean Long wrote: > It looks like Zip_lock could be a MutexLocker instead of a MonitorLocker. > > dl > > On 5/1/20 10:24 AM, Yumin Qi wrote: >> Hi, please review: >> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >> Summary: >> ? zip library is loaded unconditionally at start up but it is only >> needed when >> ? 1) bootclasspath contains zip or >> ? 2) UseAOT enabled or >> ? 3) VerifySharedArchive turned on or >> ? 4) CDS archives custom loaded classes >> ?? If none of above in java application, it is not necessary to have >> zip library loaded. >> >> ? Solution by loading zip library on demand when needed. >> >> Performance for java -Xint version: >> >> Results of " perf stat -r 50 bin/java -Xshare:on >> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >> ?? 1:???? 59611556??? 59450206 (-161350)????? -----???? 39.799 40.274 >> (? 0.475)??? ++ >> ?? 2:???? 59602708??? 59425234 (-177474)????? -----???? 40.591 41.183 >> (? 0.592)??? ++ >> ?? 3:???? 59579718??? 59441272 (-138446)????? ----????? 40.777 40.471 >> ( -0.306)????? - >> ?? 4:???? 59584882??? 59410155 (-174727)????? -----???? 40.824 40.233 >> ( -0.591)????? -- >> ?? 5:???? 59590998??? 59447252 (-143746)????? ----????? 40.400 40.493 >> (? 0.093) >> ?? 6:???? 59589523??? 59441934 (-147589)????? ----????? 40.475 40.064 >> ( -0.411)????? -- >> ?? 7:???? 59581820??? 59413612 (-168208)????? -----???? 39.763 40.077 >> (? 0.314)???? + >> ?? 8:???? 59593678??? 59418738 (-174940)????? -----???? 40.912 39.724 >> ( -1.188)????? ----- >> ?? 9:???? 59573058??? 59412554 (-160504)????? -----???? 40.126 40.033 >> ( -0.093) >> ? 10:???? 59591469??? 59419291 (-172178)????? -----???? 40.731 40.689 >> ( -0.042) >> ============================================================ >> ????????? 59589940??? 59428022 (-161917)????? -----???? 40.438 40.322 >> ( -0.116) >> instr delta =????? -161917??? -0.2717% >> time? delta =?????? -0.116 ms -0.2859% >> >> Tests: hs-tier1-4. >> Due to zip library not loaded at default, I removed 'zip' from pmap >> list in test case: *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >> >> ** >> * Thanks >> Yumin > From alexey.menkov at oracle.com Fri May 1 22:22:31 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 1 May 2020 15:22:31 -0700 Subject: RFR: JDK-8243012: Fix issues in j.l.i package info Message-ID: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> Hi all, Please review the fix for https://bugs.openjdk.java.net/browse/JDK-8243012 The change fixes security note in the java.lang.instrument javadoc. webrev: http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.1/ --alex From leonid.mesnik at oracle.com Fri May 1 22:42:22 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 1 May 2020 15:42:22 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> Message-ID: Hi > On May 1, 2020, at 1:12 PM, Chris Plummer wrote: > > Hi Leonid, > > Overall looks good. > > How did you verify that all settingBreakpoint() implementations were the same? How did you find that some of the breakpointForCommunication() implementations were different? Unfortunately I didn't find a good way to check that all these method are exactly the same. I just go through diff without lines common for all methods and verify that nothing unexpected was removed. Using this I found the different breakpointForCommunication() and settingBreakpoint() implementation. I found 2 variants of settingBreakpoint(). The one which set breakpointLocation (or location) and one which don't set. Additionally I grepped new files to check that lines like ' static final int PASSED' and other known patterns are removed. > > A few minor things in JDIBase: > > 108 throws JDITestRuntimeException { > > I think it would look better with the { on a new line. > > 80 // used by tests with breakpoint communication > 81 public EventRequestManager eventRManager; > 82 public EventQueue eventQueue; > 83 public EventSet eventSet; > 84 public EventIterator eventIterator; > 85 > 86 // additional fields initialized during breakpoint communication > 87 public Location breakpLocation = null; > 88 public BreakpointEvent bpEvent; > > Do these fields need to be public and not just protected? Public fields in java are not the norm. > > 91 /* > 92 * private BreakpointRequest settingBreakpoint(ThreadReference, ReferenceType, > 93 * String, String, String) > 94 * > 95 * It sets up a breakpoint at given line number within a given method in a given class > 96 * for a given thread. > 97 * > 98 * Return value: BreakpointRequest object in case of success > 99 * > 100 * JDITestRuntimeException in case of an Exception thrown within the method > 101 */ > 123 int n = > 124 ((IntegerValue) testedClass.getValue(testedClass.fieldByName(bpLine))).value(); > > I'm not sure why the comment includes the prototype, but in any case it is out of date. I suggest just removing it. > > Also, there are double-spaces in lines 98 and 100. > I could make fields protected and remove comments. But I want to review base class and might be make more refactoring. Also there are a couple of tests with strange formatting, so I am going to add them separately. I just don't want to put all this in single change. So diff contains mostly removal of same lines and it is easier to check it. Leonid > thanks, > > Chris > > On 4/29/20 4:45 PM, Leonid Mesnik wrote: >> Hi >> >> Could you please review following fix which remove code duplication by moving methods BreakpointRequest settingBreakpoint, getEventSet, breakpointForCommunication and log1,2,3 in base class. >> >> The fix is huge but pretty straight forward, I tried to keep changes as small as possible so most tests just changed to extends JDIBase and corresponding methods and fields were deleted. >> >> Some tests used slightly different 'breakpointForCommunication()' implementation so they override it now. >> >> Some tests contain change like this: >> - eventRequest1 = eventRManager.createBreakpointRequest(location); >> + eventRequest1 = eventRManager.createBreakpointRequest(breakpLocation); >> it is because they had 'location' and not 'breakpLocation' which is set by settingBreakpoint(...). So just merged location and breakpLocation. >> >> All tests passed, so the main goal is to ensure that changes don't introduce false positive results ignoring exit codes and statuses. >> >> I quickly verified that updated files don't contain declarations of variable like PASSED, testExitCode and other. >> >> The JDIBase and all tests could be improved, also there are still a lot of tests which could use it as a base as well as other base classes for idi debuggers. However I want to keep this fix as simple as possible. >> >> werbev: http://cr.openjdk.java.net/~lmesnik/8244133/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8244133 -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Fri May 1 23:05:26 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 May 2020 16:05:26 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From dean.long at oracle.com Fri May 1 23:24:35 2020 From: dean.long at oracle.com (Dean Long) Date: Fri, 1 May 2020 16:24:35 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> Message-ID: <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. dl On 5/1/20 2:42 PM, Yumin Qi wrote: > Hi, Dean > ? Thanks for the review. I will try MutextLocker, for AOT, I think > currently the tests are not using CDS then it will load classes from > stream, that will use libzip's Crc32. > ? I will check for AOT to see if it really loads libzip with the > patch. For test compiler/aot/DeoptimizationTest.java: > > #0? ClassLoader::load_zip_library () at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 > #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, > buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 > #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint > (this=0x7ffff0245640) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 > #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass > (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., > __the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 > #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream > (stream=0x7ffff0245640, name=0x7ffff40550f0, > loader_data=0x7ffff022dbc0, cl_info=..., __the_thread__=0x7ffff0033000) > ??? at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 > #5? 0x00007ffff57d53e4 in ClassLoader::load_class > (name=0x7ffff40550f0, search_append_only=false, > __the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 > #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class > (class_name=0x7ffff40550f0, class_loader=..., > __the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 > #7? 0x00007ffff6232d0a in > SystemDictionary::resolve_instance_class_or_null (name=0x7ffff40550f0, > class_loader=..., protection_domain=..., __the_thread__=0x7ffff0033000) > ??? at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 > #8? 0x00007ffff623137e in > SystemDictionary::resolve_instance_class_or_null_helper > (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., > __the_thread__=0x7ffff0033000) > ??? at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 > #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null > (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., > __the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 > #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail > (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., > throw_error=true, __the_thread__=0x7ffff0033000) > ??? at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 > #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail > (class_name=0x7ffff40550f0, throw_error=true, > __the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 > #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass > (id=SystemDictionary::Object_klass_knum, > __the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 > #13 0x00007ffff623681e in SystemDictionary::resolve_wk_klasses_until > (limit_id=SystemDictionary::Cloneable_klass_knum, > start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, > __the_thread__=0x7ffff0033000) > ??? at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 > #14 0x00007ffff623ac01 in SystemDictionary::resolve_wk_klasses_through > (end_id=SystemDictionary::Class_klass_knum, start_id=@0x7ffff7fc79d4: > SystemDictionary::Object_klass_knum, __the_thread__=0x7ffff0033000) > ??? at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 > #15 0x00007ffff62369b0 in SystemDictionary::resolve_well_known_classes > (__the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 > #16 0x00007ffff6236517 in SystemDictionary::initialize > (__the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 > #17 0x00007ffff62afe15 in Universe::genesis > (__the_thread__=0x7ffff0033000) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 > #18 0x00007ffff62b1e51 in universe2_init () at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 > #19 0x00007ffff5af21ed in init_globals () at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 > #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, > canTryAgain=0x7ffff7fc7d5b) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 > #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner (vm=0x7ffff7fc7ea8, > penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 > #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, > penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at > /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 > #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, > penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at > /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 > #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at > /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 > #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at > /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 > > #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at > pthread_create.c:333 > #27 0x00007ffff790041d in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 > > This is not with CDS. > > Thanks > Yumin > > On 5/1/20 2:11 PM, Dean Long wrote: >> It looks like Zip_lock could be a MutexLocker instead of a >> MonitorLocker. >> >> dl >> >> On 5/1/20 10:24 AM, Yumin Qi wrote: >>> Hi, please review: >>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>> Summary: >>> ? zip library is loaded unconditionally at start up but it is only >>> needed when >>> ? 1) bootclasspath contains zip or >>> ? 2) UseAOT enabled or >>> ? 3) VerifySharedArchive turned on or >>> ? 4) CDS archives custom loaded classes >>> ?? If none of above in java application, it is not necessary to have >>> zip library loaded. >>> >>> ? Solution by loading zip library on demand when needed. >>> >>> Performance for java -Xint version: >>> >>> Results of " perf stat -r 50 bin/java -Xshare:on >>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>> ?? 1:???? 59611556??? 59450206 (-161350)????? -----???? 39.799 >>> 40.274 (? 0.475)??? ++ >>> ?? 2:???? 59602708??? 59425234 (-177474)????? -----???? 40.591 >>> 41.183 (? 0.592)??? ++ >>> ?? 3:???? 59579718??? 59441272 (-138446)????? ----????? 40.777 >>> 40.471 ( -0.306)????? - >>> ?? 4:???? 59584882??? 59410155 (-174727)????? -----???? 40.824 >>> 40.233 ( -0.591)????? -- >>> ?? 5:???? 59590998??? 59447252 (-143746)????? ----????? 40.400 >>> 40.493 (? 0.093) >>> ?? 6:???? 59589523??? 59441934 (-147589)????? ----????? 40.475 >>> 40.064 ( -0.411)????? -- >>> ?? 7:???? 59581820??? 59413612 (-168208)????? -----???? 39.763 >>> 40.077 (? 0.314)???? + >>> ?? 8:???? 59593678??? 59418738 (-174940)????? -----???? 40.912 >>> 39.724 ( -1.188)????? ----- >>> ?? 9:???? 59573058??? 59412554 (-160504)????? -----???? 40.126 >>> 40.033 ( -0.093) >>> ? 10:???? 59591469??? 59419291 (-172178)????? -----???? 40.731 >>> 40.689 ( -0.042) >>> ============================================================ >>> ????????? 59589940??? 59428022 (-161917)????? -----???? 40.438 >>> 40.322 ( -0.116) >>> instr delta =????? -161917??? -0.2717% >>> time? delta =?????? -0.116 ms -0.2859% >>> >>> Tests: hs-tier1-4. >>> Due to zip library not loaded at default, I removed 'zip' from pmap >>> list in test case: >>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>> >>> ** >>> * Thanks >>> Yumin >> > From igor.ignatyev at oracle.com Fri May 1 23:26:18 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 1 May 2020 16:26:18 -0700 Subject: RFR(S) : 8243435 : use reproducible random in :vmTestbase_nsk_jvmti In-Reply-To: <90346959-fa85-7121-bcc5-688933dd069a@oracle.com> References: <90346959-fa85-7121-bcc5-688933dd069a@oracle.com> Message-ID: Thanks Chris, pushed. -- Igor > On May 1, 2020, at 11:52 AM, Chris Plummer wrote: > > Looks good. > > Chris > > On 5/1/20 9:33 AM, Igor Ignatyev wrote: >> Hi Serguei, >> >> thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? >> >> -- Igor >> >>> On Apr 30, 2020, at 11:24 PM, serguei.spitsyn at oracle.com wrote: >>> >>> This one looks good too. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 4/30/20 16:46, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev/8243435/webrev.00 >>>>> 49 lines changed: 19 ins; 5 del; 25 mod; >>>> Hi all, >>>> >>>> could you please review this patch? >>>> from JBS: >>>>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jvmti test group and marking the tests which make use of "randomness" with a proper k/w. >>>> testing: : vmTestbase_nsk_jvmti test group >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243435 >>>> webrevs: >>>> - code changes: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.code >>>>> 7 lines changed: 1 ins; 5 del; 1 mod; >>>> - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00.kw >>>>> 18 lines changed: 18 ins; 0 del; 0 mod; >>>> - full: http://cr.openjdk.java.net/~iignatyev//8243435/webrev.00 >>>>> 49 lines changed: 19 ins; 5 del; 25 mod; >>>> Thanks, >>>> -- Igor > From igor.ignatyev at oracle.com Fri May 1 23:26:16 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 1 May 2020 16:26:16 -0700 Subject: RFR(S/M) : 8243436 : use reproducible random in :vmTestbase_nsk_monitoring In-Reply-To: References: <4ed57228-efa0-593d-eedd-c06deaf41749@oracle.com> <0772099A-CBF7-468C-9FB6-4715B2DF2C9B@oracle.com> Message-ID: Thanks Chris, pushed. -- Igor > On May 1, 2020, at 11:20 AM, Chris Plummer wrote: > > Looks good. > > Chris > > On 5/1/20 9:32 AM, Igor Ignatyev wrote: >> Hi Serguei, >> >> thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? >> >> -- Igor >> >>> On Apr 30, 2020, at 11:22 PM, serguei.spitsyn at oracle.com wrote: >>> >>> Hi Igor, >>> >>> It looks good. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 4/30/20 16:30, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev/8243436/webrev.00 >>>>> 305 lines changed: 76 ins; 11 del; 218 mod; >>>> Hi all, >>>> >>>> could you please review this patch? >>>> from JBS: >>>>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_monitoring test group and marking the tests which make use of "randomness" with a proper k/w. >>>> testing: : vmTestbase_nsk_monitoring test group >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243436 >>>> webrevs: >>>> - code changes: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.code >>>>> 19 lines changed: 5 ins; 11 del; 3 mod; >>>> - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00.kw >>>>> 141 lines changed: 71 ins; 0 del; 70 mod; >>>> - full: http://cr.openjdk.java.net/~iignatyev//8243436/webrev.00 >>>>> 305 lines changed: 76 ins; 11 del; 218 mod; >>>> Thanks, >>>> -- Igor > From igor.ignatyev at oracle.com Fri May 1 23:26:20 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 1 May 2020 16:26:20 -0700 Subject: RFR(S) : 8243437 : use reproducible random in :vmTestbase_nsk_jdi In-Reply-To: References: <32CF125B-6481-48EA-873B-194537179776@oracle.com> <7E157D44-29B0-41B2-95B5-CA24F3F3B60C@oracle.com> Message-ID: <9E133053-82E0-471E-8A12-9B5CD7CF6200@oracle.com> Thanks Chris, pushed. -- Igor > On May 1, 2020, at 12:04 PM, Chris Plummer wrote: > > Looks good. > > Chris > > On 5/1/20 9:33 AM, Igor Ignatyev wrote: >> Hi Serguei, >> >> thanks for the review. can I get 2nd review or ack. that this is trivial-ish and fine to be pushed w/ one reviewer? >> >> -- Igor >> >>> On Apr 30, 2020, at 11:27 PM, serguei.spitsyn at oracle.com wrote: >>> >>> LGTM >>> >>> Thanks, >>> Serguei >>> >>> >>> On 4/30/20 16:52, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 >>>>> 62 lines changed: 26 ins; 0 del; 36 mod; >>>> Hi all, >>>> >>>> could you please review this small patch? >>>> from JBS: >>>>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in : vmTestbase_nsk_jdi test group and marking the tests which make use of "randomness" with a proper k/w. >>>> testing: : vmTestbase_nsk_jdi test group >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243437 >>>> webrev: http://cr.openjdk.java.net/~iignatyev/8243437/webrev.00 >>>> (all code changes have already happened in 8242314 'use reproducible random in vmTestbase shared code', so just k/w and copyright years this time) >>>> >>>> Thanks, >>>> -- Igor > From leonid.mesnik at oracle.com Fri May 1 23:51:13 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 1 May 2020 16:51:13 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> Message-ID: <580F3B60-33D3-4AD1-B8F7-AE9DF30A1427@oracle.com> Yes, the protected should be enough. I've made all non constant fields and methods protected and remove comment. The updated webrev (JDIBase only) http://cr.openjdk.java.net/~lmesnik/8244133/webrev.01/ Leonid > On May 1, 2020, at 4:05 PM, Chris Plummer wrote: > > On 5/1/20 3:42 PM, Leonid Mesnik wrote: >> Hi >> >>> On May 1, 2020, at 1:12 PM, Chris Plummer > wrote: >>> >>> Hi Leonid, >>> >>> Overall looks good. >>> >>> How did you verify that all settingBreakpoint() implementations were the same? How did you find that some of the breakpointForCommunication() implementations were different? >> >> >> Unfortunately I didn't find a good way to check that all these method are exactly the same. I just go through diff without lines common for all methods and verify that nothing unexpected was removed. >> Using this I found the different breakpointForCommunication() and settingBreakpoint() implementation. I found 2 variants of settingBreakpoint(). The one which set breakpointLocation (or location) and one which don't set. >> >> >> Additionally I grepped new files to check that lines like ' static final int PASSED' and other known patterns are removed. > Ok. >>> >>> A few minor things in JDIBase: >>> >>> 108 throws JDITestRuntimeException { >>> >>> I think it would look better with the { on a new line. >>> >>> 80 // used by tests with breakpoint communication >>> 81 public EventRequestManager eventRManager; >>> 82 public EventQueue eventQueue; >>> 83 public EventSet eventSet; >>> 84 public EventIterator eventIterator; >>> 85 >>> 86 // additional fields initialized during breakpoint communication >>> 87 public Location breakpLocation = null; >>> 88 public BreakpointEvent bpEvent; >>> >>> Do these fields need to be public and not just protected? Public fields in java are not the norm. >>> >>> 91 /* >>> 92 * private BreakpointRequest settingBreakpoint(ThreadReference, ReferenceType, >>> 93 * String, String, String) >>> 94 * >>> 95 * It sets up a breakpoint at given line number within a given method in a given class >>> 96 * for a given thread. >>> 97 * >>> 98 * Return value: BreakpointRequest object in case of success >>> 99 * >>> 100 * JDITestRuntimeException in case of an Exception thrown within the method >>> 101 */ >>> 123 int n = >>> 124 ((IntegerValue) testedClass.getValue(testedClass.fieldByName(bpLine))).value(); >>> >>> I'm not sure why the comment includes the prototype, but in any case it is out of date. I suggest just removing it. >>> >>> Also, there are double-spaces in lines 98 and 100. >>> >> I could make fields protected and remove comments. But I want to review base class and might be make more refactoring. Also there are a couple of tests with strange formatting, so I am going to add them separately. >> I just don't want to put all this in single change. So diff contains mostly removal of same lines and it is easier to check it. > But I'm just referring to the one copy in JDIBase.java. It seems getting the format and commenting of this entirely new class is something to do now, not at some later point when you cleanup other tests. > > Also, the fields were not public before being moved to JDIBase, they were package private. So unless there is a reason for them to be public, I'm not sure why you would not be more restrictive as long as it does not impact the tests. Do the tests need for the fields to be public or else would need to be modified? > > thanks, > > Chris >> >> Leonid >>> thanks, >>> >>> Chris >>> >>> On 4/29/20 4:45 PM, Leonid Mesnik wrote: >>>> Hi >>>> >>>> Could you please review following fix which remove code duplication by moving methods BreakpointRequest settingBreakpoint, getEventSet, breakpointForCommunication and log1,2,3 in base class. >>>> >>>> The fix is huge but pretty straight forward, I tried to keep changes as small as possible so most tests just changed to extends JDIBase and corresponding methods and fields were deleted. >>>> >>>> Some tests used slightly different 'breakpointForCommunication()' implementation so they override it now. >>>> >>>> Some tests contain change like this: >>>> - eventRequest1 = eventRManager.createBreakpointRequest(location); >>>> + eventRequest1 = eventRManager.createBreakpointRequest(breakpLocation); >>>> it is because they had 'location' and not 'breakpLocation' which is set by settingBreakpoint(...). So just merged location and breakpLocation. >>>> >>>> All tests passed, so the main goal is to ensure that changes don't introduce false positive results ignoring exit codes and statuses. >>>> >>>> I quickly verified that updated files don't contain declarations of variable like PASSED, testExitCode and other. >>>> >>>> The JDIBase and all tests could be improved, also there are still a lot of tests which could use it as a base as well as other base classes for idi debuggers. However I want to keep this fix as simple as possible. >>>> >>>> werbev: http://cr.openjdk.java.net/~lmesnik/8244133/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244133 >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Sat May 2 02:45:31 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 1 May 2020 19:45:31 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: <580F3B60-33D3-4AD1-B8F7-AE9DF30A1427@oracle.com> References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> <580F3B60-33D3-4AD1-B8F7-AE9DF30A1427@oracle.com> Message-ID: <1761865a-cdfe-9071-fe28-d65e40f4a38c@oracle.com> An HTML attachment was scrubbed... URL: From yumin.qi at oracle.com Sat May 2 03:00:04 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 1 May 2020 20:00:04 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> Message-ID: <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> Dean, ? I have updated to use MutexLocker instead at same link: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ ? Tested locally, passed jtreg/runtime. Thanks Yumin On 5/1/20 4:24 PM, Dean Long wrote: > OK, I didn't realize compute_fingerprint is using ZIP_CRC32. > > dl > > On 5/1/20 2:42 PM, Yumin Qi wrote: >> Hi, Dean >> ? Thanks for the review. I will try MutextLocker, for AOT, I think >> currently the tests are not using CDS then it will load classes from >> stream, that will use libzip's Crc32. >> ? I will check for AOT to see if it really loads libzip with the >> patch. For test compiler/aot/DeoptimizationTest.java: >> >> #0? ClassLoader::load_zip_library () at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >> (this=0x7ffff0245640) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >> __the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >> (stream=0x7ffff0245640, name=0x7ffff40550f0, >> loader_data=0x7ffff022dbc0, cl_info=..., __the_thread__=0x7ffff0033000) >> ??? at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >> (name=0x7ffff40550f0, search_append_only=false, >> __the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >> (class_name=0x7ffff40550f0, class_loader=..., >> __the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >> #7? 0x00007ffff6232d0a in >> SystemDictionary::resolve_instance_class_or_null >> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >> __the_thread__=0x7ffff0033000) >> ??? at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >> #8? 0x00007ffff623137e in >> SystemDictionary::resolve_instance_class_or_null_helper >> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >> __the_thread__=0x7ffff0033000) >> ??? at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >> __the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >> throw_error=true, __the_thread__=0x7ffff0033000) >> ??? at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >> (class_name=0x7ffff40550f0, throw_error=true, >> __the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >> (id=SystemDictionary::Object_klass_knum, >> __the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >> #13 0x00007ffff623681e in SystemDictionary::resolve_wk_klasses_until >> (limit_id=SystemDictionary::Cloneable_klass_knum, >> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >> __the_thread__=0x7ffff0033000) >> ??? at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >> #14 0x00007ffff623ac01 in >> SystemDictionary::resolve_wk_klasses_through >> (end_id=SystemDictionary::Class_klass_knum, start_id=@0x7ffff7fc79d4: >> SystemDictionary::Object_klass_knum, __the_thread__=0x7ffff0033000) >> ??? at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >> #15 0x00007ffff62369b0 in >> SystemDictionary::resolve_well_known_classes >> (__the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >> #16 0x00007ffff6236517 in SystemDictionary::initialize >> (__the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >> #17 0x00007ffff62afe15 in Universe::genesis >> (__the_thread__=0x7ffff0033000) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >> #18 0x00007ffff62b1e51 in universe2_init () at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >> #19 0x00007ffff5af21ed in init_globals () at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >> canTryAgain=0x7ffff7fc7d5b) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner (vm=0x7ffff7fc7ea8, >> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >> >> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >> pthread_create.c:333 >> #27 0x00007ffff790041d in clone () at >> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >> >> This is not with CDS. >> >> Thanks >> Yumin >> >> On 5/1/20 2:11 PM, Dean Long wrote: >>> It looks like Zip_lock could be a MutexLocker instead of a >>> MonitorLocker. >>> >>> dl >>> >>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>> Hi, please review: >>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>> Summary: >>>> ? zip library is loaded unconditionally at start up but it is only >>>> needed when >>>> ? 1) bootclasspath contains zip or >>>> ? 2) UseAOT enabled or >>>> ? 3) VerifySharedArchive turned on or >>>> ? 4) CDS archives custom loaded classes >>>> ?? If none of above in java application, it is not necessary to >>>> have zip library loaded. >>>> >>>> ? Solution by loading zip library on demand when needed. >>>> >>>> Performance for java -Xint version: >>>> >>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 40.274 >>>> (? 0.475)??? ++ >>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 41.183 >>>> (? 0.592)??? ++ >>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 ( >>>> -0.306)????? - >>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 40.233 ( >>>> -0.591)????? -- >>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 (? >>>> 0.093) >>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 ( >>>> -0.411)????? -- >>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 40.077 >>>> (? 0.314)???? + >>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 39.724 ( >>>> -1.188)????? ----- >>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 40.033 ( >>>> -0.093) >>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 40.689 ( >>>> -0.042) >>>> ============================================================ >>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 40.322 ( >>>> -0.116) >>>> instr delta =????? -161917??? -0.2717% >>>> time? delta =?????? -0.116 ms -0.2859% >>>> >>>> Tests: hs-tier1-4. >>>> Due to zip library not loaded at default, I removed 'zip' from pmap >>>> list in test case: >>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>> >>>> ** >>>> * Thanks >>>> Yumin >>> >> > From dean.long at oracle.com Sat May 2 07:05:22 2020 From: dean.long at oracle.com (Dean Long) Date: Sat, 2 May 2020 00:05:22 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> Message-ID: <40de653e-6586-f300-4dba-fe1526abcf03@oracle.com> In ClassLoader::crc32, instead of checking if the library is loaded every time, how about initializing the Crc32 function pointer to a function that calls load_zip_library()? Something like: static jint initial_Crc32(jint crc, const jbyte *buf, jint len) { ??? load_zip_library(); ??? assert(Crc32 != initial_Crc32, "load_zip_library did not update Crc32"); ??? return crc32(crc, buf, len); // ZIP_CRC32 } static Crc32_t?????????? Crc32????????????? = initial_Crc32; I also suggest putting ClassLoader::crc32 in classLoader.hpp so it can be inlined. dl On 5/1/20 8:00 PM, Yumin Qi wrote: > Dean, > > ? I have updated to use MutexLocker instead at same link: > http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ > > ? Tested locally, passed jtreg/runtime. > > Thanks > Yumin > > On 5/1/20 4:24 PM, Dean Long wrote: >> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >> >> dl >> >> On 5/1/20 2:42 PM, Yumin Qi wrote: >>> Hi, Dean >>> ? Thanks for the review. I will try MutextLocker, for AOT, I think >>> currently the tests are not using CDS then it will load classes from >>> stream, that will use libzip's Crc32. >>> ? I will check for AOT to see if it really loads libzip with the >>> patch. For test compiler/aot/DeoptimizationTest.java: >>> >>> #0? ClassLoader::load_zip_library () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>> >>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>> (this=0x7ffff0245640) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>> loader_data=0x7ffff022dbc0, cl_info=..., __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>> (name=0x7ffff40550f0, search_append_only=false, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>> (class_name=0x7ffff40550f0, class_loader=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>> #7? 0x00007ffff6232d0a in >>> SystemDictionary::resolve_instance_class_or_null >>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>> #8? 0x00007ffff623137e in >>> SystemDictionary::resolve_instance_class_or_null_helper >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> throw_error=true, __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>> (class_name=0x7ffff40550f0, throw_error=true, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>> (id=SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>> #13 0x00007ffff623681e in SystemDictionary::resolve_wk_klasses_until >>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>> #14 0x00007ffff623ac01 in >>> SystemDictionary::resolve_wk_klasses_through >>> (end_id=SystemDictionary::Class_klass_knum, >>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>> #15 0x00007ffff62369b0 in >>> SystemDictionary::resolve_well_known_classes >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>> #17 0x00007ffff62afe15 in Universe::genesis >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>> #18 0x00007ffff62b1e51 in universe2_init () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>> #19 0x00007ffff5af21ed in init_globals () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >>> canTryAgain=0x7ffff7fc7d5b) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner (vm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>> >>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>> pthread_create.c:333 >>> #27 0x00007ffff790041d in clone () at >>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>> >>> This is not with CDS. >>> >>> Thanks >>> Yumin >>> >>> On 5/1/20 2:11 PM, Dean Long wrote: >>>> It looks like Zip_lock could be a MutexLocker instead of a >>>> MonitorLocker. >>>> >>>> dl >>>> >>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>> Hi, please review: >>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>> Summary: >>>>> ? zip library is loaded unconditionally at start up but it is only >>>>> needed when >>>>> ? 1) bootclasspath contains zip or >>>>> ? 2) UseAOT enabled or >>>>> ? 3) VerifySharedArchive turned on or >>>>> ? 4) CDS archives custom loaded classes >>>>> ?? If none of above in java application, it is not necessary to >>>>> have zip library loaded. >>>>> >>>>> ? Solution by loading zip library on demand when needed. >>>>> >>>>> Performance for java -Xint version: >>>>> >>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 40.274 >>>>> (? 0.475)??? ++ >>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 41.183 >>>>> (? 0.592)??? ++ >>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 ( >>>>> -0.306)????? - >>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 40.233 >>>>> ( -0.591)????? -- >>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 >>>>> (? 0.093) >>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 ( >>>>> -0.411)????? -- >>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 40.077 >>>>> (? 0.314)???? + >>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 39.724 >>>>> ( -1.188)????? ----- >>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 40.033 >>>>> ( -0.093) >>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 40.689 >>>>> ( -0.042) >>>>> ============================================================ >>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 40.322 >>>>> ( -0.116) >>>>> instr delta =????? -161917??? -0.2717% >>>>> time? delta =?????? -0.116 ms -0.2859% >>>>> >>>>> Tests: hs-tier1-4. >>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>> pmap list in test case: >>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>> >>>>> ** >>>>> * Thanks >>>>> Yumin >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sat May 2 14:58:31 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 2 May 2020 15:58:31 +0100 Subject: 8244284: Two tests in test/hotspot/jtreg/vmTestbase fail with --illegal-access=deny Message-ID: <36a8c736-e3b3-0662-5eac-83ea82e2aa46@oracle.com> I need a reviewer for an tiny update to two tests in test/hotspot/jtreg/vmTestbase/ndk/jdi. Both tests need the non-exported package com.sun.tools.jdi to be opened because they instantiate ObjectReferenceImpl via its non-public constructor. This change is needed in advance of denying illegal access by default. -Alan diff --git a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java --- a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2018, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -47,6 +47,7 @@ ? * @run driver jdk.test.lib.FileInstaller . . ? * @build nsk.jdi.ClassType.invokeMethod.invokemethod009 ? *??????? nsk.jdi.ClassType.invokeMethod.invokemethod009t + * @modules jdk.jdi/com.sun.tools.jdi:open ? * @run main/othervm PropertyResolvingWrapper ? *????? nsk.jdi.ClassType.invokeMethod.invokemethod009 ? *????? -verbose diff --git a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java --- a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2018, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -48,6 +48,7 @@ ? * @clean nsk.jdi.ObjectReference.invokeMethod.invokemethod006t ? * @compile -g:lines,source,vars ../invokemethod006t.java ? * + * @modules jdk.jdi/com.sun.tools.jdi:open ? * @run main/othervm PropertyResolvingWrapper ? *????? nsk.jdi.ObjectReference.invokeMethod.invokemethod006 ? *????? -verbose From igor.ignatyev at oracle.com Sat May 2 15:39:24 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sat, 2 May 2020 08:39:24 -0700 Subject: 8244284: Two tests in test/hotspot/jtreg/vmTestbase fail with --illegal-access=deny In-Reply-To: <36a8c736-e3b3-0662-5eac-83ea82e2aa46@oracle.com> References: <36a8c736-e3b3-0662-5eac-83ea82e2aa46@oracle.com> Message-ID: <64D6621F-DE32-4950-B0FB-62B3B09AA3CB@oracle.com> Hi Alan, it looks good, but could you please put @modules tags right before @library? we are trying to follow the recommend order ( http://hg.openjdk.java.net/code-tools/jtreg/raw-file/e235ed803708/src/share/doc/javatest/regtest/tag-spec.html#ORDER ) -- Igor > On May 2, 2020, at 7:58 AM, Alan Bateman wrote: > > I need a reviewer for an tiny update to two tests in test/hotspot/jtreg/vmTestbase/ndk/jdi. Both tests need the non-exported package com.sun.tools.jdi to be opened because they instantiate ObjectReferenceImpl via its non-public constructor. This change is needed in advance of denying illegal access by default. > > -Alan > > > diff --git a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java > --- a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java > +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ClassType/invokeMethod/invokemethod009/TestDescription.java > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2018, 2020, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -47,6 +47,7 @@ > * @run driver jdk.test.lib.FileInstaller . . > * @build nsk.jdi.ClassType.invokeMethod.invokemethod009 > * nsk.jdi.ClassType.invokeMethod.invokemethod009t > + * @modules jdk.jdi/com.sun.tools.jdi:open > * @run main/othervm PropertyResolvingWrapper > * nsk.jdi.ClassType.invokeMethod.invokemethod009 > * -verbose > diff --git a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java > --- a/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java > +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdi/ObjectReference/invokeMethod/invokemethod006/TestDescription.java > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2018, 2020, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -48,6 +48,7 @@ > * @clean nsk.jdi.ObjectReference.invokeMethod.invokemethod006t > * @compile -g:lines,source,vars ../invokemethod006t.java > * > + * @modules jdk.jdi/com.sun.tools.jdi:open > * @run main/othervm PropertyResolvingWrapper > * nsk.jdi.ObjectReference.invokeMethod.invokemethod006 > * -verbose -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Sat May 2 15:46:04 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 2 May 2020 16:46:04 +0100 Subject: 8244284: Two tests in test/hotspot/jtreg/vmTestbase fail with --illegal-access=deny In-Reply-To: <64D6621F-DE32-4950-B0FB-62B3B09AA3CB@oracle.com> References: <36a8c736-e3b3-0662-5eac-83ea82e2aa46@oracle.com> <64D6621F-DE32-4950-B0FB-62B3B09AA3CB@oracle.com> Message-ID: <35ffbd79-19c5-ec14-c390-3aba67fb4567@oracle.com> On 02/05/2020 16:39, Igor Ignatyev wrote: > Hi Alan, > > it looks good, but could you please put @modules tags right before > @library? Okay but the reason I moved it down here is because there's an @run driver in the sequence of actions and the @modules that we are adding here is only interesting for the @run main action. -Alan From igor.ignatyev at oracle.com Sat May 2 15:49:54 2020 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Sat, 2 May 2020 08:49:54 -0700 Subject: 8244284: Two tests in test/hotspot/jtreg/vmTestbase fail with --illegal-access=deny In-Reply-To: <35ffbd79-19c5-ec14-c390-3aba67fb4567@oracle.com> References: <35ffbd79-19c5-ec14-c390-3aba67fb4567@oracle.com> Message-ID: B/c @modules affect all actions, such location can confuse people into thinking that it affects only the next one (or all the following). ? Igor > On May 2, 2020, at 8:46 AM, Alan Bateman wrote: > > ?On 02/05/2020 16:39, Igor Ignatyev wrote: >> Hi Alan, >> >> it looks good, but could you please put @modules tags right before @library? > Okay but the reason I moved it down here is because there's an @run driver in the sequence of actions and the @modules that we are adding here is only interesting for the @run main action. > > -Alan From ioi.lam at oracle.com Mon May 4 05:01:46 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 3 May 2020 22:01:46 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> Message-ID: <2dd3da73-7e60-3317-8fc8-2ac92d65f0b0@oracle.com> Hi Dean, The code uses double-checked locking so we can be thread-safe when loading the zip library, while avoiding using the MutexLock every time: https://en.wikipedia.org/wiki/Double-checked_locking To work correctly on modern memory architectures, we need the memory barriers: ?977 void ClassLoader::load_zip_library() { ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { ?979???? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); ?980???? if (libzip_loaded == 0) { ?981?????? load_zip_library0(); ?982?????? Atomic::release_store(&libzip_loaded, 1); ?983???? } ?984?? } ?985 } (Atomic::load_acquire is much cheaper than MutexLocker). If we read Crc32 without memory barriers, as in the original code, in two concurrent threads, I think there may be a chance for one of the threads to read an invalid value of Crc32. int ClassLoader::crc32(int crc, const char* buf, int len) { ? return (*Crc32)(crc, (const jbyte*)buf, len); } Thanks - Ioi On 5/1/20 8:00 PM, Yumin Qi wrote: > Dean, > > ? I have updated to use MutexLocker instead at same link: > http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ > > ? Tested locally, passed jtreg/runtime. > > Thanks > Yumin > > On 5/1/20 4:24 PM, Dean Long wrote: >> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >> >> dl >> >> On 5/1/20 2:42 PM, Yumin Qi wrote: >>> Hi, Dean >>> ? Thanks for the review. I will try MutextLocker, for AOT, I think >>> currently the tests are not using CDS then it will load classes from >>> stream, that will use libzip's Crc32. >>> ? I will check for AOT to see if it really loads libzip with the >>> patch. For test compiler/aot/DeoptimizationTest.java: >>> >>> #0? ClassLoader::load_zip_library () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>> >>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>> (this=0x7ffff0245640) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>> loader_data=0x7ffff022dbc0, cl_info=..., __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>> (name=0x7ffff40550f0, search_append_only=false, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>> (class_name=0x7ffff40550f0, class_loader=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>> #7? 0x00007ffff6232d0a in >>> SystemDictionary::resolve_instance_class_or_null >>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>> #8? 0x00007ffff623137e in >>> SystemDictionary::resolve_instance_class_or_null_helper >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> throw_error=true, __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>> (class_name=0x7ffff40550f0, throw_error=true, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>> (id=SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>> #13 0x00007ffff623681e in SystemDictionary::resolve_wk_klasses_until >>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>> #14 0x00007ffff623ac01 in >>> SystemDictionary::resolve_wk_klasses_through >>> (end_id=SystemDictionary::Class_klass_knum, >>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>> #15 0x00007ffff62369b0 in >>> SystemDictionary::resolve_well_known_classes >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>> #17 0x00007ffff62afe15 in Universe::genesis >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>> #18 0x00007ffff62b1e51 in universe2_init () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>> #19 0x00007ffff5af21ed in init_globals () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >>> canTryAgain=0x7ffff7fc7d5b) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner (vm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>> >>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>> pthread_create.c:333 >>> #27 0x00007ffff790041d in clone () at >>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>> >>> This is not with CDS. >>> >>> Thanks >>> Yumin >>> >>> On 5/1/20 2:11 PM, Dean Long wrote: >>>> It looks like Zip_lock could be a MutexLocker instead of a >>>> MonitorLocker. >>>> >>>> dl >>>> >>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>> Hi, please review: >>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>> Summary: >>>>> ? zip library is loaded unconditionally at start up but it is only >>>>> needed when >>>>> ? 1) bootclasspath contains zip or >>>>> ? 2) UseAOT enabled or >>>>> ? 3) VerifySharedArchive turned on or >>>>> ? 4) CDS archives custom loaded classes >>>>> ?? If none of above in java application, it is not necessary to >>>>> have zip library loaded. >>>>> >>>>> ? Solution by loading zip library on demand when needed. >>>>> >>>>> Performance for java -Xint version: >>>>> >>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 40.274 >>>>> (? 0.475)??? ++ >>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 41.183 >>>>> (? 0.592)??? ++ >>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 ( >>>>> -0.306)????? - >>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 40.233 >>>>> ( -0.591)????? -- >>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 >>>>> (? 0.093) >>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 ( >>>>> -0.411)????? -- >>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 40.077 >>>>> (? 0.314)???? + >>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 39.724 >>>>> ( -1.188)????? ----- >>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 40.033 >>>>> ( -0.093) >>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 40.689 >>>>> ( -0.042) >>>>> ============================================================ >>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 40.322 >>>>> ( -0.116) >>>>> instr delta =????? -161917??? -0.2717% >>>>> time? delta =?????? -0.116 ms -0.2859% >>>>> >>>>> Tests: hs-tier1-4. >>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>> pmap list in test case: >>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>> >>>>> ** >>>>> * Thanks >>>>> Yumin >>>> >>> >> > From ioi.lam at oracle.com Mon May 4 05:02:58 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 3 May 2020 22:02:58 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <40de653e-6586-f300-4dba-fe1526abcf03@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <40de653e-6586-f300-4dba-fe1526abcf03@oracle.com> Message-ID: <57f339dd-d7b0-044d-0976-42e967cf2b2e@oracle.com> Hi Dean, The code uses double-checked locking so we can be thread-safe when loading the zip library, while avoiding using the MutexLock every time: https://en.wikipedia.org/wiki/Double-checked_locking To work correctly on modern memory architectures, we need the memory barriers: ?977 void ClassLoader::load_zip_library() { ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { ?979???? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); ?980???? if (libzip_loaded == 0) { ?981?????? load_zip_library0(); ?982?????? Atomic::release_store(&libzip_loaded, 1); ?983???? } ?984?? } ?985 } (Atomic::load_acquire is much cheaper than MutexLocker). If we read Crc32 without memory barriers, as in the original code, in two concurrent threads, I think there may be a chance for one of the threads to read an invalid value of Crc32. int ClassLoader::crc32(int crc, const char* buf, int len) { ? return (*Crc32)(crc, (const jbyte*)buf, len); } Thanks - Ioi On 5/2/20 12:05 AM, Dean Long wrote: > In ClassLoader::crc32, instead of checking if the library is loaded > every time, how about initializing the Crc32 function pointer to a > function that calls load_zip_library()? > Something like: > > static jint initial_Crc32(jint crc, const jbyte *buf, jint len) { > ??? load_zip_library(); > ??? assert(Crc32 != initial_Crc32, "load_zip_library did not update > Crc32"); > ??? return crc32(crc, buf, len); // ZIP_CRC32 > } > > static Crc32_t?????????? Crc32????????????? = initial_Crc32; > > I also suggest putting > ClassLoader::crc32 in classLoader.hpp so it can be inlined. > > dl > > On 5/1/20 8:00 PM, Yumin Qi wrote: >> Dean, >> >> ? I have updated to use MutexLocker instead at same link: >> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >> >> ? Tested locally, passed jtreg/runtime. >> >> Thanks >> Yumin >> >> On 5/1/20 4:24 PM, Dean Long wrote: >>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>> >>> dl >>> >>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>> Hi, Dean >>>> ? Thanks for the review. I will try MutextLocker, for AOT, I think >>>> currently the tests are not using CDS then it will load classes >>>> from stream, that will use libzip's Crc32. >>>> ? I will check for AOT to see if it really loads libzip with the >>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>> >>>> #0? ClassLoader::load_zip_library () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>> >>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>> (this=0x7ffff0245640) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>> (name=0x7ffff40550f0, search_append_only=false, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>> #7? 0x00007ffff6232d0a in >>>> SystemDictionary::resolve_instance_class_or_null >>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>> #8? 0x00007ffff623137e in >>>> SystemDictionary::resolve_instance_class_or_null_helper >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., throw_error=true, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>> (class_name=0x7ffff40550f0, throw_error=true, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>> (id=SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>> #13 0x00007ffff623681e in >>>> SystemDictionary::resolve_wk_klasses_until >>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>> #14 0x00007ffff623ac01 in >>>> SystemDictionary::resolve_wk_klasses_through >>>> (end_id=SystemDictionary::Class_klass_knum, >>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>> #15 0x00007ffff62369b0 in >>>> SystemDictionary::resolve_well_known_classes >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>> #17 0x00007ffff62afe15 in Universe::genesis >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>> #19 0x00007ffff5af21ed in init_globals () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >>>> canTryAgain=0x7ffff7fc7d5b) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>> >>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>> pthread_create.c:333 >>>> #27 0x00007ffff790041d in clone () at >>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>> >>>> This is not with CDS. >>>> >>>> Thanks >>>> Yumin >>>> >>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>> MonitorLocker. >>>>> >>>>> dl >>>>> >>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>> Hi, please review: >>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>> Summary: >>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>> only needed when >>>>>> ? 1) bootclasspath contains zip or >>>>>> ? 2) UseAOT enabled or >>>>>> ? 3) VerifySharedArchive turned on or >>>>>> ? 4) CDS archives custom loaded classes >>>>>> ?? If none of above in java application, it is not necessary to >>>>>> have zip library loaded. >>>>>> >>>>>> ? Solution by loading zip library on demand when needed. >>>>>> >>>>>> Performance for java -Xint version: >>>>>> >>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 40.274 >>>>>> (? 0.475)??? ++ >>>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 41.183 >>>>>> (? 0.592)??? ++ >>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 >>>>>> ( -0.306)????? - >>>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 40.233 >>>>>> ( -0.591)????? -- >>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 >>>>>> (? 0.093) >>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 >>>>>> ( -0.411)????? -- >>>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 40.077 >>>>>> (? 0.314)???? + >>>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 39.724 >>>>>> ( -1.188)????? ----- >>>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 40.033 >>>>>> ( -0.093) >>>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 40.689 >>>>>> ( -0.042) >>>>>> ============================================================ >>>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 40.322 >>>>>> ( -0.116) >>>>>> instr delta =????? -161917??? -0.2717% >>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>> >>>>>> Tests: hs-tier1-4. >>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>> pmap list in test case: >>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>> >>>>>> ** >>>>>> * Thanks >>>>>> Yumin >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikael.vidstedt at oracle.com Mon May 4 05:12:25 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Sun, 3 May 2020 22:12:25 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) Message-ID: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> Please review this change which implements part of JEP 381: JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! Background: Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. Testing: A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. Cheers, Mikael [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ From mikael.vidstedt at oracle.com Mon May 4 05:12:20 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Sun, 3 May 2020 22:12:20 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) Message-ID: Please review this change which implements part of JEP 381: JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! Background: Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. Testing: A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. Cheers, Mikael [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ From stefan.karlsson at oracle.com Mon May 4 06:30:19 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 4 May 2020 08:30:19 +0200 Subject: RFR: 8244078: ProcessTools executeTestJvm and createJavaProcessBuilder have inconsistent handling of test.*.opts In-Reply-To: References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com> Message-ID: <13c1f541-ac07-c925-26ee-574d5bba65df@oracle.com> On 2020-05-01 21:34, Chris Plummer wrote: > On 4/30/20 2:07 AM, Stefan Karlsson wrote: ... >> There was one odd thing in jdi that requires extra scrutiny: >> https://cr.openjdk.java.net/~stefank/8244078/webrev.01/test/jdk/com/sun/jdi/lib/jdb/Debuggee.java.udiff.html >> > Yes, that did look a odd at first glance, but I think that fact that > your removed addTestVmAndJavaOptions() and everything still built is > proof enough that it was just bit rotted code and not needed. Thanks for checking on this. StefanK > > Chris >> >> I've run this through tier1-3, and are currently running this through >> higher tiers. >> >> Thanks, >> StefanK > > From serguei.spitsyn at oracle.com Mon May 4 07:20:45 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 4 May 2020 00:20:45 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> Message-ID: <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> HI Mikael, Some quick comments. Some extra references to Solaris/solaris, SunOS or SPARC are listed below: src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. src/hotspot/share/prims/jvmti.xml:??? On the Solaris Operating Environment, an agent library is a shared src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating src/hotspot/share/prims/jvmti.xml:????? example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris src/hotspot/share/prims/methodHandles.cpp:#undef CS? // Solaris builds complain Thanks, Serguei On 5/3/20 22:12, Mikael Vidstedt wrote: > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From stefan.karlsson at oracle.com Mon May 4 08:28:27 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 4 May 2020 10:28:27 +0200 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: Hi Mikael, On 2020-05-04 07:12, Mikael Vidstedt wrote: > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ I went over this patch and collected some comments: src/hotspot/share/adlc/output_c.cpp src/hotspot/share/adlc/output_h.cpp Awkward code layout after change to. src/hotspot/share/c1/c1_Runtime1.cpp src/hotspot/share/classfile/classListParser.cpp src/hotspot/share/memory/arena.hpp src/hotspot/share/opto/chaitin.cpp test/hotspot/jtreg/gc/TestCardTablePageCommits.java Surrounding comments still refers to Sparc and/or Solaris. There are even more places if you search in the rest of the HotSpot source. Are we leaving those for a separate cleanup pass? src/hotspot/share/gc/g1/g1HeapRegionAttr.hpp Remove comment: // We use different types to represent the state value depending on platform as // some have issues loading parts of words. src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp Fuse the declaration and definition, now that we only have one implementation. Maybe even remove function/file at some point. src/hotspot/share/utilities/globalDefinitions.hpp Now that STACK_BIAS is always 0, should we remove its usages? Follow-up RFE? src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/meta/HotSpotGraphBuilderPlugins.java Maybe remove decryptSuffix? src/utils/hsdis/Makefile Is this really correct? Shouldn't: ARCH1=$(CPU:x86_64=amd64) ARCH2=$(ARCH1:i686=i386) ARCH=$(ARCH2:sparc64=sparcv9) be changed to: ARCH1=$(CPU:x86_64=amd64) ARCH=$(ARCH1:i686=i386) so that we have ARCH defined? Other than that this looks good to me. StefanK > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! > > Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From kim.barrett at oracle.com Mon May 4 08:54:25 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 4 May 2020 04:54:25 -0400 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: > On May 4, 2020, at 4:28 AM, Stefan Karlsson wrote: > > Hi Mikael, > > On 2020-05-04 07:12, Mikael Vidstedt wrote: >> Please review this change which implements part of JEP 381: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ > > [?] > > src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp > > Fuse the declaration and definition, now that we only have one implementation. Maybe even remove function/file at some point. I think that cleanup is covered by JDK-8142349. From thomas.schatzl at oracle.com Mon May 4 09:11:56 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 4 May 2020 11:11:56 +0200 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: Hi, On 04.05.20 10:28, Stefan Karlsson wrote: > Hi Mikael, > > On 2020-05-04 07:12, Mikael Vidstedt wrote: >> >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >> > > I went over this patch and collected some comments: > > src/hotspot/share/adlc/output_c.cpp > src/hotspot/share/adlc/output_h.cpp > > Awkward code layout after change to. > > > src/hotspot/share/c1/c1_Runtime1.cpp > src/hotspot/share/classfile/classListParser.cpp > src/hotspot/share/memory/arena.hpp > src/hotspot/share/opto/chaitin.cpp > test/hotspot/jtreg/gc/TestCardTablePageCommits.jav > > Surrounding comments still refers to Sparc and/or Solaris. > > There are even more places if you search in the rest of the HotSpot > source. Are we leaving those for a separate cleanup pass? In addition to "sparc", "solaris", also "solstudio"/"Sun Studio"/"SS compiler bug"/"niagara" yield some search (=grep) results. Some of these locations look like additional RFEs. Thanks, Thomas From richard.reingruber at sap.com Mon May 4 10:33:08 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 4 May 2020 10:33:08 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> Message-ID: // Trimmed the list of recipients. If the list gets too long then the message needs to be approved // by a moderator. Hi David, > On 28/04/2020 12:09 am, Reingruber, Richard wrote: > > Hi David, > > > >> Not a review but some general commentary ... > > > > That's welcome. > Having had to take an even closer look now I have a review comment too :) > src/hotspot/share/prims/jvmtiThreadState.cpp > void JvmtiThreadState::invalidate_cur_stack_depth() { > ! assert(SafepointSynchronize::is_at_safepoint() || > ! (Thread::current()->is_VM_thread() && > get_thread()->is_vmthread_processing_handshake()) || > (JavaThread *)Thread::current() == get_thread(), > "must be current thread or at safepoint"); You're looking at an outdated webrev, I'm afraid. This would be the post with the current webrev.1 http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html Thanks, Richard. -----Original Message----- From: David Holmes Sent: Montag, 4. Mai 2020 08:51 To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant Hi Richard, On 28/04/2020 12:09 am, Reingruber, Richard wrote: > Hi David, > >> Not a review but some general commentary ... > > That's welcome. Having had to take an even closer look now I have a review comment too :) src/hotspot/share/prims/jvmtiThreadState.cpp void JvmtiThreadState::invalidate_cur_stack_depth() { ! assert(SafepointSynchronize::is_at_safepoint() || ! (Thread::current()->is_VM_thread() && get_thread()->is_vmthread_processing_handshake()) || (JavaThread *)Thread::current() == get_thread(), "must be current thread or at safepoint"); The message needs updating to include handshakes. More below ... >> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>> Hi Yasumasa, Patricio, >>> >>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>> >>>> Thanks for your information. >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>> I will modify and will test it after yours. >>> >>> Thanks :) >>> >>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>> to me, how this has to be handled. >>> >>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>> >>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>> also I'm unsure if a thread should do safepoint checks while executing a handshake. > >> I'm growing increasingly concerned that use of direct handshakes to >> replace VM operations needs a much greater examination for correctness >> than might initially be thought. I see a number of issues: > > I agree. I'll address your concerns in the context of this review thread for JDK-8238585 below. > > In addition I would suggest to take the general part of the discussion to a dedicated thread or to > the review thread for JDK-8242427. I would like to keep this thread closer to its subject. I will focus on the issues in the context of this particular change then, though the issues themselves are applicable to all handshake situations (and more so with direct handshakes). This is mostly just discussion. >> First, the VMThread executes (most) VM operations with a clean stack in >> a clean state, so it has lots of room to work. If we now execute the >> same logic in a JavaThread then we risk hitting stackoverflows if >> nothing else. But we are also now executing code in a JavaThread and so >> we have to be sure that code is not going to act differently (in a bad >> way) if executed by a JavaThread rather than the VMThread. For example, >> may it be possible that if executing in the VMThread we defer some >> activity that might require execution of Java code, or else hand it off >> to one of the service threads? If we execute that code directly in the >> current JavaThread instead we may not be in a valid state (e.g. consider >> re-entrancy to various subsystems that is not allowed). > > It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is doing. I already added a > paragraph to the JBS-Item [1] explaining why the direct handshake is sufficient from a > synchronization point of view. Just to be clear, your proposed change is not using a direct handshake. > Furthermore the stack is walked and the return pc of compiled frames is replaced with the address of > the deopt handler. > > I can't see why this cannot be done with a direct handshake. Something very similar is already done > in JavaThread::deoptimize_marked_methods() which is executed as part of an ordinary handshake. Note that existing non-direct handshakes may also have issues that not have been fully investigated. > The demand on stack-space should be very modest. I would not expect a higher risk for stackoverflow. For the target thread if you use more stack than would be used stopping at a safepoint then you are at risk. For the thread initiating the direct handshake if you use more stack than would be used enqueuing a VM operation, then you are at risk. As we have not quantified these numbers, nor have any easy way to establish the stack use of the actual code to be executed, we're really just hoping for the best. This is a general problem with handshakes that needs to be investigated more deeply. As a simple, general, example just imagine if the code involves logging that might utilise an on-stack buffer. >> Second, we have this question mark over what happens if the operation >> hits further safepoint or handshake polls/checks? Are there constraints >> on what is allowed here? How can we recognise this problem may exist and >> so deal with it? > > The thread in EnterInterpOnlyModeClosure::do_thread() can't become safepoint/handshake safe. I > tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a NoSafepointVerifier. That's good to hear but such tests are not exhaustive, they will detect if you do reach a safepoint/handshake but they can't prove that you cannot reach one. What you have done is necessary but may not be sufficient. Plus you didn't actually add the NSV to the code - is there a reason we can't actually keep it in do_thread? (I'm not sure if the NSV also acts as a NoHandshakeVerifier?) >> Third, while we are generally considering what appear to be >> single-thread operations, which should be amenable to a direct >> handshake, we also have to be careful that some of the code involved >> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >> not need to take a lock where a direct handshake might! > > See again my arguments in the JBS item [1]. Yes I see the reasoning and that is good. My point is a general one as it may not be obvious when such assumptions exist in the current code. Thanks, David > Thanks, > Richard. > > [1] https://bugs.openjdk.java.net/browse/JDK-8238585 > > -----Original Message----- > From: David Holmes > Sent: Montag, 27. April 2020 07:16 > To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi all, > > Not a review but some general commentary ... > > On 25/04/2020 2:08 am, Reingruber, Richard wrote: >> Hi Yasumasa, Patricio, >> >>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>> Does it help you? I think it gives you to remove workaround. >>>> >>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >> >>> Thanks for your information. >>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>> I will modify and will test it after yours. >> >> Thanks :) >> >>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>> to me, how this has to be handled. >> >>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >> >> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >> also I'm unsure if a thread should do safepoint checks while executing a handshake. > > I'm growing increasingly concerned that use of direct handshakes to > replace VM operations needs a much greater examination for correctness > than might initially be thought. I see a number of issues: > > First, the VMThread executes (most) VM operations with a clean stack in > a clean state, so it has lots of room to work. If we now execute the > same logic in a JavaThread then we risk hitting stackoverflows if > nothing else. But we are also now executing code in a JavaThread and so > we have to be sure that code is not going to act differently (in a bad > way) if executed by a JavaThread rather than the VMThread. For example, > may it be possible that if executing in the VMThread we defer some > activity that might require execution of Java code, or else hand it off > to one of the service threads? If we execute that code directly in the > current JavaThread instead we may not be in a valid state (e.g. consider > re-entrancy to various subsystems that is not allowed). > > Second, we have this question mark over what happens if the operation > hits further safepoint or handshake polls/checks? Are there constraints > on what is allowed here? How can we recognise this problem may exist and > so deal with it? > > Third, while we are generally considering what appear to be > single-thread operations, which should be amenable to a direct > handshake, we also have to be careful that some of the code involved > doesn't already expect/assume we are at a safepoint - e.g. a VM op may > not need to take a lock where a direct handshake might! > > Cheers, > David > ----- > >> @Patricio, coming back to my question [1]: >> >> In the example you gave in your answer [2]: the java thread would execute a vm operation during a >> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads >> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The >> handshakee would be safepoint safe, wouldn't it? >> >> Thanks, Richard. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677 >> >> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763 >> >> -----Original Message----- >> From: Yasumasa Suenaga >> Sent: Freitag, 24. April 2020 17:23 >> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi Richard, >> >> On 2020/04/24 23:44, Reingruber, Richard wrote: >>> Hi Yasumasa, >>> >>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>> Does it help you? I think it gives you to remove workaround. >>> >>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >> >> Thanks for your information. >> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >> I will modify and will test it after yours. >> >> >>> Also my first impression was that it won't be that easy from a synchronization point of view to >>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>> to me, how this has to be handled. >> >> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >> >> >> Thanks, >> >> Yasumasa >> >> >>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585). >>> >>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>> >>> Would be interesting to see how you handled the issues above :) >>> >>> Thanks, Richard. >>> >>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030 >>> >>> -----Original Message----- >>> From: Yasumasa Suenaga >>> Sent: Freitag, 24. April 2020 13:34 >>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>> >>> Hi Richard, >>> >>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>> Does it help you? I think it gives you to remove workaround. >>> >>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/04/24 17:18, Reingruber, Richard wrote: >>>> Hi Patricio, Vladimir, and Serguei, >>>> >>>> now that direct handshakes are available, I've updated the patch to make use of them. >>>> >>>> In addition I have done some clean-up changes I missed in the first webrev. >>>> >>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake >>>> into the vm operation VM_SetFramePop [1] >>>> >>>> Kindly review again: >>>> >>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ >>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ >>>> >>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a >>>> direct handshake: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>> >>>> Testing: >>>> >>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>> >>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 >>>> >>>> Thanks, >>>> Richard. >>>> >>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. >>>> >>>> -----Original Message----- >>>> From: hotspot-dev On Behalf Of Reingruber, Richard >>>> Sent: Freitag, 14. Februar 2020 19:47 >>>> To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Patricio, >>>> >>>> > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>> > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>> > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>> > > >>>> > > > Alternatively I think you could do something similar to what we do in >>>> > > > Deoptimization::deoptimize_all_marked(): >>>> > > > >>>> > > > EnterInterpOnlyModeClosure hs; >>>> > > > if (SafepointSynchronize::is_at_safepoint()) { >>>> > > > hs.do_thread(state->get_thread()); >>>> > > > } else { >>>> > > > Handshake::execute(&hs, state->get_thread()); >>>> > > > } >>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>> > > > HandshakeClosure() constructor) >>>> > > >>>> > > Maybe this could be used also in the Handshake::execute() methods as general solution? >>>> > Right, we could also do that. Avoiding to clear the polling page in >>>> > HandshakeState::clear_handshake() should be enough to fix this issue and >>>> > execute a handshake inside a safepoint, but adding that "if" statement >>>> > in Hanshake::execute() sounds good to avoid all the extra code that we >>>> > go through when executing a handshake. I filed 8239084 to make that change. >>>> >>>> Thanks for taking care of this and creating the RFE. >>>> >>>> > >>>> > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>> > > > always called in a nested operation or just sometimes. >>>> > > >>>> > > At least one execution path without vm operation exists: >>>> > > >>>> > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>> > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void >>>> > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>> > > >>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>> > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>> > > encouraged to do it with a handshake :) >>>> > Ah! I think you can still do it with a handshake with the >>>> > Deoptimization::deoptimize_all_marked() like solution. I can change the >>>> > if-else statement with just the Handshake::execute() call in 8239084. >>>> > But up to you. : ) >>>> >>>> Well, I think that's enough encouragement :) >>>> I'll wait for 8239084 and try then again. >>>> (no urgency and all) >>>> >>>> Thanks, >>>> Richard. >>>> >>>> -----Original Message----- >>>> From: Patricio Chilano >>>> Sent: Freitag, 14. Februar 2020 15:54 >>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Richard, >>>> >>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: >>>>> Hi Patricio, >>>>> >>>>> thanks for having a look. >>>>> >>>>> > I?m only commenting on the handshake changes. >>>>> > I see that operation VM_EnterInterpOnlyMode can be called inside >>>>> > operation VM_SetFramePop which also allows nested operations. Here is a >>>>> > comment in VM_SetFramePop definition: >>>>> > >>>>> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>> > >>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>> > could have a handshake inside a safepoint operation. The issue I see >>>>> > there is that at the end of the handshake the polling page of the target >>>>> > thread could be disarmed. So if the target thread happens to be in a >>>>> > blocked state just transiently and wakes up then it will not stop for >>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>> > polling page is armed at the beginning of disarm_safepoint(). >>>>> >>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>> >>>>> > Alternatively I think you could do something similar to what we do in >>>>> > Deoptimization::deoptimize_all_marked(): >>>>> > >>>>> > EnterInterpOnlyModeClosure hs; >>>>> > if (SafepointSynchronize::is_at_safepoint()) { >>>>> > hs.do_thread(state->get_thread()); >>>>> > } else { >>>>> > Handshake::execute(&hs, state->get_thread()); >>>>> > } >>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>> > HandshakeClosure() constructor) >>>>> >>>>> Maybe this could be used also in the Handshake::execute() methods as general solution? >>>> Right, we could also do that. Avoiding to clear the polling page in >>>> HandshakeState::clear_handshake() should be enough to fix this issue and >>>> execute a handshake inside a safepoint, but adding that "if" statement >>>> in Hanshake::execute() sounds good to avoid all the extra code that we >>>> go through when executing a handshake. I filed 8239084 to make that change. >>>> >>>>> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>> > always called in a nested operation or just sometimes. >>>>> >>>>> At least one execution path without vm operation exists: >>>>> >>>>> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>> JvmtiEventControllerPrivate::recompute_enabled() : void >>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>> >>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>> encouraged to do it with a handshake :) >>>> Ah! I think you can still do it with a handshake with the >>>> Deoptimization::deoptimize_all_marked() like solution. I can change the >>>> if-else statement with just the Handshake::execute() call in 8239084. >>>> But up to you.? : ) >>>> >>>> Thanks, >>>> Patricio >>>>> Thanks again, >>>>> Richard. >>>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Donnerstag, 13. Februar 2020 18:47 >>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Richard, >>>>> >>>>> I?m only commenting on the handshake changes. >>>>> I see that operation VM_EnterInterpOnlyMode can be called inside >>>>> operation VM_SetFramePop which also allows nested operations. Here is a >>>>> comment in VM_SetFramePop definition: >>>>> >>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>> >>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>> could have a handshake inside a safepoint operation. The issue I see >>>>> there is that at the end of the handshake the polling page of the target >>>>> thread could be disarmed. So if the target thread happens to be in a >>>>> blocked state just transiently and wakes up then it will not stop for >>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>> polling page is armed at the beginning of disarm_safepoint(). >>>>> >>>>> I think one option could be to remove >>>>> SafepointMechanism::disarm_if_needed() in >>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself >>>>> for the handshake case. >>>>> >>>>> Alternatively I think you could do something similar to what we do in >>>>> Deoptimization::deoptimize_all_marked(): >>>>> >>>>> ? EnterInterpOnlyModeClosure hs; >>>>> ? if (SafepointSynchronize::is_at_safepoint()) { >>>>> ??? hs.do_thread(state->get_thread()); >>>>> ? } else { >>>>> ??? Handshake::execute(&hs, state->get_thread()); >>>>> ? } >>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>> HandshakeClosure() constructor) >>>>> >>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>> always called in a nested operation or just sometimes. >>>>> >>>>> Thanks, >>>>> Patricio >>>>> >>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>>>>> // Repost including hotspot runtime and gc lists. >>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>>>>> // with a handshake. >>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>>>>> >>>>>> Hi, >>>>>> >>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>> >>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>>>>> >>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>>>>> >>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>> >>>>>> Thanks, Richard. >>>>>> >>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html >>>> From kim.barrett at oracle.com Mon May 4 10:47:11 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 4 May 2020 06:47:11 -0400 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: <553D0344-188E-455B-A03E-D080C1484B41@oracle.com> > On May 4, 2020, at 1:12 AM, Mikael Vidstedt wrote: > > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 I've only looked at the src/hotspot changes so far. I've not duplicated comments already made by Stefan. Looks good, other than a few very minor issues, some of which might already be covered by planned followup RFEs. ------------------------------------------------------------------------------ I think with sparc removal, c1's pack64/unpack64 stuff is no longer used. So I think that can be removed from c1_LIR.[ch]pp too. ------------------------------------------------------------------------------ src/hotspot/share/opto/generateOptoStub.cpp 225 // Clear last_Java_pc and (optionally)_flags The sparc-specific clearing of "flags" is gone. ------------------------------------------------------------------------------ src/hotspot/share/runtime/deoptimization.cpp 1086 *((jlong *) check_alignment_get_addr(obj, index, 8)) = (jlong) *((jlong *) &val); [pre-existing] The rhs cast to jlong is unnecessary, since it's dereferencing a jlong*. ------------------------------------------------------------------------------ src/hotspot/share/runtime/flags/jvmFlagConstraintsCompiler.cpp 236 JVMFlag::Error CompilerThreadPriorityConstraintFunc(intx value, bool verbose) { 237 return JVMFlag::SUCCESS; 238 } After SOLARIS code removal we no longer need this constraint function. ------------------------------------------------------------------------------ src/hotspot/share/runtime/globals.hpp 2392 experimental(size_t, ArrayAllocatorMallocLimit, \ 2393 (size_t)-1, \ Combine these lines. ------------------------------------------------------------------------------ src/hotspot/share/utilities/dtrace.hpp Shuold just eliminate all traces of HS_DTRACE_WORKAROUND_TAIL_CALL_BUG. ------------------------------------------------------------------------------ From david.holmes at oracle.com Mon May 4 06:50:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 May 2020 16:50:49 +1000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> Message-ID: <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> Hi Richard, On 28/04/2020 12:09 am, Reingruber, Richard wrote: > Hi David, > >> Not a review but some general commentary ... > > That's welcome. Having had to take an even closer look now I have a review comment too :) src/hotspot/share/prims/jvmtiThreadState.cpp void JvmtiThreadState::invalidate_cur_stack_depth() { ! assert(SafepointSynchronize::is_at_safepoint() || ! (Thread::current()->is_VM_thread() && get_thread()->is_vmthread_processing_handshake()) || (JavaThread *)Thread::current() == get_thread(), "must be current thread or at safepoint"); The message needs updating to include handshakes. More below ... >> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>> Hi Yasumasa, Patricio, >>> >>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>> >>>> Thanks for your information. >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>> I will modify and will test it after yours. >>> >>> Thanks :) >>> >>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>> to me, how this has to be handled. >>> >>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>> >>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>> also I'm unsure if a thread should do safepoint checks while executing a handshake. > >> I'm growing increasingly concerned that use of direct handshakes to >> replace VM operations needs a much greater examination for correctness >> than might initially be thought. I see a number of issues: > > I agree. I'll address your concerns in the context of this review thread for JDK-8238585 below. > > In addition I would suggest to take the general part of the discussion to a dedicated thread or to > the review thread for JDK-8242427. I would like to keep this thread closer to its subject. I will focus on the issues in the context of this particular change then, though the issues themselves are applicable to all handshake situations (and more so with direct handshakes). This is mostly just discussion. >> First, the VMThread executes (most) VM operations with a clean stack in >> a clean state, so it has lots of room to work. If we now execute the >> same logic in a JavaThread then we risk hitting stackoverflows if >> nothing else. But we are also now executing code in a JavaThread and so >> we have to be sure that code is not going to act differently (in a bad >> way) if executed by a JavaThread rather than the VMThread. For example, >> may it be possible that if executing in the VMThread we defer some >> activity that might require execution of Java code, or else hand it off >> to one of the service threads? If we execute that code directly in the >> current JavaThread instead we may not be in a valid state (e.g. consider >> re-entrancy to various subsystems that is not allowed). > > It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is doing. I already added a > paragraph to the JBS-Item [1] explaining why the direct handshake is sufficient from a > synchronization point of view. Just to be clear, your proposed change is not using a direct handshake. > Furthermore the stack is walked and the return pc of compiled frames is replaced with the address of > the deopt handler. > > I can't see why this cannot be done with a direct handshake. Something very similar is already done > in JavaThread::deoptimize_marked_methods() which is executed as part of an ordinary handshake. Note that existing non-direct handshakes may also have issues that not have been fully investigated. > The demand on stack-space should be very modest. I would not expect a higher risk for stackoverflow. For the target thread if you use more stack than would be used stopping at a safepoint then you are at risk. For the thread initiating the direct handshake if you use more stack than would be used enqueuing a VM operation, then you are at risk. As we have not quantified these numbers, nor have any easy way to establish the stack use of the actual code to be executed, we're really just hoping for the best. This is a general problem with handshakes that needs to be investigated more deeply. As a simple, general, example just imagine if the code involves logging that might utilise an on-stack buffer. >> Second, we have this question mark over what happens if the operation >> hits further safepoint or handshake polls/checks? Are there constraints >> on what is allowed here? How can we recognise this problem may exist and >> so deal with it? > > The thread in EnterInterpOnlyModeClosure::do_thread() can't become safepoint/handshake safe. I > tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a NoSafepointVerifier. That's good to hear but such tests are not exhaustive, they will detect if you do reach a safepoint/handshake but they can't prove that you cannot reach one. What you have done is necessary but may not be sufficient. Plus you didn't actually add the NSV to the code - is there a reason we can't actually keep it in do_thread? (I'm not sure if the NSV also acts as a NoHandshakeVerifier?) >> Third, while we are generally considering what appear to be >> single-thread operations, which should be amenable to a direct >> handshake, we also have to be careful that some of the code involved >> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >> not need to take a lock where a direct handshake might! > > See again my arguments in the JBS item [1]. Yes I see the reasoning and that is good. My point is a general one as it may not be obvious when such assumptions exist in the current code. Thanks, David > Thanks, > Richard. > > [1] https://bugs.openjdk.java.net/browse/JDK-8238585 > > -----Original Message----- > From: David Holmes > Sent: Montag, 27. April 2020 07:16 > To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi all, > > Not a review but some general commentary ... > > On 25/04/2020 2:08 am, Reingruber, Richard wrote: >> Hi Yasumasa, Patricio, >> >>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>> Does it help you? I think it gives you to remove workaround. >>>> >>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >> >>> Thanks for your information. >>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>> I will modify and will test it after yours. >> >> Thanks :) >> >>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>> to me, how this has to be handled. >> >>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >> >> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >> also I'm unsure if a thread should do safepoint checks while executing a handshake. > > I'm growing increasingly concerned that use of direct handshakes to > replace VM operations needs a much greater examination for correctness > than might initially be thought. I see a number of issues: > > First, the VMThread executes (most) VM operations with a clean stack in > a clean state, so it has lots of room to work. If we now execute the > same logic in a JavaThread then we risk hitting stackoverflows if > nothing else. But we are also now executing code in a JavaThread and so > we have to be sure that code is not going to act differently (in a bad > way) if executed by a JavaThread rather than the VMThread. For example, > may it be possible that if executing in the VMThread we defer some > activity that might require execution of Java code, or else hand it off > to one of the service threads? If we execute that code directly in the > current JavaThread instead we may not be in a valid state (e.g. consider > re-entrancy to various subsystems that is not allowed). > > Second, we have this question mark over what happens if the operation > hits further safepoint or handshake polls/checks? Are there constraints > on what is allowed here? How can we recognise this problem may exist and > so deal with it? > > Third, while we are generally considering what appear to be > single-thread operations, which should be amenable to a direct > handshake, we also have to be careful that some of the code involved > doesn't already expect/assume we are at a safepoint - e.g. a VM op may > not need to take a lock where a direct handshake might! > > Cheers, > David > ----- > >> @Patricio, coming back to my question [1]: >> >> In the example you gave in your answer [2]: the java thread would execute a vm operation during a >> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads >> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The >> handshakee would be safepoint safe, wouldn't it? >> >> Thanks, Richard. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677 >> >> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763 >> >> -----Original Message----- >> From: Yasumasa Suenaga >> Sent: Freitag, 24. April 2020 17:23 >> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi Richard, >> >> On 2020/04/24 23:44, Reingruber, Richard wrote: >>> Hi Yasumasa, >>> >>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>> Does it help you? I think it gives you to remove workaround. >>> >>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >> >> Thanks for your information. >> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >> I will modify and will test it after yours. >> >> >>> Also my first impression was that it won't be that easy from a synchronization point of view to >>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>> to me, how this has to be handled. >> >> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >> >> >> Thanks, >> >> Yasumasa >> >> >>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585). >>> >>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>> >>> Would be interesting to see how you handled the issues above :) >>> >>> Thanks, Richard. >>> >>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030 >>> >>> -----Original Message----- >>> From: Yasumasa Suenaga >>> Sent: Freitag, 24. April 2020 13:34 >>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>> >>> Hi Richard, >>> >>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>> Does it help you? I think it gives you to remove workaround. >>> >>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/04/24 17:18, Reingruber, Richard wrote: >>>> Hi Patricio, Vladimir, and Serguei, >>>> >>>> now that direct handshakes are available, I've updated the patch to make use of them. >>>> >>>> In addition I have done some clean-up changes I missed in the first webrev. >>>> >>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake >>>> into the vm operation VM_SetFramePop [1] >>>> >>>> Kindly review again: >>>> >>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ >>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ >>>> >>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a >>>> direct handshake: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>> >>>> Testing: >>>> >>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>> >>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 >>>> >>>> Thanks, >>>> Richard. >>>> >>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. >>>> >>>> -----Original Message----- >>>> From: hotspot-dev On Behalf Of Reingruber, Richard >>>> Sent: Freitag, 14. Februar 2020 19:47 >>>> To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Patricio, >>>> >>>> > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>> > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>> > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>> > > >>>> > > > Alternatively I think you could do something similar to what we do in >>>> > > > Deoptimization::deoptimize_all_marked(): >>>> > > > >>>> > > > EnterInterpOnlyModeClosure hs; >>>> > > > if (SafepointSynchronize::is_at_safepoint()) { >>>> > > > hs.do_thread(state->get_thread()); >>>> > > > } else { >>>> > > > Handshake::execute(&hs, state->get_thread()); >>>> > > > } >>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>> > > > HandshakeClosure() constructor) >>>> > > >>>> > > Maybe this could be used also in the Handshake::execute() methods as general solution? >>>> > Right, we could also do that. Avoiding to clear the polling page in >>>> > HandshakeState::clear_handshake() should be enough to fix this issue and >>>> > execute a handshake inside a safepoint, but adding that "if" statement >>>> > in Hanshake::execute() sounds good to avoid all the extra code that we >>>> > go through when executing a handshake. I filed 8239084 to make that change. >>>> >>>> Thanks for taking care of this and creating the RFE. >>>> >>>> > >>>> > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>> > > > always called in a nested operation or just sometimes. >>>> > > >>>> > > At least one execution path without vm operation exists: >>>> > > >>>> > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>> > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void >>>> > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>> > > >>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>> > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>> > > encouraged to do it with a handshake :) >>>> > Ah! I think you can still do it with a handshake with the >>>> > Deoptimization::deoptimize_all_marked() like solution. I can change the >>>> > if-else statement with just the Handshake::execute() call in 8239084. >>>> > But up to you. : ) >>>> >>>> Well, I think that's enough encouragement :) >>>> I'll wait for 8239084 and try then again. >>>> (no urgency and all) >>>> >>>> Thanks, >>>> Richard. >>>> >>>> -----Original Message----- >>>> From: Patricio Chilano >>>> Sent: Freitag, 14. Februar 2020 15:54 >>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Richard, >>>> >>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: >>>>> Hi Patricio, >>>>> >>>>> thanks for having a look. >>>>> >>>>> > I?m only commenting on the handshake changes. >>>>> > I see that operation VM_EnterInterpOnlyMode can be called inside >>>>> > operation VM_SetFramePop which also allows nested operations. Here is a >>>>> > comment in VM_SetFramePop definition: >>>>> > >>>>> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>> > >>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>> > could have a handshake inside a safepoint operation. The issue I see >>>>> > there is that at the end of the handshake the polling page of the target >>>>> > thread could be disarmed. So if the target thread happens to be in a >>>>> > blocked state just transiently and wakes up then it will not stop for >>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>> > polling page is armed at the beginning of disarm_safepoint(). >>>>> >>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>> >>>>> > Alternatively I think you could do something similar to what we do in >>>>> > Deoptimization::deoptimize_all_marked(): >>>>> > >>>>> > EnterInterpOnlyModeClosure hs; >>>>> > if (SafepointSynchronize::is_at_safepoint()) { >>>>> > hs.do_thread(state->get_thread()); >>>>> > } else { >>>>> > Handshake::execute(&hs, state->get_thread()); >>>>> > } >>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>> > HandshakeClosure() constructor) >>>>> >>>>> Maybe this could be used also in the Handshake::execute() methods as general solution? >>>> Right, we could also do that. Avoiding to clear the polling page in >>>> HandshakeState::clear_handshake() should be enough to fix this issue and >>>> execute a handshake inside a safepoint, but adding that "if" statement >>>> in Hanshake::execute() sounds good to avoid all the extra code that we >>>> go through when executing a handshake. I filed 8239084 to make that change. >>>> >>>>> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>> > always called in a nested operation or just sometimes. >>>>> >>>>> At least one execution path without vm operation exists: >>>>> >>>>> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>> JvmtiEventControllerPrivate::recompute_enabled() : void >>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>> >>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>> encouraged to do it with a handshake :) >>>> Ah! I think you can still do it with a handshake with the >>>> Deoptimization::deoptimize_all_marked() like solution. I can change the >>>> if-else statement with just the Handshake::execute() call in 8239084. >>>> But up to you.? : ) >>>> >>>> Thanks, >>>> Patricio >>>>> Thanks again, >>>>> Richard. >>>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Donnerstag, 13. Februar 2020 18:47 >>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Richard, >>>>> >>>>> I?m only commenting on the handshake changes. >>>>> I see that operation VM_EnterInterpOnlyMode can be called inside >>>>> operation VM_SetFramePop which also allows nested operations. Here is a >>>>> comment in VM_SetFramePop definition: >>>>> >>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>> >>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>> could have a handshake inside a safepoint operation. The issue I see >>>>> there is that at the end of the handshake the polling page of the target >>>>> thread could be disarmed. So if the target thread happens to be in a >>>>> blocked state just transiently and wakes up then it will not stop for >>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>> polling page is armed at the beginning of disarm_safepoint(). >>>>> >>>>> I think one option could be to remove >>>>> SafepointMechanism::disarm_if_needed() in >>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself >>>>> for the handshake case. >>>>> >>>>> Alternatively I think you could do something similar to what we do in >>>>> Deoptimization::deoptimize_all_marked(): >>>>> >>>>> ? EnterInterpOnlyModeClosure hs; >>>>> ? if (SafepointSynchronize::is_at_safepoint()) { >>>>> ??? hs.do_thread(state->get_thread()); >>>>> ? } else { >>>>> ??? Handshake::execute(&hs, state->get_thread()); >>>>> ? } >>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>> HandshakeClosure() constructor) >>>>> >>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>> always called in a nested operation or just sometimes. >>>>> >>>>> Thanks, >>>>> Patricio >>>>> >>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>>>>> // Repost including hotspot runtime and gc lists. >>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>>>>> // with a handshake. >>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>>>>> >>>>>> Hi, >>>>>> >>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>> >>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>>>>> >>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>>>>> >>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>> >>>>>> Thanks, Richard. >>>>>> >>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html >>>> From ioi.lam at oracle.com Mon May 4 15:31:02 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 4 May 2020 08:31:02 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> Message-ID: <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> Hi Yumin, ?977 void ClassLoader::load_zip_library() { ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { ?979???? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); ?980???? if (libzip_loaded == 0) { ?981?????? load_zip_library0(); ?982?????? Atomic::release_store(&libzip_loaded, 1); ?983???? } ?984?? } ?985 } 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { 1027?? if (libzip_loaded == 0) { 1028???? load_zip_library(); 1029?? } 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); 1031 } ClassLoader::crc32 access libzip_loaded without a memory barrier, so it may read libzip_loaded out-of-order from Crc32. This means, thread 1 may be writing the memory in this order: ??? Crc32 = ; ??? libzip_loaded = 1; but the order observed in thread 2 may be ??? libzip_loaded = 1; ???? >> ClassLoader::crc32 called here in thread 2 ??? Crc32 = ; as a result, thread 2 may read libzip_loaded = 1, but still reads a NULL from Crc32. To ensure the memory barrier is used, I think we should do this: // private inline void ClassLoader::load_zip_library_if_needed() { ? if (Atomic::load_acquire(&libzip_loaded) == 0) { ??? release_load_zip_library(); ? } } // private void ClassLoader::release_load_zip_library() { ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); ? if (libzip_loaded == 0) { ??? load_zip_library0(); ??? Atomic::release_store(&libzip_loaded, 1); ? } } int ClassLoader::crc32(int crc, const char* buf, int len) { ? load_zip_library_if_needed(); ? return (*Crc32)(crc, (const jbyte*)buf, len); } HotSpot code uses the "release_" prefix to indicate that the function must be used as a part of an acquire/release memory barrier. (E.g., InstanceKlass::release_set_array_klasses()). Some backgrounds: https://preshing.com/20130922/acquire-and-release-fences/ Thanks - Ioi On 5/1/20 8:00 PM, Yumin Qi wrote: > Dean, > > ? I have updated to use MutexLocker instead at same link: > http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ > > ? Tested locally, passed jtreg/runtime. > > Thanks > Yumin > > On 5/1/20 4:24 PM, Dean Long wrote: >> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >> >> dl >> >> On 5/1/20 2:42 PM, Yumin Qi wrote: >>> Hi, Dean >>> ? Thanks for the review. I will try MutextLocker, for AOT, I think >>> currently the tests are not using CDS then it will load classes from >>> stream, that will use libzip's Crc32. >>> ? I will check for AOT to see if it really loads libzip with the >>> patch. For test compiler/aot/DeoptimizationTest.java: >>> >>> #0? ClassLoader::load_zip_library () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>> >>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>> (this=0x7ffff0245640) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>> loader_data=0x7ffff022dbc0, cl_info=..., __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>> (name=0x7ffff40550f0, search_append_only=false, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>> (class_name=0x7ffff40550f0, class_loader=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>> #7? 0x00007ffff6232d0a in >>> SystemDictionary::resolve_instance_class_or_null >>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>> #8? 0x00007ffff623137e in >>> SystemDictionary::resolve_instance_class_or_null_helper >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>> (class_name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>> throw_error=true, __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>> (class_name=0x7ffff40550f0, throw_error=true, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>> (id=SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>> #13 0x00007ffff623681e in SystemDictionary::resolve_wk_klasses_until >>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>> #14 0x00007ffff623ac01 in >>> SystemDictionary::resolve_wk_klasses_through >>> (end_id=SystemDictionary::Class_klass_knum, >>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>> __the_thread__=0x7ffff0033000) >>> ??? at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>> #15 0x00007ffff62369b0 in >>> SystemDictionary::resolve_well_known_classes >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>> #17 0x00007ffff62afe15 in Universe::genesis >>> (__the_thread__=0x7ffff0033000) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>> #18 0x00007ffff62b1e51 in universe2_init () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>> #19 0x00007ffff5af21ed in init_globals () at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >>> canTryAgain=0x7ffff7fc7d5b) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner (vm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>> >>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>> pthread_create.c:333 >>> #27 0x00007ffff790041d in clone () at >>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>> >>> This is not with CDS. >>> >>> Thanks >>> Yumin >>> >>> On 5/1/20 2:11 PM, Dean Long wrote: >>>> It looks like Zip_lock could be a MutexLocker instead of a >>>> MonitorLocker. >>>> >>>> dl >>>> >>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>> Hi, please review: >>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>> Summary: >>>>> ? zip library is loaded unconditionally at start up but it is only >>>>> needed when >>>>> ? 1) bootclasspath contains zip or >>>>> ? 2) UseAOT enabled or >>>>> ? 3) VerifySharedArchive turned on or >>>>> ? 4) CDS archives custom loaded classes >>>>> ?? If none of above in java application, it is not necessary to >>>>> have zip library loaded. >>>>> >>>>> ? Solution by loading zip library on demand when needed. >>>>> >>>>> Performance for java -Xint version: >>>>> >>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 40.274 >>>>> (? 0.475)??? ++ >>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 41.183 >>>>> (? 0.592)??? ++ >>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 ( >>>>> -0.306)????? - >>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 40.233 >>>>> ( -0.591)????? -- >>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 >>>>> (? 0.093) >>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 ( >>>>> -0.411)????? -- >>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 40.077 >>>>> (? 0.314)???? + >>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 39.724 >>>>> ( -1.188)????? ----- >>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 40.033 >>>>> ( -0.093) >>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 40.689 >>>>> ( -0.042) >>>>> ============================================================ >>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 40.322 >>>>> ( -0.116) >>>>> instr delta =????? -161917??? -0.2717% >>>>> time? delta =?????? -0.116 ms -0.2859% >>>>> >>>>> Tests: hs-tier1-4. >>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>> pmap list in test case: >>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>> >>>>> ** >>>>> * Thanks >>>>> Yumin >>>> >>> >> > From chris.plummer at oracle.com Mon May 4 18:32:01 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 4 May 2020 11:32:01 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: <580F3B60-33D3-4AD1-B8F7-AE9DF30A1427@oracle.com> References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> <580F3B60-33D3-4AD1-B8F7-AE9DF30A1427@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From leonid.mesnik at oracle.com Mon May 4 18:43:42 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 4 May 2020 11:43:42 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> <580F3B60-33D3-4AD1-B8F7-AE9DF30A1427@oracle.com> Message-ID: Thank you for review and feedback. Leonid On 5/4/20 11:32 AM, Chris Plummer wrote: > Hi Leonid, > > I wasn't suggesting to remove the entire comment, just to remove the > prototype from the comment. If you do choose to restore the comment, > make sure you fix the two places where there are double spaces. I'm > also ok if you want to leave the comment out. I don't need to see > another webrev. > > thanks, > > Chris > > On 5/1/20 4:51 PM, Leonid Mesnik wrote: >> Yes, the protected should be enough. I've made all non constant >> fields and methods protected and remove comment. >> >> The updated webrev (JDIBase only) >> http://cr.openjdk.java.net/~lmesnik/8244133/webrev.01/ >> >> Leonid >> >>> On May 1, 2020, at 4:05 PM, Chris Plummer >> > wrote: >>> >>> On 5/1/20 3:42 PM, Leonid Mesnik wrote: >>>> Hi >>>> >>>>> On May 1, 2020, at 1:12 PM, Chris Plummer >>>>> > wrote: >>>>> >>>>> Hi Leonid, >>>>> >>>>> Overall looks good. >>>>> >>>>> How did you verify that all settingBreakpoint() implementations >>>>> were the same? How did you find that some of the >>>>> breakpointForCommunication() implementations were different? >>>> >>>> Unfortunately I didn't find a good way to check that all these >>>> method are exactly the same. I just go through diff without lines >>>> common for all methods and verify that nothing unexpected was removed. >>>> Using this I found the different ?breakpointForCommunication() and >>>> settingBreakpoint() implementation. I found 2 ?variants of >>>> ??settingBreakpoint(). The one which set breakpointLocation (or >>>> location) and one which don't set. >>>> >>>> >>>> Additionally I grepped new files to check that lines like '?static >>>> final int PASSED' and other known patterns are removed. >>> Ok. >>>>> >>>>> A few minor things in JDIBase: >>>>> >>>>> ?108???????????? throws JDITestRuntimeException { >>>>> >>>>> I think it would look better with the { on a new line. >>>>> >>>>> ? 80???? // used by tests with breakpoint communication >>>>> ? 81???? public EventRequestManager eventRManager; >>>>> ? 82???? public EventQueue eventQueue; >>>>> ? 83???? public EventSet eventSet; >>>>> ? 84???? public EventIterator eventIterator; >>>>> ? 85 >>>>> ? 86???? // additional fields initialized during breakpoint >>>>> communication >>>>> ? 87???? public Location breakpLocation = null; >>>>> ? 88???? public BreakpointEvent bpEvent; >>>>> >>>>> Do these fields need to be public and not just protected? Public >>>>> fields in java are not the norm. >>>>> >>>>> ? 91???? /* >>>>> ? 92????? * private BreakpointRequest >>>>> settingBreakpoint(ThreadReference, ReferenceType, >>>>> ? 93 * String, String, String) >>>>> ? 94????? * >>>>> ? 95????? * It sets up a breakpoint at given line number within a >>>>> given method in a given class >>>>> ? 96????? * for a given thread. >>>>> ? 97????? * >>>>> ? 98????? * Return value: BreakpointRequest object? in case of success >>>>> ? 99????? * >>>>> ?100????? * JDITestRuntimeException? in case of an Exception >>>>> thrown within the method >>>>> ?101????? */ >>>>> ?123???????????? int n = >>>>> ?124???????????????????? ((IntegerValue) >>>>> testedClass.getValue(testedClass.fieldByName(bpLine))).value(); >>>>> >>>>> I'm not sure why the comment includes the prototype, but in any >>>>> case it is out of date. I suggest just removing it. >>>>> >>>>> Also, there are double-spaces in lines 98 and 100. >>>>> >>>> I could make fields protected and remove comments. But I want to >>>> review base class and ?might be make more refactoring. ?Also there >>>> are a couple of tests with strange formatting, so I am going to add >>>> them separately. >>>> I just don't want to put all this in single change. So diff >>>> contains mostly removal of same lines and it is easier to check it. >>> But I'm just referring to the one copy in JDIBase.java. It seems >>> getting the format and commenting of this entirely new class is >>> something to do now, not at some later point when you cleanup other >>> tests. >>> >>> Also, the fields were not public before being moved to JDIBase, they >>> were package private. So unless there is a reason for them to be >>> public, I'm not sure why you would not be more restrictive as long >>> as it does not impact the tests. Do the tests need for the fields to >>> be public or else would need to be modified? >>> >>> thanks, >>> >>> Chris >>>> >>>> Leonid >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 4/29/20 4:45 PM, Leonid Mesnik wrote: >>>>>> Hi >>>>>> >>>>>> Could you please review following fix which remove code >>>>>> duplication by moving methods?BreakpointRequest >>>>>> settingBreakpoint,?getEventSet,?breakpointForCommunication and >>>>>> log1,2,3 in base class. >>>>>> >>>>>> The fix is huge but pretty straight forward, I tried to keep >>>>>> changes as small as possible so most tests just changed to >>>>>> extends JDIBase and corresponding methods and fields were deleted. >>>>>> >>>>>> Some tests used slightly different 'breakpointForCommunication()' >>>>>> implementation so they override it now. >>>>>> >>>>>> Some tests contain change like this: >>>>>> - eventRequest1 = eventRManager.createBreakpointRequest(location); >>>>>> + eventRequest1 = >>>>>> eventRManager.createBreakpointRequest(breakpLocation); >>>>>> it is because they had 'location' and not 'breakpLocation' which >>>>>> is set by settingBreakpoint(...). So just merged location and >>>>>> breakpLocation. >>>>>> >>>>>> All tests passed, so the main goal is to ensure that changes >>>>>> don't introduce false positive results ignoring exit codes and >>>>>> statuses. >>>>>> >>>>>> I quickly verified that updated files don't contain declarations >>>>>> of variable like PASSED, testExitCode and other. >>>>>> >>>>>> The JDIBase and all tests could be improved, also there are still >>>>>> a lot of tests which could use it as a base as well as other base >>>>>> classes for idi debuggers. However I ?want to keep this fix as >>>>>> simple as possible. >>>>>> >>>>>> werbev: http://cr.openjdk.java.net/~lmesnik/8244133/webrev.00/ >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244133 >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From leonid.mesnik at oracle.com Mon May 4 18:43:53 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 4 May 2020 11:43:53 -0700 Subject: RFR: 8244133: Refactor nsk/jdi tests to reduce code duplication in settingBreakpoint communication In-Reply-To: <1761865a-cdfe-9071-fe28-d65e40f4a38c@oracle.com> References: <53C1DDD1-5FB7-47E8-9DE3-1DA0DE84579B@oracle.com> <580F3B60-33D3-4AD1-B8F7-AE9DF30A1427@oracle.com> <1761865a-cdfe-9071-fe28-d65e40f4a38c@oracle.com> Message-ID: <398484e2-7926-e598-2978-3708dc5eac69@oracle.com> Thank you for review and feedback. Leonid On 5/1/20 7:45 PM, serguei.spitsyn at oracle.com wrote: > Hi Leonid, > > I'd suggest to align parameters and remove two commented lines: > ?90 protected final BreakpointRequest settingBreakpoint(ThreadReference thread, > 91 ReferenceType testedClass, > 92 String methodName, > 93 String bpLine, > 94 String property) > ?. . . > 141 protected final void getEventSet() throws JDITestRuntimeException { > 142 try { > 143 // jdiHelper.log2(" eventSet = eventQueue.remove(waitTime);"); > 144 eventSet = eventQueue.remove(waitTime); > 145 if (eventSet == null) { > 146 throw new JDITestRuntimeException("** TIMEOUT while waiting for event **"); > 147 } > 148 // jdiHelper.log2(" eventIterator = eventSet.eventIterator;"); > > Otherwise, looks good. > No need in new webrev if you fix it. > > Thanks, > Serguei > > > On 5/1/20 16:51, Leonid Mesnik wrote: >> Yes, the protected should be enough. I've made all non constant >> fields and methods protected and remove comment. >> >> The updated webrev (JDIBase only) >> http://cr.openjdk.java.net/~lmesnik/8244133/webrev.01/ >> >> Leonid >> >>> On May 1, 2020, at 4:05 PM, Chris Plummer >> > wrote: >>> >>> On 5/1/20 3:42 PM, Leonid Mesnik wrote: >>>> Hi >>>> >>>>> On May 1, 2020, at 1:12 PM, Chris Plummer >>>>> > wrote: >>>>> >>>>> Hi Leonid, >>>>> >>>>> Overall looks good. >>>>> >>>>> How did you verify that all settingBreakpoint() implementations >>>>> were the same? How did you find that some of the >>>>> breakpointForCommunication() implementations were different? >>>> >>>> Unfortunately I didn't find a good way to check that all these >>>> method are exactly the same. I just go through diff without lines >>>> common for all methods and verify that nothing unexpected was removed. >>>> Using this I found the different ?breakpointForCommunication() and >>>> settingBreakpoint() implementation. I found 2 ?variants of >>>> ??settingBreakpoint(). The one which set breakpointLocation (or >>>> location) and one which don't set. >>>> >>>> >>>> Additionally I grepped new files to check that lines like '?static >>>> final int PASSED' and other known patterns are removed. >>> Ok. >>>>> >>>>> A few minor things in JDIBase: >>>>> >>>>> ?108???????????? throws JDITestRuntimeException { >>>>> >>>>> I think it would look better with the { on a new line. >>>>> >>>>> ? 80???? // used by tests with breakpoint communication >>>>> ? 81???? public EventRequestManager eventRManager; >>>>> ? 82???? public EventQueue eventQueue; >>>>> ? 83???? public EventSet eventSet; >>>>> ? 84???? public EventIterator eventIterator; >>>>> ? 85 >>>>> ? 86???? // additional fields initialized during breakpoint >>>>> communication >>>>> ? 87???? public Location breakpLocation = null; >>>>> ? 88???? public BreakpointEvent bpEvent; >>>>> >>>>> Do these fields need to be public and not just protected? Public >>>>> fields in java are not the norm. >>>>> >>>>> ? 91???? /* >>>>> ? 92????? * private BreakpointRequest >>>>> settingBreakpoint(ThreadReference, ReferenceType, >>>>> ? 93 * String, String, String) >>>>> ? 94????? * >>>>> ? 95????? * It sets up a breakpoint at given line number within a >>>>> given method in a given class >>>>> ? 96????? * for a given thread. >>>>> ? 97????? * >>>>> ? 98????? * Return value: BreakpointRequest object? in case of success >>>>> ? 99????? * >>>>> ?100????? * JDITestRuntimeException? in case of an Exception >>>>> thrown within the method >>>>> ?101????? */ >>>>> ?123???????????? int n = >>>>> ?124???????????????????? ((IntegerValue) >>>>> testedClass.getValue(testedClass.fieldByName(bpLine))).value(); >>>>> >>>>> I'm not sure why the comment includes the prototype, but in any >>>>> case it is out of date. I suggest just removing it. >>>>> >>>>> Also, there are double-spaces in lines 98 and 100. >>>>> >>>> I could make fields protected and remove comments. But I want to >>>> review base class and ?might be make more refactoring. ?Also there >>>> are a couple of tests with strange formatting, so I am going to add >>>> them separately. >>>> I just don't want to put all this in single change. So diff >>>> contains mostly removal of same lines and it is easier to check it. >>> But I'm just referring to the one copy in JDIBase.java. It seems >>> getting the format and commenting of this entirely new class is >>> something to do now, not at some later point when you cleanup other >>> tests. >>> >>> Also, the fields were not public before being moved to JDIBase, they >>> were package private. So unless there is a reason for them to be >>> public, I'm not sure why you would not be more restrictive as long >>> as it does not impact the tests. Do the tests need for the fields to >>> be public or else would need to be modified? >>> >>> thanks, >>> >>> Chris >>>> >>>> Leonid >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 4/29/20 4:45 PM, Leonid Mesnik wrote: >>>>>> Hi >>>>>> >>>>>> Could you please review following fix which remove code >>>>>> duplication by moving methods?BreakpointRequest >>>>>> settingBreakpoint,?getEventSet,?breakpointForCommunication and >>>>>> log1,2,3 in base class. >>>>>> >>>>>> The fix is huge but pretty straight forward, I tried to keep >>>>>> changes as small as possible so most tests just changed to >>>>>> extends JDIBase and corresponding methods and fields were deleted. >>>>>> >>>>>> Some tests used slightly different 'breakpointForCommunication()' >>>>>> implementation so they override it now. >>>>>> >>>>>> Some tests contain change like this: >>>>>> - eventRequest1 = eventRManager.createBreakpointRequest(location); >>>>>> + eventRequest1 = >>>>>> eventRManager.createBreakpointRequest(breakpLocation); >>>>>> it is because they had 'location' and not 'breakpLocation' which >>>>>> is set by settingBreakpoint(...). So just merged location and >>>>>> breakpLocation. >>>>>> >>>>>> All tests passed, so the main goal is to ensure that changes >>>>>> don't introduce false positive results ignoring exit codes and >>>>>> statuses. >>>>>> >>>>>> I quickly verified that updated files don't contain declarations >>>>>> of variable like PASSED, testExitCode and other. >>>>>> >>>>>> The JDIBase and all tests could be improved, also there are still >>>>>> a lot of tests which could use it as a base as well as other base >>>>>> classes for idi debuggers. However I ?want to keep this fix as >>>>>> simple as possible. >>>>>> >>>>>> werbev: http://cr.openjdk.java.net/~lmesnik/8244133/webrev.00/ >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244133 >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Mon May 4 19:01:17 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 4 May 2020 12:01:17 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: <25d5e2fa-8909-0b94-e0b2-6b5aaa224492@oracle.com> JIT, AOT, JVMCI and Graal changes seem fine to me. It would be interesting to see shared code execution coverage change. There are places where we use flags and setting instead of #ifdef SPARC which may not be executed now or executed partially. We may simplify such code too. Thanks, Vladimir On 5/3/20 10:12 PM, Mikael Vidstedt wrote: > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! > > Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From daniil.x.titov at oracle.com Mon May 4 20:05:47 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Mon, 04 May 2020 13:05:47 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> Message-ID: Hi Chris, Please review a new version of webrev [1] that addresses your comments. For the following 3 tests that showed the increase of the execution time with -Xcomp more than 5 minutes this version of the change strips -Xcomp option when forwarding VM arguments to j*-tools : -- serviceability/sa/sadebugd/SADebugDTest.java, -- serviceability/sa/sadebugd/DebugdConnectTest.java, -- serviceability/sa/ClhsdbJstackXcompStress.java The execution time for the rest of the tests when running with -Xcomp was increased within 1 and half minute. [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ [2] https://bugs.openjdk.java.net/browse/JDK-8242009 Thank you, Daniil ?On 4/27/20, 12:55 PM, "Chris Plummer" wrote: Hi Daniil, Overall it looks good. A few comments below. Can you add a comment to TestSysProps.java indicating the reason for checking if the line starts with "[". In JDKToolLauncher you have an extra empty line: 112 * Any platform specific arguments required for running the tool are 113 * automatically added. 114 * 115 * 116 * @param args In OutputAnalyzer.java, I wonder if all these matching APIs you updated should by default just include the version output in their filtering. As for the added time to execute, I would suggest possibly stripping -Xcomp from the few outliers, and I would mostly focus on how much longer it takes, not how many times longer. For example, increasing from 10 seconds to 40 seconds is not a big deal. Increasing from 10 minutes to 20 minutes is. SADebugDTest creates 8 tool processes so, that's probably the reason for the big increase, although I'm not sure why it is kind of slow in the first place. It does nothing more on each iteration than launch "jhsdb debugd", which will connect to the debuggee, and then is killed off. Maybe there is something slow with connecting, especial with RMI. thanks, Chris On 4/27/20 12:07 PM, Daniil Titov wrote: > Please review the change [1] that ensures that VM and test options are forwarded to > j*-tools when they are launched from serviceability/sa tests. > > The tests that expect an empty output were corrected to ignore the product version printed > in the error stream since in some tiers the tests are run with ' -showversion' VM option (tier3). > > In test serviceability/sa/TestSysProps.java the code that counts the system properties was corrected > to ignore the debug output when the test is run with " -Xlog:cds=debug" option (tier4). > > Testing: Mach5 tests for tier1 - tier7 passed. > > I also run the test with -XComp at Mach5 linux-x64-debug builds before and after the changes > and for the most of the tests the overhead is about 2 times although for > serviceability/sa/sadebugd/SADebugDTest.java it spikes up to 5 times. Probably at least for some tests > it makes to filter out some properties (e.g. -Xcomp) before forwarding them to j*-tools. > > serviceability/sa/sadebugd/SADebugDTest.java, before : 2m 23s , after:11m 07s > serviceability/sa/sadebugd/TestJmapCore.java, before : 42s , after:1m 09s > serviceability/sa/TestSysProps.java, before : 36s , after: 1m 27s > > > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 > > Thank you, > Daniil > > From dean.long at oracle.com Mon May 4 20:49:35 2020 From: dean.long at oracle.com (Dean Long) Date: Mon, 4 May 2020 13:49:35 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <2dd3da73-7e60-3317-8fc8-2ac92d65f0b0@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <2dd3da73-7e60-3317-8fc8-2ac92d65f0b0@oracle.com> Message-ID: <0c31a4a8-8ef2-3d75-c7ef-e698f52ad13c@oracle.com> Hi Ioi.? I think you're right.? Good catch. dl On 5/3/20 10:01 PM, Ioi Lam wrote: > Hi Dean, > > The code uses double-checked locking so we can be thread-safe when > loading the zip library, while avoiding using the MutexLock every time: > > https://en.wikipedia.org/wiki/Double-checked_locking > > To work correctly on modern memory architectures, we need the memory > barriers: > > ?977 void ClassLoader::load_zip_library() { > ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { > ?979???? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); > ?980???? if (libzip_loaded == 0) { > ?981?????? load_zip_library0(); > ?982?????? Atomic::release_store(&libzip_loaded, 1); > ?983???? } > ?984?? } > ?985 } > > (Atomic::load_acquire is much cheaper than MutexLocker). > > If we read Crc32 without memory barriers, as in the original code, in > two concurrent threads, I think there may be a chance for one of the > threads to read an invalid value of Crc32. > > int ClassLoader::crc32(int crc, const char* buf, int len) { > ? return (*Crc32)(crc, (const jbyte*)buf, len); > } > > Thanks > - Ioi > > > On 5/1/20 8:00 PM, Yumin Qi wrote: >> Dean, >> >> ? I have updated to use MutexLocker instead at same link: >> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >> >> ? Tested locally, passed jtreg/runtime. >> >> Thanks >> Yumin >> >> On 5/1/20 4:24 PM, Dean Long wrote: >>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>> >>> dl >>> >>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>> Hi, Dean >>>> ? Thanks for the review. I will try MutextLocker, for AOT, I think >>>> currently the tests are not using CDS then it will load classes >>>> from stream, that will use libzip's Crc32. >>>> ? I will check for AOT to see if it really loads libzip with the >>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>> >>>> #0? ClassLoader::load_zip_library () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>> >>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>> (this=0x7ffff0245640) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>> (name=0x7ffff40550f0, search_append_only=false, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>> #7? 0x00007ffff6232d0a in >>>> SystemDictionary::resolve_instance_class_or_null >>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>> #8? 0x00007ffff623137e in >>>> SystemDictionary::resolve_instance_class_or_null_helper >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., throw_error=true, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>> (class_name=0x7ffff40550f0, throw_error=true, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>> (id=SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>> #13 0x00007ffff623681e in >>>> SystemDictionary::resolve_wk_klasses_until >>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>> #14 0x00007ffff623ac01 in >>>> SystemDictionary::resolve_wk_klasses_through >>>> (end_id=SystemDictionary::Class_klass_knum, >>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>> #15 0x00007ffff62369b0 in >>>> SystemDictionary::resolve_well_known_classes >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>> #17 0x00007ffff62afe15 in Universe::genesis >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>> #19 0x00007ffff5af21ed in init_globals () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >>>> canTryAgain=0x7ffff7fc7d5b) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>> >>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>> pthread_create.c:333 >>>> #27 0x00007ffff790041d in clone () at >>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>> >>>> This is not with CDS. >>>> >>>> Thanks >>>> Yumin >>>> >>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>> MonitorLocker. >>>>> >>>>> dl >>>>> >>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>> Hi, please review: >>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>> Summary: >>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>> only needed when >>>>>> ? 1) bootclasspath contains zip or >>>>>> ? 2) UseAOT enabled or >>>>>> ? 3) VerifySharedArchive turned on or >>>>>> ? 4) CDS archives custom loaded classes >>>>>> ?? If none of above in java application, it is not necessary to >>>>>> have zip library loaded. >>>>>> >>>>>> ? Solution by loading zip library on demand when needed. >>>>>> >>>>>> Performance for java -Xint version: >>>>>> >>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 40.274 >>>>>> (? 0.475)??? ++ >>>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 41.183 >>>>>> (? 0.592)??? ++ >>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 >>>>>> ( -0.306)????? - >>>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 40.233 >>>>>> ( -0.591)????? -- >>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 >>>>>> (? 0.093) >>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 >>>>>> ( -0.411)????? -- >>>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 40.077 >>>>>> (? 0.314)???? + >>>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 39.724 >>>>>> ( -1.188)????? ----- >>>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 40.033 >>>>>> ( -0.093) >>>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 40.689 >>>>>> ( -0.042) >>>>>> ============================================================ >>>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 40.322 >>>>>> ( -0.116) >>>>>> instr delta =????? -161917??? -0.2717% >>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>> >>>>>> Tests: hs-tier1-4. >>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>> pmap list in test case: >>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>> >>>>>> ** >>>>>> * Thanks >>>>>> Yumin >>>>> >>>> >>> >> > From igor.ignatyev at oracle.com Mon May 4 21:29:39 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 4 May 2020 14:29:39 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: Hi Mikael, the changes in /test/ look good to me. I have a question regarding src/jdk.internal.vm.compiler/*, aren't these files part of graal-compiler and hence will be brought back by the next graal update? Thanks, -- Igor > On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: > > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! > > Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From vladimir.kozlov at oracle.com Mon May 4 21:49:47 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 4 May 2020 14:49:47 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: <60d3a22f-d4b4-280d-a8a7-1a306b7ad483@oracle.com> I filed Graal issue to change mx script to filter out SPARC code when we do sync Graal changes into JDK. For Graal shared code we may need to have versioning for latest JDK as we do in other cases. Regards, Vladimir On 5/4/20 2:29 PM, Igor Ignatyev wrote: > Hi Mikael, > > the changes in /test/ look good to me. > > I have a question regarding src/jdk.internal.vm.compiler/*, aren't these files part of graal-compiler and hence will be brought back by the next graal update? > > Thanks, > -- Igor > >> On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: >> >> >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> >> >> Background: >> >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> >> A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! >> >> Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. >> >> Testing: >> >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> >> Cheers, >> Mikael >> >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >> > From yumin.qi at oracle.com Mon May 4 23:40:51 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Mon, 4 May 2020 16:40:51 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> Message-ID: <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> Hi, Ioi ? Thanks. Updated webrev at: http://cr.openjdk.java.net/~minqi/8237750/webrev-02/ ? libzip_loaded now is a class member of ClassLoader, no longer a file private integer since it is used in both .hpp and .cpp files. Thanks ?Yumin On 5/4/20 8:31 AM, Ioi Lam wrote: > Hi Yumin, > > ?977 void ClassLoader::load_zip_library() { > ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { > ?979???? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); > ?980???? if (libzip_loaded == 0) { > ?981?????? load_zip_library0(); > ?982?????? Atomic::release_store(&libzip_loaded, 1); > ?983???? } > ?984?? } > ?985 } > > 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { > 1027?? if (libzip_loaded == 0) { > 1028???? load_zip_library(); > 1029?? } > 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); > 1031 } > > ClassLoader::crc32 access libzip_loaded without a memory barrier, so > it may read libzip_loaded out-of-order from Crc32. This means, thread > 1 may be writing the memory in this order: > > ??? Crc32 = ; > ??? libzip_loaded = 1; > > but the order observed in thread 2 may be > > ??? libzip_loaded = 1; > ???? >> ClassLoader::crc32 called here in thread 2 > ??? Crc32 = ; > > as a result, thread 2 may read libzip_loaded = 1, but still reads a > NULL from Crc32. > > To ensure the memory barrier is used, I think we should do this: > > // private > inline void ClassLoader::load_zip_library_if_needed() { > ? if (Atomic::load_acquire(&libzip_loaded) == 0) { > ??? release_load_zip_library(); > ? } > } > > // private > void ClassLoader::release_load_zip_library() { > ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); > ? if (libzip_loaded == 0) { > ??? load_zip_library0(); > ??? Atomic::release_store(&libzip_loaded, 1); > ? } > } > > int ClassLoader::crc32(int crc, const char* buf, int len) { > ? load_zip_library_if_needed(); > ? return (*Crc32)(crc, (const jbyte*)buf, len); > } > > HotSpot code uses the "release_" prefix to indicate that the function > must be used as a part of an acquire/release memory barrier. (E.g., > InstanceKlass::release_set_array_klasses()). > > Some backgrounds: > https://preshing.com/20130922/acquire-and-release-fences/ > > Thanks > - Ioi > > > > On 5/1/20 8:00 PM, Yumin Qi wrote: >> Dean, >> >> ? I have updated to use MutexLocker instead at same link: >> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >> >> ? Tested locally, passed jtreg/runtime. >> >> Thanks >> Yumin >> >> On 5/1/20 4:24 PM, Dean Long wrote: >>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>> >>> dl >>> >>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>> Hi, Dean >>>> ? Thanks for the review. I will try MutextLocker, for AOT, I think >>>> currently the tests are not using CDS then it will load classes >>>> from stream, that will use libzip's Crc32. >>>> ? I will check for AOT to see if it really loads libzip with the >>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>> >>>> #0? ClassLoader::load_zip_library () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>> >>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>> (this=0x7ffff0245640) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>> (name=0x7ffff40550f0, search_append_only=false, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>> #7? 0x00007ffff6232d0a in >>>> SystemDictionary::resolve_instance_class_or_null >>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>> #8? 0x00007ffff623137e in >>>> SystemDictionary::resolve_instance_class_or_null_helper >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>> (class_name=0x7ffff40550f0, class_loader=..., >>>> protection_domain=..., throw_error=true, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>> (class_name=0x7ffff40550f0, throw_error=true, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>> (id=SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>> #13 0x00007ffff623681e in >>>> SystemDictionary::resolve_wk_klasses_until >>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>> #14 0x00007ffff623ac01 in >>>> SystemDictionary::resolve_wk_klasses_through >>>> (end_id=SystemDictionary::Class_klass_knum, >>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>> __the_thread__=0x7ffff0033000) >>>> ??? at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>> #15 0x00007ffff62369b0 in >>>> SystemDictionary::resolve_well_known_classes >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>> #17 0x00007ffff62afe15 in Universe::genesis >>>> (__the_thread__=0x7ffff0033000) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>> #19 0x00007ffff5af21ed in init_globals () at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >>>> canTryAgain=0x7ffff7fc7d5b) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>> >>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>> pthread_create.c:333 >>>> #27 0x00007ffff790041d in clone () at >>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>> >>>> This is not with CDS. >>>> >>>> Thanks >>>> Yumin >>>> >>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>> MonitorLocker. >>>>> >>>>> dl >>>>> >>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>> Hi, please review: >>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>> Summary: >>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>> only needed when >>>>>> ? 1) bootclasspath contains zip or >>>>>> ? 2) UseAOT enabled or >>>>>> ? 3) VerifySharedArchive turned on or >>>>>> ? 4) CDS archives custom loaded classes >>>>>> ?? If none of above in java application, it is not necessary to >>>>>> have zip library loaded. >>>>>> >>>>>> ? Solution by loading zip library on demand when needed. >>>>>> >>>>>> Performance for java -Xint version: >>>>>> >>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 40.274 >>>>>> (? 0.475)??? ++ >>>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 41.183 >>>>>> (? 0.592)??? ++ >>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 >>>>>> ( -0.306)????? - >>>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 40.233 >>>>>> ( -0.591)????? -- >>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 >>>>>> (? 0.093) >>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 >>>>>> ( -0.411)????? -- >>>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 40.077 >>>>>> (? 0.314)???? + >>>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 39.724 >>>>>> ( -1.188)????? ----- >>>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 40.033 >>>>>> ( -0.093) >>>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 40.689 >>>>>> ( -0.042) >>>>>> ============================================================ >>>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 40.322 >>>>>> ( -0.116) >>>>>> instr delta =????? -161917??? -0.2717% >>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>> >>>>>> Tests: hs-tier1-4. >>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>> pmap list in test case: >>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>> >>>>>> ** >>>>>> * Thanks >>>>>> Yumin >>>>> >>>> >>> >> > From leonid.mesnik at oracle.com Tue May 5 01:15:37 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 4 May 2020 18:15:37 -0700 Subject: RFR: 8244267: Improve serviceability task definitions in CI Message-ID: <5c9f78d6-6fc7-b981-3ad4-2daee90190f9@oracle.com> Hi Could you please review following fix which add all quick and stable serviceability tests in tier1. The tier1_serviceability still takes less than 15 minutes on 2 cores i5. The tests with @intermittent key are not included. bug: https://bugs.openjdk.java.net/browse/JDK-8244267 diff -r d26f79a1edea test/hotspot/jtreg/TEST.groups --- a/test/hotspot/jtreg/TEST.groups??? Wed Apr 29 19:51:45 2020 -0700 +++ b/test/hotspot/jtreg/TEST.groups??? Mon May 04 16:52:39 2020 -0700 @@ -375,17 +375,15 @@ ? -runtime/cds/appcds/jigsaw/modulepath/MainModuleOnly.java ?tier1_serviceability = \ -? serviceability/dcmd/compiler \ +? serviceability/ \ ?? -serviceability/dcmd/compiler/CompilerQueueTest.java \ -? serviceability/jvmti/RedefineClasses \ ?? -serviceability/jvmti/RedefineClasses/RedefineLeak.java \ -serviceability/jvmti/RedefineClasses/RedefinePreviousVersions.java \ -serviceability/jvmti/RedefineClasses/RedefineRunningMethods.java \ -serviceability/jvmti/RedefineClasses/RedefineRunningMethodsWithBacktrace.java \ ?? -serviceability/jvmti/RedefineClasses/TestRedefineObject.java \ -? serviceability/logging \ -? serviceability/sa \ ?? -serviceability/sa/ClhsdbScanOops.java \ +? -serviceability/sa/ClhsdbJstackXcompStress.java \ ?? -serviceability/sa/TestJmapCore.java \ ?? -serviceability/sa/TestJmapCoreMetaspace.java Leonid From chris.plummer at oracle.com Tue May 5 17:52:46 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 5 May 2020 10:52:46 -0700 Subject: RFR: 8244267: Improve serviceability task definitions in CI In-Reply-To: <5c9f78d6-6fc7-b981-3ad4-2daee90190f9@oracle.com> References: <5c9f78d6-6fc7-b981-3ad4-2daee90190f9@oracle.com> Message-ID: <488bede1-f599-5ff4-c29a-6decfb47975b@oracle.com> Hi Leonid, Looks good. Thanks for fixing this. Chris On 5/4/20 6:15 PM, Leonid Mesnik wrote: > Hi > > Could you please review following fix which add all quick and stable > serviceability tests in tier1. The tier1_serviceability still takes > less than 15 minutes on 2 cores i5. > > The tests with @intermittent key are not included. > > bug: https://bugs.openjdk.java.net/browse/JDK-8244267 > > diff -r d26f79a1edea test/hotspot/jtreg/TEST.groups > --- a/test/hotspot/jtreg/TEST.groups??? Wed Apr 29 19:51:45 2020 -0700 > +++ b/test/hotspot/jtreg/TEST.groups??? Mon May 04 16:52:39 2020 -0700 > @@ -375,17 +375,15 @@ > ? -runtime/cds/appcds/jigsaw/modulepath/MainModuleOnly.java > > ?tier1_serviceability = \ > -? serviceability/dcmd/compiler \ > +? serviceability/ \ > ?? -serviceability/dcmd/compiler/CompilerQueueTest.java \ > -? serviceability/jvmti/RedefineClasses \ > ?? -serviceability/jvmti/RedefineClasses/RedefineLeak.java \ > -serviceability/jvmti/RedefineClasses/RedefinePreviousVersions.java \ > -serviceability/jvmti/RedefineClasses/RedefineRunningMethods.java \ > -serviceability/jvmti/RedefineClasses/RedefineRunningMethodsWithBacktrace.java > \ > ?? -serviceability/jvmti/RedefineClasses/TestRedefineObject.java \ > -? serviceability/logging \ > -? serviceability/sa \ > ?? -serviceability/sa/ClhsdbScanOops.java \ > +? -serviceability/sa/ClhsdbJstackXcompStress.java \ > ?? -serviceability/sa/TestJmapCore.java \ > ?? -serviceability/sa/TestJmapCoreMetaspace.java > > > Leonid > From ioi.lam at oracle.com Tue May 5 17:59:28 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 5 May 2020 10:59:28 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> Message-ID: <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> Hi Yumin, Looks good. Just small nits. No need for new webrev. [1] I think this should be named _libzip_loaded to be consistent with other fields ? static int? libzip_loaded;?????????? // used to sync loading zip. [2] This introduces dependency on atomic.hpp to classLoader.hpp ? static void load_zip_library_if_needed() { ??? if (Atomic::load_acquire(&libzip_loaded) == 0) { ????? release_load_zip_library(); ??? } ? } Since this function is used only privately in the cpp file, I think it's better to change the hpp to this (to reduce unnecessary header file dependencies) ? inline static void load_zip_library_if_needed(); and then move the implementation to the classLoader.cpp file. You also need to add include "runtime/atomic.hpp" to classLoader.cpp. Thanks - Ioi On 5/4/20 4:40 PM, Yumin Qi wrote: > Hi, Ioi > > ? Thanks. Updated webrev at: > http://cr.openjdk.java.net/~minqi/8237750/webrev-02/ > ? libzip_loaded now is a class member of ClassLoader, no longer a file > private integer since it is used in both .hpp and .cpp files. > > Thanks > ?Yumin > > On 5/4/20 8:31 AM, Ioi Lam wrote: >> Hi Yumin, >> >> ?977 void ClassLoader::load_zip_library() { >> ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { >> ?979???? MutexLocker locker(Zip_lock, >> Monitor::_no_safepoint_check_flag); >> ?980???? if (libzip_loaded == 0) { >> ?981?????? load_zip_library0(); >> ?982?????? Atomic::release_store(&libzip_loaded, 1); >> ?983???? } >> ?984?? } >> ?985 } >> >> 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { >> 1027?? if (libzip_loaded == 0) { >> 1028???? load_zip_library(); >> 1029?? } >> 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); >> 1031 } >> >> ClassLoader::crc32 access libzip_loaded without a memory barrier, so >> it may read libzip_loaded out-of-order from Crc32. This means, thread >> 1 may be writing the memory in this order: >> >> ??? Crc32 = ; >> ??? libzip_loaded = 1; >> >> but the order observed in thread 2 may be >> >> ??? libzip_loaded = 1; >> ???? >> ClassLoader::crc32 called here in thread 2 >> ??? Crc32 = ; >> >> as a result, thread 2 may read libzip_loaded = 1, but still reads a >> NULL from Crc32. >> >> To ensure the memory barrier is used, I think we should do this: >> >> // private >> inline void ClassLoader::load_zip_library_if_needed() { >> ? if (Atomic::load_acquire(&libzip_loaded) == 0) { >> ??? release_load_zip_library(); >> ? } >> } >> >> // private >> void ClassLoader::release_load_zip_library() { >> ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); >> ? if (libzip_loaded == 0) { >> ??? load_zip_library0(); >> ??? Atomic::release_store(&libzip_loaded, 1); >> ? } >> } >> >> int ClassLoader::crc32(int crc, const char* buf, int len) { >> ? load_zip_library_if_needed(); >> ? return (*Crc32)(crc, (const jbyte*)buf, len); >> } >> >> HotSpot code uses the "release_" prefix to indicate that the function >> must be used as a part of an acquire/release memory barrier. (E.g., >> InstanceKlass::release_set_array_klasses()). >> >> Some backgrounds: >> https://preshing.com/20130922/acquire-and-release-fences/ >> >> Thanks >> - Ioi >> >> >> >> On 5/1/20 8:00 PM, Yumin Qi wrote: >>> Dean, >>> >>> ? I have updated to use MutexLocker instead at same link: >>> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>> >>> ? Tested locally, passed jtreg/runtime. >>> >>> Thanks >>> Yumin >>> >>> On 5/1/20 4:24 PM, Dean Long wrote: >>>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>>> >>>> dl >>>> >>>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>>> Hi, Dean >>>>> ? Thanks for the review. I will try MutextLocker, for AOT, I think >>>>> currently the tests are not using CDS then it will load classes >>>>> from stream, that will use libzip's Crc32. >>>>> ? I will check for AOT to see if it really loads libzip with the >>>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>>> >>>>> #0? ClassLoader::load_zip_library () at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>>> >>>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>>> (this=0x7ffff0245640) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, cl_inst_info=..., >>>>> __the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>>> __the_thread__=0x7ffff0033000) >>>>> ??? at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>>> (name=0x7ffff40550f0, search_append_only=false, >>>>> __the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>> __the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>>> #7? 0x00007ffff6232d0a in >>>>> SystemDictionary::resolve_instance_class_or_null >>>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>>> __the_thread__=0x7ffff0033000) >>>>> ??? at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>>> #8? 0x00007ffff623137e in >>>>> SystemDictionary::resolve_instance_class_or_null_helper >>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>>> ??? at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>> protection_domain=..., throw_error=true, >>>>> __the_thread__=0x7ffff0033000) >>>>> ??? at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>>> (class_name=0x7ffff40550f0, throw_error=true, >>>>> __the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>>> (id=SystemDictionary::Object_klass_knum, >>>>> __the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>>> #13 0x00007ffff623681e in >>>>> SystemDictionary::resolve_wk_klasses_until >>>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>> __the_thread__=0x7ffff0033000) >>>>> ??? at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>>> #14 0x00007ffff623ac01 in >>>>> SystemDictionary::resolve_wk_klasses_through >>>>> (end_id=SystemDictionary::Class_klass_knum, >>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>> __the_thread__=0x7ffff0033000) >>>>> ??? at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>>> #15 0x00007ffff62369b0 in >>>>> SystemDictionary::resolve_well_known_classes >>>>> (__the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>>> (__the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>>> #17 0x00007ffff62afe15 in Universe::genesis >>>>> (__the_thread__=0x7ffff0033000) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>>> #19 0x00007ffff5af21ed in init_globals () at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>>> #20 0x00007ffff627feff in Threads::create_vm (args=0x7ffff7fc7e50, >>>>> canTryAgain=0x7ffff7fc7d5b) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>>> >>>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>>> >>>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>>> pthread_create.c:333 >>>>> #27 0x00007ffff790041d in clone () at >>>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>>> >>>>> This is not with CDS. >>>>> >>>>> Thanks >>>>> Yumin >>>>> >>>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>>> MonitorLocker. >>>>>> >>>>>> dl >>>>>> >>>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>>> Hi, please review: >>>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>>> Summary: >>>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>>> only needed when >>>>>>> ? 1) bootclasspath contains zip or >>>>>>> ? 2) UseAOT enabled or >>>>>>> ? 3) VerifySharedArchive turned on or >>>>>>> ? 4) CDS archives custom loaded classes >>>>>>> ?? If none of above in java application, it is not necessary to >>>>>>> have zip library loaded. >>>>>>> >>>>>>> ? Solution by loading zip library on demand when needed. >>>>>>> >>>>>>> Performance for java -Xint version: >>>>>>> >>>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 >>>>>>> 40.274 (? 0.475)??? ++ >>>>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 >>>>>>> 41.183 (? 0.592)??? ++ >>>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 40.471 >>>>>>> ( -0.306)????? - >>>>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 >>>>>>> 40.233 ( -0.591)????? -- >>>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 40.493 >>>>>>> (? 0.093) >>>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 40.064 >>>>>>> ( -0.411)????? -- >>>>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 >>>>>>> 40.077 (? 0.314)???? + >>>>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 >>>>>>> 39.724 ( -1.188)????? ----- >>>>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 >>>>>>> 40.033 ( -0.093) >>>>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 >>>>>>> 40.689 ( -0.042) >>>>>>> ============================================================ >>>>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 >>>>>>> 40.322 ( -0.116) >>>>>>> instr delta =????? -161917??? -0.2717% >>>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>>> >>>>>>> Tests: hs-tier1-4. >>>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>>> pmap list in test case: >>>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>>> >>>>>>> ** >>>>>>> * Thanks >>>>>>> Yumin >>>>>> >>>>> >>>> >>> >> > From serguei.spitsyn at oracle.com Tue May 5 18:23:08 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 5 May 2020 11:23:08 -0700 Subject: RFR: 8244267: Improve serviceability task definitions in CI In-Reply-To: <5c9f78d6-6fc7-b981-3ad4-2daee90190f9@oracle.com> References: <5c9f78d6-6fc7-b981-3ad4-2daee90190f9@oracle.com> Message-ID: Hi Leonid, It looks okay to me. Thanks, Serguei On 5/4/20 18:15, Leonid Mesnik wrote: > Hi > > Could you please review following fix which add all quick and stable > serviceability tests in tier1. The tier1_serviceability still takes > less than 15 minutes on 2 cores i5. > > The tests with @intermittent key are not included. > > bug: https://bugs.openjdk.java.net/browse/JDK-8244267 > > diff -r d26f79a1edea test/hotspot/jtreg/TEST.groups > --- a/test/hotspot/jtreg/TEST.groups??? Wed Apr 29 19:51:45 2020 -0700 > +++ b/test/hotspot/jtreg/TEST.groups??? Mon May 04 16:52:39 2020 -0700 > @@ -375,17 +375,15 @@ > ? -runtime/cds/appcds/jigsaw/modulepath/MainModuleOnly.java > > ?tier1_serviceability = \ > -? serviceability/dcmd/compiler \ > +? serviceability/ \ > ?? -serviceability/dcmd/compiler/CompilerQueueTest.java \ > -? serviceability/jvmti/RedefineClasses \ > ?? -serviceability/jvmti/RedefineClasses/RedefineLeak.java \ > -serviceability/jvmti/RedefineClasses/RedefinePreviousVersions.java \ > -serviceability/jvmti/RedefineClasses/RedefineRunningMethods.java \ > -serviceability/jvmti/RedefineClasses/RedefineRunningMethodsWithBacktrace.java > \ > ?? -serviceability/jvmti/RedefineClasses/TestRedefineObject.java \ > -? serviceability/logging \ > -? serviceability/sa \ > ?? -serviceability/sa/ClhsdbScanOops.java \ > +? -serviceability/sa/ClhsdbJstackXcompStress.java \ > ?? -serviceability/sa/TestJmapCore.java \ > ?? -serviceability/sa/TestJmapCoreMetaspace.java > > > Leonid > From calvin.cheung at oracle.com Tue May 5 19:18:15 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 5 May 2020 12:18:15 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> Message-ID: <228c6d92-9422-d27e-d5c5-e9a29fb3e9bb@oracle.com> On 5/5/20 10:59 AM, Ioi Lam wrote: > Hi Yumin, > > Looks good. > > Just small nits. No need for new webrev. > > [1] I think this should be named _libzip_loaded to be consistent with > other fields > > ? static int? libzip_loaded;?????????? // used to sync loading zip. > > > [2] This introduces dependency on atomic.hpp to classLoader.hpp > > ? static void load_zip_library_if_needed() { > ??? if (Atomic::load_acquire(&libzip_loaded) == 0) { > ????? release_load_zip_library(); > ??? } > ? } > > Since this function is used only privately in the cpp file, I think > it's better to change the hpp to this (to reduce unnecessary header > file dependencies) > > ? inline static void load_zip_library_if_needed(); > > and then move the implementation to the classLoader.cpp file. You also > need to add include "runtime/atomic.hpp" to classLoader.cpp. How about putting the function in classloader.inline.hpp which already includes runtime/atomic.hpp? thanks, Calvin > > Thanks > - Ioi > > > > > > On 5/4/20 4:40 PM, Yumin Qi wrote: >> Hi, Ioi >> >> ? Thanks. Updated webrev at: >> http://cr.openjdk.java.net/~minqi/8237750/webrev-02/ >> ? libzip_loaded now is a class member of ClassLoader, no longer a >> file private integer since it is used in both .hpp and .cpp files. >> >> Thanks >> ?Yumin >> >> On 5/4/20 8:31 AM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> ?977 void ClassLoader::load_zip_library() { >>> ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>> ?979???? MutexLocker locker(Zip_lock, >>> Monitor::_no_safepoint_check_flag); >>> ?980???? if (libzip_loaded == 0) { >>> ?981?????? load_zip_library0(); >>> ?982?????? Atomic::release_store(&libzip_loaded, 1); >>> ?983???? } >>> ?984?? } >>> ?985 } >>> >>> 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { >>> 1027?? if (libzip_loaded == 0) { >>> 1028???? load_zip_library(); >>> 1029?? } >>> 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); >>> 1031 } >>> >>> ClassLoader::crc32 access libzip_loaded without a memory barrier, so >>> it may read libzip_loaded out-of-order from Crc32. This means, >>> thread 1 may be writing the memory in this order: >>> >>> ??? Crc32 = ; >>> ??? libzip_loaded = 1; >>> >>> but the order observed in thread 2 may be >>> >>> ??? libzip_loaded = 1; >>> ???? >> ClassLoader::crc32 called here in thread 2 >>> ??? Crc32 = ; >>> >>> as a result, thread 2 may read libzip_loaded = 1, but still reads a >>> NULL from Crc32. >>> >>> To ensure the memory barrier is used, I think we should do this: >>> >>> // private >>> inline void ClassLoader::load_zip_library_if_needed() { >>> ? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>> ??? release_load_zip_library(); >>> ? } >>> } >>> >>> // private >>> void ClassLoader::release_load_zip_library() { >>> ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); >>> ? if (libzip_loaded == 0) { >>> ??? load_zip_library0(); >>> ??? Atomic::release_store(&libzip_loaded, 1); >>> ? } >>> } >>> >>> int ClassLoader::crc32(int crc, const char* buf, int len) { >>> ? load_zip_library_if_needed(); >>> ? return (*Crc32)(crc, (const jbyte*)buf, len); >>> } >>> >>> HotSpot code uses the "release_" prefix to indicate that the >>> function must be used as a part of an acquire/release memory >>> barrier. (E.g., InstanceKlass::release_set_array_klasses()). >>> >>> Some backgrounds: >>> https://preshing.com/20130922/acquire-and-release-fences/ >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> On 5/1/20 8:00 PM, Yumin Qi wrote: >>>> Dean, >>>> >>>> ? I have updated to use MutexLocker instead at same link: >>>> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>> >>>> ? Tested locally, passed jtreg/runtime. >>>> >>>> Thanks >>>> Yumin >>>> >>>> On 5/1/20 4:24 PM, Dean Long wrote: >>>>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>>>> >>>>> dl >>>>> >>>>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>>>> Hi, Dean >>>>>> ? Thanks for the review. I will try MutextLocker, for AOT, I >>>>>> think currently the tests are not using CDS then it will load >>>>>> classes from stream, that will use libzip's Crc32. >>>>>> ? I will check for AOT to see if it really loads libzip with the >>>>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>>>> >>>>>> #0? ClassLoader::load_zip_library () at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>>>> >>>>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>>>> (this=0x7ffff0245640) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, >>>>>> cl_inst_info=..., __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>>>> (name=0x7ffff40550f0, search_append_only=false, >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>>>> #7? 0x00007ffff6232d0a in >>>>>> SystemDictionary::resolve_instance_class_or_null >>>>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>>>> #8? 0x00007ffff623137e in >>>>>> SystemDictionary::resolve_instance_class_or_null_helper >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>>>> >>>>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> protection_domain=..., throw_error=true, >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>>>> (class_name=0x7ffff40550f0, throw_error=true, >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>>>> (id=SystemDictionary::Object_klass_knum, >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>>>> #13 0x00007ffff623681e in >>>>>> SystemDictionary::resolve_wk_klasses_until >>>>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>>>> #14 0x00007ffff623ac01 in >>>>>> SystemDictionary::resolve_wk_klasses_through >>>>>> (end_id=SystemDictionary::Class_klass_knum, >>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>>>> #15 0x00007ffff62369b0 in >>>>>> SystemDictionary::resolve_well_known_classes >>>>>> (__the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>>>> (__the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>>>> #17 0x00007ffff62afe15 in Universe::genesis >>>>>> (__the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>>>> #19 0x00007ffff5af21ed in init_globals () at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>>>> #20 0x00007ffff627feff in Threads::create_vm >>>>>> (args=0x7ffff7fc7e50, canTryAgain=0x7ffff7fc7d5b) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>>>> >>>>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>>>> >>>>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>>>> >>>>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>>>> pthread_create.c:333 >>>>>> #27 0x00007ffff790041d in clone () at >>>>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>>>> >>>>>> This is not with CDS. >>>>>> >>>>>> Thanks >>>>>> Yumin >>>>>> >>>>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>>>> MonitorLocker. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>>>> Hi, please review: >>>>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>>>> Summary: >>>>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>>>> only needed when >>>>>>>> ? 1) bootclasspath contains zip or >>>>>>>> ? 2) UseAOT enabled or >>>>>>>> ? 3) VerifySharedArchive turned on or >>>>>>>> ? 4) CDS archives custom loaded classes >>>>>>>> ?? If none of above in java application, it is not necessary to >>>>>>>> have zip library loaded. >>>>>>>> >>>>>>>> ? Solution by loading zip library on demand when needed. >>>>>>>> >>>>>>>> Performance for java -Xint version: >>>>>>>> >>>>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 >>>>>>>> 40.274 (? 0.475)??? ++ >>>>>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 >>>>>>>> 41.183 (? 0.592)??? ++ >>>>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 >>>>>>>> 40.471 ( -0.306)????? - >>>>>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 >>>>>>>> 40.233 ( -0.591)????? -- >>>>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 >>>>>>>> 40.493 (? 0.093) >>>>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 >>>>>>>> 40.064 ( -0.411)????? -- >>>>>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 >>>>>>>> 40.077 (? 0.314)???? + >>>>>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 >>>>>>>> 39.724 ( -1.188)????? ----- >>>>>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 >>>>>>>> 40.033 ( -0.093) >>>>>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 >>>>>>>> 40.689 ( -0.042) >>>>>>>> ============================================================ >>>>>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 >>>>>>>> 40.322 ( -0.116) >>>>>>>> instr delta =????? -161917??? -0.2717% >>>>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>>>> >>>>>>>> Tests: hs-tier1-4. >>>>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>>>> pmap list in test case: >>>>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>>>> >>>>>>>> ** >>>>>>>> * Thanks >>>>>>>> Yumin >>>>>>> >>>>>> >>>>> >>>> >>> >> > From yumin.qi at oracle.com Tue May 5 20:32:39 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 5 May 2020 13:32:39 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> Message-ID: Hi, Ioi and Calvin ? There is an file classLoader.inline.hpp which also includes atomic.hpp. I put load_zip_library_if_needed in it. Updated webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-03/ ? Also, double checked the function is not in libjvm.so by ? nm libjvm.so | grep load_zip_library_if_needed Thanks Yumin On 5/5/20 10:59 AM, Ioi Lam wrote: > Hi Yumin, > > Looks good. > > Just small nits. No need for new webrev. > > [1] I think this should be named _libzip_loaded to be consistent with > other fields > > ? static int? libzip_loaded;?????????? // used to sync loading zip. > > > [2] This introduces dependency on atomic.hpp to classLoader.hpp > > ? static void load_zip_library_if_needed() { > ??? if (Atomic::load_acquire(&libzip_loaded) == 0) { > ????? release_load_zip_library(); > ??? } > ? } > > Since this function is used only privately in the cpp file, I think > it's better to change the hpp to this (to reduce unnecessary header > file dependencies) > > ? inline static void load_zip_library_if_needed(); > > and then move the implementation to the classLoader.cpp file. You also > need to add include "runtime/atomic.hpp" to classLoader.cpp. > > Thanks > - Ioi > > > > > > On 5/4/20 4:40 PM, Yumin Qi wrote: >> Hi, Ioi >> >> ? Thanks. Updated webrev at: >> http://cr.openjdk.java.net/~minqi/8237750/webrev-02/ >> ? libzip_loaded now is a class member of ClassLoader, no longer a >> file private integer since it is used in both .hpp and .cpp files. >> >> Thanks >> ?Yumin >> >> On 5/4/20 8:31 AM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> ?977 void ClassLoader::load_zip_library() { >>> ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>> ?979???? MutexLocker locker(Zip_lock, >>> Monitor::_no_safepoint_check_flag); >>> ?980???? if (libzip_loaded == 0) { >>> ?981?????? load_zip_library0(); >>> ?982?????? Atomic::release_store(&libzip_loaded, 1); >>> ?983???? } >>> ?984?? } >>> ?985 } >>> >>> 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { >>> 1027?? if (libzip_loaded == 0) { >>> 1028???? load_zip_library(); >>> 1029?? } >>> 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); >>> 1031 } >>> >>> ClassLoader::crc32 access libzip_loaded without a memory barrier, so >>> it may read libzip_loaded out-of-order from Crc32. This means, >>> thread 1 may be writing the memory in this order: >>> >>> ??? Crc32 = ; >>> ??? libzip_loaded = 1; >>> >>> but the order observed in thread 2 may be >>> >>> ??? libzip_loaded = 1; >>> ???? >> ClassLoader::crc32 called here in thread 2 >>> ??? Crc32 = ; >>> >>> as a result, thread 2 may read libzip_loaded = 1, but still reads a >>> NULL from Crc32. >>> >>> To ensure the memory barrier is used, I think we should do this: >>> >>> // private >>> inline void ClassLoader::load_zip_library_if_needed() { >>> ? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>> ??? release_load_zip_library(); >>> ? } >>> } >>> >>> // private >>> void ClassLoader::release_load_zip_library() { >>> ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); >>> ? if (libzip_loaded == 0) { >>> ??? load_zip_library0(); >>> ??? Atomic::release_store(&libzip_loaded, 1); >>> ? } >>> } >>> >>> int ClassLoader::crc32(int crc, const char* buf, int len) { >>> ? load_zip_library_if_needed(); >>> ? return (*Crc32)(crc, (const jbyte*)buf, len); >>> } >>> >>> HotSpot code uses the "release_" prefix to indicate that the >>> function must be used as a part of an acquire/release memory >>> barrier. (E.g., InstanceKlass::release_set_array_klasses()). >>> >>> Some backgrounds: >>> https://preshing.com/20130922/acquire-and-release-fences/ >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> On 5/1/20 8:00 PM, Yumin Qi wrote: >>>> Dean, >>>> >>>> ? I have updated to use MutexLocker instead at same link: >>>> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>> >>>> ? Tested locally, passed jtreg/runtime. >>>> >>>> Thanks >>>> Yumin >>>> >>>> On 5/1/20 4:24 PM, Dean Long wrote: >>>>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>>>> >>>>> dl >>>>> >>>>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>>>> Hi, Dean >>>>>> ? Thanks for the review. I will try MutextLocker, for AOT, I >>>>>> think currently the tests are not using CDS then it will load >>>>>> classes from stream, that will use libzip's Crc32. >>>>>> ? I will check for AOT to see if it really loads libzip with the >>>>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>>>> >>>>>> #0? ClassLoader::load_zip_library () at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>>>> >>>>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>>>> (this=0x7ffff0245640) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, >>>>>> cl_inst_info=..., __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>>>> (name=0x7ffff40550f0, search_append_only=false, >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>>>> #7? 0x00007ffff6232d0a in >>>>>> SystemDictionary::resolve_instance_class_or_null >>>>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>>>> #8? 0x00007ffff623137e in >>>>>> SystemDictionary::resolve_instance_class_or_null_helper >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>>>> >>>>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>> protection_domain=..., throw_error=true, >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>>>> (class_name=0x7ffff40550f0, throw_error=true, >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>>>> (id=SystemDictionary::Object_klass_knum, >>>>>> __the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>>>> #13 0x00007ffff623681e in >>>>>> SystemDictionary::resolve_wk_klasses_until >>>>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>>>> #14 0x00007ffff623ac01 in >>>>>> SystemDictionary::resolve_wk_klasses_through >>>>>> (end_id=SystemDictionary::Class_klass_knum, >>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>> __the_thread__=0x7ffff0033000) >>>>>> ??? at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>>>> #15 0x00007ffff62369b0 in >>>>>> SystemDictionary::resolve_well_known_classes >>>>>> (__the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>>>> (__the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>>>> #17 0x00007ffff62afe15 in Universe::genesis >>>>>> (__the_thread__=0x7ffff0033000) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>>>> #19 0x00007ffff5af21ed in init_globals () at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>>>> #20 0x00007ffff627feff in Threads::create_vm >>>>>> (args=0x7ffff7fc7e50, canTryAgain=0x7ffff7fc7d5b) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>>>> >>>>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>>>> >>>>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) at >>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>>>> >>>>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>>>> pthread_create.c:333 >>>>>> #27 0x00007ffff790041d in clone () at >>>>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>>>> >>>>>> This is not with CDS. >>>>>> >>>>>> Thanks >>>>>> Yumin >>>>>> >>>>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>>>> MonitorLocker. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>>>> Hi, please review: >>>>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>>>> Summary: >>>>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>>>> only needed when >>>>>>>> ? 1) bootclasspath contains zip or >>>>>>>> ? 2) UseAOT enabled or >>>>>>>> ? 3) VerifySharedArchive turned on or >>>>>>>> ? 4) CDS archives custom loaded classes >>>>>>>> ?? If none of above in java application, it is not necessary to >>>>>>>> have zip library loaded. >>>>>>>> >>>>>>>> ? Solution by loading zip library on demand when needed. >>>>>>>> >>>>>>>> Performance for java -Xint version: >>>>>>>> >>>>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>>>> ?? 1:???? 59611556??? 59450206 (-161350)????? ----- 39.799 >>>>>>>> 40.274 (? 0.475)??? ++ >>>>>>>> ?? 2:???? 59602708??? 59425234 (-177474)????? ----- 40.591 >>>>>>>> 41.183 (? 0.592)??? ++ >>>>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 >>>>>>>> 40.471 ( -0.306)????? - >>>>>>>> ?? 4:???? 59584882??? 59410155 (-174727)????? ----- 40.824 >>>>>>>> 40.233 ( -0.591)????? -- >>>>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 >>>>>>>> 40.493 (? 0.093) >>>>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 >>>>>>>> 40.064 ( -0.411)????? -- >>>>>>>> ?? 7:???? 59581820??? 59413612 (-168208)????? ----- 39.763 >>>>>>>> 40.077 (? 0.314)???? + >>>>>>>> ?? 8:???? 59593678??? 59418738 (-174940)????? ----- 40.912 >>>>>>>> 39.724 ( -1.188)????? ----- >>>>>>>> ?? 9:???? 59573058??? 59412554 (-160504)????? ----- 40.126 >>>>>>>> 40.033 ( -0.093) >>>>>>>> ? 10:???? 59591469??? 59419291 (-172178)????? ----- 40.731 >>>>>>>> 40.689 ( -0.042) >>>>>>>> ============================================================ >>>>>>>> ????????? 59589940??? 59428022 (-161917)????? ----- 40.438 >>>>>>>> 40.322 ( -0.116) >>>>>>>> instr delta =????? -161917??? -0.2717% >>>>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>>>> >>>>>>>> Tests: hs-tier1-4. >>>>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>>>> pmap list in test case: >>>>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>>>> >>>>>>>> ** >>>>>>>> * Thanks >>>>>>>> Yumin >>>>>>> >>>>>> >>>>> >>>> >>> >> > From calvin.cheung at oracle.com Tue May 5 20:58:46 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 5 May 2020 13:58:46 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> Message-ID: Looks good. thanks, Calvin On 5/5/20 1:32 PM, Yumin Qi wrote: > Hi, Ioi and Calvin > > ? There is an file classLoader.inline.hpp which also includes > atomic.hpp. I put load_zip_library_if_needed in it. Updated webrev: > http://cr.openjdk.java.net/~minqi/8237750/webrev-03/ > ? Also, double checked the function is not in libjvm.so by > ? nm libjvm.so | grep load_zip_library_if_needed > > Thanks > Yumin > > On 5/5/20 10:59 AM, Ioi Lam wrote: >> Hi Yumin, >> >> Looks good. >> >> Just small nits. No need for new webrev. >> >> [1] I think this should be named _libzip_loaded to be consistent with >> other fields >> >> ? static int? libzip_loaded;?????????? // used to sync loading zip. >> >> >> [2] This introduces dependency on atomic.hpp to classLoader.hpp >> >> ? static void load_zip_library_if_needed() { >> ??? if (Atomic::load_acquire(&libzip_loaded) == 0) { >> ????? release_load_zip_library(); >> ??? } >> ? } >> >> Since this function is used only privately in the cpp file, I think >> it's better to change the hpp to this (to reduce unnecessary header >> file dependencies) >> >> ? inline static void load_zip_library_if_needed(); >> >> and then move the implementation to the classLoader.cpp file. You >> also need to add include "runtime/atomic.hpp" to classLoader.cpp. >> >> Thanks >> - Ioi >> >> >> >> >> >> On 5/4/20 4:40 PM, Yumin Qi wrote: >>> Hi, Ioi >>> >>> ? Thanks. Updated webrev at: >>> http://cr.openjdk.java.net/~minqi/8237750/webrev-02/ >>> ? libzip_loaded now is a class member of ClassLoader, no longer a >>> file private integer since it is used in both .hpp and .cpp files. >>> >>> Thanks >>> ?Yumin >>> >>> On 5/4/20 8:31 AM, Ioi Lam wrote: >>>> Hi Yumin, >>>> >>>> ?977 void ClassLoader::load_zip_library() { >>>> ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>>> ?979???? MutexLocker locker(Zip_lock, >>>> Monitor::_no_safepoint_check_flag); >>>> ?980???? if (libzip_loaded == 0) { >>>> ?981?????? load_zip_library0(); >>>> ?982?????? Atomic::release_store(&libzip_loaded, 1); >>>> ?983???? } >>>> ?984?? } >>>> ?985 } >>>> >>>> 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { >>>> 1027?? if (libzip_loaded == 0) { >>>> 1028???? load_zip_library(); >>>> 1029?? } >>>> 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); >>>> 1031 } >>>> >>>> ClassLoader::crc32 access libzip_loaded without a memory barrier, >>>> so it may read libzip_loaded out-of-order from Crc32. This means, >>>> thread 1 may be writing the memory in this order: >>>> >>>> ??? Crc32 = ; >>>> ??? libzip_loaded = 1; >>>> >>>> but the order observed in thread 2 may be >>>> >>>> ??? libzip_loaded = 1; >>>> ???? >> ClassLoader::crc32 called here in thread 2 >>>> ??? Crc32 = ; >>>> >>>> as a result, thread 2 may read libzip_loaded = 1, but still reads a >>>> NULL from Crc32. >>>> >>>> To ensure the memory barrier is used, I think we should do this: >>>> >>>> // private >>>> inline void ClassLoader::load_zip_library_if_needed() { >>>> ? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>>> ??? release_load_zip_library(); >>>> ? } >>>> } >>>> >>>> // private >>>> void ClassLoader::release_load_zip_library() { >>>> ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); >>>> ? if (libzip_loaded == 0) { >>>> ??? load_zip_library0(); >>>> ??? Atomic::release_store(&libzip_loaded, 1); >>>> ? } >>>> } >>>> >>>> int ClassLoader::crc32(int crc, const char* buf, int len) { >>>> ? load_zip_library_if_needed(); >>>> ? return (*Crc32)(crc, (const jbyte*)buf, len); >>>> } >>>> >>>> HotSpot code uses the "release_" prefix to indicate that the >>>> function must be used as a part of an acquire/release memory >>>> barrier. (E.g., InstanceKlass::release_set_array_klasses()). >>>> >>>> Some backgrounds: >>>> https://preshing.com/20130922/acquire-and-release-fences/ >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> On 5/1/20 8:00 PM, Yumin Qi wrote: >>>>> Dean, >>>>> >>>>> ? I have updated to use MutexLocker instead at same link: >>>>> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>> >>>>> ? Tested locally, passed jtreg/runtime. >>>>> >>>>> Thanks >>>>> Yumin >>>>> >>>>> On 5/1/20 4:24 PM, Dean Long wrote: >>>>>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>>>>> >>>>>> dl >>>>>> >>>>>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>>>>> Hi, Dean >>>>>>> ? Thanks for the review. I will try MutextLocker, for AOT, I >>>>>>> think currently the tests are not using CDS then it will load >>>>>>> classes from stream, that will use libzip's Crc32. >>>>>>> ? I will check for AOT to see if it really loads libzip with the >>>>>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>>>>> >>>>>>> #0? ClassLoader::load_zip_library () at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>>>>> >>>>>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>>>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>>>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>>>>> (this=0x7ffff0245640) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>>>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>>>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, >>>>>>> cl_inst_info=..., __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>>>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>>>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>>>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>>>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>>>>> (name=0x7ffff40550f0, search_append_only=false, >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>>>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>>>>> #7? 0x00007ffff6232d0a in >>>>>>> SystemDictionary::resolve_instance_class_or_null >>>>>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>>>>> #8? 0x00007ffff623137e in >>>>>>> SystemDictionary::resolve_instance_class_or_null_helper >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>>>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>>>>> >>>>>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> protection_domain=..., throw_error=true, >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>>>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>>>>> (class_name=0x7ffff40550f0, throw_error=true, >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>>>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>>>>> (id=SystemDictionary::Object_klass_knum, >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>>>>> #13 0x00007ffff623681e in >>>>>>> SystemDictionary::resolve_wk_klasses_until >>>>>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>>>>> #14 0x00007ffff623ac01 in >>>>>>> SystemDictionary::resolve_wk_klasses_through >>>>>>> (end_id=SystemDictionary::Class_klass_knum, >>>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>>>>> #15 0x00007ffff62369b0 in >>>>>>> SystemDictionary::resolve_well_known_classes >>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>>>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>>>>> #17 0x00007ffff62afe15 in Universe::genesis >>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>>>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>>>>> #19 0x00007ffff5af21ed in init_globals () at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>>>>> #20 0x00007ffff627feff in Threads::create_vm >>>>>>> (args=0x7ffff7fc7e50, canTryAgain=0x7ffff7fc7d5b) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>>>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>>>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>>>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>>>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>>>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>>>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>>>>> >>>>>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>>>>> >>>>>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) >>>>>>> at >>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>>>>> >>>>>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>>>>> pthread_create.c:333 >>>>>>> #27 0x00007ffff790041d in clone () at >>>>>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>>>>> >>>>>>> This is not with CDS. >>>>>>> >>>>>>> Thanks >>>>>>> Yumin >>>>>>> >>>>>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>>>>> MonitorLocker. >>>>>>>> >>>>>>>> dl >>>>>>>> >>>>>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>>>>> Hi, please review: >>>>>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>>>>> Summary: >>>>>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>>>>> only needed when >>>>>>>>> ? 1) bootclasspath contains zip or >>>>>>>>> ? 2) UseAOT enabled or >>>>>>>>> ? 3) VerifySharedArchive turned on or >>>>>>>>> ? 4) CDS archives custom loaded classes >>>>>>>>> ?? If none of above in java application, it is not necessary >>>>>>>>> to have zip library loaded. >>>>>>>>> >>>>>>>>> ? Solution by loading zip library on demand when needed. >>>>>>>>> >>>>>>>>> Performance for java -Xint version: >>>>>>>>> >>>>>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>>>>> ?? 1:???? 59611556??? 59450206 (-161350) ----- 39.799 40.274 >>>>>>>>> (? 0.475)??? ++ >>>>>>>>> ?? 2:???? 59602708??? 59425234 (-177474) ----- 40.591 41.183 >>>>>>>>> (? 0.592)??? ++ >>>>>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 >>>>>>>>> 40.471 ( -0.306)????? - >>>>>>>>> ?? 4:???? 59584882??? 59410155 (-174727) ----- 40.824 40.233 ( >>>>>>>>> -0.591)????? -- >>>>>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 >>>>>>>>> 40.493 (? 0.093) >>>>>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 >>>>>>>>> 40.064 ( -0.411)????? -- >>>>>>>>> ?? 7:???? 59581820??? 59413612 (-168208) ----- 39.763 40.077 >>>>>>>>> (? 0.314)???? + >>>>>>>>> ?? 8:???? 59593678??? 59418738 (-174940) ----- 40.912 39.724 ( >>>>>>>>> -1.188)????? ----- >>>>>>>>> ?? 9:???? 59573058??? 59412554 (-160504) ----- 40.126 40.033 ( >>>>>>>>> -0.093) >>>>>>>>> ? 10:???? 59591469??? 59419291 (-172178) ----- 40.731 40.689 ( >>>>>>>>> -0.042) >>>>>>>>> ============================================================ >>>>>>>>> ????????? 59589940??? 59428022 (-161917) ----- 40.438 40.322 ( >>>>>>>>> -0.116) >>>>>>>>> instr delta =????? -161917??? -0.2717% >>>>>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>>>>> >>>>>>>>> Tests: hs-tier1-4. >>>>>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>>>>> pmap list in test case: >>>>>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>>>>> >>>>>>>>> ** >>>>>>>>> * Thanks >>>>>>>>> Yumin >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Tue May 5 21:23:02 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 5 May 2020 14:23:02 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> Message-ID: <8bbcdefe-ceeb-a9a7-af9d-ea3939df68fe@oracle.com> Hi Yumin, 258?? static void load_zip_library_if_needed(); I think "inline" should be added to the declaration. It probably doesn't matter, but we usually add 'inline' to the declaration of functions that appear in xxx_inline.hpp files. Thanks - Ioi On 5/5/20 1:32 PM, Yumin Qi wrote: > Hi, Ioi and Calvin > > ? There is an file classLoader.inline.hpp which also includes > atomic.hpp. I put load_zip_library_if_needed in it. Updated webrev: > http://cr.openjdk.java.net/~minqi/8237750/webrev-03/ > ? Also, double checked the function is not in libjvm.so by > ? nm libjvm.so | grep load_zip_library_if_needed > > Thanks > Yumin > > On 5/5/20 10:59 AM, Ioi Lam wrote: >> Hi Yumin, >> >> Looks good. >> >> Just small nits. No need for new webrev. >> >> [1] I think this should be named _libzip_loaded to be consistent with >> other fields >> >> ? static int? libzip_loaded;?????????? // used to sync loading zip. >> >> >> [2] This introduces dependency on atomic.hpp to classLoader.hpp >> >> ? static void load_zip_library_if_needed() { >> ??? if (Atomic::load_acquire(&libzip_loaded) == 0) { >> ????? release_load_zip_library(); >> ??? } >> ? } >> >> Since this function is used only privately in the cpp file, I think >> it's better to change the hpp to this (to reduce unnecessary header >> file dependencies) >> >> ? inline static void load_zip_library_if_needed(); >> >> and then move the implementation to the classLoader.cpp file. You >> also need to add include "runtime/atomic.hpp" to classLoader.cpp. >> >> Thanks >> - Ioi >> >> >> >> >> >> On 5/4/20 4:40 PM, Yumin Qi wrote: >>> Hi, Ioi >>> >>> ? Thanks. Updated webrev at: >>> http://cr.openjdk.java.net/~minqi/8237750/webrev-02/ >>> ? libzip_loaded now is a class member of ClassLoader, no longer a >>> file private integer since it is used in both .hpp and .cpp files. >>> >>> Thanks >>> ?Yumin >>> >>> On 5/4/20 8:31 AM, Ioi Lam wrote: >>>> Hi Yumin, >>>> >>>> ?977 void ClassLoader::load_zip_library() { >>>> ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>>> ?979???? MutexLocker locker(Zip_lock, >>>> Monitor::_no_safepoint_check_flag); >>>> ?980???? if (libzip_loaded == 0) { >>>> ?981?????? load_zip_library0(); >>>> ?982?????? Atomic::release_store(&libzip_loaded, 1); >>>> ?983???? } >>>> ?984?? } >>>> ?985 } >>>> >>>> 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { >>>> 1027?? if (libzip_loaded == 0) { >>>> 1028???? load_zip_library(); >>>> 1029?? } >>>> 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); >>>> 1031 } >>>> >>>> ClassLoader::crc32 access libzip_loaded without a memory barrier, >>>> so it may read libzip_loaded out-of-order from Crc32. This means, >>>> thread 1 may be writing the memory in this order: >>>> >>>> ??? Crc32 = ; >>>> ??? libzip_loaded = 1; >>>> >>>> but the order observed in thread 2 may be >>>> >>>> ??? libzip_loaded = 1; >>>> ???? >> ClassLoader::crc32 called here in thread 2 >>>> ??? Crc32 = ; >>>> >>>> as a result, thread 2 may read libzip_loaded = 1, but still reads a >>>> NULL from Crc32. >>>> >>>> To ensure the memory barrier is used, I think we should do this: >>>> >>>> // private >>>> inline void ClassLoader::load_zip_library_if_needed() { >>>> ? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>>> ??? release_load_zip_library(); >>>> ? } >>>> } >>>> >>>> // private >>>> void ClassLoader::release_load_zip_library() { >>>> ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); >>>> ? if (libzip_loaded == 0) { >>>> ??? load_zip_library0(); >>>> ??? Atomic::release_store(&libzip_loaded, 1); >>>> ? } >>>> } >>>> >>>> int ClassLoader::crc32(int crc, const char* buf, int len) { >>>> ? load_zip_library_if_needed(); >>>> ? return (*Crc32)(crc, (const jbyte*)buf, len); >>>> } >>>> >>>> HotSpot code uses the "release_" prefix to indicate that the >>>> function must be used as a part of an acquire/release memory >>>> barrier. (E.g., InstanceKlass::release_set_array_klasses()). >>>> >>>> Some backgrounds: >>>> https://preshing.com/20130922/acquire-and-release-fences/ >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> On 5/1/20 8:00 PM, Yumin Qi wrote: >>>>> Dean, >>>>> >>>>> ? I have updated to use MutexLocker instead at same link: >>>>> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>> >>>>> ? Tested locally, passed jtreg/runtime. >>>>> >>>>> Thanks >>>>> Yumin >>>>> >>>>> On 5/1/20 4:24 PM, Dean Long wrote: >>>>>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>>>>> >>>>>> dl >>>>>> >>>>>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>>>>> Hi, Dean >>>>>>> ? Thanks for the review. I will try MutextLocker, for AOT, I >>>>>>> think currently the tests are not using CDS then it will load >>>>>>> classes from stream, that will use libzip's Crc32. >>>>>>> ? I will check for AOT to see if it really loads libzip with the >>>>>>> patch. For test compiler/aot/DeoptimizationTest.java: >>>>>>> >>>>>>> #0? ClassLoader::load_zip_library () at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>>>>> >>>>>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>>>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>>>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>>>>> (this=0x7ffff0245640) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>>>>> #3? 0x00007ffff57c40ed in ClassFileParser::create_instance_klass >>>>>>> (this=0x7ffff7fc6fc0, changed_by_loadhook=false, >>>>>>> cl_inst_info=..., __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>>>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>>>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>>>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>>>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>>>>> (name=0x7ffff40550f0, search_append_only=false, >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>>>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>>>>> #7? 0x00007ffff6232d0a in >>>>>>> SystemDictionary::resolve_instance_class_or_null >>>>>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>>>>> #8? 0x00007ffff623137e in >>>>>>> SystemDictionary::resolve_instance_class_or_null_helper >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>>>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>>>>> >>>>>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>> protection_domain=..., throw_error=true, >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>>>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>>>>> (class_name=0x7ffff40550f0, throw_error=true, >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>>>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>>>>> (id=SystemDictionary::Object_klass_knum, >>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>>>>> #13 0x00007ffff623681e in >>>>>>> SystemDictionary::resolve_wk_klasses_until >>>>>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>>>>> #14 0x00007ffff623ac01 in >>>>>>> SystemDictionary::resolve_wk_klasses_through >>>>>>> (end_id=SystemDictionary::Class_klass_knum, >>>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>>> __the_thread__=0x7ffff0033000) >>>>>>> ??? at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>>>>> #15 0x00007ffff62369b0 in >>>>>>> SystemDictionary::resolve_well_known_classes >>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>>>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>>>>> #17 0x00007ffff62afe15 in Universe::genesis >>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>>>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>>>>> #19 0x00007ffff5af21ed in init_globals () at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>>>>> #20 0x00007ffff627feff in Threads::create_vm >>>>>>> (args=0x7ffff7fc7e50, canTryAgain=0x7ffff7fc7d5b) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>>>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>>>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>>>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>>>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>>>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>>>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>>>>> >>>>>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>>>>> >>>>>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) >>>>>>> at >>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>>>>> >>>>>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>>>>> pthread_create.c:333 >>>>>>> #27 0x00007ffff790041d in clone () at >>>>>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>>>>> >>>>>>> This is not with CDS. >>>>>>> >>>>>>> Thanks >>>>>>> Yumin >>>>>>> >>>>>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>>>>> MonitorLocker. >>>>>>>> >>>>>>>> dl >>>>>>>> >>>>>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>>>>> Hi, please review: >>>>>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>>>>> Summary: >>>>>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>>>>> only needed when >>>>>>>>> ? 1) bootclasspath contains zip or >>>>>>>>> ? 2) UseAOT enabled or >>>>>>>>> ? 3) VerifySharedArchive turned on or >>>>>>>>> ? 4) CDS archives custom loaded classes >>>>>>>>> ?? If none of above in java application, it is not necessary >>>>>>>>> to have zip library loaded. >>>>>>>>> >>>>>>>>> ? Solution by loading zip library on demand when needed. >>>>>>>>> >>>>>>>>> Performance for java -Xint version: >>>>>>>>> >>>>>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>>>>> ?? 1:???? 59611556??? 59450206 (-161350) ----- 39.799 40.274 >>>>>>>>> (? 0.475)??? ++ >>>>>>>>> ?? 2:???? 59602708??? 59425234 (-177474) ----- 40.591 41.183 >>>>>>>>> (? 0.592)??? ++ >>>>>>>>> ?? 3:???? 59579718??? 59441272 (-138446)????? ---- 40.777 >>>>>>>>> 40.471 ( -0.306)????? - >>>>>>>>> ?? 4:???? 59584882??? 59410155 (-174727) ----- 40.824 40.233 ( >>>>>>>>> -0.591)????? -- >>>>>>>>> ?? 5:???? 59590998??? 59447252 (-143746)????? ---- 40.400 >>>>>>>>> 40.493 (? 0.093) >>>>>>>>> ?? 6:???? 59589523??? 59441934 (-147589)????? ---- 40.475 >>>>>>>>> 40.064 ( -0.411)????? -- >>>>>>>>> ?? 7:???? 59581820??? 59413612 (-168208) ----- 39.763 40.077 >>>>>>>>> (? 0.314)???? + >>>>>>>>> ?? 8:???? 59593678??? 59418738 (-174940) ----- 40.912 39.724 ( >>>>>>>>> -1.188)????? ----- >>>>>>>>> ?? 9:???? 59573058??? 59412554 (-160504) ----- 40.126 40.033 ( >>>>>>>>> -0.093) >>>>>>>>> ? 10:???? 59591469??? 59419291 (-172178) ----- 40.731 40.689 ( >>>>>>>>> -0.042) >>>>>>>>> ============================================================ >>>>>>>>> ????????? 59589940??? 59428022 (-161917) ----- 40.438 40.322 ( >>>>>>>>> -0.116) >>>>>>>>> instr delta =????? -161917??? -0.2717% >>>>>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>>>>> >>>>>>>>> Tests: hs-tier1-4. >>>>>>>>> Due to zip library not loaded at default, I removed 'zip' from >>>>>>>>> pmap list in test case: >>>>>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>>>>> >>>>>>>>> ** >>>>>>>>> * Thanks >>>>>>>>> Yumin >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From mikael.vidstedt at oracle.com Tue May 5 21:34:34 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 5 May 2020 14:34:34 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> Message-ID: <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> All good points! I deliberately chose *not* to update comments where it wasn?t immediately 100% obvious exactly how to update them. For example, in many cases I found that the comments are already incomplete or stale, and for each such case we?ll want to consider how exactly to update the comment (remove/switch to Unix/etc.). I would prefer to handle this as follow-up(s), separate from the JEP. Does that sounds reasonable? Apart from the comments - do the changes look good? If so, can I count you as a reviewer? Cheers, Mikael > On May 4, 2020, at 12:20 AM, serguei.spitsyn at oracle.com wrote: > > HI Mikael, > > Some quick comments. > > Some extra references to Solaris/solaris, SunOS or SPARC are listed below: > > src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) > src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) > src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) > > src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent > src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. > src/hotspot/share/prims/jvmti.xml: On the Solaris Operating Environment, an agent library is a shared > src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating > src/hotspot/share/prims/jvmti.xml: example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris > src/hotspot/share/prims/methodHandles.cpp:#undef CS // Solaris builds complain > > > Thanks, > Serguei > > > On 5/3/20 22:12, Mikael Vidstedt wrote: >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> >> >> Background: >> >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> >> >> Testing: >> >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> >> Cheers, >> Mikael >> >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >> > From yumin.qi at oracle.com Tue May 5 22:30:02 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 5 May 2020 15:30:02 -0700 Subject: (S) 8237750: Load libzip.so only if necessary In-Reply-To: <8bbcdefe-ceeb-a9a7-af9d-ea3939df68fe@oracle.com> References: <3d8c886b-ac12-ad39-e3c3-43f36b570f01@oracle.com> <54cdc1ed-baaa-d46e-626b-6caf60e0825a@oracle.com> <64ed5d10-7500-da2c-91e0-84f6b280b3a8@oracle.com> <921eea6b-e92e-7e99-e376-bff32c3af9fe@oracle.com> <02454183-8899-6178-a9d2-8ab7030d524c@oracle.com> <00cd070b-7ae2-9355-e3cc-6633a0b07460@oracle.com> <948e391a-ed33-9a7e-eae2-8aa4a04aef93@oracle.com> <8bbcdefe-ceeb-a9a7-af9d-ea3939df68fe@oracle.com> Message-ID: <310992f5-d266-5d05-5815-7b74155ae8d8@oracle.com> Ioi, Calvin ? Thanks. The inline sometime prefix to function declaration sometimes not. I will put inline without updating webrev. Thanks Yumin On 5/5/20 2:23 PM, Ioi Lam wrote: > Hi Yumin, > > 258?? static void load_zip_library_if_needed(); > > I think "inline" should be added to the declaration. It probably > doesn't matter, but we usually add 'inline' to the declaration of > functions that appear in xxx_inline.hpp files. > > Thanks > - Ioi > > On 5/5/20 1:32 PM, Yumin Qi wrote: >> Hi, Ioi and Calvin >> >> ? There is an file classLoader.inline.hpp which also includes >> atomic.hpp. I put load_zip_library_if_needed in it. Updated webrev: >> http://cr.openjdk.java.net/~minqi/8237750/webrev-03/ >> ? Also, double checked the function is not in libjvm.so by >> ? nm libjvm.so | grep load_zip_library_if_needed >> >> Thanks >> Yumin >> >> On 5/5/20 10:59 AM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> Looks good. >>> >>> Just small nits. No need for new webrev. >>> >>> [1] I think this should be named _libzip_loaded to be consistent >>> with other fields >>> >>> ? static int? libzip_loaded;?????????? // used to sync loading zip. >>> >>> >>> [2] This introduces dependency on atomic.hpp to classLoader.hpp >>> >>> ? static void load_zip_library_if_needed() { >>> ??? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>> ????? release_load_zip_library(); >>> ??? } >>> ? } >>> >>> Since this function is used only privately in the cpp file, I think >>> it's better to change the hpp to this (to reduce unnecessary header >>> file dependencies) >>> >>> ? inline static void load_zip_library_if_needed(); >>> >>> and then move the implementation to the classLoader.cpp file. You >>> also need to add include "runtime/atomic.hpp" to classLoader.cpp. >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> >>> On 5/4/20 4:40 PM, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? Thanks. Updated webrev at: >>>> http://cr.openjdk.java.net/~minqi/8237750/webrev-02/ >>>> ? libzip_loaded now is a class member of ClassLoader, no longer a >>>> file private integer since it is used in both .hpp and .cpp files. >>>> >>>> Thanks >>>> ?Yumin >>>> >>>> On 5/4/20 8:31 AM, Ioi Lam wrote: >>>>> Hi Yumin, >>>>> >>>>> ?977 void ClassLoader::load_zip_library() { >>>>> ?978?? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>>>> ?979???? MutexLocker locker(Zip_lock, >>>>> Monitor::_no_safepoint_check_flag); >>>>> ?980???? if (libzip_loaded == 0) { >>>>> ?981?????? load_zip_library0(); >>>>> ?982?????? Atomic::release_store(&libzip_loaded, 1); >>>>> ?983???? } >>>>> ?984?? } >>>>> ?985 } >>>>> >>>>> 1026 int ClassLoader::crc32(int crc, const char* buf, int len) { >>>>> 1027?? if (libzip_loaded == 0) { >>>>> 1028???? load_zip_library(); >>>>> 1029?? } >>>>> 1030?? return (*Crc32)(crc, (const jbyte*)buf, len); >>>>> 1031 } >>>>> >>>>> ClassLoader::crc32 access libzip_loaded without a memory barrier, >>>>> so it may read libzip_loaded out-of-order from Crc32. This means, >>>>> thread 1 may be writing the memory in this order: >>>>> >>>>> ??? Crc32 = ; >>>>> ??? libzip_loaded = 1; >>>>> >>>>> but the order observed in thread 2 may be >>>>> >>>>> ??? libzip_loaded = 1; >>>>> ???? >> ClassLoader::crc32 called here in thread 2 >>>>> ??? Crc32 = ; >>>>> >>>>> as a result, thread 2 may read libzip_loaded = 1, but still reads >>>>> a NULL from Crc32. >>>>> >>>>> To ensure the memory barrier is used, I think we should do this: >>>>> >>>>> // private >>>>> inline void ClassLoader::load_zip_library_if_needed() { >>>>> ? if (Atomic::load_acquire(&libzip_loaded) == 0) { >>>>> ??? release_load_zip_library(); >>>>> ? } >>>>> } >>>>> >>>>> // private >>>>> void ClassLoader::release_load_zip_library() { >>>>> ? MutexLocker locker(Zip_lock, Monitor::_no_safepoint_check_flag); >>>>> ? if (libzip_loaded == 0) { >>>>> ??? load_zip_library0(); >>>>> ??? Atomic::release_store(&libzip_loaded, 1); >>>>> ? } >>>>> } >>>>> >>>>> int ClassLoader::crc32(int crc, const char* buf, int len) { >>>>> ? load_zip_library_if_needed(); >>>>> ? return (*Crc32)(crc, (const jbyte*)buf, len); >>>>> } >>>>> >>>>> HotSpot code uses the "release_" prefix to indicate that the >>>>> function must be used as a part of an acquire/release memory >>>>> barrier. (E.g., InstanceKlass::release_set_array_klasses()). >>>>> >>>>> Some backgrounds: >>>>> https://preshing.com/20130922/acquire-and-release-fences/ >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> On 5/1/20 8:00 PM, Yumin Qi wrote: >>>>>> Dean, >>>>>> >>>>>> ? I have updated to use MutexLocker instead at same link: >>>>>> http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>> >>>>>> ? Tested locally, passed jtreg/runtime. >>>>>> >>>>>> Thanks >>>>>> Yumin >>>>>> >>>>>> On 5/1/20 4:24 PM, Dean Long wrote: >>>>>>> OK, I didn't realize compute_fingerprint is using ZIP_CRC32. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> On 5/1/20 2:42 PM, Yumin Qi wrote: >>>>>>>> Hi, Dean >>>>>>>> ? Thanks for the review. I will try MutextLocker, for AOT, I >>>>>>>> think currently the tests are not using CDS then it will load >>>>>>>> classes from stream, that will use libzip's Crc32. >>>>>>>> ? I will check for AOT to see if it really loads libzip with >>>>>>>> the patch. For test compiler/aot/DeoptimizationTest.java: >>>>>>>> >>>>>>>> #0? ClassLoader::load_zip_library () at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:978 >>>>>>>> >>>>>>>> #1? 0x00007ffff57d4693 in ClassLoader::crc32 (crc=0, >>>>>>>> buf=0x7ffff0244ee0 "\312\376\272\276", len=1888) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1028 >>>>>>>> >>>>>>>> #2? 0x00007ffff57cef5d in ClassFileStream::compute_fingerprint >>>>>>>> (this=0x7ffff0245640) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileStream.cpp:80 >>>>>>>> #3? 0x00007ffff57c40ed in >>>>>>>> ClassFileParser::create_instance_klass (this=0x7ffff7fc6fc0, >>>>>>>> changed_by_loadhook=false, cl_inst_info=..., >>>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classFileParser.cpp:5630 >>>>>>>> >>>>>>>> #4? 0x00007ffff5dea647 in KlassFactory::create_from_stream >>>>>>>> (stream=0x7ffff0245640, name=0x7ffff40550f0, >>>>>>>> loader_data=0x7ffff022dbc0, cl_info=..., >>>>>>>> __the_thread__=0x7ffff0033000) >>>>>>>> ??? at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/klassFactory.cpp:207 >>>>>>>> #5? 0x00007ffff57d53e4 in ClassLoader::load_class >>>>>>>> (name=0x7ffff40550f0, search_append_only=false, >>>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/classLoader.cpp:1285 >>>>>>>> #6? 0x00007ffff6234fcf in SystemDictionary::load_instance_class >>>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1550 >>>>>>>> #7? 0x00007ffff6232d0a in >>>>>>>> SystemDictionary::resolve_instance_class_or_null >>>>>>>> (name=0x7ffff40550f0, class_loader=..., protection_domain=..., >>>>>>>> __the_thread__=0x7ffff0033000) >>>>>>>> ??? at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:882 >>>>>>>> #8? 0x00007ffff623137e in >>>>>>>> SystemDictionary::resolve_instance_class_or_null_helper >>>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) >>>>>>>> ??? at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:302 >>>>>>>> #9? 0x00007ffff623124c in SystemDictionary::resolve_or_null >>>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>>> protection_domain=..., __the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:285 >>>>>>>> >>>>>>>> #10 0x00007ffff6230f57 in SystemDictionary::resolve_or_fail >>>>>>>> (class_name=0x7ffff40550f0, class_loader=..., >>>>>>>> protection_domain=..., throw_error=true, >>>>>>>> __the_thread__=0x7ffff0033000) >>>>>>>> ??? at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:233 >>>>>>>> #11 0x00007ffff62311eb in SystemDictionary::resolve_or_fail >>>>>>>> (class_name=0x7ffff40550f0, throw_error=true, >>>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:275 >>>>>>>> #12 0x00007ffff6236722 in SystemDictionary::resolve_wk_klass >>>>>>>> (id=SystemDictionary::Object_klass_knum, >>>>>>>> __the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2029 >>>>>>>> #13 0x00007ffff623681e in >>>>>>>> SystemDictionary::resolve_wk_klasses_until >>>>>>>> (limit_id=SystemDictionary::Cloneable_klass_knum, >>>>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>>>> __the_thread__=0x7ffff0033000) >>>>>>>> ??? at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2039 >>>>>>>> #14 0x00007ffff623ac01 in >>>>>>>> SystemDictionary::resolve_wk_klasses_through >>>>>>>> (end_id=SystemDictionary::Class_klass_knum, >>>>>>>> start_id=@0x7ffff7fc79d4: SystemDictionary::Object_klass_knum, >>>>>>>> __the_thread__=0x7ffff0033000) >>>>>>>> ??? at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.hpp:418 >>>>>>>> #15 0x00007ffff62369b0 in >>>>>>>> SystemDictionary::resolve_well_known_classes >>>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:2082 >>>>>>>> #16 0x00007ffff6236517 in SystemDictionary::initialize >>>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/classfile/systemDictionary.cpp:1985 >>>>>>>> #17 0x00007ffff62afe15 in Universe::genesis >>>>>>>> (__the_thread__=0x7ffff0033000) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:326 >>>>>>>> #18 0x00007ffff62b1e51 in universe2_init () at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/memory/universe.cpp:847 >>>>>>>> #19 0x00007ffff5af21ed in init_globals () at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/init.cpp:128 >>>>>>>> #20 0x00007ffff627feff in Threads::create_vm >>>>>>>> (args=0x7ffff7fc7e50, canTryAgain=0x7ffff7fc7d5b) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/runtime/thread.cpp:3879 >>>>>>>> #21 0x00007ffff5bf537d in JNI_CreateJavaVM_inner >>>>>>>> (vm=0x7ffff7fc7ea8, penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) >>>>>>>> at /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3789 >>>>>>>> #22 0x00007ffff5bf5677 in JNI_CreateJavaVM (vm=0x7ffff7fc7ea8, >>>>>>>> penv=0x7ffff7fc7eb0, args=0x7ffff7fc7e50) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/hotspot/share/prims/jni.cpp:3872 >>>>>>>> #23 0x00007ffff7bca4c9 in InitializeJVM (pvm=0x7ffff7fc7ea8, >>>>>>>> penv=0x7ffff7fc7eb0, ifn=0x7ffff7fc7f00) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:1538 >>>>>>>> >>>>>>>> #24 0x00007ffff7bc70b5 in JavaMain (_args=0x7fffffffab10) at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/share/native/libjli/java.c:417 >>>>>>>> >>>>>>>> #25 0x00007ffff7bceb5f in ThreadJavaMain (args=0x7fffffffab10) >>>>>>>> at >>>>>>>> /home/minqi/ws/jdk15/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 >>>>>>>> >>>>>>>> #26 0x00007ffff71c56ba in start_thread (arg=0x7ffff7fc8700) at >>>>>>>> pthread_create.c:333 >>>>>>>> #27 0x00007ffff790041d in clone () at >>>>>>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 >>>>>>>> >>>>>>>> This is not with CDS. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Yumin >>>>>>>> >>>>>>>> On 5/1/20 2:11 PM, Dean Long wrote: >>>>>>>>> It looks like Zip_lock could be a MutexLocker instead of a >>>>>>>>> MonitorLocker. >>>>>>>>> >>>>>>>>> dl >>>>>>>>> >>>>>>>>> On 5/1/20 10:24 AM, Yumin Qi wrote: >>>>>>>>>> Hi, please review: >>>>>>>>>> bug 8237750: https://bugs.openjdk.java.net/browse/JDK-8237750 >>>>>>>>>> webrev: http://cr.openjdk.java.net/~minqi/8237750/webrev-01/ >>>>>>>>>> Summary: >>>>>>>>>> ? zip library is loaded unconditionally at start up but it is >>>>>>>>>> only needed when >>>>>>>>>> ? 1) bootclasspath contains zip or >>>>>>>>>> ? 2) UseAOT enabled or >>>>>>>>>> ? 3) VerifySharedArchive turned on or >>>>>>>>>> ? 4) CDS archives custom loaded classes >>>>>>>>>> ?? If none of above in java application, it is not necessary >>>>>>>>>> to have zip library loaded. >>>>>>>>>> >>>>>>>>>> ? Solution by loading zip library on demand when needed. >>>>>>>>>> >>>>>>>>>> Performance for java -Xint version: >>>>>>>>>> >>>>>>>>>> Results of " perf stat -r 50 bin/java -Xshare:on >>>>>>>>>> -XX:SharedArchiveFile=jdk2.jsa -Xint -version " >>>>>>>>>> ?? 1:???? 59611556??? 59450206 (-161350) ----- 39.799 40.274 >>>>>>>>>> (? 0.475)??? ++ >>>>>>>>>> ?? 2:???? 59602708??? 59425234 (-177474) ----- 40.591 41.183 >>>>>>>>>> (? 0.592)??? ++ >>>>>>>>>> ?? 3:???? 59579718??? 59441272 (-138446) ---- 40.777 40.471 ( >>>>>>>>>> -0.306)????? - >>>>>>>>>> ?? 4:???? 59584882??? 59410155 (-174727) ----- 40.824 40.233 >>>>>>>>>> ( -0.591)????? -- >>>>>>>>>> ?? 5:???? 59590998??? 59447252 (-143746) ---- 40.400 40.493 >>>>>>>>>> (? 0.093) >>>>>>>>>> ?? 6:???? 59589523??? 59441934 (-147589) ---- 40.475 40.064 ( >>>>>>>>>> -0.411)????? -- >>>>>>>>>> ?? 7:???? 59581820??? 59413612 (-168208) ----- 39.763 40.077 >>>>>>>>>> (? 0.314)???? + >>>>>>>>>> ?? 8:???? 59593678??? 59418738 (-174940) ----- 40.912 39.724 >>>>>>>>>> ( -1.188)????? ----- >>>>>>>>>> ?? 9:???? 59573058??? 59412554 (-160504) ----- 40.126 40.033 >>>>>>>>>> ( -0.093) >>>>>>>>>> ? 10:???? 59591469??? 59419291 (-172178) ----- 40.731 40.689 >>>>>>>>>> ( -0.042) >>>>>>>>>> ============================================================ >>>>>>>>>> ????????? 59589940??? 59428022 (-161917) ----- 40.438 40.322 >>>>>>>>>> ( -0.116) >>>>>>>>>> instr delta =????? -161917??? -0.2717% >>>>>>>>>> time? delta =?????? -0.116 ms -0.2859% >>>>>>>>>> >>>>>>>>>> Tests: hs-tier1-4. >>>>>>>>>> Due to zip library not loaded at default, I removed 'zip' >>>>>>>>>> from pmap list in test case: >>>>>>>>>> *test/hotspot/jtreg/serviceability/sa/ClhsdbPmap.java >>>>>>>>>> >>>>>>>>>> ** >>>>>>>>>> * Thanks >>>>>>>>>> Yumin >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From serguei.spitsyn at oracle.com Tue May 5 23:48:32 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 5 May 2020 16:48:32 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> Message-ID: <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> Hi Mikael, The fixes in webrev look good to me. I've just noticed a couple of more serviceability related things can be missed. (Not sure if they are included into different chunk of fixes.) One is libjvm_db.so which is for Solaris Pstack support: ? make/hotspot/lib/CompileDtraceLibraries.gmk:??? # Note that libjvm_db.c has tests for COMPILER2, but this was never set by ? make/hotspot/lib/CompileDtraceLibraries.gmk: LIBJVM_DB_OUTPUTDIR := $(JVM_VARIANT_OUTPUTDIR)/libjvm_db ? make/hotspot/lib/CompileDtraceLibraries.gmk:??????? NAME := jvm_db, \ ? make/hotspot/lib/CompileDtraceLibraries.gmk:??????? SRC := $(TOPDIR)/src/java.base/solaris/native/libjvm_db, \ Another is DTrace support which also includes jhelper.d (support for DTrace jstack action which is for Solaris only): ? make/hotspot/gensrc/GensrcDtrace.gmk ? make/hotspot/lib/JvmDtraceObjects.gmk It also impacts some other make files. Also, these lines can be just removed: ? open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#ifndef _JAVASOFT_SOLARIS_PATH_MD_H_ ? open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#define _JAVASOFT_SOLARIS_PATH_MD_H_ ? open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#endif /* !_JAVASOFT_SOLARIS_PATH_MD_H_ */ Thanks, Serguei On 5/5/20 14:34, Mikael Vidstedt wrote: > All good points! I deliberately chose *not* to update comments where it wasn?t immediately 100% obvious exactly how to update them. For example, in many cases I found that the comments are already incomplete or stale, and for each such case we?ll want to consider how exactly to update the comment (remove/switch to Unix/etc.). I would prefer to handle this as follow-up(s), separate from the JEP. Does that sounds reasonable? > > Apart from the comments - do the changes look good? If so, can I count you as a reviewer? > > Cheers, > Mikael > > >> On May 4, 2020, at 12:20 AM, serguei.spitsyn at oracle.com wrote: >> >> HI Mikael, >> >> Some quick comments. >> >> Some extra references to Solaris/solaris, SunOS or SPARC are listed below: >> >> src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) >> src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) >> src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) >> >> src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent >> src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. >> src/hotspot/share/prims/jvmti.xml: On the Solaris Operating Environment, an agent library is a shared >> src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating >> src/hotspot/share/prims/jvmti.xml: example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris >> src/hotspot/share/prims/methodHandles.cpp:#undef CS // Solaris builds complain >> >> >> Thanks, >> Serguei >> >> >> On 5/3/20 22:12, Mikael Vidstedt wrote: >>> Please review this change which implements part of JEP 381: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >>> >>> >>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>> >>> >>> Background: >>> >>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>> >>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>> >>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>> >>> >>> Testing: >>> >>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>> >>> Cheers, >>> Mikael >>> >>> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>> From mikael.vidstedt at oracle.com Wed May 6 00:04:53 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 5 May 2020 17:04:53 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> Message-ID: <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> > On May 5, 2020, at 4:48 PM, serguei.spitsyn at oracle.com wrote: > > Hi Mikael, > > The fixes in webrev look good to me. > > I've just noticed a couple of more serviceability related things can be missed. > (Not sure if they are included into different chunk of fixes.) > > One is libjvm_db.so which is for Solaris Pstack support: > make/hotspot/lib/CompileDtraceLibraries.gmk: # Note that libjvm_db.c has tests for COMPILER2, but this was never set by > make/hotspot/lib/CompileDtraceLibraries.gmk: LIBJVM_DB_OUTPUTDIR := $(JVM_VARIANT_OUTPUTDIR)/libjvm_db > make/hotspot/lib/CompileDtraceLibraries.gmk: NAME := jvm_db, \ > make/hotspot/lib/CompileDtraceLibraries.gmk: SRC := $(TOPDIR)/src/java.base/solaris/native/libjvm_db, \ > > Another is DTrace support which also includes jhelper.d (support for DTrace jstack action which is for Solaris only): > make/hotspot/gensrc/GensrcDtrace.gmk > make/hotspot/lib/JvmDtraceObjects.gmk > It also impacts some other make files. I believe you?ll find that these changes were included in the build system patch: https://mail.openjdk.java.net/pipermail/build-dev/2020-May/027342.html http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/build/open/webrev/ Let me know if I missed something. > Also, these lines can be just removed: > open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#ifndef _JAVASOFT_SOLARIS_PATH_MD_H_ > open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#define _JAVASOFT_SOLARIS_PATH_MD_H_ > open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#endif /* !_JAVASOFT_SOLARIS_PATH_MD_H_ */ Remove or rather rename? The corresponding Windows header also has the guard, any particular reason why it should be *removed*? Cheers, Mikael > > Thanks, > Serguei > > > On 5/5/20 14:34, Mikael Vidstedt wrote: >> All good points! I deliberately chose *not* to update comments where it wasn?t immediately 100% obvious exactly how to update them. For example, in many cases I found that the comments are already incomplete or stale, and for each such case we?ll want to consider how exactly to update the comment (remove/switch to Unix/etc.). I would prefer to handle this as follow-up(s), separate from the JEP. Does that sounds reasonable? >> >> Apart from the comments - do the changes look good? If so, can I count you as a reviewer? >> >> Cheers, >> Mikael >> >> >>> On May 4, 2020, at 12:20 AM, serguei.spitsyn at oracle.com wrote: >>> >>> HI Mikael, >>> >>> Some quick comments. >>> >>> Some extra references to Solaris/solaris, SunOS or SPARC are listed below: >>> >>> src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) >>> src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) >>> src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) >>> >>> src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent >>> src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. >>> src/hotspot/share/prims/jvmti.xml: On the Solaris Operating Environment, an agent library is a shared >>> src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating >>> src/hotspot/share/prims/jvmti.xml: example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris >>> src/hotspot/share/prims/methodHandles.cpp:#undef CS // Solaris builds complain >>> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 5/3/20 22:12, Mikael Vidstedt wrote: >>>> Please review this change which implements part of JEP 381: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >>>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >>>> >>>> >>>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>>> >>>> >>>> Background: >>>> >>>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>>> >>>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>>> >>>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>>> >>>> >>>> Testing: >>>> >>>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>>> >>>> Cheers, >>>> Mikael >>>> >>>> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>>> > From serguei.spitsyn at oracle.com Wed May 6 00:33:18 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 5 May 2020 17:33:18 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> Message-ID: On 5/5/20 17:04, Mikael Vidstedt wrote: > >> On May 5, 2020, at 4:48 PM, serguei.spitsyn at oracle.com wrote: >> >> Hi Mikael, >> >> The fixes in webrev look good to me. >> >> I've just noticed a couple of more serviceability related things can be missed. >> (Not sure if they are included into different chunk of fixes.) >> >> One is libjvm_db.so which is for Solaris Pstack support: >> make/hotspot/lib/CompileDtraceLibraries.gmk: # Note that libjvm_db.c has tests for COMPILER2, but this was never set by >> make/hotspot/lib/CompileDtraceLibraries.gmk: LIBJVM_DB_OUTPUTDIR := $(JVM_VARIANT_OUTPUTDIR)/libjvm_db >> make/hotspot/lib/CompileDtraceLibraries.gmk: NAME := jvm_db, \ >> make/hotspot/lib/CompileDtraceLibraries.gmk: SRC := $(TOPDIR)/src/java.base/solaris/native/libjvm_db, \ >> >> Another is DTrace support which also includes jhelper.d (support for DTrace jstack action which is for Solaris only): >> make/hotspot/gensrc/GensrcDtrace.gmk >> make/hotspot/lib/JvmDtraceObjects.gmk >> It also impacts some other make files. > I believe you?ll find that these changes were included in the build system patch: > > https://mail.openjdk.java.net/pipermail/build-dev/2020-May/027342.html > http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/build/open/webrev/ > > Let me know if I missed something. Thanks, I'll look at at. >> Also, these lines can be just removed: >> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#ifndef _JAVASOFT_SOLARIS_PATH_MD_H_ >> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#define _JAVASOFT_SOLARIS_PATH_MD_H_ >> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#endif /* !_JAVASOFT_SOLARIS_PATH_MD_H_ */ > Remove or rather rename? The corresponding Windows header also has the guard, any particular reason why it should be *removed*? You are right, we have to rename it: _JAVASOFT_SOLARIS_PATH_MD_H_ => _UNIX_PATH_MD_H_ No need in another webrev. Thanks, Serguei > Cheers, > Mikael > >> Thanks, >> Serguei >> >> >> On 5/5/20 14:34, Mikael Vidstedt wrote: >>> All good points! I deliberately chose *not* to update comments where it wasn?t immediately 100% obvious exactly how to update them. For example, in many cases I found that the comments are already incomplete or stale, and for each such case we?ll want to consider how exactly to update the comment (remove/switch to Unix/etc.). I would prefer to handle this as follow-up(s), separate from the JEP. Does that sounds reasonable? >>> >>> Apart from the comments - do the changes look good? If so, can I count you as a reviewer? >>> >>> Cheers, >>> Mikael >>> >>> >>>> On May 4, 2020, at 12:20 AM, serguei.spitsyn at oracle.com wrote: >>>> >>>> HI Mikael, >>>> >>>> Some quick comments. >>>> >>>> Some extra references to Solaris/solaris, SunOS or SPARC are listed below: >>>> >>>> src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) >>>> src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) >>>> src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) >>>> >>>> src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent >>>> src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. >>>> src/hotspot/share/prims/jvmti.xml: On the Solaris Operating Environment, an agent library is a shared >>>> src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating >>>> src/hotspot/share/prims/jvmti.xml: example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris >>>> src/hotspot/share/prims/methodHandles.cpp:#undef CS // Solaris builds complain >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/3/20 22:12, Mikael Vidstedt wrote: >>>>> Please review this change which implements part of JEP 381: >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >>>>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >>>>> >>>>> >>>>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>>>> >>>>> >>>>> Background: >>>>> >>>>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>>>> >>>>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>>>> >>>>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>>>> >>>>> >>>>> Testing: >>>>> >>>>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>>>> >>>>> Cheers, >>>>> Mikael >>>>> >>>>> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>>>> From serguei.spitsyn at oracle.com Wed May 6 06:09:18 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 5 May 2020 23:09:18 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> Message-ID: <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Wed May 6 10:27:43 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 6 May 2020 10:27:43 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, I had a look at your change. It's complex, but not that big. A lot of code is just passing info through layers of abstraction. Also, one can tell this went through some iterations by now, I think it's very well engineered. I had a look at webrev.05 Unfortunately "8242425: JVMTI monitor operations should use Thread-Local Handshakes" breaks webrev.05. I updated to before that change and took that as base of my review. I see four parts of the change that can be looked at rather individually. * Refactoring the scopeDesc constructors. Trivial. * Persisting information about the optimizations done by the compilers. Large and mostly trivial. * Deoptimizing. The most complicated part. Really well abstracted, though. * DeoptimizeObjectsALot for testing and the tests. Review of compiler changes: I understand you annotate at safepoints where the escape analysis finds out that an object is "better" than global escape. This are the cases where the analysis identifies optimization opportunities. These annotations are then used to deoptimize frames and the objects referenced by them. Doesn't this overestimate the optimized objects? E.g., eliminate_alloc_node has many cases where it bails out. c1_IR.hpp OK, nothing to do for C1, just adapt to extended method signature. Break line once more so that it matches above line length. ciEnv.h|cpp Pass through another jvmti capability. Trivial & good. debugInfoRec.hpp Pass through escape info that must be recorded. OK. pcDesc.hpp I would like to see some documentation of the methods. Maybe: // There is an object in the scope that does not escape globally. // It either does not escape at all or it escapes as arguemnt. and // One of the arguments is an object that is not globally visible // but escapes to the callee. scopeDesc.cpp Besides refactoring copy escape info from pcDesc to scopeDesc and add accessors. Trivial. In scopeDesc.hpp you talk about NoEscape and ArgEscape. This are opto terms, but scopeDesc is a shared datastructure that does not depend on a specific compiler. Please explain what is going on without using these terms. jvmciCodeInstaller.cpp OK, nothing for JVMCI. Here support for Object Optimizations for JVMCI compilers could be added. Leave this to graal people. callnode.hpp You add functionality to annotate callnodes with escape information This is carried through code generation to final output where it is added to the compiled methods meta information. At Safepoints in general jvmti can access - Objects that were scalar replaced. They must be reallocated. (Flag EliminateAllocations) - Objects that should be locked but are not because they never escape the thread. They need to be relocked. At calls, Objects where locks have been removed escape to callees. We must persist this information so that if jvmti accesses the object in a callee, we can determine by looking at the caller that it needs to be relocked. A side comment: I think the flage handling in Opto is not very intuitive. DoEscapeAnalysis depends on the jvmti capabilities. This makes no sense. It is only an analysis. The optimizations should depend on the jvmti capabilities. The correct setup would be to handle this in CompilerConfig::ergo_initialize(): If the jvmti capabilities allow, enable the optimizations EliminateAllocations or EliminateLocks/EliminateNestedLocks. If one of these optimizations is on, enable EscapeAnalysis. -- end side comment. So I would propose the following comments: // In the scope of this safepoints there are objects // that do not globally escape. They are either NoEscape or // ArgEscape. As such, they might be subject to optimizations. // Persist this information here so that the frame an the // Objects in scope can // be deoptimized if jvmti accesses an object at this safepoint. void set_not_global_escape_in_scope(bool b) { // This call passes objects that do not globally escape // to its callee. The object might be subject to optimization, // e.g. a lock might be omitted. Persist this information here // so that on a jvmti access to the callee frame we can deoptimize // the object and this frame. void set_arg_escape(bool f) { _arg_escape = f; } Actuall I am not sure whether the name of these fields (and all the others in the course of this change) should refer to escape analysis. I think the term "Object deoptimization" you also use is much better. You could call these properties (througout the whole change) set_optimized_objects_in_scope() and set_passes_optimized_objects(). I think this would make the whole matter much easier to understand. Anyways, locks can already be removed without running escape analysis at all. C2 recognizes some local patterns that allow this. escape.h|cpp The code looks good. Line 325: The comment could be a bit more elaborate: // Annotate at safepoints if they have <= ArgEscape objects in their // scope. Additionally, if the safepoint is a java call, annotate // whether it passes ArgEscape objects as parameters. And maybe add these comments?: // Returns true if an oop in the scope of sfn does not escape // globally. bool ConnectionGraph::has_not_global_escape_in_scope(SafePointNode* sfn) { // Returns true if at least one of the arguments to the call is an oop // that does not escape globally. bool ConnectionGraph::has_arg_escape(CallJavaNode* call) { General question: You collect the information you want to annotate to the method during escape analysis. Don't you overestimate the optimized objects by this? E.g. elimination of allocations does bail out for various reasons. At the end, no optimization might have happened, but then during runtime the frame is deoptimized nevertheless. machnode.hpp: Extends MachSafePointNode similar to the ideal version. Good. matcher.cpp Copy info from ideal to mach node. good. output.cpp Now finally the information is written to the debug info. Good. --------------------------------------------------------- So now let's have a look at the runtime part (including relaxing constraints to escape analysis): rootResolver.cpp Adapt to changed interface. good. c2compiler.cpp / macro.cpp Make EscpaeAnlysis independent of jvmti capabilities. Good. jvmtiEnv.cpp/jvmtiEnvBase.cpp You add deoptimization of objects where they are accessed. good. jvmtiImpl.cpp In deoptimize_objects, you check for DoEscapeAnalysis. This is correct given the current design of the flag handling in the compiler. It's not really nice to have a dependency to C2 here, though. I understand it's an optimization, the code could be run anyways, it would check but not find anything. But actually I would excpect dependencies on EliminateLocks and EliminateAllocations (if they were set according to jvmti capabilitiers as I elaborated above.) Would it make sense to protect the ArgEscape loop by if (EliminateLocks)? jvmtiTagMap.cpp Deoptimize for jvmti operations. Good. deoptimization.cpp I guess this is the core of your work. You add a new mode that just deoptimizes objects but not frames. Good idea. You have to use reallocated objects in upper frames, or by jvmti accesses to inner frames, which can not easily be replaced by interpreter frames. This way you can wait with replacing the frame until just before execution returns. eliminate_allocations(): (Strange method name, should at least be in past tense, even better reallocate_eliminated_allocations() or allocate_scalarized_objects(). Confused me until I groked the code. Legacy though, not your business.) It's not that nice to return whether you only deoptimized objects by the boolean reference argument. After all, it again depends on the mode you pass in. A different design would be to clone the method and have an eliminate_allocations_no_unpack() variant, but that would not be better as some code would be duplicated. Maybe a comment for argument eliminate_allocations: // deoptimized_objects is set to true if objects were deoptimized // but not the frame. It is unchanged if there are no objects to // be deoptimized, or if the frame was deoptim Similar for eliminate_locks(): // deoptimized_objects is set to true if objects were relocked, // else it is left unchanged. You reuse and extend the existing realloc/relock_objects, but extended it. deoptimize_objects_internal() Simple version of fetch_unroll_info_helper for EscapeBarrier. Good. I attributed the comment "Then relock objects if synchronization on them was eliminated." to the if() just below. Add an empty line to make clear the comment refers to the next 10 lines. Alternatively, replace the whole comment by // At first, reallocate the non-escaping objects and restore their fields // so they are available for relocking. And add // Now relock objects with eliminated locks. befor the if ((DoEscape... below. In fetch_unroll_info_helper, I don't understand why you need && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { for eliminated locks, but not for skalar replaced objects? I would guess it is because the eliminated locks can be applied to argEscape, but scalar replacement only to noescape objects? I.e. it might have been done before? But why isn't this the case for eliminate_allocations? deoptimize_objects_internal does both unconditionally, so both can happen to inner frames, right? relock_objects() Ok, you need to undo biased locking. Also, you remember the lock nesting for later relocking if waiting for lock. revoke_for_object_deoptimization() I like if boolean operators are at the beginning of broken lines, but I think hotspot convention is to have them at the end. Code will get much more simple if BiasedLocking is removed. EscapeBarrier:: ... (This class maybe would qualify for a file of its own.) deoptimize_objects() I would mention escape analysis only as side remark. Also, as I understand, there is only one frame at given depth? // Deoptimize frames with optimized objects. This can be omitted locks and // objects not allocated but replaced by scalars. In C2, these optimizations // are based on escape analysis. // Up to depth, deoptimize frames with any optimized objects. // From depth to entry_frame, deoptimize only frames that // pass optimized objects to their callees. (First part similar for the comment above EscapeBarrier::deoptimize_objects_internal().) What is the check (cur_depth <= depth) good for? Can you ever walk past entry_frame? Isn't vf->is_compiled_frame() prerequisite that "Move to next physical frame" is needed? You could move it into the other check. If so, similar for deoptimize_objects_all_threads(). Syncronization: looks good. I think others had a look at this before. EscapeBarrier::deoptimize_objects_internal() The method name is misleading, it is not used by deoptimize_objects(). Also, method with the same name is in Deopitmization. Proposal: deoptimize_objects_thread() ? C1 stubs: this really shows you tested all configurations, great! mutexLocker: ok. objectMonitor.cpp: ok stackValue.hpp Is this missing clearing a bug? thread.hpp I would remove "_ea" from the flag and method names. Renaming deferred_locals to deferred_updates is good, as well as adding a datastructure for it. (Adding this data structure might be a breakout, too.) good. thread.cpp good. vframe.cpp Is this a bug in existing code? Makes sense. vframe_hp.hpp (What stands _hp for? helper? The file should be named compiledVFrame ...) not_global_escape_in_scope() ... Again, you mention escape analysis here. Comments above hold, too. You introduce JvmtiDeferredUpdates. Good. vframe_hp.cpp Changes for JvmtiDeferredUpdates, escape state accessors, line 422: Would an assertion assert(!info->owner_is_scalar_replaced(), ...) hold here? macros.hpp Good. Test coding ============ compileBroker.h|cpp You introduce a third class of threads handled here and add a new flag to distinguish it. Before, the two kinds of threads were distinguished implicitly by passing in a compiler for compiler threads. The new thread kind is only used for testing in debug. make_thread: You could assert (comp != NULL...) to assure previous conditions. line 989 indentation broken escape.cpp You enable the optimization in case of testruns. good. whitebox.cpp ok. deoptimization.cpp deoptimize_objects_alot_loop() Good. globals.hpp Nice docu of flags, but pleas mention "for testing purposes" or the like in DeoptimizeObjectsALot. I would place the flags next to each other. interfaceSupport.cpp: good. I'll look at the test themselves in an extra mail (learning from Martin ??) Best regards, Goetz. > -----Original Message----- > From: Reingruber, Richard > Sent: Wednesday, April 1, 2020 8:15 AM > To: Doerr, Martin ; 'Robbin Ehn' > ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) ; > serviceability-dev at openjdk.java.net; hotspot-compiler- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Martin, > > > thanks for addressing all my points. I've looked over webrev.5 and I'm > satisfied with your changes. > > Thanks! > > > I had also promised to review the tests. > > Thanks++ > I appreciate it very much, the tests are many lines of code. > > > test/jdk/com/sun/jdi/EATests.java > > This is a substantial amount of tests which is appropriate for a such a large > change. Skipping some subtests with UseJVMCICompiler makes sense > because it doesn't provide the necessary JVMTI functionality, yet. > > Nice work! > > I also like that you test with and without BiasedLocking. Your tests will still > be fine after BiasedLocking deprecation. > > Hope so :) > > > Very minor nits: > > - 2 typos in comment above EARelockingNestedInflatedTarget: "lockes are > ommitted" (sounds funny) > > - You sometimes write "graal" and sometimes "Graal". I guess the capital G > is better. (Also in EATestsJVMCI.java.) > > > test/jdk/com/sun/jdi/EATestsJVMCI.java > > EATests with Graal enabled. Nice that you support Graal to some extent. > Maybe Graal folks want to enhance them in the future. I think this is a good > starting point. > > Will change this in the next webrev. > > > Conclusion: Looks good and not trivial :-) > > Now, you have one full review. I'd be ok with covering 2nd review by partial > reviews. > > Compiler and JVMTI parts are not too complicated IMHO. > > Runtime part should get at least one additional careful review. > > Thanks a lot, > Richard. > > -----Original Message----- > From: Doerr, Martin > Sent: Dienstag, 31. M?rz 2020 16:01 > To: Reingruber, Richard ; 'Robbin Ehn' > ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) ; > serviceability-dev at openjdk.java.net; hotspot-compiler- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Richard, > > thanks for addressing all my points. I've looked over webrev.5 and I'm > satisfied with your changes. > > > I had also promised to review the tests. > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysis > Enabled.java > Thanks for updating the @summary comment. Looks good in webrev.5. > > test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnaly > sisEnabled.c > JVMTI agent for object tagging and heap iteration. Good. > > test/jdk/com/sun/jdi/EATests.java > This is a substantial amount of tests which is appropriate for a such a large > change. Skipping some subtests with UseJVMCICompiler makes sense > because it doesn't provide the necessary JVMTI functionality, yet. > Nice work! > I also like that you test with and without BiasedLocking. Your tests will still be > fine after BiasedLocking deprecation. > > Very minor nits: > - 2 typos in comment above EARelockingNestedInflatedTarget: "lockes are > ommitted" (sounds funny) > - You sometimes write "graal" and sometimes "Graal". I guess the capital G is > better. (Also in EATestsJVMCI.java.) > > test/jdk/com/sun/jdi/EATestsJVMCI.java > EATests with Graal enabled. Nice that you support Graal to some extent. > Maybe Graal folks want to enhance them in the future. I think this is a good > starting point. > > > Conclusion: Looks good and not trivial :-) > Now, you have one full review. I'd be ok with covering 2nd review by partial > reviews. > Compiler and JVMTI parts are not too complicated IMHO. > Runtime part should get at least one additional careful review. > > Best regards, > Martin > > > > -----Original Message----- > > From: Reingruber, Richard > > Sent: Montag, 30. M?rz 2020 10:32 > > To: Doerr, Martin ; 'Robbin Ehn' > > ; Lindenmaier, Goetz > > ; David Holmes > ; > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > ; serviceability-dev at openjdk.java.net; > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > dev at openjdk.java.net > > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > > in the Presence of JVMTI Agents > > > > Hi, > > > > this is webrev.5 based on Robbin's feedback and Martin's review - thanks! :) > > > > The change affects jvmti, hotspot and c2. Partial reviews are very welcome > > too. > > > > Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/ > > Delta: > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5.inc/ > > > > Robbin, Martin, please let me know, if anything shouldn't be quite as you > > wanted it. Also find my > > comments on your feedback below. > > > > Robbin, can I count you as Reviewer for the runtime part? > > > > Thanks, Richard. > > > > -- > > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > > You can move both declaration and definition to that file, no need to > > clobber > > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > Done. > > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in > it's > > own > > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > I moved JvmtiDeferredUpdates to vframe_hp.hpp where preexisting > > jvmtiDeferredLocalVariableSet is > > declared. > > > > > src/hotspot/share/code/compiledMethod.cpp > > > Nice cleanup! > > > > Thanks :) > > > > > src/hotspot/share/code/debugInfoRec.cpp > > > src/hotspot/share/code/debugInfoRec.hpp > > > Additional parmeters. (Remark: I think "non_global_escape_in_scope" > > would read better than "not_global_escape_in_scope", but your version is > > consistent with existing code, so no change request from my side.) Ok. > > > > I've been thinking about this too and finally stayed with > > not_global_escape_in_scope. It's supposed > > to mean an object whose escape state is not GlobalEscape is in scope. > > > > > src/hotspot/share/compiler/compileBroker.cpp > > > src/hotspot/share/compiler/compileBroker.hpp > > > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into > > a follow up change together with the test in order to make this webrev > > smaller, but since it is included, I'm reviewing everything at once. Not a big > > deal.) Ok. > > > > Yes the change would be a little smaller. And if it helps I'll split it off. In > > general I prefer > > patches that bring along a suitable amount of tests. > > > > > src/hotspot/share/opto/c2compiler.cpp > > > Make do_escape_analysis independent of JVMCI capabilities. Nice! > > > > It is the main goal of the enhancement. It is done for C2, but could be done > > for JVMCI compilers > > with just a small effort as well. > > > > > src/hotspot/share/opto/escape.cpp > > > Annotation for MachSafePointNodes. Your added functionality looks > > correct. > > > But I'd prefer to move the bulky code out of the large function. > > > I suggest to factor out something like has_not_global_escape and > > has_arg_escape. So the code could look like this: > > > SafePointNode* sfn = sfn_worklist.at(next); > > > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > > > if (sfn->is_CallJava()) { > > > CallJavaNode* call = sfn->as_CallJava(); > > > call->set_arg_escape(has_arg_escape(call)); > > > } > > > This would also allow us to get rid of the found_..._escape_in_args > > variables making the loops better readable. > > > > Done. > > > > > It's kind of ugly to use strcmp to recognize uncommon trap, but that > seems > > to be the way to do it (there are more such places). So it's ok. > > > > Yeah. I copied the snippet. > > > > > src/hotspot/share/prims/jvmtiImpl.cpp > > > src/hotspot/share/prims/jvmtiImpl.hpp > > > The sequence is pretty complex: > > > VM_GetOrSetLocal element initialization executes EscapeBarrier code > > which suspends the target thread (extra VM Operation). > > > > Note that the target threads have to be suspended already for > > VM_GetOrSetLocal*. So it's mainly the > > synchronization effect of EscapeBarrier::sync_and_suspend_one() that is > > required here. Also no extra > > _handshake_ is executed, since sync_and_suspend_one() will find the > > target threads already > > suspended. > > > > > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by > VM > > Thread to prepare VM Operation with frame deoptimization). > > > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor > > which resumes the target thread. > > > But I don't have any improvement proposal. Performance is probably not > a > > concern, here. So it's ok. > > > > > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it > > has non-globally escaping objects and other frames if they have arg > escaping > > ones. Good. > > > > It's not specifically the top frame, but the frame that is accessed. > > > > > src/hotspot/share/runtime/deoptimization.cpp > > > Object deoptimization. I have more comments and proposals, here. > > > First of all, handling recursive and waiting locks in relock_objects is tricky, > > but looks correct. > > > Comments are sufficient to understand why things are done as they are > > implemented. > > > > > BiasedLocking related parts are complex, but we may get rid of them in > the > > future (with BiasedLocking removal). > > > Anyway, looks correct, too. > > > > > Typo in comment: "regularily" => "regularly" > > > > > Deoptimization::fetch_unroll_info_helper is the only place where > > _jvmti_deferred_updates get deallocated (except JavaThread destructor). > > But I think we always go through it, so I can't see a memory leak or such > kind > > of issues. > > > > That's correct. The compiled frame for which deferred updates are > allocated > > is always deoptimized > > before (see EscapeBarrier::deoptimize_objects()). This is also asserted in > > compiledVFrame::update_deferred_value(). I've added the same assertion > > to > > Deoptimization::relock_objects(). So we can be sure that > > _jvmti_deferred_updates are deallocated > > again in fetch_unroll_info_helper(). > > > > > EscapeBarrier::deoptimize_objects: ResourceMark should use > > calling_thread(). > > > > Sure, well spotted! > > > > > You can use MutexLocker and MonitorLocker with Thread* to save the > > Thread::current() call. > > > > Right, good hint. This was recently introduced with 8235678. I even had to > > resolve conflicts. Should > > have done this then. > > > > > I'd make set_objs_are_deoptimized static and remove it from the > > EscapeBarrier interface because I think it shouldn't be used outside of > > EscapeBarrier::deoptimize_objects. > > > > Done. > > > > > Typo in comment: "we must only deoptimize" => "we only have to > > deoptimize" > > > > Replaced with "[...] we deoptimize iff local objects are passed as args" > > > > > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and > > barrier_active() is redundant. Implementation can get moved to hpp file. > > > > Ok. Done. > > > > > I'll get back to suspend flags, later. > > > > > There are weird cases regarding _self_deoptimization_in_progress. > > > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. > > C can set _self_deoptimization_in_progress while A performs the > handshake > > for suspending C. I think this doesn't lead to errors, but it's probably not > > desired. > > > I think it would be better to use only one "wait" call in > > sync_and_suspend_one and sync_and_suspend_all. > > > > You're right. We've discussed that face-to-face, but couldn't find a real > issue. > > But now, thinking again, a reckon I found one: > > > > 2808 // Sync with other threads that might be doing deoptimizations > > 2809 { > > 2810 // Need to switch to _thread_blocked for the wait() call > > 2811 ThreadBlockInVM tbivm(_calling_thread); > > 2812 MonitorLocker ml(EscapeBarrier_lock, > > Mutex::_no_safepoint_check_flag); > > 2813 while (_self_deoptimization_in_progress) { > > 2814 ml.wait(); > > 2815 } > > 2816 > > 2817 if (self_deopt()) { > > 2818 _self_deoptimization_in_progress = true; > > 2819 } > > 2820 > > 2821 while (_deoptee_thread->is_ea_obj_deopt_suspend()) { > > 2822 ml.wait(); > > 2823 } > > 2824 > > 2825 if (self_deopt()) { > > 2826 return; > > 2827 } > > 2828 > > 2829 // set suspend flag for target thread > > 2830 _deoptee_thread->set_ea_obj_deopt_flag(); > > 2831 } > > > > - A waits in 2822 > > - C is suspended > > - B notifies all in resume_one() > > - A and C wake up > > - C wins over A and sets _self_deoptimization_in_progress = true in 2818 > > - C does the self deoptimization > > - A executes 2830 _deoptee_thread->set_ea_obj_deopt_flag() > > > > C will self suspend at some undefined point. The resulting state is illegal. > > > > > I first thought it'd be better to move ThreadBlockInVM before wait() to > > reduce thread state transitions, but that seems to be problematic because > > ThreadBlockInVM destructor contains a safepoint check which we > shouldn't > > do while holding EscapeBarrier_lock. So no change request. > > > > Yes, would be nice to have the state change only if needed, but for the > > reason you mentioned it is > > not quite as easy as it seems to be. I experimented as well with a second > > lock, but did not succeed. > > > > > Change in thred_added: > > > I think the sequence would be more comprehensive if we waited for > > deopt_all_threads in Thread::start and all other places where a new thread > > can run into Java code (e.g. JVMTI attach). > > > Your version makes new threads come up with suspend flag set. That > looks > > correct, too. Advantage is that you only have to change one place > > (thread_added). It'll be interesting to see how it will look like when we use > > async handshakes instead of suspend flags. > > > For now, I'm ok with your version. > > > > I had a version that did what you are suggesting. The current version also > has > > the advantage, that > > there are fewer places where a thread has to wait for ongoing object > > deoptimization. This means > > viewer places where you have to worry about correct thread state > > transitions, possible deadlocks, > > and if all oops are properly Handle'ed. > > > > > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt- > > >is_hidden_from_external_view()). > > > > Done. > > > > > Having 4 different deoptimize_objects functions makes it a little hard to > > keep an overview of which one is used for what. > > > Maybe adding suffixes would help a little bit, but I can also live with what > > you have. > > > Implementation looks correct to me. > > > > 2 are internal. I added the suffix _internal to them. This leaves 2 to choose > > from. > > > > > src/hotspot/share/runtime/deoptimization.hpp > > > Escape barriers and object deoptimization functions. > > > Typo in comment: "helt" => "held" > > > > Done in place already. > > > > > src/hotspot/share/runtime/interfaceSupport.cpp > > > InterfaceSupport::deoptimizeAllObjects() is only used for > > DeoptimizeObjectsALot = 1. > > > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not > bad > > to have DeoptimizeObjectsALot = 1 in addition. Ok. > > > > I never used DeoptimizeObjectsALot = 1 that much. It could be more > > deterministic in single threaded > > scenarios. I wouldn't object to get rid of it though. > > > > > src/hotspot/share/runtime/stackValue.hpp > > > Better reinitilization in StackValue. Good. > > > > StackValue::obj_is_scalar_replaced() should not return true after calling > > set_obj(). > > > > > src/hotspot/share/runtime/thread.cpp > > > src/hotspot/share/runtime/thread.hpp > > > src/hotspot/share/runtime/thread.inline.hpp > > > wait_for_object_deoptimization, suspend flag, deferred updates and test > > feature to deoptimize objects. > > > > > In the long term, we want to get rid of suspend flags, so it's not so nice to > > introduce a new one. But I agree with G?tz that it should be acceptable as > > temporary solution until async handshakes are available (which takes more > > time). So I'm ok with your change. > > > > I'm keen to build the feature on async handshakes when the arive. > > > > > You can use MutexLocker with Thread*. > > > > Done. > > > > > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class > > out of thread.hpp. > > > > Done. > > > > > src/hotspot/share/runtime/vframe.cpp > > > Added support for entry frame to new_vframe. Ok. > > > > > > > src/hotspot/share/runtime/vframe_hp.cpp > > > src/hotspot/share/runtime/vframe_hp.hpp > > > > > I think code()->as_nmethod() in not_global_escape_in_scope() and > > arg_escape() should better be under #ifdef ASSERT or inside the assert > > statement (no need for code cache walking in product build). > > > > Done. > > > > > jvmtiDeferredLocalVariableSet::update_monitors: > > > Please add a comment explaining that owner referenced by original info > > may be scalar replaced, but it is deoptimized in the vframe. > > > > Done. > > > > -----Original Message----- > > From: Doerr, Martin > > Sent: Donnerstag, 12. M?rz 2020 17:28 > > To: Reingruber, Richard ; 'Robbin Ehn' > > ; Lindenmaier, Goetz > > ; David Holmes > ; > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > ; serviceability-dev at openjdk.java.net; > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > dev at openjdk.java.net > > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > > in the Presence of JVMTI Agents > > > > Hi Richard, > > > > > > I managed to find time for a (almost) complete review of webrev.4. (I'll > > review the tests separately.) > > > > First of all, the change seems to be in pretty good quality for its significant > > complexity. I couldn't find any real bugs. But I'd like to propose minor > > improvements. > > I'm convinced that it's mature because we did substantial testing. > > > > I like the new functionality for object deoptimization. It can possibly be > > reused for future escape analysis based optimizations. So I appreciate > having > > it available in the code base. > > In addition to that, your change makes the JVMTI implementation better > > integrated into the VM. > > > > > > Now to the details: > > > > > > src/hotspot/share/c1/c1_IR.hpp > > describe_scope parameters. Ok. > > > > > > src/hotspot/share/ci/ciEnv.cpp > > src/hotspot/share/ci/ciEnv.hpp > > Fix for JvmtiExport::can_walk_any_space() capability. Ok. > > > > > > src/hotspot/share/code/compiledMethod.cpp > > Nice cleanup! > > > > > > src/hotspot/share/code/debugInfoRec.cpp > > src/hotspot/share/code/debugInfoRec.hpp > > Additional parmeters. (Remark: I think "non_global_escape_in_scope" > > would read better than "not_global_escape_in_scope", but your version is > > consistent with existing code, so no change request from my side.) Ok. > > > > > > src/hotspot/share/code/nmethod.cpp > > Nice cleanup! > > > > > > src/hotspot/share/code/pcDesc.hpp > > Additional parameters. Ok. > > > > > > src/hotspot/share/code/scopeDesc.cpp > > src/hotspot/share/code/scopeDesc.hpp > > Improved implementation + additional parameters. Ok. > > > > > > src/hotspot/share/compiler/compileBroker.cpp > > src/hotspot/share/compiler/compileBroker.hpp > > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a > > follow up change together with the test in order to make this webrev > > smaller, but since it is included, I'm reviewing everything at once. Not a big > > deal.) Ok. > > > > > > src/hotspot/share/jvmci/jvmciCodeInstaller.cpp > > Additional parameters. Ok. > > > > > > src/hotspot/share/opto/c2compiler.cpp > > Make do_escape_analysis independent of JVMCI capabilities. Nice! > > > > > > src/hotspot/share/opto/callnode.hpp > > Additional fields for MachSafePointNodes. Ok. > > > > > > src/hotspot/share/opto/escape.cpp > > Annotation for MachSafePointNodes. Your added functionality looks > correct. > > But I'd prefer to move the bulky code out of the large function. > > I suggest to factor out something like has_not_global_escape and > > has_arg_escape. So the code could look like this: > > SafePointNode* sfn = sfn_worklist.at(next); > > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > > if (sfn->is_CallJava()) { > > CallJavaNode* call = sfn->as_CallJava(); > > call->set_arg_escape(has_arg_escape(call)); > > } > > This would also allow us to get rid of the found_..._escape_in_args > variables > > making the loops better readable. > > > > It's kind of ugly to use strcmp to recognize uncommon trap, but that seems > > to be the way to do it (there are more such places). So it's ok. > > > > > > src/hotspot/share/opto/machnode.hpp > > Additional fields for MachSafePointNodes. Ok. > > > > > > src/hotspot/share/opto/macro.cpp > > Allow elimination of non-escaping allocations. Ok. > > > > > > src/hotspot/share/opto/matcher.cpp > > src/hotspot/share/opto/output.cpp > > Copy attribute / pass parameters. Ok. > > > > > > src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp > > Nice cleanup! > > > > > > src/hotspot/share/prims/jvmtiEnv.cpp > > src/hotspot/share/prims/jvmtiEnvBase.cpp > > Escape barriers + deoptimize objects for target thread. Good. > > > > > > src/hotspot/share/prims/jvmtiImpl.cpp > > src/hotspot/share/prims/jvmtiImpl.hpp > > The sequence is pretty complex: > > VM_GetOrSetLocal element initialization executes EscapeBarrier code > which > > suspends the target thread (extra VM Operation). > > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM > > Thread to prepare VM Operation with frame deoptimization). > > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor > which > > resumes the target thread. > > But I don't have any improvement proposal. Performance is probably not a > > concern, here. So it's ok. > > > > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has > > non-globally escaping objects and other frames if they have arg escaping > > ones. Good. > > > > > > src/hotspot/share/prims/jvmtiTagMap.cpp > > Escape barriers + deoptimize objects for all threads. Ok. > > > > > > src/hotspot/share/prims/whitebox.cpp > > Added WB_IsFrameDeoptimized to API. Ok. > > > > > > src/hotspot/share/runtime/deoptimization.cpp > > Object deoptimization. I have more comments and proposals, here. > > First of all, handling recursive and waiting locks in relock_objects is tricky, > but > > looks correct. > > Comments are sufficient to understand why things are done as they are > > implemented. > > > > BiasedLocking related parts are complex, but we may get rid of them in the > > future (with BiasedLocking removal). > > Anyway, looks correct, too. > > > > Typo in comment: "regularily" => "regularly" > > > > Deoptimization::fetch_unroll_info_helper is the only place where > > _jvmti_deferred_updates get deallocated (except JavaThread destructor). > > But I think we always go through it, so I can't see a memory leak or such > kind > > of issues. > > > > EscapeBarrier::deoptimize_objects: ResourceMark should use > > calling_thread(). > > > > You can use MutexLocker and MonitorLocker with Thread* to save the > > Thread::current() call. > > > > I'd make set_objs_are_deoptimized static and remove it from the > > EscapeBarrier interface because I think it shouldn't be used outside of > > EscapeBarrier::deoptimize_objects. > > > > Typo in comment: "we must only deoptimize" => "we only have to > > deoptimize" > > > > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and > > barrier_active() is redundant. Implementation can get moved to hpp file. > > > > I'll get back to suspend flags, later. > > > > There are weird cases regarding _self_deoptimization_in_progress. > > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. > C > > can set _self_deoptimization_in_progress while A performs the handshake > > for suspending C. I think this doesn't lead to errors, but it's probably not > > desired. > > I think it would be better to use only one "wait" call in > > sync_and_suspend_one and sync_and_suspend_all. > > > > I first thought it'd be better to move ThreadBlockInVM before wait() to > > reduce thread state transitions, but that seems to be problematic because > > ThreadBlockInVM destructor contains a safepoint check which we > shouldn't > > do while holding EscapeBarrier_lock. So no change request. > > > > Change in thred_added: > > I think the sequence would be more comprehensive if we waited for > > deopt_all_threads in Thread::start and all other places where a new thread > > can run into Java code (e.g. JVMTI attach). > > Your version makes new threads come up with suspend flag set. That looks > > correct, too. Advantage is that you only have to change one place > > (thread_added). It'll be interesting to see how it will look like when we use > > async handshakes instead of suspend flags. > > For now, I'm ok with your version. > > > > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt- > > >is_hidden_from_external_view()). > > > > Having 4 different deoptimize_objects functions makes it a little hard to > keep > > an overview of which one is used for what. > > Maybe adding suffixes would help a little bit, but I can also live with what > you > > have. > > Implementation looks correct to me. > > > > > > src/hotspot/share/runtime/deoptimization.hpp > > Escape barriers and object deoptimization functions. > > Typo in comment: "helt" => "held" > > > > > > src/hotspot/share/runtime/globals.hpp > > Addition of develop flag DeoptimizeObjectsALotInterval. Ok. > > > > > > src/hotspot/share/runtime/interfaceSupport.cpp > > InterfaceSupport::deoptimizeAllObjects() is only used for > > DeoptimizeObjectsALot = 1. > > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad > > to have DeoptimizeObjectsALot = 1 in addition. Ok. > > > > > > src/hotspot/share/runtime/interfaceSupport.inline.hpp > > Addition of deoptimizeAllObjects. Ok. > > > > > > src/hotspot/share/runtime/mutexLocker.cpp > > src/hotspot/share/runtime/mutexLocker.hpp > > Addition of EscapeBarrier_lock. Ok. > > > > > > src/hotspot/share/runtime/objectMonitor.cpp > > Make recursion count relock aware. Ok. > > > > > > src/hotspot/share/runtime/stackValue.hpp > > Better reinitilization in StackValue. Good. > > > > > > src/hotspot/share/runtime/thread.cpp > > src/hotspot/share/runtime/thread.hpp > > src/hotspot/share/runtime/thread.inline.hpp > > wait_for_object_deoptimization, suspend flag, deferred updates and test > > feature to deoptimize objects. > > > > In the long term, we want to get rid of suspend flags, so it's not so nice to > > introduce a new one. But I agree with G?tz that it should be acceptable as > > temporary solution until async handshakes are available (which takes more > > time). So I'm ok with your change. > > > > You can use MutexLocker with Thread*. > > > > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class > out > > of thread.hpp. > > > > > > src/hotspot/share/runtime/vframe.cpp > > Added support for entry frame to new_vframe. Ok. > > > > > > src/hotspot/share/runtime/vframe_hp.cpp > > src/hotspot/share/runtime/vframe_hp.hpp > > > > I think code()->as_nmethod() in not_global_escape_in_scope() and > > arg_escape() should better be under #ifdef ASSERT or inside the assert > > statement (no need for code cache walking in product build). > > > > jvmtiDeferredLocalVariableSet::update_monitors: > > Please add a comment explaining that owner referenced by original info > may > > be scalar replaced, but it is deoptimized in the vframe. > > > > > > src/hotspot/share/utilities/macros.hpp > > Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. > > > > > > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysi > > sEnabled.java > > > test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnal > > ysisEnabled.c > > New test. Will review separately. > > > > > > test/jdk/TEST.ROOT > > Addition of vm.jvmci as required property. Ok. > > > > > > test/jdk/com/sun/jdi/EATests.java > > test/jdk/com/sun/jdi/EATestsJVMCI.java > > New test. Will review separately. > > > > > > test/lib/sun/hotspot/WhiteBox.java > > Added isFrameDeoptimized to API. Ok. > > > > > > That was it. Best regards, > > Martin > > > > > > > -----Original Message----- > > > From: hotspot-compiler-dev > > bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > > > Sent: Dienstag, 3. M?rz 2020 21:23 > > > To: 'Robbin Ehn' ; Lindenmaier, Goetz > > > ; David Holmes > > ; > > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > > ; serviceability-dev at openjdk.java.net; > > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > > dev at openjdk.java.net > > > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better > > > Performance in the Presence of JVMTI Agents > > > > > > Hi Robbin, > > > > > > > > I understand that Robbin proposed to replace the usage of > > > > > _suspend_flag with handshakes. Apparently, async handshakes > > > > > are needed to do so. We have been waiting a while for removal > > > > > of the _suspend_flag / introduction of async handshakes [2]. > > > > > What is the status here? > > > > > > > I have an old prototype which I would like to continue to work on. > > > > So do not assume asynch handshakes will make 15. > > > > Even if it would, I think there are a lot more investigate work to remove > > > > _suspend_flag. > > > > > > Let us know, if we can be of any help to you and be it only testing. > > > > > > > >> Full: > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > > > You can move both declaration and definition to that file, no need to > > > clobber > > > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > > > Will do. > > > > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in > > it's > > > own > > > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > > > You are right. It shouldn't be declared in thread.hpp. I will look into that. > > > > > > > Note that we also think we may have a bug in deopt: > > > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > > > > > I think it would be best, if possible, to push after that is resolved. > > > > > > Sure. > > > > > > > Not even nearly a full review :) > > > > > > I know :) > > > > > > Anyways, thanks a lot, > > > Richard. > > > > > > > > > -----Original Message----- > > > From: Robbin Ehn > > > Sent: Monday, March 2, 2020 11:17 AM > > > To: Lindenmaier, Goetz ; Reingruber, > > Richard > > > ; David Holmes > > ; > > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > > ; serviceability-dev at openjdk.java.net; > > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > > dev at openjdk.java.net > > > Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance > > > in the Presence of JVMTI Agents > > > > > > Hi, > > > > > > On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > > > > Hi, > > > > > > > > I had a look at the progress of this change. Nothing > > > > happened since Richard posted his update using more > > > > handshakes [1]. > > > > But we (SAP) would appreciate a lot if this change could > > > > be successfully reviewed and pushed. > > > > > > > > I think there is basic understanding that this > > > > change is helpful. It fixes a number of issues with JVMTI, > > > > and will deliver the same performance benefits as EA > > > > does in current production mode for debugging scenarios. > > > > > > > > This is important for us as we run our VMs prepared > > > > for debugging in production mode. > > > > > > > > I understand that Robbin proposed to replace the usage of > > > > _suspend_flag with handshakes. Apparently, async handshakes > > > > are needed to do so. We have been waiting a while for removal > > > > of the _suspend_flag / introduction of async handshakes [2]. > > > > What is the status here? > > > > > > I have an old prototype which I would like to continue to work on. > > > So do not assume asynch handshakes will make 15. > > > Even if it would, I think there are a lot more investigate work to remove > > > _suspend_flag. > > > > > > > > > > > I think we should no longer wait, but proceed with > > > > this change. We will look into removing the usage of > > > > suspend_flag introduced here once it is possible to implement > > > > it with handshakes. > > > > > > Yes, sure. > > > > > > >> Full: > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > > You can move both declaration and definition to that file, no need to > > clobber > > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in > it's > > > own > > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > > > Note that we also think we may have a bug in deopt: > > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > > > > I think it would be best, if possible, to push after that is resolved. > > > > > > Not even nearly a full review :) > > > > > > Thanks, Robbin > > > > > > > > > >> Incremental: > > > >> > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ > > > >> > > > >> I was not able to eliminate the additional suspend flag now. I'll take > care > > > of this > > > >> as soon as the > > > >> existing suspend-resume-mechanism is reworked. > > > >> > > > >> Testing: > > > >> > > > >> Nightly tests @SAP: > > > >> > > > >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > > > Renaissance > > > >> Suite, SAP specific tests > > > >> with fastdebug and release builds on all platforms > > > >> > > > >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x > > > parallel > > > >> for 24h > > > >> > > > >> Thanks, Richard. > > > >> > > > >> > > > >> More details on the changes: > > > >> > > > >> * Hide DeoptimizeObjectsALotThread from external view. > > > >> > > > >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. > > > >> It used to be _safepoint_check_sometimes, which will be eliminated > > > sooner or > > > >> later. > > > >> I added explicit thread state changes with ThreadBlockInVM to code > > > paths > > > >> where we can wait() > > > >> on EscapeBarrier_lock to become safepoint safe. > > > >> > > > >> * Use handshake EscapeBarrierSuspendHandshake to suspend target > > > threads > > > >> instead of vm operation > > > >> VM_ThreadSuspendAllForObjDeopt. > > > >> > > > >> * Removed uses of Threads_lock. When adding a new thread we > > suspend > > > it iff > > > >> EA optimizations are > > > >> being reverted. In the previous version we were waiting on > > > Threads_lock > > > >> while EA optimizations > > > >> were reverted. See EscapeBarrier::thread_added(). > > > >> > > > >> * Made tests require Xmixed compilation mode. > > > >> > > > >> * Made tests agnostic regarding tiered compilation. > > > >> I.e. tc isn't disabled anymore, and the tests can be run with tc > enabled > > or > > > >> disabled. > > > >> > > > >> * Exercising EATests.java as well with stress test options > > > >> DeoptimizeObjectsALot* > > > >> Due to the non-deterministic deoptimizations some tests need to be > > > skipped. > > > >> We do this to prevent bit-rot of the stress test code. > > > >> > > > >> * Executing EATests.java as well with graal if available. Driver for this is > > > >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not > > > provide all > > > >> the new debug info > > > >> (namely not_global_escape_in_scope and arg_escape in > > > scopeDesc.hpp). > > > >> And graal does not yet support the JVMTI operations force early > > return > > > and > > > >> pop frame. > > > >> > > > >> * Removed tracing from new jdi tests in EATests.java. Too much trace > > > output > > > >> before the debugging > > > >> connection is established can cause deadlock because output buffers > > fill > > > up. > > > >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) > > > >> > > > >> * Many copyright year changes and smaller clean-up changes of > testing > > > code > > > >> (trailing white-space and > > > >> the like). > > > >> > > > >> > > > >> -----Original Message----- > > > >> From: David Holmes > > > >> Sent: Donnerstag, 19. Dezember 2019 03:12 > > > >> To: Reingruber, Richard ; serviceability- > > > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > > > hotspot- > > > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > > > (vladimir.kozlov at oracle.com) > > > >> > > > >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > Performance in > > > >> the Presence of JVMTI Agents > > > >> > > > >> Hi Richard, > > > >> > > > >> I think my issue is with the way EliminateNestedLocks works so I'm > going > > > >> to look into that more deeply. > > > >> > > > >> Thanks for the explanations. > > > >> > > > >> David > > > >> > > > >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: > > > >>> Hi David, > > > >>> > > > >>> > > > Some further queries/concerns: > > > >>> > > > > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp > > > >>> > > > > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: > > > >>> > > > > > > >>> > > > ! _recursions = save // restore the old recursion count > > > >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // > > > >>> > > > increased by the deferred relock count > > > >>> > > > > > > >>> > > > what is the "deferred relock count"? I gather it relates to > > > >>> > > > > > > >>> > > > "The code was extended to be able to deoptimize objects of > a > > > >>> > > frame that > > > >>> > > > is not the top frame and to let another thread than the > > owning > > > >>> > > thread do > > > >>> > > > it." > > > >>> > > > > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, > > > when a > > > >> compiled frame is > > > >>> > > replaced with corresponding interpreter frames. Part of this is > > > relocking > > > >> objects with eliminated > > > >>> > > locking. New with the enhancement is that we do this also just > > > before > > > >> object references are > > > >>> > > acquired through JVMTI. In this case we deoptimize also the > > > owning > > > >> compiled frame C and we > > > >>> > > register deoptimized objects as deferred updates. When control > > > returns > > > >> to C it gets deoptimized, > > > >>> > > we notice that objects are already deoptimized (reallocated and > > > >> relocked), so we don't do it again > > > >>> > > (relocking twice would be incorrect of course). Deferred > updates > > > are > > > >> copied into the new > > > >>> > > interpreter frames. > > > >>> > > > > > >>> > > Problem: relocking is not possible if the target thread T is > waiting > > > on the > > > >> monitor that needs to > > > >>> > > be relocked. This happens only with non-local objects with > > > >> EliminateNestedLocks. Instead relocking > > > >>> > > is deferred until T owns the monitor again. This is what the > piece > > of > > > >> code above does. > > > >>> > > > > >>> > Sorry I need some more detail here. How can you wait() on an > > > object > > > >>> > monitor if the object allocation and/or locking was optimised > > away? > > > And > > > >>> > what is a "non-local object" in this context? Isn't EA restricted to > > > >>> > thread-confined objects? > > > >>> > > > >>> "Non-local object" is an object that escapes its thread. The issue I'm > > > >> addressing with the changes > > > >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused > by > > > >> EliminateNestedLocks, where C2 > > > >>> eliminates recursive locking of an already owned lock. The lock > owning > > > object > > > >> exists on the heap, it > > > >>> is locked and you can call wait() on it. > > > >>> > > > >>> EliminateLocks is the C2 option that controls lock elimination based > on > > > EA. > > > >> Both optimizations have > > > >>> in common that objects with eliminated locking need to be relocked > > > when > > > >> deoptimizing a frame, > > > >>> i.e. when replacing a compiled frame with equivalent interpreter > > > >>> frames. Deoptimization::relock_objects does that job for /all/ > > eliminated > > > >> locks in scope. /All/ can > > > >>> be a mix of eliminated nested locks and locks of not-escaping objects. > > > >>> > > > >>> New with the enhancement: I call relock_objects earlier, just before > > > objects > > > >> pontentially > > > >>> escape. But then later when the owning compiled frame gets > > > deoptimized, I > > > >> must not do it again: > > > >>> > > > >>> See call to EscapeBarrier::objs_are_deoptimized in > > deoptimization.cpp: > > > >>> > > > >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || > > > EliminateNestedLocks) && > > > >> EliminateLocks)) > > > >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, > > > deoptee.id())) { > > > >>> 375 bool unused; > > > >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, > > > exec_mode, > > > >> unused); > > > >>> 377 } > > > >>> > > > >>> Now when calling relock_objects early it is quiet possible that I have > to > > > relock > > > >> an object the > > > >>> target thread currently waits for. Obviously I cannot relock in this > case, > > > >> instead I chose to > > > >>> introduce relock_count_after_wait to JavaThread. > > > >>> > > > >>> > Is it just that some of the locking gets optimized away e.g. > > > >>> > > > > >>> > synchronised(obj) { > > > >>> > synchronised(obj) { > > > >>> > synchronised(obj) { > > > >>> > obj.wait(); > > > >>> > } > > > >>> > } > > > >>> > } > > > >>> > > > > >>> > If this is reduced to a form as-if it were a single lock of the > monitor > > > >>> > (due to EA) and the wait() triggers a JVM TI event which leads to > > the > > > >>> > escape of "obj" then we need to reconstruct the true lock state, > > and > > > so > > > >>> > when the wait() internally unblocks and reacquires the monitor it > > > has to > > > >>> > set the true recursion count to 3, not the 1 that it appeared to be > > > when > > > >>> > wait() was initially called. Is that the scenario? > > > >>> > > > >>> Kind of... except that the locking is not eliminated due to EA and > there > > is > > > no > > > >> JVM TI event > > > >>> triggered by wait. > > > >>> > > > >>> Add > > > >>> > > > >>> LocalObject l1 = new LocalObject(); > > > >>> > > > >>> in front of the synchrnized blocks and assume a JVM TI agent > acquires > > l1. > > > This > > > >> triggers the code in > > > >>> question. > > > >>> > > > >>> See that relocking/reallocating is transactional. If it is done then for > > /all/ > > > >> objects in scope and it is > > > >>> done at most once. It wouldn't be quite so easy to split this in > relocking > > > of > > > >> nested/EA-based > > > >>> eliminated locks. > > > >>> > > > >>> > If so I find this truly awful. Anyone using wait() in a realistic form > > > >>> > requires a notification and so the object cannot be thread > > confined. > > > In > > > >>> > > > >>> It is not thread confined. > > > >>> > > > >>> > which case I would strongly argue that upon hitting the wait() the > > > deopt > > > >>> > should occur unconditionally and so the lock state is correct > before > > > we > > > >>> > wait and so we don't need to mess with the recursion count > > > internally > > > >>> > when we reacquire the monitor. > > > >>> > > > > >>> > > > > > >>> > > > which I don't like the sound of at all when it comes to > > > ObjectMonitor > > > >>> > > > state. So I'd like to understand in detail exactly what is going > > on > > > here > > > >>> > > > and why. This is a very intrusive change that seems to badly > > > break > > > >>> > > > encapsulation and impacts future changes to ObjectMonitor > > > that are > > > >> under > > > >>> > > > investigation. > > > >>> > > > > > >>> > > I would not regard this as breaking encapsulation. Certainly not > > > badly. > > > >>> > > > > > >>> > > I've added a property relock_count_after_wait to JavaThread. > > The > > > >> property is well > > > >>> > > encapsulated. Future ObjectMonitor implementations have to > > deal > > > with > > > >> recursion too. They are free > > > >>> > > in choosing a way to do that as long as that property is taken > into > > > >> account. This is hardly a > > > >>> > > limitation. > > > >>> > > > > >>> > I do think this badly breaks encapsulation as you have to add a > > > callout > > > >>> > from the guts of the ObjectMonitor code to reach into the thread > > to > > > get > > > >>> > this lock count adjustment. I understand why you have had to do > > > this but > > > >>> > I would much rather see a change to the EA optimisation strategy > > so > > > that > > > >>> > this is not needed. > > > >>> > > > > >>> > > Note also that the property is a straight forward extension of > the > > > >> existing concept of deferred > > > >>> > > local updates. It is embedded into the structure holding them. > So > > > not > > > >> even the footprint of a > > > >>> > > JavaThread is enlarged if no deferred updates are generated. > > > >>> > > > > >>> > [...] > > > >>> > > > > >>> > > > > > >>> > > I'm actually duplicating the existing external suspend > mechanism, > > > >> because a thread can be > > > >>> > > suspended at most once. And hey, and don't like that either! > But > > it > > > >> seems not unlikely that the > > > >>> > > duplicate can be removed together with the original and the > new > > > type > > > >> of handshakes that will be > > > >>> > > used for thread suspend can be used for object deoptimization > > > too. See > > > >> today's discussion in > > > >>> > > JDK-8227745 [2]. > > > >>> > > > > >>> > I hope that discussion bears some fruit, at the moment it seems > > not > > > to > > > >>> > be possible to use handshakes here. :( > > > >>> > > > > >>> > The external suspend mechanism is a royal pain in the proverbial > > > that we > > > >>> > have to carefully live with. The idea that we're duplicating that > for > > > >>> > use in another fringe area of functionality does not thrill me at all. > > > >>> > > > > >>> > To be clear, I understand the problem that exists and that you > > wish > > > to > > > >>> > solve, but for the runtime parts I balk at the complexity cost of > > > >>> > solving it. > > > >>> > > > >>> I know it's complex, but by far no rocket science. > > > >>> > > > >>> Also I find it hard to imagine another fix for JDK-8233915 besides > > > changing > > > >> the JVM TI specification. > > > >>> > > > >>> Thanks, Richard. > > > >>> > > > >>> -----Original Message----- > > > >>> From: David Holmes > > > >>> Sent: Dienstag, 17. Dezember 2019 08:03 > > > >>> To: Reingruber, Richard ; > serviceability- > > > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > > > hotspot- > > > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > > > (vladimir.kozlov at oracle.com) > > > >> > > > >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > Performance > > > >> in the Presence of JVMTI Agents > > > >>> > > > >>> > > > >>> > > > >>> David > > > >>> > > > >>> On 17/12/2019 4:57 pm, David Holmes wrote: > > > >>>> Hi Richard, > > > >>>> > > > >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: > > > >>>>> Hi David, > > > >>>>> > > > >>>>> ?? > Some further queries/concerns: > > > >>>>> ?? > > > > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp > > > >>>>> ?? > > > > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: > > > >>>>> ?? > > > > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count > > > >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > > > >>>>> ?? > increased by the deferred relock count > > > >>>>> ?? > > > > >>>>> ?? > what is the "deferred relock count"? I gather it relates to > > > >>>>> ?? > > > > >>>>> ?? > "The code was extended to be able to deoptimize objects of a > > > >>>>> frame that > > > >>>>> ?? > is not the top frame and to let another thread than the owning > > > >>>>> thread do > > > >>>>> ?? > it." > > > >>>>> > > > >>>>> Yes, these relate. Currently EA based optimizations are reverted, > > > when > > > >>>>> a compiled frame is replaced > > > >>>>> with corresponding interpreter frames. Part of this is relocking > > > >>>>> objects with eliminated > > > >>>>> locking. New with the enhancement is that we do this also just > > before > > > >>>>> object references are acquired > > > >>>>> through JVMTI. In this case we deoptimize also the owning > compiled > > > >>>>> frame C and we register > > > >>>>> deoptimized objects as deferred updates. When control returns to > > C > > > it > > > >>>>> gets deoptimized, we notice > > > >>>>> that objects are already deoptimized (reallocated and relocked), so > > > we > > > >>>>> don't do it again (relocking > > > >>>>> twice would be incorrect of course). Deferred updates are copied > > into > > > >>>>> the new interpreter frames. > > > >>>>> > > > >>>>> Problem: relocking is not possible if the target thread T is waiting > > > >>>>> on the monitor that needs to be > > > >>>>> relocked. This happens only with non-local objects with > > > >>>>> EliminateNestedLocks. Instead relocking is > > > >>>>> deferred until T owns the monitor again. This is what the piece of > > > >>>>> code above does. > > > >>>> > > > >>>> Sorry I need some more detail here. How can you wait() on an > object > > > >>>> monitor if the object allocation and/or locking was optimised away? > > > And > > > >>>> what is a "non-local object" in this context? Isn't EA restricted to > > > >>>> thread-confined objects? > > > >>>> > > > >>>> Is it just that some of the locking gets optimized away e.g. > > > >>>> > > > >>>> synchronised(obj) { > > > >>>> ? synchronised(obj) { > > > >>>> ??? synchronised(obj) { > > > >>>> ????? obj.wait(); > > > >>>> ??? } > > > >>>> ? } > > > >>>> } > > > >>>> > > > >>>> If this is reduced to a form as-if it were a single lock of the monitor > > > >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the > > > >>>> escape of "obj" then we need to reconstruct the true lock state, and > > so > > > >>>> when the wait() internally unblocks and reacquires the monitor it > has > > to > > > >>>> set the true recursion count to 3, not the 1 that it appeared to be > > when > > > >>>> wait() was initially called. Is that the scenario? > > > >>>> > > > >>>> If so I find this truly awful. Anyone using wait() in a realistic form > > > >>>> requires a notification and so the object cannot be thread confined. > > In > > > >>>> which case I would strongly argue that upon hitting the wait() the > > > deopt > > > >>>> should occur unconditionally and so the lock state is correct before > > we > > > >>>> wait and so we don't need to mess with the recursion count > internally > > > >>>> when we reacquire the monitor. > > > >>>> > > > >>>>> > > > >>>>> ?? > which I don't like the sound of at all when it comes to > > > >>>>> ObjectMonitor > > > >>>>> ?? > state. So I'd like to understand in detail exactly what is going > > > >>>>> on here > > > >>>>> ?? > and why.? This is a very intrusive change that seems to badly > > > break > > > >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor > > that > > > >>>>> are under > > > >>>>> ?? > investigation. > > > >>>>> > > > >>>>> I would not regard this as breaking encapsulation. Certainly not > > badly. > > > >>>>> > > > >>>>> I've added a property relock_count_after_wait to JavaThread. The > > > >>>>> property is well > > > >>>>> encapsulated. Future ObjectMonitor implementations have to deal > > > with > > > >>>>> recursion too. They are free in > > > >>>>> choosing a way to do that as long as that property is taken into > > > >>>>> account. This is hardly a > > > >>>>> limitation. > > > >>>> > > > >>>> I do think this badly breaks encapsulation as you have to add a > callout > > > >>>> from the guts of the ObjectMonitor code to reach into the thread to > > > get > > > >>>> this lock count adjustment. I understand why you have had to do > this > > > but > > > >>>> I would much rather see a change to the EA optimisation strategy so > > > that > > > >>>> this is not needed. > > > >>>> > > > >>>>> Note also that the property is a straight forward extension of the > > > >>>>> existing concept of deferred > > > >>>>> local updates. It is embedded into the structure holding them. So > > not > > > >>>>> even the footprint of a > > > >>>>> JavaThread is enlarged if no deferred updates are generated. > > > >>>>> > > > >>>>> ?? > --- > > > >>>>> ?? > > > > >>>>> ?? > src/hotspot/share/runtime/thread.cpp > > > >>>>> ?? > > > > >>>>> ?? > Can you please explain why > > > >>>>> JavaThread::wait_for_object_deoptimization > > > >>>>> ?? > has to be handcrafted in this way rather than using proper > > > >>>>> transitions. > > > >>>>> ?? > > > > >>>>> > > > >>>>> I wrote wait_for_object_deoptimization taking > > > >>>>> JavaThread::java_suspend_self_with_safepoint_check > > > >>>>> as template. So in short: for the same reasons :) > > > >>>>> > > > >>>>> Threads reach both methods as part of thread state transitions, > > > >>>>> therefore special handling is > > > >>>>> required to change thread state on top of ongoing transitions. > > > >>>>> > > > >>>>> ?? > We got rid of "deopt suspend" some time ago and it is > > disturbing > > > >>>>> to see > > > >>>>> ?? > it being added back (effectively). This seems like it may be > > > >>>>> something > > > >>>>> ?? > that handshakes could be used for. > > > >>>>> > > > >>>>> Deopt suspend used to be something rather different with a > similar > > > >>>>> name[1]. It is not being added back. > > > >>>> > > > >>>> I stand corrected. Despite comments in the code to the contrary > > > >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a > lot > > of > > > >>>> cleanup in this area 13 years ago :) > > > >>>> > > > >>>>> > > > >>>>> I'm actually duplicating the existing external suspend mechanism, > > > >>>>> because a thread can be suspended > > > >>>>> at most once. And hey, and don't like that either! But it seems not > > > >>>>> unlikely that the duplicate can > > > >>>>> be removed together with the original and the new type of > > > handshakes > > > >>>>> that will be used for > > > >>>>> thread suspend can be used for object deoptimization too. See > > > today's > > > >>>>> discussion in JDK-8227745 [2]. > > > >>>> > > > >>>> I hope that discussion bears some fruit, at the moment it seems not > > to > > > >>>> be possible to use handshakes here. :( > > > >>>> > > > >>>> The external suspend mechanism is a royal pain in the proverbial > that > > > we > > > >>>> have to carefully live with. The idea that we're duplicating that for > > > >>>> use in another fringe area of functionality does not thrill me at all. > > > >>>> > > > >>>> To be clear, I understand the problem that exists and that you wish > to > > > >>>> solve, but for the runtime parts I balk at the complexity cost of > > > >>>> solving it. > > > >>>> > > > >>>> Thanks, > > > >>>> David > > > >>>> ----- > > > >>>> > > > >>>>> Thanks, Richard. > > > >>>>> > > > >>>>> [1] Deopt suspend was something like an async. handshake for > > > >>>>> architectures with register windows, > > > >>>>> ???? where patching the return pc for deoptimization of a compiled > > > >>>>> frame was racy if the owner thread > > > >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on > > > >>>>> which the thread patched its own > > > >>>>> ???? frame upon return from native. So no thread was suspended. > It > > > got > > > >>>>> its name only from the name of > > > >>>>> ???? the flags. > > > >>>>> > > > >>>>> [2] Discussion about using handshakes to sync. with the target > > thread: > > > >>>>> > > > >>>>> https://bugs.openjdk.java.net/browse/JDK- > > > >> > > > > > > 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst > > > e > > > >> m.issuetabpanels:comment-tabpanel#comment-14306727 > > > >>>>> > > > >>>>> > > > >>>>> -----Original Message----- > > > >>>>> From: David Holmes > > > >>>>> Sent: Freitag, 13. Dezember 2019 00:56 > > > >>>>> To: Reingruber, Richard ; > > > >>>>> serviceability-dev at openjdk.java.net; > > > >>>>> hotspot-compiler-dev at openjdk.java.net; > > > >>>>> hotspot-runtime-dev at openjdk.java.net > > > >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > >>>>> Performance in the Presence of JVMTI Agents > > > >>>>> > > > >>>>> Hi Richard, > > > >>>>> > > > >>>>> Some further queries/concerns: > > > >>>>> > > > >>>>> src/hotspot/share/runtime/objectMonitor.cpp > > > >>>>> > > > >>>>> Can you please explain the changes to ObjectMonitor::wait: > > > >>>>> > > > >>>>> !?? _recursions = save????? // restore the old recursion count > > > >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > > > >>>>> increased by the deferred relock count > > > >>>>> > > > >>>>> what is the "deferred relock count"? I gather it relates to > > > >>>>> > > > >>>>> "The code was extended to be able to deoptimize objects of a > > frame > > > that > > > >>>>> is not the top frame and to let another thread than the owning > > thread > > > do > > > >>>>> it." > > > >>>>> > > > >>>>> which I don't like the sound of at all when it comes to > ObjectMonitor > > > >>>>> state. So I'd like to understand in detail exactly what is going on > here > > > >>>>> and why.? This is a very intrusive change that seems to badly break > > > >>>>> encapsulation and impacts future changes to ObjectMonitor that > > are > > > under > > > >>>>> investigation. > > > >>>>> > > > >>>>> --- > > > >>>>> > > > >>>>> src/hotspot/share/runtime/thread.cpp > > > >>>>> > > > >>>>> Can you please explain why > > > JavaThread::wait_for_object_deoptimization > > > >>>>> has to be handcrafted in this way rather than using proper > > transitions. > > > >>>>> > > > >>>>> We got rid of "deopt suspend" some time ago and it is disturbing > to > > > see > > > >>>>> it being added back (effectively). This seems like it may be > > something > > > >>>>> that handshakes could be used for. > > > >>>>> > > > >>>>> Thanks, > > > >>>>> David > > > >>>>> ----- > > > >>>>> > > > >>>>> On 12/12/2019 7:02 am, David Holmes wrote: > > > >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: > > > >>>>>>> Hi David, > > > >>>>>>> > > > >>>>>>> ??? > Most of the details here are in areas I can comment on in > > > detail, > > > >>>>>>> but I > > > >>>>>>> ??? > did take an initial general look at things. > > > >>>>>>> > > > >>>>>>> Thanks for taking the time! > > > >>>>>> > > > >>>>>> Apologies the above should read: > > > >>>>>> > > > >>>>>> "Most of the details here are in areas I *can't* comment on in > > detail > > > >>>>>> ..." > > > >>>>>> > > > >>>>>> David > > > >>>>>> > > > >>>>>>> ??? > The only thing that jumped out at me is that I think the > > > >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. > > > >>>>>>> ??? > > > > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; > > } > > > >>>>>>> > > > >>>>>>> Yes, it should. Will add the method like above. > > > >>>>>>> > > > >>>>>>> ??? > Also I don't see any testing of the > > > DeoptimizeObjectsALotThread. > > > >>>>>>> Without > > > >>>>>>> ??? > active testing this will just bit-rot. > > > >>>>>>> > > > >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger > > > >>>>>>> workload. I will add a minimal test > > > >>>>>>> to keep it fresh. > > > >>>>>>> > > > >>>>>>> ??? > Also on the tests I don't understand your @requires clause: > > > >>>>>>> ??? > > > > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & > > > vm.compiler2.enabled > > > >> & > > > >>>>>>> ??? > (vm.opt.TieredCompilation != true)) > > > >>>>>>> ??? > > > > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, > but > > > >>>>>>> tiered is > > > >>>>>>> ??? > our normal mode of operation. ?? > > > >>>>>>> ??? > > > > >>>>>>> > > > >>>>>>> I removed the clause. I guess I wanted to target the tests > towards > > > the > > > >>>>>>> code they are supposed to > > > >>>>>>> test, and it's easier to analyze failures w/o tiered compilation > and > > > >>>>>>> with just one compiler thread. > > > >>>>>>> > > > >>>>>>> Additionally I will make use of > > > >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the > > tests. > > > >>>>>>> > > > >>>>>>> Thanks, > > > >>>>>>> Richard. > > > >>>>>>> > > > >>>>>>> -----Original Message----- > > > >>>>>>> From: David Holmes > > > >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 > > > >>>>>>> To: Reingruber, Richard ; > > > >>>>>>> serviceability-dev at openjdk.java.net; > > > >>>>>>> hotspot-compiler-dev at openjdk.java.net; > > > >>>>>>> hotspot-runtime-dev at openjdk.java.net > > > >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > >>>>>>> Performance in the Presence of JVMTI Agents > > > >>>>>>> > > > >>>>>>> Hi Richard, > > > >>>>>>> > > > >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: > > > >>>>>>>> Hi, > > > >>>>>>>> > > > >>>>>>>> I would like to get reviews please for > > > >>>>>>>> > > > >>>>>>>> > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > > > >>>>>>>> > > > >>>>>>>> Corresponding RFE: > > > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 > > > >>>>>>>> > > > >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > > > >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- > > > 8214584 [1] > > > >>>>>>>> > > > >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing > > > without > > > >>>>>>>> issues (thanks!). In addition the > > > >>>>>>>> change is being tested at SAP since I posted the first RFR some > > > >>>>>>>> months ago. > > > >>>>>>>> > > > >>>>>>>> The intention of this enhancement is to benefit performance > > wise > > > from > > > >>>>>>>> escape analysis even if JVMTI > > > >>>>>>>> agents request capabilities that allow them to access local > > variable > > > >>>>>>>> values. E.g. if you start-up > > > >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, > > > then > > > >>>>>>>> escape analysis is disabled right > > > >>>>>>>> from the beginning, well before a debugger attaches -- if ever > > one > > > >>>>>>>> should do so. With the > > > >>>>>>>> enhancement, escape analysis will remain enabled until and > > after > > > a > > > >>>>>>>> debugger attaches. EA based > > > >>>>>>>> optimizations are reverted just before an agent acquires the > > > >>>>>>>> reference to an object. In the JBS item > > > >>>>>>>> you'll find more details. > > > >>>>>>> > > > >>>>>>> Most of the details here are in areas I can comment on in detail, > > but > > > I > > > >>>>>>> did take an initial general look at things. > > > >>>>>>> > > > >>>>>>> The only thing that jumped out at me is that I think the > > > >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. > > > >>>>>>> > > > >>>>>>> +? bool is_hidden_from_external_view() const { return true; } > > > >>>>>>> > > > >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. > > > >>>>>>> Without > > > >>>>>>> active testing this will just bit-rot. > > > >>>>>>> > > > >>>>>>> Also on the tests I don't understand your @requires clause: > > > >>>>>>> > > > >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & > > > vm.compiler2.enabled & > > > >>>>>>> (vm.opt.TieredCompilation != true)) > > > >>>>>>> > > > >>>>>>> This seems to require that TieredCompilation is disabled, but > > tiered > > > is > > > >>>>>>> our normal mode of operation. ?? > > > >>>>>>> > > > >>>>>>> Thanks, > > > >>>>>>> David > > > >>>>>>> > > > >>>>>>>> Thanks, > > > >>>>>>>> Richard. > > > >>>>>>>> > > > >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 > > > >>>>>>>> > > > >> > > > > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa > > > tc > > > >> h > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> From coleen.phillimore at oracle.com Wed May 6 10:40:56 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 6 May 2020 06:40:56 -0400 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> Message-ID: <4087edea-8762-99ba-a74e-132768013136@oracle.com> On 5/6/20 2:09 AM, serguei.spitsyn at oracle.com wrote: > On 5/5/20 17:04, Mikael Vidstedt wrote: >>> On May 5, 2020, at 4:48 PM,serguei.spitsyn at oracle.com wrote: >>> >>> Hi Mikael, >>> >>> The fixes in webrev look good to me. >>> >>> I've just noticed a couple of more serviceability related things can be missed. >>> (Not sure if they are included into different chunk of fixes.) >>> >>> One is libjvm_db.so which is for Solaris Pstack support: >>> make/hotspot/lib/CompileDtraceLibraries.gmk: # Note that libjvm_db.c has tests for COMPILER2, but this was never set by >>> make/hotspot/lib/CompileDtraceLibraries.gmk: LIBJVM_DB_OUTPUTDIR := $(JVM_VARIANT_OUTPUTDIR)/libjvm_db >>> make/hotspot/lib/CompileDtraceLibraries.gmk: NAME := jvm_db, \ >>> make/hotspot/lib/CompileDtraceLibraries.gmk: SRC := $(TOPDIR)/src/java.base/solaris/native/libjvm_db, \ >>> >>> Another is DTrace support which also includes jhelper.d (support for DTrace jstack action which is for Solaris only): >>> make/hotspot/gensrc/GensrcDtrace.gmk >>> make/hotspot/lib/JvmDtraceObjects.gmk >>> It also impacts some other make files. >> I believe you?ll find that these changes were included in the build system patch: >> >> https://mail.openjdk.java.net/pipermail/build-dev/2020-May/027342.html >> http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/build/open/webrev/ >> >> Let me know if I missed something. > > The file || make/hotspot/src/native/dtrace/generateJvmOffsets.cpp is > for Solaris only, and so, can be removed. > It is for libjvm_db.so (provider for Solaris Pstack) and jhelper.d > (provider for Solaris DTrace jstack action). > The jstack action (prints mixed java+native stack traces) was never > implemented other than for Solaris. I wonder if this can be used to implement the same thing on linux and if we can keep this?? Thoughts? thanks, Coleen > > I do not see other problems but looked only at the Serviceability > related files. > > Thanks, > Serguei > >>> Also, these lines can be just removed: >>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#ifndef _JAVASOFT_SOLARIS_PATH_MD_H_ >>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#define _JAVASOFT_SOLARIS_PATH_MD_H_ >>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#endif /* !_JAVASOFT_SOLARIS_PATH_MD_H_ */ >> Remove or rather rename? The corresponding Windows header also has the guard, any particular reason why it should be *removed*? >> >> Cheers, >> Mikael >> >>> Thanks, >>> Serguei >>> >>> >>> On 5/5/20 14:34, Mikael Vidstedt wrote: >>>> All good points! I deliberately chose *not* to update comments where it wasn?t immediately 100% obvious exactly how to update them. For example, in many cases I found that the comments are already incomplete or stale, and for each such case we?ll want to consider how exactly to update the comment (remove/switch to Unix/etc.). I would prefer to handle this as follow-up(s), separate from the JEP. Does that sounds reasonable? >>>> >>>> Apart from the comments - do the changes look good? If so, can I count you as a reviewer? >>>> >>>> Cheers, >>>> Mikael >>>> >>>> >>>>> On May 4, 2020, at 12:20 AM,serguei.spitsyn at oracle.com wrote: >>>>> >>>>> HI Mikael, >>>>> >>>>> Some quick comments. >>>>> >>>>> Some extra references to Solaris/solaris, SunOS or SPARC are listed below: >>>>> >>>>> src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) >>>>> src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) >>>>> src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) >>>>> >>>>> src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent >>>>> src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. >>>>> src/hotspot/share/prims/jvmti.xml: On the Solaris Operating Environment, an agent library is a shared >>>>> src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating >>>>> src/hotspot/share/prims/jvmti.xml: example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris >>>>> src/hotspot/share/prims/methodHandles.cpp:#undef CS // Solaris builds complain >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 5/3/20 22:12, Mikael Vidstedt wrote: >>>>>> Please review this change which implements part of JEP 381: >>>>>> >>>>>> JBS:https://bugs.openjdk.java.net/browse/JDK-8244224 >>>>>> webrev:http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >>>>>> JEP:https://bugs.openjdk.java.net/browse/JDK-8241787 >>>>>> >>>>>> >>>>>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>>>>> >>>>>> >>>>>> Background: >>>>>> >>>>>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>>>>> >>>>>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>>>>> >>>>>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>>>>> >>>>>> >>>>>> Testing: >>>>>> >>>>>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>>>>> >>>>>> Cheers, >>>>>> Mikael >>>>>> >>>>>> [1]http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>>>>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.ihse.bursie at oracle.com Wed May 6 14:27:13 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 6 May 2020 16:27:13 +0200 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <4087edea-8762-99ba-a74e-132768013136@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> <4087edea-8762-99ba-a74e-132768013136@oracle.com> Message-ID: On 2020-05-06 12:40, coleen.phillimore at oracle.com wrote: > > > On 5/6/20 2:09 AM, serguei.spitsyn at oracle.com wrote: >> On 5/5/20 17:04, Mikael Vidstedt wrote: >>>> On May 5, 2020, at 4:48 PM,serguei.spitsyn at oracle.com wrote: >>>> >>>> Hi Mikael, >>>> >>>> The fixes in webrev look good to me. >>>> >>>> I've just noticed a couple of more serviceability related things can be missed. >>>> (Not sure if they are included into different chunk of fixes.) >>>> >>>> One is libjvm_db.so which is for Solaris Pstack support: >>>> make/hotspot/lib/CompileDtraceLibraries.gmk: # Note that libjvm_db.c has tests for COMPILER2, but this was never set by >>>> make/hotspot/lib/CompileDtraceLibraries.gmk: LIBJVM_DB_OUTPUTDIR := $(JVM_VARIANT_OUTPUTDIR)/libjvm_db >>>> make/hotspot/lib/CompileDtraceLibraries.gmk: NAME := jvm_db, \ >>>> make/hotspot/lib/CompileDtraceLibraries.gmk: SRC := $(TOPDIR)/src/java.base/solaris/native/libjvm_db, \ >>>> >>>> Another is DTrace support which also includes jhelper.d (support for DTrace jstack action which is for Solaris only): >>>> make/hotspot/gensrc/GensrcDtrace.gmk >>>> make/hotspot/lib/JvmDtraceObjects.gmk >>>> It also impacts some other make files. >>> I believe you?ll find that these changes were included in the build system patch: >>> >>> https://mail.openjdk.java.net/pipermail/build-dev/2020-May/027342.html >>> http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/build/open/webrev/ >>> >>> Let me know if I missed something. >> >> The file || make/hotspot/src/native/dtrace/generateJvmOffsets.cpp is >> for Solaris only, and so, can be removed. >> It is for libjvm_db.so (provider for Solaris Pstack) and jhelper.d >> (provider for Solaris DTrace jstack action). >> The jstack action (prints mixed java+native stack traces) was never >> implemented other than for Solaris. > > I wonder if this can be used to implement the same thing on linux and > if we can keep this?? Thoughts? Is anyone seriously using dtrace to get stack traces? Or more cynically, is anyone seriouslly using dtrace with Java at all? /Magnus > > thanks, > Coleen >> >> I do not see other problems but looked only at the Serviceability >> related files. >> >> Thanks, >> Serguei >> >>>> Also, these lines can be just removed: >>>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#ifndef _JAVASOFT_SOLARIS_PATH_MD_H_ >>>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#define _JAVASOFT_SOLARIS_PATH_MD_H_ >>>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#endif /* !_JAVASOFT_SOLARIS_PATH_MD_H_ */ >>> Remove or rather rename? The corresponding Windows header also has the guard, any particular reason why it should be *removed*? >>> >>> Cheers, >>> Mikael >>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/5/20 14:34, Mikael Vidstedt wrote: >>>>> All good points! I deliberately chose *not* to update comments where it wasn?t immediately 100% obvious exactly how to update them. For example, in many cases I found that the comments are already incomplete or stale, and for each such case we?ll want to consider how exactly to update the comment (remove/switch to Unix/etc.). I would prefer to handle this as follow-up(s), separate from the JEP. Does that sounds reasonable? >>>>> >>>>> Apart from the comments - do the changes look good? If so, can I count you as a reviewer? >>>>> >>>>> Cheers, >>>>> Mikael >>>>> >>>>> >>>>>> On May 4, 2020, at 12:20 AM,serguei.spitsyn at oracle.com wrote: >>>>>> >>>>>> HI Mikael, >>>>>> >>>>>> Some quick comments. >>>>>> >>>>>> Some extra references to Solaris/solaris, SunOS or SPARC are listed below: >>>>>> >>>>>> src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) >>>>>> src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) >>>>>> src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) >>>>>> >>>>>> src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent >>>>>> src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. >>>>>> src/hotspot/share/prims/jvmti.xml: On the Solaris Operating Environment, an agent library is a shared >>>>>> src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating >>>>>> src/hotspot/share/prims/jvmti.xml: example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris >>>>>> src/hotspot/share/prims/methodHandles.cpp:#undef CS // Solaris builds complain >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 5/3/20 22:12, Mikael Vidstedt wrote: >>>>>>> Please review this change which implements part of JEP 381: >>>>>>> >>>>>>> JBS:https://bugs.openjdk.java.net/browse/JDK-8244224 >>>>>>> webrev:http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >>>>>>> JEP:https://bugs.openjdk.java.net/browse/JDK-8241787 >>>>>>> >>>>>>> >>>>>>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>>>>>> >>>>>>> >>>>>>> Background: >>>>>>> >>>>>>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>>>>>> >>>>>>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>>>>>> >>>>>>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>>>>>> >>>>>>> >>>>>>> Testing: >>>>>>> >>>>>>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>>>>>> >>>>>>> Cheers, >>>>>>> Mikael >>>>>>> >>>>>>> [1]http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>>>>>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed May 6 17:04:43 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 May 2020 10:04:43 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <4087edea-8762-99ba-a74e-132768013136@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> <4087edea-8762-99ba-a74e-132768013136@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed May 6 17:15:06 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 May 2020 10:15:06 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> <4087edea-8762-99ba-a74e-132768013136@oracle.com> Message-ID: <8e76194e-dc06-4323-f65f-c29bd620923b@oracle.com> An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Wed May 6 17:58:01 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 6 May 2020 13:58:01 -0400 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> <4087edea-8762-99ba-a74e-132768013136@oracle.com> Message-ID: <6bb57ba8-6596-ceb8-22c0-8d1a73ecae68@oracle.com> On 5/6/20 1:04 PM, serguei.spitsyn at oracle.com wrote: > On 5/6/20 03:40, coleen.phillimore at oracle.com wrote: >> >> >> On 5/6/20 2:09 AM, serguei.spitsyn at oracle.com wrote: >>> On 5/5/20 17:04, Mikael Vidstedt wrote: >>>>> On May 5, 2020, at 4:48 PM,serguei.spitsyn at oracle.com wrote: >>>>> >>>>> Hi Mikael, >>>>> >>>>> The fixes in webrev look good to me. >>>>> >>>>> I've just noticed a couple of more serviceability related things can be missed. >>>>> (Not sure if they are included into different chunk of fixes.) >>>>> >>>>> One is libjvm_db.so which is for Solaris Pstack support: >>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: # Note that libjvm_db.c has tests for COMPILER2, but this was never set by >>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: LIBJVM_DB_OUTPUTDIR := $(JVM_VARIANT_OUTPUTDIR)/libjvm_db >>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: NAME := jvm_db, \ >>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: SRC := $(TOPDIR)/src/java.base/solaris/native/libjvm_db, \ >>>>> >>>>> Another is DTrace support which also includes jhelper.d (support for DTrace jstack action which is for Solaris only): >>>>> make/hotspot/gensrc/GensrcDtrace.gmk >>>>> make/hotspot/lib/JvmDtraceObjects.gmk >>>>> It also impacts some other make files. >>>> I believe you?ll find that these changes were included in the build system patch: >>>> >>>> https://mail.openjdk.java.net/pipermail/build-dev/2020-May/027342.html >>>> http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/build/open/webrev/ >>>> >>>> Let me know if I missed something. >>> >>> The file || make/hotspot/src/native/dtrace/generateJvmOffsets.cpp is >>> for Solaris only, and so, can be removed. >>> It is for libjvm_db.so (provider for Solaris Pstack) and jhelper.d >>> (provider for Solaris DTrace jstack action). >>> The jstack action (prints mixed java+native stack traces) was never >>> implemented other than for Solaris. >> >> I wonder if this can be used to implement the same thing on linux and >> if we can keep this?? Thoughts? > > I was also thinking about it. And DTrace kind of exists on Mac OS as well. > Yes, it can be used. But It will require the jstack action > implementation and the jhelper use on the DTrace side. > The same is true for the libjvm_db.so. It could be used in the Linux > Pstack utility to print mixed stack traces. > (I see some pstack projects like this: https://github.com/peadar/pstack ) Yes, I was thinking specifically of the pstack utility and not DTrace, just to get Java frames in ptrace.? Could libjvm_db.so be used for that? Coleen > > But as we already discussed the GDB has a different approach to > register code: > https://sourceware.org/gdb/onlinedocs/gdb/JIT-Interface.html#JIT-Interface > > Thanks, > Serguei > >> >> thanks, >> Coleen >>> >>> I do not see other problems but looked only at the Serviceability >>> related files. >>> >>> Thanks, >>> Serguei >>> >>>>> Also, these lines can be just removed: >>>>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#ifndef _JAVASOFT_SOLARIS_PATH_MD_H_ >>>>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#define _JAVASOFT_SOLARIS_PATH_MD_H_ >>>>> open/src/jdk.jdwp.agent/unix/native/libjdwp/path_md.h:#endif /* !_JAVASOFT_SOLARIS_PATH_MD_H_ */ >>>> Remove or rather rename? The corresponding Windows header also has the guard, any particular reason why it should be *removed*? >>>> >>>> Cheers, >>>> Mikael >>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 5/5/20 14:34, Mikael Vidstedt wrote: >>>>>> All good points! I deliberately chose *not* to update comments where it wasn?t immediately 100% obvious exactly how to update them. For example, in many cases I found that the comments are already incomplete or stale, and for each such case we?ll want to consider how exactly to update the comment (remove/switch to Unix/etc.). I would prefer to handle this as follow-up(s), separate from the JEP. Does that sounds reasonable? >>>>>> >>>>>> Apart from the comments - do the changes look good? If so, can I count you as a reviewer? >>>>>> >>>>>> Cheers, >>>>>> Mikael >>>>>> >>>>>> >>>>>>> On May 4, 2020, at 12:20 AM,serguei.spitsyn at oracle.com wrote: >>>>>>> >>>>>>> HI Mikael, >>>>>>> >>>>>>> Some quick comments. >>>>>>> >>>>>>> Some extra references to Solaris/solaris, SunOS or SPARC are listed below: >>>>>>> >>>>>>> src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c (2 refs to Solaris/solaris) >>>>>>> src/java.management/share/classes/javax/management/loading/MLet.java (refs to Solaris, SPARC/sparc and SunOS) >>>>>>> src/jdk.management.agent/unix/classes/jdk/internal/agent/FileSystemImpl.java (ref to Solaris) >>>>>>> >>>>>>> src/hotspot/share/prims/forte.cpp:// Solaris SPARC Compiler1 needs an additional check on the grandparent >>>>>>> src/hotspot/share/prims/forte.cpp:// on Linux X86, Solaris SPARC and Solaris X86. >>>>>>> src/hotspot/share/prims/jvmti.xml: On the Solaris Operating Environment, an agent library is a shared >>>>>>> src/hotspot/share/prims/jvmti.xml: LD_LIBRARY_PATH under the Solaris operating >>>>>>> src/hotspot/share/prims/jvmti.xml: example, in the Java 2 SDK a CTRL-Break on Win32 and a CTRL-\ on Solaris >>>>>>> src/hotspot/share/prims/methodHandles.cpp:#undef CS // Solaris builds complain >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 5/3/20 22:12, Mikael Vidstedt wrote: >>>>>>>> Please review this change which implements part of JEP 381: >>>>>>>> >>>>>>>> JBS:https://bugs.openjdk.java.net/browse/JDK-8244224 >>>>>>>> webrev:http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >>>>>>>> JEP:https://bugs.openjdk.java.net/browse/JDK-8241787 >>>>>>>> >>>>>>>> >>>>>>>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>>>>>>> >>>>>>>> >>>>>>>> Background: >>>>>>>> >>>>>>>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>>>>>>> >>>>>>>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>>>>>>> >>>>>>>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>>>>>>> >>>>>>>> >>>>>>>> Testing: >>>>>>>> >>>>>>>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Mikael >>>>>>>> >>>>>>>> [1]http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>>>>>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed May 6 18:12:10 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 May 2020 11:12:10 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <6bb57ba8-6596-ceb8-22c0-8d1a73ecae68@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> <4087edea-8762-99ba-a74e-132768013136@oracle.com> <6bb57ba8-6596-ceb8-22c0-8d1a73ecae68@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed May 6 22:19:32 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 May 2020 15:19:32 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> <4087edea-8762-99ba-a74e-132768013136@oracle.com> <6bb57ba8-6596-ceb8-22c0-8d1a73ecae68@oracle.com> Message-ID: <3f647684-4255-1217-917a-be2eeb8da88b@oracle.com> An HTML attachment was scrubbed... URL: From mikael.vidstedt at oracle.com Thu May 7 05:16:27 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:16:27 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: Thank you for the review! Comments inline.. > On May 4, 2020, at 1:28 AM, Stefan Karlsson wrote: > > Hi Mikael, > > On 2020-05-04 07:12, Mikael Vidstedt wrote: >> Please review this change which implements part of JEP 381: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ > > I went over this patch and collected some comments: > > src/hotspot/share/adlc/output_c.cpp > src/hotspot/share/adlc/output_h.cpp > > Awkward code layout after change to. Indeed - fixed! > src/hotspot/share/c1/c1_Runtime1.cpp > src/hotspot/share/classfile/classListParser.cpp > src/hotspot/share/memory/arena.hpp > src/hotspot/share/opto/chaitin.cpp > test/hotspot/jtreg/gc/TestCardTablePageCommits.java > > Surrounding comments still refers to Sparc and/or Solaris. > > There are even more places if you search in the rest of the HotSpot source. Are we leaving those for a separate cleanup pass? Correct - I deliberately avoided changing comments that were not immediately ?obvious? how to address and/or that were pre-existing issues, since it?s not necessarily wrong for a comment to refer to Solaris or SPARC even after these changes. I would prefer to do that as follow-ups. Fair? > src/hotspot/share/gc/g1/g1HeapRegionAttr.hpp > > Remove comment: > // We use different types to represent the state value depending on platform as > // some have issues loading parts of words. Fixed. > src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp > > Fuse the declaration and definition, now that we only have one implementation. Maybe even remove function/file at some point. Fixed (fused). > src/hotspot/share/utilities/globalDefinitions.hpp > > Now that STACK_BIAS is always 0, should we remove its usages? Follow-up RFE? Yes, this is one of the things I have on my list to file a follow-up for. > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/meta/HotSpotGraphBuilderPlugins.java > > Maybe remove decryptSuffix? Fixed. > src/utils/hsdis/Makefile > > Is this really correct? > > Shouldn't: > ARCH1=$(CPU:x86_64=amd64) > ARCH2=$(ARCH1:i686=i386) > ARCH=$(ARCH2:sparc64=sparcv9) > > be changed to: > ARCH1=$(CPU:x86_64=amd64) > ARCH=$(ARCH1:i686=i386) > > so that we have ARCH defined? Very good catch! This Makefile could use some indentation love or just a plain rewrite.. In either case I fixed the ARCH definition and tested it to make sure the end result seemed to do the right thing (and AFAICT it does). > Other than that this looks good to me. Thank you! Cheers, Mikael > >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> Background: >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! >> Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. >> Testing: >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> Cheers, >> Mikael >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ From mikael.vidstedt at oracle.com Thu May 7 05:19:07 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:19:07 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: <2DDEE6DB-577A-4E23-B78D-0D19A15083F2@oracle.com> > On May 4, 2020, at 2:11 AM, Thomas Schatzl wrote: > > Hi, > > On 04.05.20 10:28, Stefan Karlsson wrote: >> Hi Mikael, >> On 2020-05-04 07:12, Mikael Vidstedt wrote: >>> >>> Please review this change which implements part of JEP 381: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >> I went over this patch and collected some comments: >> src/hotspot/share/adlc/output_c.cpp >> src/hotspot/share/adlc/output_h.cpp >> Awkward code layout after change to. >> src/hotspot/share/c1/c1_Runtime1.cpp >> src/hotspot/share/classfile/classListParser.cpp >> src/hotspot/share/memory/arena.hpp >> src/hotspot/share/opto/chaitin.cpp >> test/hotspot/jtreg/gc/TestCardTablePageCommits.jav > > > Surrounding comments still refers to Sparc and/or Solaris. > > > > There are even more places if you search in the rest of the HotSpot > > source. Are we leaving those for a separate cleanup pass? > > In addition to "sparc", "solaris", also "solstudio"/"Sun Studio"/"SS compiler bug"/"niagara" yield some search (=grep) results. > > Some of these locations look like additional RFEs. Ah good! I found and fixed a few additional places based on those strings, but would like to handling the remaining comment updates as RFEs. Thank you for having a look! Cheers, Mikael -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikael.vidstedt at oracle.com Thu May 7 05:23:13 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:23:13 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: <553D0344-188E-455B-A03E-D080C1484B41@oracle.com> References: <553D0344-188E-455B-A03E-D080C1484B41@oracle.com> Message-ID: <40FCCD0E-5C84-482D-9231-428898E1E0AD@oracle.com> Kim, thank you for the review! Comments inline.. > On May 4, 2020, at 3:47 AM, Kim Barrett wrote: > >> On May 4, 2020, at 1:12 AM, Mikael Vidstedt wrote: >> >> >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > I've only looked at the src/hotspot changes so far. I've not > duplicated comments already made by Stefan. > > Looks good, other than a few very minor issues, some of which might > already be covered by planned followup RFEs. > > ------------------------------------------------------------------------------ > > I think with sparc removal, c1's pack64/unpack64 stuff is no longer > used. So I think that can be removed from c1_LIR.[ch]pp too. Good catch. Fixed. > ------------------------------------------------------------------------------ > src/hotspot/share/opto/generateOptoStub.cpp > 225 // Clear last_Java_pc and (optionally)_flags > > The sparc-specific clearing of "flags" is gone. Fixed. > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/deoptimization.cpp > 1086 *((jlong *) check_alignment_get_addr(obj, index, 8)) = (jlong) *((jlong *) &val); > > [pre-existing] > The rhs cast to jlong is unnecessary, since it's dereferencing a jlong*. When I first updated the code I actually remove the cast, but it just ended up looking asymmetrical so I chose to leave it there. Let me know if you feel strongly that it should go. (I don?t like these casts in general). > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/flags/jvmFlagConstraintsCompiler.cpp > 236 JVMFlag::Error CompilerThreadPriorityConstraintFunc(intx value, bool verbose) { > 237 return JVMFlag::SUCCESS; > 238 } > > After SOLARIS code removal we no longer need this constraint function. Fixed. (I had that on my follow-up list, but included it in the upcoming webrev.) > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/globals.hpp > 2392 experimental(size_t, ArrayAllocatorMallocLimit, \ > 2393 (size_t)-1, \ > > Combine these lines. Fixed. > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/dtrace.hpp > > Shuold just eliminate all traces of HS_DTRACE_WORKAROUND_TAIL_CALL_BUG. Fixed - more code removed! Cheers, Mikael From mikael.vidstedt at oracle.com Thu May 7 05:25:22 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:25:22 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: <25d5e2fa-8909-0b94-e0b2-6b5aaa224492@oracle.com> References: <25d5e2fa-8909-0b94-e0b2-6b5aaa224492@oracle.com> Message-ID: <4B24DD51-8AA4-4896-B1F2-59FA5233F125@oracle.com> Vladimir, thank you for the review! Note that based on Stefan?s comments I have removed the decryptSuffix variable in src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/meta/HotSpotGraphBuilderPlugins.java in the upcoming webrev. Cheers, Mikael > On May 4, 2020, at 12:01 PM, Vladimir Kozlov wrote: > > JIT, AOT, JVMCI and Graal changes seem fine to me. > > It would be interesting to see shared code execution coverage change. There are places where we use flags and setting instead of #ifdef SPARC which may not be executed now or executed partially. We may simplify such code too. > > Thanks, > Vladimir > > On 5/3/20 10:12 PM, Mikael Vidstedt wrote: >> Please review this change which implements part of JEP 381: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> Background: >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! >> Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. >> Testing: >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> Cheers, >> Mikael >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ From mikael.vidstedt at oracle.com Thu May 7 05:27:04 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:27:04 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: <7F17A7EF-C9B3-40A0-816B-53614A56B7CA@oracle.com> Igor, thank you for the review, and again for helping make the test changes in the first place! :) I hope Vladimir?s reply clarifies how we?re planning on handling the Graal related changes. Cheers, Mikael > On May 4, 2020, at 2:29 PM, Igor Ignatyev wrote: > > Hi Mikael, > > the changes in /test/ look good to me. > > I have a question regarding src/jdk.internal.vm.compiler/*, aren't these files part of graal-compiler and hence will be brought back by the next graal update? > > Thanks, > -- Igor > >> On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: >> >> >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> >> >> Background: >> >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> >> A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! >> >> Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. >> >> Testing: >> >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> >> Cheers, >> Mikael >> >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >> > From mikael.vidstedt at oracle.com Thu May 7 05:35:30 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:35:30 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: References: Message-ID: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> New webrev here: webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot/open/webrev/ incremental: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot.incr/open/webrev/ Remaining items: * File follow-up to remove STACK_BIAS * File follow-ups to change/update/remove flags and/or flag documentation: UseLWPSynchronization, BranchOnRegister, LIRFillDelaySlots, ArrayAllocatorMallocLimit, ThreadPriorityPolicy * File follow-up(s) to update comments ("solaris", ?sparc?, ?solstudio?, ?sunos?, ?sun studio?, ?s compiler bug?, ?niagara?, ?) Please let me know if there?s something I have missed! Cheers, Mikael > On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: > > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! > > Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From mikael.vidstedt at oracle.com Thu May 7 05:42:38 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:42:38 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <3f647684-4255-1217-917a-be2eeb8da88b@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <45d37ff7-dfab-2632-9fb7-a219505a437a@oracle.com> <48001AE8-9CF4-4E62-98E2-B46BE231D0A4@oracle.com> <10d4a8d7-36a4-9ba5-3b76-bed238f976c5@oracle.com> <6E6BB0A8-BC8B-48D5-A615-2739F078A2B5@oracle.com> <0129393d-c328-6a32-5186-cc42aeb40863@oracle.com> <4087edea-8762-99ba-a74e-132768013136@oracle.com> <6bb57ba8-6596-ceb8-22c0-8d1a73ecae68@oracle.com> <3f647684-4255-1217-917a-be2eeb8da88b@oracle.com> Message-ID: > On May 6, 2020, at 3:19 PM, serguei.spitsyn at oracle.com wrote: > > > > On 5/6/20 11:12, serguei.spitsyn at oracle.com wrote: >> On 5/6/20 10:58, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 5/6/20 1:04 PM, serguei.spitsyn at oracle.com wrote: >>>> On 5/6/20 03:40, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> On 5/6/20 2:09 AM, serguei.spitsyn at oracle.com wrote: >>>>>> On 5/5/20 17:04, Mikael Vidstedt wrote: >>>>>>>> On May 5, 2020, at 4:48 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> >>>>>>>> Hi Mikael, >>>>>>>> >>>>>>>> The fixes in webrev look good to me. >>>>>>>> >>>>>>>> I've just noticed a couple of more serviceability related things can be missed. >>>>>>>> (Not sure if they are included into different chunk of fixes.) >>>>>>>> >>>>>>>> One is libjvm_db.so which is for Solaris Pstack support: >>>>>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: # Note that libjvm_db.c has tests for COMPILER2, but this was never set by >>>>>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: LIBJVM_DB_OUTPUTDIR := $(JVM_VARIANT_OUTPUTDIR)/libjvm_db >>>>>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: NAME := jvm_db, \ >>>>>>>> make/hotspot/lib/CompileDtraceLibraries.gmk: SRC := $(TOPDIR)/src/java.base/solaris/native/libjvm_db, \ >>>>>>>> >>>>>>>> Another is DTrace support which also includes jhelper.d (support for DTrace jstack action which is for Solaris only): >>>>>>>> make/hotspot/gensrc/GensrcDtrace.gmk >>>>>>>> make/hotspot/lib/JvmDtraceObjects.gmk >>>>>>>> It also impacts some other make files. >>>>>>> I believe you?ll find that these changes were included in the build system patch: >>>>>>> >>>>>>> https://mail.openjdk.java.net/pipermail/build-dev/2020-May/027342.html >>>>>>> http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/build/open/webrev/ >>>>>>> >>>>>>> Let me know if I missed something. >>>>>> >>>>>> The file make/hotspot/src/native/dtrace/generateJvmOffsets.cpp is for Solaris only, and so, can be removed. >>>>>> It is for libjvm_db.so (provider for Solaris Pstack) and jhelper.d (provider for Solaris DTrace jstack action). >>>>>> The jstack action (prints mixed java+native stack traces) was never implemented other than for Solaris. >>>>> >>>>> I wonder if this can be used to implement the same thing on linux and if we can keep this? Thoughts? >>>> >>>> I was also thinking about it. And DTrace kind of exists on Mac OS as well. >>>> Yes, it can be used. But It will require the jstack action implementation and the jhelper use on the DTrace side. >>> >>>> The same is true for the libjvm_db.so. It could be used in the Linux Pstack utility to print mixed stack traces. >>>> (I see some pstack projects like this: https://github.com/peadar/pstack ) >>> Yes, I was thinking specifically of the pstack utility and not DTrace, just to get Java frames in ptrace. Could libjvm_db.so be used for that? >> >> Yes, it can be. >> The work needs to be done on the Linux side to get use of the libjvm_db.so. >> We worked with the Solaris team in the past to make the Pstack to print java frames (including virtual ones). >> The libjvm_db API came out from discussions with them. >> The interface is pretty simple: >> The Pstack does one initialization call and then in a loop requests the java frame strings from the register context (pc, sp and fp). >> Each time the libjvm_db returns corrected register state to help with the stack walking steps. >> It is kind of assisted stack walking. > > We had a local chat with Mikael and Coleen on this. > The conclusion is that we?ll remove it for now and restore it later if needed. > This folder has to be also removed with this: > src/java.base/solaris/native/libjvm_db Thank you Serguei & Coleen for the discussion! As we concluded the code will still be there in the history in case we need inspiration, leaving it in (without actually being used) will just lead to inevitable bit-rot. Cheers, Mikael -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikael.vidstedt at oracle.com Thu May 7 05:44:29 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 6 May 2020 22:44:29 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> Message-ID: <4BE043F6-A526-4B1A-8A7F-672270D398EA@oracle.com> To summarize the status: I have not made any changes to the serviceability patch itself, but I did remove generateJvmOffsets.cpp as part of the build system patch. I believe the following items remain: * File follow-up(s) to update comments * File follow-up to rename the #ifdef guards in src/jdk/jdwp.agent/*/native/libjdwp/path_md.h * File follow-up to remove "#undef CS? in src/hotspot/share/prims/methodHandles.cpp * File follow-up to investigate what to do about dtrace going forward (on linux & mac) Please let me know if I missed anything! Cheers, Mikael > On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: > > > Please review this change which implements part of JEP 381: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ > JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 > > > Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! > > > Background: > > Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. > > For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. > > In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. > > > Testing: > > A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. > > Cheers, > Mikael > > [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ > From kim.barrett at oracle.com Thu May 7 06:35:22 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 7 May 2020 02:35:22 -0400 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> References: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> Message-ID: > On May 7, 2020, at 1:35 AM, Mikael Vidstedt wrote: > > > New webrev here: > > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot/open/webrev/ > incremental: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot.incr/open/webrev/ > > Remaining items: > > * File follow-up to remove STACK_BIAS > > * File follow-ups to change/update/remove flags and/or flag documentation: UseLWPSynchronization, BranchOnRegister, LIRFillDelaySlots, ArrayAllocatorMallocLimit, ThreadPriorityPolicy > > * File follow-up(s) to update comments ("solaris", ?sparc?, ?solstudio?, ?sunos?, ?sun studio?, ?s compiler bug?, ?niagara?, ?) > > > Please let me know if there?s something I have missed! Looks good. From igor.ignatyev at oracle.com Thu May 7 14:39:50 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 7 May 2020 07:39:50 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: <7F17A7EF-C9B3-40A0-816B-53614A56B7CA@oracle.com> References: <7F17A7EF-C9B3-40A0-816B-53614A56B7CA@oracle.com> Message-ID: <95E6D883-D696-466A-80BD-4ADE420DBA43@oracle.com> Hi Mikael, yes, Vladimir's reply made it clear, let's hope all the needed changes got upstream before the next graal update so it goes smoothly. Cheers, -- Igor > On May 6, 2020, at 10:27 PM, Mikael Vidstedt wrote: > > > Igor, thank you for the review, and again for helping make the test changes in the first place! :) > > I hope Vladimir?s reply clarifies how we?re planning on handling the Graal related changes. > > Cheers, > Mikael > >> On May 4, 2020, at 2:29 PM, Igor Ignatyev wrote: >> >> Hi Mikael, >> >> the changes in /test/ look good to me. >> >> I have a question regarding src/jdk.internal.vm.compiler/*, aren't these files part of graal-compiler and hence will be brought back by the next graal update? >> >> Thanks, >> -- Igor >> >>> On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: >>> >>> >>> Please review this change which implements part of JEP 381: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >>> >>> >>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>> >>> >>> Background: >>> >>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>> >>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>> >>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>> >>> A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! >>> >>> Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. >>> >>> Testing: >>> >>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>> >>> Cheers, >>> Mikael >>> >>> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>> >> > From serguei.spitsyn at oracle.com Thu May 7 21:43:50 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 7 May 2020 14:43:50 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (serviceability) In-Reply-To: <4BE043F6-A526-4B1A-8A7F-672270D398EA@oracle.com> References: <9D521D7D-CE0E-44AD-828E-B2AC1AD31198@oracle.com> <4BE043F6-A526-4B1A-8A7F-672270D398EA@oracle.com> Message-ID: Hi Mikael, The summary looks correct. The fix looks good to me. Thanks, Serguei On 5/6/20 22:44, Mikael Vidstedt wrote: > To summarize the status: > > I have not made any changes to the serviceability patch itself, but I did remove generateJvmOffsets.cpp as part of the build system patch. I believe the following items remain: > > * File follow-up(s) to update comments > > * File follow-up to rename the #ifdef guards in src/jdk/jdwp.agent/*/native/libjdwp/path_md.h > > * File follow-up to remove "#undef CS? in src/hotspot/share/prims/methodHandles.cpp > > * File follow-up to investigate what to do about dtrace going forward (on linux & mac) > > Please let me know if I missed anything! > > Cheers, > Mikael > >> On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: >> >> >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/serviceability/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> >> >> Background: >> >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> >> >> Testing: >> >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> >> Cheers, >> Mikael >> >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >> From alexey.menkov at oracle.com Fri May 8 01:19:00 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 7 May 2020 18:19:00 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> Message-ID: <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> On 05/01/2020 15:22, Alex Menkov wrote: > Hi all, > > Please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8243012 > > The change fixes security note in the java.lang.instrument javadoc. > > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.1/ > > --alex From chris.plummer at oracle.com Fri May 8 06:44:03 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 7 May 2020 23:44:03 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> Message-ID: <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> Hi Daniil, The changes look good. thanks, Chris On 5/4/20 1:05 PM, Daniil Titov wrote: > Hi Chris, > > Please review a new version of webrev [1] that addresses your comments. > > For the following 3 tests that showed the increase of the execution time with -Xcomp > more than 5 minutes this version of the change strips -Xcomp option when > forwarding VM arguments to j*-tools : > > -- serviceability/sa/sadebugd/SADebugDTest.java, > -- serviceability/sa/sadebugd/DebugdConnectTest.java, > -- serviceability/sa/ClhsdbJstackXcompStress.java > > The execution time for the rest of the tests when running with -Xcomp was increased > within 1 and half minute. > > > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 > > Thank you, > Daniil > > > ?On 4/27/20, 12:55 PM, "Chris Plummer" wrote: > > Hi Daniil, > > Overall it looks good. A few comments below. > > Can you add a comment to TestSysProps.java indicating the reason for > checking if the line starts with "[". > > In JDKToolLauncher you have an extra empty line: > > 112 * Any platform specific arguments required for running the > tool are > 113 * automatically added. > 114 * > 115 * > 116 * @param args > > In OutputAnalyzer.java, I wonder if all these matching APIs you updated > should by default just include the version output in their filtering. > > As for the added time to execute, I would suggest possibly stripping > -Xcomp from the few outliers, and I would mostly focus on how much > longer it takes, not how many times longer. For example, increasing from > 10 seconds to 40 seconds is not a big deal. Increasing from 10 minutes > to 20 minutes is. > > SADebugDTest creates 8 tool processes so, that's probably the reason for > the big increase, although I'm not sure why it is kind of slow in the > first place. It does nothing more on each iteration than launch "jhsdb > debugd", which will connect to the debuggee, and then is killed off. > Maybe there is something slow with connecting, especial with RMI. > > thanks, > > Chris > > On 4/27/20 12:07 PM, Daniil Titov wrote: > > Please review the change [1] that ensures that VM and test options are forwarded to > > j*-tools when they are launched from serviceability/sa tests. > > > > The tests that expect an empty output were corrected to ignore the product version printed > > in the error stream since in some tiers the tests are run with ' -showversion' VM option (tier3). > > > > In test serviceability/sa/TestSysProps.java the code that counts the system properties was corrected > > to ignore the debug output when the test is run with " -Xlog:cds=debug" option (tier4). > > > > Testing: Mach5 tests for tier1 - tier7 passed. > > > > I also run the test with -XComp at Mach5 linux-x64-debug builds before and after the changes > > and for the most of the tests the overhead is about 2 times although for > > serviceability/sa/sadebugd/SADebugDTest.java it spikes up to 5 times. Probably at least for some tests > > it makes to filter out some properties (e.g. -Xcomp) before forwarding them to j*-tools. > > > > serviceability/sa/sadebugd/SADebugDTest.java, before : 2m 23s , after:11m 07s > > serviceability/sa/sadebugd/TestJmapCore.java, before : 42s , after:1m 09s > > serviceability/sa/TestSysProps.java, before : 36s , after: 1m 27s > > > > > > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 > > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 > > > > Thank you, > > Daniil > > > > > > > > From chris.plummer at oracle.com Fri May 8 07:21:20 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 8 May 2020 00:21:20 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> Message-ID: Hi Daniil, I just noticed you didn't update the tests in jdk/sun/tools/jhsdb. Do you think these should be done also? Chris On 5/7/20 11:44 PM, Chris Plummer wrote: > Hi Daniil, > > The changes look good. > > thanks, > > Chris > > On 5/4/20 1:05 PM, Daniil Titov wrote: >> Hi Chris, >> >> Please review a new version of webrev [1] that addresses your comments. >> >> For the following 3 tests that showed the increase of the execution >> time with -Xcomp >> more than 5 minutes this version of the change? strips -Xcomp option >> when >> forwarding VM? arguments to? j*-tools? : >> >> ??? -- serviceability/sa/sadebugd/SADebugDTest.java, >> ??? -- serviceability/sa/sadebugd/DebugdConnectTest.java, >> ??? -- serviceability/sa/ClhsdbJstackXcompStress.java >> >> The execution time for the rest of the tests when running with -Xcomp >> was increased >> within 1 and half minute. >> >> >> [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ >> [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> >> Thank you, >> ? Daniil >> >> >> ?On 4/27/20, 12:55 PM, "Chris Plummer" wrote: >> >> ???? Hi Daniil, >> >> ???? Overall it looks good. A few comments below. >> >> ???? Can you add a comment to TestSysProps.java indicating the reason >> for >> ???? checking if the line starts with "[". >> >> ???? In JDKToolLauncher you have an extra empty line: >> >> ?????? 112????? * Any platform specific arguments required for >> running the >> ???? tool are >> ?????? 113????? * automatically added. >> ?????? 114????? * >> ?????? 115????? * >> ?????? 116????? * @param args >> >> ???? In OutputAnalyzer.java, I wonder if all these matching APIs you >> updated >> ???? should by default just include the version output in their >> filtering. >> >> ???? As for the added time to execute, I would suggest possibly >> stripping >> ???? -Xcomp from the few outliers, and I would mostly focus on how much >> ???? longer it takes, not how many times longer. For example, >> increasing from >> ???? 10 seconds to 40 seconds is not a big deal. Increasing from 10 >> minutes >> ???? to 20 minutes is. >> >> ???? SADebugDTest creates 8 tool processes so, that's probably the >> reason for >> ???? the big increase, although I'm not sure why it is kind of slow >> in the >> ???? first place. It does nothing more on each iteration than launch >> "jhsdb >> ???? debugd", which will connect to the debuggee, and then is killed >> off. >> ???? Maybe there is something slow with connecting, especial with RMI. >> >> ???? thanks, >> >> ???? Chris >> >> ???? On 4/27/20 12:07 PM, Daniil Titov wrote: >> ???? > Please review the change [1] that ensures that VM and test >> options are forwarded to >> ???? > j*-tools when they are launched from serviceability/sa tests. >> ???? > >> ???? > The tests that expect an empty output? were corrected to >> ignore the product version printed >> ???? > in the error stream since in some? tiers the tests are run >> with ' -showversion' VM option (tier3). >> ???? > >> ???? > In test serviceability/sa/TestSysProps.java the code that >> counts the system properties? was? corrected >> ???? > to ignore the debug output when the test is run with " >> -Xlog:cds=debug" option (tier4). >> ???? > >> ???? > Testing:? Mach5 tests for tier1 - tier7 passed. >> ???? > >> ???? > I also run the test with -XComp at Mach5 linux-x64-debug >> builds before and after the changes >> ???? > and for? the most of the tests the? overhead is about 2 times >> although for >> ???? > serviceability/sa/sadebugd/SADebugDTest.java it spikes up to 5 >> times.? Probably at least for some tests >> ???? > it makes to filter out some properties (e.g. -Xcomp) before >> forwarding them to j*-tools. >> ???? > >> ???? > serviceability/sa/sadebugd/SADebugDTest.java, before : 2m 23s >> ,? after:11m 07s >> ???? > serviceability/sa/sadebugd/TestJmapCore.java,? before : 42s ,? >> after:1m 09s >> ???? > serviceability/sa/TestSysProps.java, before : 36s , after: 1m 27s >> ???? > >> ???? > >> ???? > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 >> ???? > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> ???? > >> ???? > Thank you, >> ???? > Daniil >> ???? > >> ???? > >> >> >> >> > > From daniil.x.titov at oracle.com Fri May 8 07:44:52 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Fri, 08 May 2020 00:44:52 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> Message-ID: Hi Chris, Yes. I think it makes sense to update these tests as well. I will include them in the new version of the webrev. Thank you, Daniil ?On 5/8/20, 12:21 AM, "Chris Plummer" wrote: Hi Daniil, I just noticed you didn't update the tests in jdk/sun/tools/jhsdb. Do you think these should be done also? Chris On 5/7/20 11:44 PM, Chris Plummer wrote: > Hi Daniil, > > The changes look good. > > thanks, > > Chris > > On 5/4/20 1:05 PM, Daniil Titov wrote: >> Hi Chris, >> >> Please review a new version of webrev [1] that addresses your comments. >> >> For the following 3 tests that showed the increase of the execution >> time with -Xcomp >> more than 5 minutes this version of the change strips -Xcomp option >> when >> forwarding VM arguments to j*-tools : >> >> -- serviceability/sa/sadebugd/SADebugDTest.java, >> -- serviceability/sa/sadebugd/DebugdConnectTest.java, >> -- serviceability/sa/ClhsdbJstackXcompStress.java >> >> The execution time for the rest of the tests when running with -Xcomp >> was increased >> within 1 and half minute. >> >> >> [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ >> [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> >> Thank you, >> Daniil >> >> >> ?On 4/27/20, 12:55 PM, "Chris Plummer" wrote: >> >> Hi Daniil, >> >> Overall it looks good. A few comments below. >> >> Can you add a comment to TestSysProps.java indicating the reason >> for >> checking if the line starts with "[". >> >> In JDKToolLauncher you have an extra empty line: >> >> 112 * Any platform specific arguments required for >> running the >> tool are >> 113 * automatically added. >> 114 * >> 115 * >> 116 * @param args >> >> In OutputAnalyzer.java, I wonder if all these matching APIs you >> updated >> should by default just include the version output in their >> filtering. >> >> As for the added time to execute, I would suggest possibly >> stripping >> -Xcomp from the few outliers, and I would mostly focus on how much >> longer it takes, not how many times longer. For example, >> increasing from >> 10 seconds to 40 seconds is not a big deal. Increasing from 10 >> minutes >> to 20 minutes is. >> >> SADebugDTest creates 8 tool processes so, that's probably the >> reason for >> the big increase, although I'm not sure why it is kind of slow >> in the >> first place. It does nothing more on each iteration than launch >> "jhsdb >> debugd", which will connect to the debuggee, and then is killed >> off. >> Maybe there is something slow with connecting, especial with RMI. >> >> thanks, >> >> Chris >> >> On 4/27/20 12:07 PM, Daniil Titov wrote: >> > Please review the change [1] that ensures that VM and test >> options are forwarded to >> > j*-tools when they are launched from serviceability/sa tests. >> > >> > The tests that expect an empty output were corrected to >> ignore the product version printed >> > in the error stream since in some tiers the tests are run >> with ' -showversion' VM option (tier3). >> > >> > In test serviceability/sa/TestSysProps.java the code that >> counts the system properties was corrected >> > to ignore the debug output when the test is run with " >> -Xlog:cds=debug" option (tier4). >> > >> > Testing: Mach5 tests for tier1 - tier7 passed. >> > >> > I also run the test with -XComp at Mach5 linux-x64-debug >> builds before and after the changes >> > and for the most of the tests the overhead is about 2 times >> although for >> > serviceability/sa/sadebugd/SADebugDTest.java it spikes up to 5 >> times. Probably at least for some tests >> > it makes to filter out some properties (e.g. -Xcomp) before >> forwarding them to j*-tools. >> > >> > serviceability/sa/sadebugd/SADebugDTest.java, before : 2m 23s >> , after:11m 07s >> > serviceability/sa/sadebugd/TestJmapCore.java, before : 42s , >> after:1m 09s >> > serviceability/sa/TestSysProps.java, before : 36s , after: 1m 27s >> > >> > >> > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 >> > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> > >> > Thank you, >> > Daniil >> > >> > >> >> >> >> > > From thomas.schatzl at oracle.com Fri May 8 09:34:15 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 8 May 2020 11:34:15 +0200 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> References: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> Message-ID: <28de2cd7-6209-d780-0975-ab701e6f24f9@oracle.com> Hi, On 07.05.20 07:35, Mikael Vidstedt wrote: > > New webrev here: > > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot/open/webrev/ > incremental: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot.incr/open/webrev/ > > Remaining items: > > * File follow-up to remove STACK_BIAS > > * File follow-ups to change/update/remove flags and/or flag documentation: UseLWPSynchronization, BranchOnRegister, LIRFillDelaySlots, ArrayAllocatorMallocLimit, ThreadPriorityPolicy > > * File follow-up(s) to update comments ("solaris", ?sparc?, ?solstudio?, ?sunos?, ?sun studio?, ?s compiler bug?, ?niagara?, ?) > > > Please let me know if there?s something I have missed! Looks good. Thomas From harold.seigel at oracle.com Fri May 8 17:22:27 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 8 May 2020 13:22:27 -0400 Subject: RFR 8244105: JDK 11.0.7 -Xlog gc issue with file path if not exist Message-ID: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> Hi, Please review this fix for JDK-8244105.? The fix continues program execution even when specified logging options are invalid.? Previously, invalid logging options terminated the program.? Now, a warning is issued.? For example: > java -Xlog:"gc*:file=/dont/exist" -version [0.001s][error][logging] Error opening log file '/dont/exist': No such file or directory [0.001s][error][logging] Initialization of output 'file=/dont/exist' using options '(null)' failed. Java HotSpot(TM) 64-Bit Server VM warning: Invalid -Xlog option '-Xlog:gc*:file=/dont/exist', see error log for details. java version "15-internal" 2020-09-15 Java(TM) SE Runtime Environment (fastdebug build 15-internal+0-2020-05-08-1313404.hseigel.bug8244105) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 15-internal+0-2020-05-08-1313404.hseigel.bug8244105, mixed mode, sharing) Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8244105/webrev/index.html JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8244105 The fix was regression tested by running Mach5 tiers 1 and 2 tests and builds on Linux-x64, Solaris, Windows, and Mac OS X and by running Mach5 tiers 3-5 tests on Linux-x64. Thanks, Harold -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Fri May 8 19:48:02 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 8 May 2020 15:48:02 -0400 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> References: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> Message-ID: <048a8e25-fec4-5304-8763-6a4472bfbf54@oracle.com> On 5/7/20 1:35 AM, Mikael Vidstedt wrote: > New webrev here: > > webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot/open/webrev/ This pretty much says it all: > Summary of changes:??? 90904 lines changed: 8 ins; 90725 del; 171 mod; 103780 unchg My review is focused on looking at the changes and not looking for missed changes. I figure there's enough work here just looking at the changes to keep me occupied for a while and enough people have posted comments about finding other things to be examined, etc... Unlike my normal reviews, I won't be listing all the touched files; (there's _only_ 427 of them...) Don't forget to make a copyright year update pass before you push. src/hotspot/os/posix/os_posix.hpp ??????? L174 ??? old L175 #ifndef SOLARIS ??????? L176 ?????? nit - on most of this style of deletion you also got rid of ?????? one of the blank lines, but not here. src/hotspot/share/utilities/dtrace.hpp ??? old L42: #elif defined(__APPLE__) ??? old L44: #include ??? old L45: #else ??? new L32: #include ??????? was previous included only for __APPLE__ and it ??????? is now there for every platform. Any particular reason? Thumbs up! Dan > incremental: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot.incr/open/webrev/ > > Remaining items: > > * File follow-up to remove STACK_BIAS > > * File follow-ups to change/update/remove flags and/or flag documentation: UseLWPSynchronization, BranchOnRegister, LIRFillDelaySlots, ArrayAllocatorMallocLimit, ThreadPriorityPolicy > > * File follow-up(s) to update comments ("solaris", ?sparc?, ?solstudio?, ?sunos?, ?sun studio?, ?s compiler bug?, ?niagara?, ?) > > > Please let me know if there?s something I have missed! > > Cheers, > Mikael > >> On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: >> >> >> Please review this change which implements part of JEP 381: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >> >> >> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >> >> >> Background: >> >> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >> >> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >> >> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >> >> A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! >> >> Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. >> >> Testing: >> >> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >> >> Cheers, >> Mikael >> >> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >> From daniil.x.titov at oracle.com Fri May 8 20:12:51 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Fri, 08 May 2020 13:12:51 -0700 Subject: RFR: 8194874 SA: Remove scripts with sa-jdi.jar dependencies. Message-ID: <0385540B-9246-47BC-BC98-26DFE9D4082D@oracle.com> Please review the change [1] that removes the scripts with dependencies to sa-jdi.jar that left after sa-jdi.jar was removed in JDK 9 [3]. Mach5 tier1-tier3 tests and doc build successfully passed. [1] Webrev: http://cr.openjdk.java.net/~dtitov/8194874/webrev.01/ [2] Jira issue: https://bugs.openjdk.java.net/browse/JDK-8194874 [3] https://bugs.openjdk.java.net/browse/JDK-8158050 Thank you, Daniil From alexey.menkov at oracle.com Fri May 8 21:27:48 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 8 May 2020 14:27:48 -0700 Subject: RFR: 8194874 SA: Remove scripts with sa-jdi.jar dependencies. In-Reply-To: <0385540B-9246-47BC-BC98-26DFE9D4082D@oracle.com> References: <0385540B-9246-47BC-BC98-26DFE9D4082D@oracle.com> Message-ID: <6863bd0a-63c7-928c-b7c6-55a1982a076b@oracle.com> Looks good --alex On 05/08/2020 13:12, Daniil Titov wrote: > Please review the change [1] that removes the scripts with dependencies to sa-jdi.jar > that left after sa-jdi.jar was removed in JDK 9 [3]. > > Mach5 tier1-tier3 tests and doc build successfully passed. > > [1] Webrev: http://cr.openjdk.java.net/~dtitov/8194874/webrev.01/ > [2] Jira issue: https://bugs.openjdk.java.net/browse/JDK-8194874 > [3] https://bugs.openjdk.java.net/browse/JDK-8158050 > > Thank you, > Daniil > > From chris.plummer at oracle.com Fri May 8 21:33:53 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 8 May 2020 14:33:53 -0700 Subject: RFR: 8194874 SA: Remove scripts with sa-jdi.jar dependencies. In-Reply-To: <0385540B-9246-47BC-BC98-26DFE9D4082D@oracle.com> References: <0385540B-9246-47BC-BC98-26DFE9D4082D@oracle.com> Message-ID: <55af7ee6-a6ae-280a-945a-af4cb599be6a@oracle.com> LGTM. Chris On 5/8/20 1:12 PM, Daniil Titov wrote: > Please review the change [1] that removes the scripts with dependencies to sa-jdi.jar > that left after sa-jdi.jar was removed in JDK 9 [3]. > > Mach5 tier1-tier3 tests and doc build successfully passed. > > [1] Webrev: http://cr.openjdk.java.net/~dtitov/8194874/webrev.01/ > [2] Jira issue: https://bugs.openjdk.java.net/browse/JDK-8194874 > [3] https://bugs.openjdk.java.net/browse/JDK-8158050 > > Thank you, > Daniil > > From alexey.menkov at oracle.com Sat May 9 01:14:32 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 8 May 2020 18:14:32 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file Message-ID: Hi all, please review the fix for https://bugs.openjdk.java.net/browse/JDK-8235211 webrev: http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ Test failures are caused by deadlock during attach listener restarting: check_socket_file function aborts socket listening and waits while attach listener state becomes AL_NOT_INITIALIZED (it happens when AttachListener::dequeue returns NULL). AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. To solve it ThreadBlockInVM was added inside waiting cycle in check_socket_file. Other changes: - made _listener (and _shutdown for aix) volatile as they are used by 2 threads (attach listener thread and signal handler thread) without synchronization; - added close() for listening socket on aix (before it had only shutdown() for it); - additional logging and some cleanup in the test; - added handling of LingeredApp hang. --alex From linzang at tencent.com Sat May 9 07:47:13 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Sat, 9 May 2020 07:47:13 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) In-Reply-To: References: <94C0D11E-F395-4FE4-9ECE-5ECC84B3AE1B@tencent.com> <09702D94-F53C-413D-A156-B7390D689BC6@tencent.com> <4751f476-1e7a-490f-80c5-96b58eb25191@oracle.com> Message-ID: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin ?On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > From suenaga at oss.nttdata.com Sat May 9 15:28:11 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Sun, 10 May 2020 00:28:11 +0900 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: Message-ID: <7c8617ec-4c13-3f34-0096-a87d6d16607b@oss.nttdata.com> Hi Alex, It seems to be hanged in check_socket_file() because AttachListener::deque() could not return due to safepoint. So it is good idea to use ThreadBlockInVM, but I think we can change as below to reduce overhead: ``` { ThreadBlockInVM tbivm(JavaThread::current()); while (AttachListener::transit_state(AL_INITIALIZING, AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { os::naked_yield(); } } ``` It might consume much CPU time with long safepoint, but I think it is corner case because this issue occurs intermittently. Thanks, Yasumasa On 2020/05/09 10:14, Alex Menkov wrote: > Hi all, > > please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8235211 > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ > > Test failures are caused by deadlock during attach listener restarting: > check_socket_file function aborts socket listening and waits while attach listener state becomes AL_NOT_INITIALIZED (it happens when AttachListener::dequeue returns NULL). > AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. > To solve it ThreadBlockInVM was added inside waiting cycle in check_socket_file. > > Other changes: > - made _listener (and _shutdown for aix) volatile as they are used by 2 threads (attach listener thread and signal handler thread) without synchronization; > - added close() for listening socket on aix (before it had only shutdown() for it); > - additional logging and some cleanup in the test; > - added handling of LingeredApp hang. > > --alex From daniil.x.titov at oracle.com Sat May 9 16:29:32 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Sat, 09 May 2020 09:29:32 -0700 Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools Message-ID: Please review a change[1] that centralizes the signature processing in serviceability tools to make it capable of being easily extensible in the future. Testing: Mach5 tier1-tier3 tests successfully passed. [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.01 [2] https://bugs.openjdk.java.net/browse/JDK-8241080 Thank you, Daniil From serguei.spitsyn at oracle.com Mon May 11 07:31:03 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 00:31:03 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: Message-ID: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> An HTML attachment was scrubbed... URL: From dmytro.sheyko.jdk at gmail.com Mon May 11 15:06:25 2020 From: dmytro.sheyko.jdk at gmail.com (dmytro sheyko) Date: Mon, 11 May 2020 18:06:25 +0300 Subject: RFR 8244105: JDK 11.0.7 -Xlog gc issue with file path if not exist In-Reply-To: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> References: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> Message-ID: Hi, I think it would be very convenient for app developers if JVM were able to create intermediate directories to gc.log file if they do not exist. I.e. $ if [ -f logs/gc.log ]; then echo "log file exists"; else echo "log file does not exist"; fi log file does not exist $ if [ -d logs ]; then echo "log directory exists"; else echo "log directory does not exist"; fi log directory does not exist $ java -Xlog:"gc*:file=logs/gc.log" -version ... $ if [ -d logs ]; then echo "log directory exists"; else echo "log directory does not exist"; fi log directory exists $ if [ -f logs/gc.log ]; then echo "log file exists"; else echo "log file does not exist"; fi log file exists Thanks, Dmytro On Fri, May 8, 2020 at 8:31 PM Harold Seigel wrote: > Hi, > > Please review this fix for JDK-8244105. The fix continues program > execution even when specified logging options are invalid. Previously, > invalid logging options terminated the program. Now, a warning is issued. > For example: > > > java -Xlog:"gc*:file=/dont/exist" -version > [0.001s][error][logging] Error opening log file '/dont/exist': No such > file or directory > [0.001s][error][logging] Initialization of output 'file=/dont/exist' using > options '(null)' failed. > Java HotSpot(TM) 64-Bit Server VM warning: Invalid -Xlog option > '-Xlog:gc*:file=/dont/exist', see error log for details. > > java version "15-internal" 2020-09-15 > Java(TM) SE Runtime Environment (fastdebug build > 15-internal+0-2020-05-08-1313404.hseigel.bug8244105) > Java HotSpot(TM) 64-Bit Server VM (fastdebug build > 15-internal+0-2020-05-08-1313404.hseigel.bug8244105, mixed mode, sharing) > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8244105/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8244105 > > The fix was regression tested by running Mach5 tiers 1 and 2 tests and > builds on Linux-x64, Solaris, Windows, and Mac OS X and by running Mach5 > tiers 3-5 tests on Linux-x64. > > Thanks, Harold > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Mon May 11 16:43:59 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Mon, 11 May 2020 09:43:59 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> Message-ID: <0E6DBBFE-4395-45F5-9CB3-C023ADD7DD2F@oracle.com> Hi Chris, Please review a new version of the fix[1] that also updates jdk/sun/tools tests. Testing: Mach5 tier1-tier7 tests successfully passed. [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.03/ [2] ] https://bugs.openjdk.java.net/browse/JDK-8242009 Thank you, Daniil ?On 5/8/20, 12:21 AM, "Chris Plummer" wrote: Hi Daniil, I just noticed you didn't update the tests in jdk/sun/tools/jhsdb. Do you think these should be done also? Chris On 5/7/20 11:44 PM, Chris Plummer wrote: > Hi Daniil, > > The changes look good. > > thanks, > > Chris > > On 5/4/20 1:05 PM, Daniil Titov wrote: >> Hi Chris, >> >> Please review a new version of webrev [1] that addresses your comments. >> >> For the following 3 tests that showed the increase of the execution >> time with -Xcomp >> more than 5 minutes this version of the change strips -Xcomp option >> when >> forwarding VM arguments to j*-tools : >> >> -- serviceability/sa/sadebugd/SADebugDTest.java, >> -- serviceability/sa/sadebugd/DebugdConnectTest.java, >> -- serviceability/sa/ClhsdbJstackXcompStress.java >> >> The execution time for the rest of the tests when running with -Xcomp >> was increased >> within 1 and half minute. >> >> >> [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ >> [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> >> Thank you, >> Daniil >> >> >> ?On 4/27/20, 12:55 PM, "Chris Plummer" wrote: >> >> Hi Daniil, >> >> Overall it looks good. A few comments below. >> >> Can you add a comment to TestSysProps.java indicating the reason >> for >> checking if the line starts with "[". >> >> In JDKToolLauncher you have an extra empty line: >> >> 112 * Any platform specific arguments required for >> running the >> tool are >> 113 * automatically added. >> 114 * >> 115 * >> 116 * @param args >> >> In OutputAnalyzer.java, I wonder if all these matching APIs you >> updated >> should by default just include the version output in their >> filtering. >> >> As for the added time to execute, I would suggest possibly >> stripping >> -Xcomp from the few outliers, and I would mostly focus on how much >> longer it takes, not how many times longer. For example, >> increasing from >> 10 seconds to 40 seconds is not a big deal. Increasing from 10 >> minutes >> to 20 minutes is. >> >> SADebugDTest creates 8 tool processes so, that's probably the >> reason for >> the big increase, although I'm not sure why it is kind of slow >> in the >> first place. It does nothing more on each iteration than launch >> "jhsdb >> debugd", which will connect to the debuggee, and then is killed >> off. >> Maybe there is something slow with connecting, especial with RMI. >> >> thanks, >> >> Chris >> >> On 4/27/20 12:07 PM, Daniil Titov wrote: >> > Please review the change [1] that ensures that VM and test >> options are forwarded to >> > j*-tools when they are launched from serviceability/sa tests. >> > >> > The tests that expect an empty output were corrected to >> ignore the product version printed >> > in the error stream since in some tiers the tests are run >> with ' -showversion' VM option (tier3). >> > >> > In test serviceability/sa/TestSysProps.java the code that >> counts the system properties was corrected >> > to ignore the debug output when the test is run with " >> -Xlog:cds=debug" option (tier4). >> > >> > Testing: Mach5 tests for tier1 - tier7 passed. >> > >> > I also run the test with -XComp at Mach5 linux-x64-debug >> builds before and after the changes >> > and for the most of the tests the overhead is about 2 times >> although for >> > serviceability/sa/sadebugd/SADebugDTest.java it spikes up to 5 >> times. Probably at least for some tests >> > it makes to filter out some properties (e.g. -Xcomp) before >> forwarding them to j*-tools. >> > >> > serviceability/sa/sadebugd/SADebugDTest.java, before : 2m 23s >> , after:11m 07s >> > serviceability/sa/sadebugd/TestJmapCore.java, before : 42s , >> after:1m 09s >> > serviceability/sa/TestSysProps.java, before : 36s , after: 1m 27s >> > >> > >> > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 >> > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> > >> > Thank you, >> > Daniil >> > >> > >> >> >> >> > > From chris.plummer at oracle.com Mon May 11 17:41:58 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 11 May 2020 10:41:58 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: Message-ID: <3d9a758b-3654-4c22-da4b-312dec96694f@oracle.com> Hi Alex, ?228???????????? // if for a reason the app hangs, we don't want to wait test timeout ?229???????????? if (!appProcess.waitFor(Utils.adjustTimeout(appWaitTime), TimeUnit.SECONDS)) { Can you fix the comment. Maybe: ?228???????????? // If the app hangs, we don't want to wait for the to test timeout. And your use of appWaitTime had me look for other uses, and I noticed a pre-existing comment there that could use some work. Perhaps you can clean it up with this RFR. ?273????? * Waits the application to start with the default timeout. Should be "Waits for the application..." thanks, Chris On 5/8/20 6:14 PM, Alex Menkov wrote: > Hi all, > > please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8235211 > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ > > > Test failures are caused by deadlock during attach listener restarting: > check_socket_file function aborts socket listening and waits while > attach listener state becomes AL_NOT_INITIALIZED (it happens when > AttachListener::dequeue returns NULL). > AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. > To solve it ThreadBlockInVM was added inside waiting cycle in > check_socket_file. > > Other changes: > - made _listener (and _shutdown for aix) volatile as they are used by > 2 threads (attach listener thread and signal handler thread) without > synchronization; > - added close() for listening socket on aix (before it had only > shutdown() for it); > - additional logging and some cleanup in the test; > - added handling of LingeredApp hang. > > --alex From chris.plummer at oracle.com Mon May 11 17:54:56 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 11 May 2020 10:54:56 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: <0E6DBBFE-4395-45F5-9CB3-C023ADD7DD2F@oracle.com> References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> <0E6DBBFE-4395-45F5-9CB3-C023ADD7DD2F@oracle.com> Message-ID: Hi Daniil, Looks good. thanks, Chris On 5/11/20 9:43 AM, Daniil Titov wrote: > Hi Chris, > > Please review a new version of the fix[1] that also updates jdk/sun/tools tests. > > Testing: Mach5 tier1-tier7 tests successfully passed. > > > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.03/ > [2] ] https://bugs.openjdk.java.net/browse/JDK-8242009 > > Thank you, > Daniil > > ?On 5/8/20, 12:21 AM, "Chris Plummer" wrote: > > Hi Daniil, > > I just noticed you didn't update the tests in jdk/sun/tools/jhsdb. Do > you think these should be done also? > > Chris > > On 5/7/20 11:44 PM, Chris Plummer wrote: > > Hi Daniil, > > > > The changes look good. > > > > thanks, > > > > Chris > > > > On 5/4/20 1:05 PM, Daniil Titov wrote: > >> Hi Chris, > >> > >> Please review a new version of webrev [1] that addresses your comments. > >> > >> For the following 3 tests that showed the increase of the execution > >> time with -Xcomp > >> more than 5 minutes this version of the change strips -Xcomp option > >> when > >> forwarding VM arguments to j*-tools : > >> > >> -- serviceability/sa/sadebugd/SADebugDTest.java, > >> -- serviceability/sa/sadebugd/DebugdConnectTest.java, > >> -- serviceability/sa/ClhsdbJstackXcompStress.java > >> > >> The execution time for the rest of the tests when running with -Xcomp > >> was increased > >> within 1 and half minute. > >> > >> > >> [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ > >> [2] https://bugs.openjdk.java.net/browse/JDK-8242009 > >> > >> Thank you, > >> Daniil > >> > >> > >> ?On 4/27/20, 12:55 PM, "Chris Plummer" wrote: > >> > >> Hi Daniil, > >> > >> Overall it looks good. A few comments below. > >> > >> Can you add a comment to TestSysProps.java indicating the reason > >> for > >> checking if the line starts with "[". > >> > >> In JDKToolLauncher you have an extra empty line: > >> > >> 112 * Any platform specific arguments required for > >> running the > >> tool are > >> 113 * automatically added. > >> 114 * > >> 115 * > >> 116 * @param args > >> > >> In OutputAnalyzer.java, I wonder if all these matching APIs you > >> updated > >> should by default just include the version output in their > >> filtering. > >> > >> As for the added time to execute, I would suggest possibly > >> stripping > >> -Xcomp from the few outliers, and I would mostly focus on how much > >> longer it takes, not how many times longer. For example, > >> increasing from > >> 10 seconds to 40 seconds is not a big deal. Increasing from 10 > >> minutes > >> to 20 minutes is. > >> > >> SADebugDTest creates 8 tool processes so, that's probably the > >> reason for > >> the big increase, although I'm not sure why it is kind of slow > >> in the > >> first place. It does nothing more on each iteration than launch > >> "jhsdb > >> debugd", which will connect to the debuggee, and then is killed > >> off. > >> Maybe there is something slow with connecting, especial with RMI. > >> > >> thanks, > >> > >> Chris > >> > >> On 4/27/20 12:07 PM, Daniil Titov wrote: > >> > Please review the change [1] that ensures that VM and test > >> options are forwarded to > >> > j*-tools when they are launched from serviceability/sa tests. > >> > > >> > The tests that expect an empty output were corrected to > >> ignore the product version printed > >> > in the error stream since in some tiers the tests are run > >> with ' -showversion' VM option (tier3). > >> > > >> > In test serviceability/sa/TestSysProps.java the code that > >> counts the system properties was corrected > >> > to ignore the debug output when the test is run with " > >> -Xlog:cds=debug" option (tier4). > >> > > >> > Testing: Mach5 tests for tier1 - tier7 passed. > >> > > >> > I also run the test with -XComp at Mach5 linux-x64-debug > >> builds before and after the changes > >> > and for the most of the tests the overhead is about 2 times > >> although for > >> > serviceability/sa/sadebugd/SADebugDTest.java it spikes up to 5 > >> times. Probably at least for some tests > >> > it makes to filter out some properties (e.g. -Xcomp) before > >> forwarding them to j*-tools. > >> > > >> > serviceability/sa/sadebugd/SADebugDTest.java, before : 2m 23s > >> , after:11m 07s > >> > serviceability/sa/sadebugd/TestJmapCore.java, before : 42s , > >> after:1m 09s > >> > serviceability/sa/TestSysProps.java, before : 36s , after: 1m 27s > >> > > >> > > >> > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 > >> > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 > >> > > >> > Thank you, > >> > Daniil > >> > > >> > > >> > >> > >> > >> > > > > > > > From chris.plummer at oracle.com Mon May 11 18:12:41 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 11 May 2020 11:12:41 -0700 Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools In-Reply-To: References: Message-ID: Hi Daniil, Overall looks good. Just one minor thing. In ClassTypeImpl.c and ObjectReferenceImpl.c I think the following would be more readable with an if/else: ? 79???????? return JNI_FUNC_PTR(env,ExceptionOccurred)(env) ? AGENT_ERROR_JNI_EXCEPTION ? 80???????????? : JVMTI_ERROR_NONE; Also, have you filed a CR for the JDWP spec issues related to missing type information in some of the protocol packet descriptions? thanks, Chris On 5/9/20 9:29 AM, Daniil Titov wrote: > Please review a change[1] that centralizes the signature processing in serviceability tools to make it capable of being easily extensible in the future. > > Testing: Mach5 tier1-tier3 tests successfully passed. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.01 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > > Thank you, > Daniil > > From mikael.vidstedt at oracle.com Mon May 11 18:18:30 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Mon, 11 May 2020 11:18:30 -0700 Subject: RFR: 8244224: Implementation of JEP 381: Remove the Solaris and SPARC Ports (hotspot) In-Reply-To: <048a8e25-fec4-5304-8763-6a4472bfbf54@oracle.com> References: <394BD86E-EC91-440E-9936-696B2B453093@oracle.com> <048a8e25-fec4-5304-8763-6a4472bfbf54@oracle.com> Message-ID: <3810FA34-8AC5-45F0-B0DB-57C28324FECB@oracle.com> > On May 8, 2020, at 12:48 PM, Daniel D. Daugherty wrote: > > On 5/7/20 1:35 AM, Mikael Vidstedt wrote: >> New webrev here: >> >> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot/open/webrev/ > > This pretty much says it all: > > > Summary of changes: 90904 lines changed: 8 ins; 90725 del; 171 mod; 103780 unchg > > > My review is focused on looking at the changes and not looking for missed > changes. I figure there's enough work here just looking at the changes to > keep me occupied for a while and enough people have posted comments about > finding other things to be examined, etc... > > Unlike my normal reviews, I won't be listing all the touched files; > (there's _only_ 427 of them...) > > Don't forget to make a copyright year update pass before you push. Yup - I have added it in 10 different places on my TODO list to maximize the likelihood of me remembering it :) > > src/hotspot/os/posix/os_posix.hpp > L174 > old L175 #ifndef SOLARIS > L176 > nit - on most of this style of deletion you also got rid of > one of the blank lines, but not here. Oops, will fix. > src/hotspot/share/utilities/dtrace.hpp > old L42: #elif defined(__APPLE__) > old L44: #include > old L45: #else > new L32: #include > was previous included only for __APPLE__ and it > is now there for every platform. Any particular reason? No particular reason other than "it looks cleaner". I guess we could see if the include can be removed altogether. > Thumbs up! Thanks for the review!! Cheers, Mikael > >> incremental: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.01/hotspot.incr/open/webrev/ >> >> Remaining items: >> >> * File follow-up to remove STACK_BIAS >> >> * File follow-ups to change/update/remove flags and/or flag documentation: UseLWPSynchronization, BranchOnRegister, LIRFillDelaySlots, ArrayAllocatorMallocLimit, ThreadPriorityPolicy >> >> * File follow-up(s) to update comments ("solaris", ?sparc?, ?solstudio?, ?sunos?, ?sun studio?, ?s compiler bug?, ?niagara?, ?) >> >> >> Please let me know if there?s something I have missed! >> >> Cheers, >> Mikael >> >>> On May 3, 2020, at 10:12 PM, Mikael Vidstedt wrote: >>> >>> >>> Please review this change which implements part of JEP 381: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8244224 >>> webrev: http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/hotspot/open/webrev/ >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8241787 >>> >>> >>> Note: When reviewing this, please be aware that this exercise was *extremely* mind-numbing, so I appreciate your help reviewing all the individual changes carefully. You may want to get that coffee cup filled up (or whatever keeps you awake)! >>> >>> >>> Background: >>> >>> Because of the size of the total patch and wide range of areas touched, this patch is one out of in total six partial patches which together make up the necessary changes to remove the Solaris and SPARC ports. The other patches are being sent out for review to mailing lists appropriate for the respective areas the touch. An email will be sent to jdk-dev summarizing all the patches/reviews. To be clear: this patch is *not* in itself complete and stand-alone - all of the (six) patches are needed to form a complete patch. Some changes in this patch may look wrong or incomplete unless also looking at the corresponding changes in other areas. >>> >>> For convenience, I?m including a link below[1] to the full webrev, but in case you have comments on changes in other areas, outside of the files included in this thread, please provide those comments directly in the thread on the appropriate mailing list for that area if possible. >>> >>> In case it helps, the changes were effectively produced by searching for and updating any code mentioning ?solaris", ?sparc?, ?solstudio?, ?sunos?, etc. More information about the areas impacted can be found in the JEP itself. >>> >>> A big thank you to Igor Ignatyev for helping make the changes to the hotspot tests! >>> >>> Also, I have a short list of follow-ups which I?m going to look at separately from this JEP/patch, mainly related to command line options/flags which are no longer relevant and should be deprecated/obsoleted/removed. >>> >>> Testing: >>> >>> A slightly earlier version of this change successfully passed tier1-8, as well as client tier1-2. Additional testing will be done after the first round of reviews has been completed. >>> >>> Cheers, >>> Mikael >>> >>> [1] http://cr.openjdk.java.net/~mikael/webrevs/8244224/webrev.00/all/open/webrev/ >>> > From serguei.spitsyn at oracle.com Mon May 11 18:21:45 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 11:21:45 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> Message-ID: <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon May 11 18:52:52 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 May 2020 19:52:52 +0100 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> Message-ID: <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> On 11/05/2020 19:21, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > There is no need to repeat this: > ? "deploy applications thatpackage an agent with the application, > ?? or use tools that load agents into a running application" > > I'd suggest to rephrase it to something like: > > ? "Agents can transform classes in arbitrary ways at load time, transform > ?? modules, or transform the bytecode of methods of already loaded > classes. > ?? Developers or administrators that deploy agents are responsible for > their > ?? trustworthiness and must therefore verify each agent including the > content > ?? and structure of its JAR file." > > > Also, could you, please, replace: > ?*

The three ways to start an agent is described below. > > with: > ?*

The three ways to start an agent are described below. Serguei's suggestions look good. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon May 11 18:53:04 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 11:53:04 -0700 Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon May 11 19:01:43 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 12:01:43 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From Roger.Riggs at oracle.com Mon May 11 19:01:39 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Mon, 11 May 2020 12:01:39 -0700 (PDT) Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools In-Reply-To: References: Message-ID: <010be3c0-e4a3-78dc-5663-3e79f54c24ef@oracle.com> Hi Daniil, In the grand scheme of things, could servicability use the signature parsing support in HotSpot? Thanks, Roger On 5/9/20 12:29 PM, Daniil Titov wrote: > Please review a change[1] that centralizes the signature processing in serviceability tools to make it capable of being easily extensible in the future. > > Testing: Mach5 tier1-tier3 tests successfully passed. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.01 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > > Thank you, > Daniil > > From alexey.menkov at oracle.com Mon May 11 19:53:28 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 11 May 2020 12:53:28 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> Message-ID: <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> Hi Yasumasa, Serguei, Chris, Thank you for the feedback updated webrev (all suggestions are applied): http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > It looks good in general. > I have a couple of minor comments. > > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html > + // if for a reason the app hangs, we don't want to wait test timeout > > Nit: replace: wait test timeout => wait for test timeout > > I hope, you remember about copyright comments update. > > > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html > > Q1: How useful is this variable? : > AixAttachListener::_shutdown = false; > > ??? Why is it needed on aix but not on other platforms? IFAIU AIX has issue with accept() - it hangs if the socket is closed. I don't think this _shutdown flag helps a lot, but I don't want to make significant changes in the AIX code as I cannot test it. --alex > > Thanks, > Serguei > > > On 5/8/20 18:14, Alex Menkov wrote: >> Hi all, >> >> please review the fix for >> https://bugs.openjdk.java.net/browse/JDK-8235211 >> webrev: >> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >> >> >> Test failures are caused by deadlock during attach listener restarting: >> check_socket_file function aborts socket listening and waits while >> attach listener state becomes AL_NOT_INITIALIZED (it happens when >> AttachListener::dequeue returns NULL). >> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >> To solve it ThreadBlockInVM was added inside waiting cycle in >> check_socket_file. >> >> Other changes: >> - made _listener (and _shutdown for aix) volatile as they are used by >> 2 threads (attach listener thread and signal handler thread) without >> synchronization; >> - added close() for listening socket on aix (before it had only >> shutdown() for it); >> - additional logging and some cleanup in the test; >> - added handling of LingeredApp hang. >> >> --alex > From chris.plummer at oracle.com Mon May 11 20:44:53 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 11 May 2020 13:44:53 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> <0E6DBBFE-4395-45F5-9CB3-C023ADD7DD2F@oracle.com> Message-ID: BTW, I just found out that due to a fix for JDK-8220295 early last year, the jdk/sun/tools tests are not run in parallel, so at least atm are not contributing to any host memory related issues. Still it is best to fix them now. Chris On 5/11/20 10:54 AM, Chris Plummer wrote: > Hi Daniil, > > Looks good. > > thanks, > > Chris > > On 5/11/20 9:43 AM, Daniil Titov wrote: >> Hi Chris, >> >> Please review a new version of the fix[1] that also updates >> jdk/sun/tools? tests. >> >> Testing: Mach5 tier1-tier7 tests successfully passed. >> >> >> [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.03/ >> [2] ] https://bugs.openjdk.java.net/browse/JDK-8242009 >> >> Thank you, >> Daniil >> >> ?On 5/8/20, 12:21 AM, "Chris Plummer" wrote: >> >> ???? Hi Daniil, >> >> ???? I just noticed you didn't update the tests in >> jdk/sun/tools/jhsdb. Do >> ???? you think these should be done also? >> >> ???? Chris >> >> ???? On 5/7/20 11:44 PM, Chris Plummer wrote: >> ???? > Hi Daniil, >> ???? > >> ???? > The changes look good. >> ???? > >> ???? > thanks, >> ???? > >> ???? > Chris >> ???? > >> ???? > On 5/4/20 1:05 PM, Daniil Titov wrote: >> ???? >> Hi Chris, >> ???? >> >> ???? >> Please review a new version of webrev [1] that addresses your >> comments. >> ???? >> >> ???? >> For the following 3 tests that showed the increase of the >> execution >> ???? >> time with -Xcomp >> ???? >> more than 5 minutes this version of the change strips -Xcomp >> option >> ???? >> when >> ???? >> forwarding VM? arguments to? j*-tools? : >> ???? >> >> ???? >>???? -- serviceability/sa/sadebugd/SADebugDTest.java, >> ???? >>???? -- serviceability/sa/sadebugd/DebugdConnectTest.java, >> ???? >>???? -- serviceability/sa/ClhsdbJstackXcompStress.java >> ???? >> >> ???? >> The execution time for the rest of the tests when running >> with -Xcomp >> ???? >> was increased >> ???? >> within 1 and half minute. >> ???? >> >> ???? >> >> ???? >> [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ >> ???? >> [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> ???? >> >> ???? >> Thank you, >> ???? >>?? Daniil >> ???? >> >> ???? >> >> ???? >> ?On 4/27/20, 12:55 PM, "Chris Plummer" >> wrote: >> ???? >> >> ???? >>????? Hi Daniil, >> ???? >> >> ???? >>????? Overall it looks good. A few comments below. >> ???? >> >> ???? >>????? Can you add a comment to TestSysProps.java indicating >> the reason >> ???? >> for >> ???? >>????? checking if the line starts with "[". >> ???? >> >> ???? >>????? In JDKToolLauncher you have an extra empty line: >> ???? >> >> ???? >>??????? 112????? * Any platform specific arguments required for >> ???? >> running the >> ???? >>????? tool are >> ???? >>??????? 113????? * automatically added. >> ???? >>??????? 114????? * >> ???? >>??????? 115????? * >> ???? >>??????? 116????? * @param args >> ???? >> >> ???? >>????? In OutputAnalyzer.java, I wonder if all these matching >> APIs you >> ???? >> updated >> ???? >>????? should by default just include the version output in their >> ???? >> filtering. >> ???? >> >> ???? >>????? As for the added time to execute, I would suggest possibly >> ???? >> stripping >> ???? >>????? -Xcomp from the few outliers, and I would mostly focus >> on how much >> ???? >>????? longer it takes, not how many times longer. For example, >> ???? >> increasing from >> ???? >>????? 10 seconds to 40 seconds is not a big deal. Increasing >> from 10 >> ???? >> minutes >> ???? >>????? to 20 minutes is. >> ???? >> >> ???? >>????? SADebugDTest creates 8 tool processes so, that's >> probably the >> ???? >> reason for >> ???? >>????? the big increase, although I'm not sure why it is kind >> of slow >> ???? >> in the >> ???? >>????? first place. It does nothing more on each iteration than >> launch >> ???? >> "jhsdb >> ???? >>????? debugd", which will connect to the debuggee, and then is >> killed >> ???? >> off. >> ???? >>????? Maybe there is something slow with connecting, especial >> with RMI. >> ???? >> >> ???? >>????? thanks, >> ???? >> >> ???? >>????? Chris >> ???? >> >> ???? >>????? On 4/27/20 12:07 PM, Daniil Titov wrote: >> ???? >>????? > Please review the change [1] that ensures that VM and >> test >> ???? >> options are forwarded to >> ???? >>????? > j*-tools when they are launched from serviceability/sa >> tests. >> ???? >>????? > >> ???? >>????? > The tests that expect an empty output were corrected to >> ???? >> ignore the product version printed >> ???? >>????? > in the error stream since in some? tiers the tests are >> run >> ???? >> with ' -showversion' VM option (tier3). >> ???? >>????? > >> ???? >>????? > In test serviceability/sa/TestSysProps.java the code that >> ???? >> counts the system properties? was? corrected >> ???? >>????? > to ignore the debug output when the test is run with " >> ???? >> -Xlog:cds=debug" option (tier4). >> ???? >>????? > >> ???? >>????? > Testing:? Mach5 tests for tier1 - tier7 passed. >> ???? >>????? > >> ???? >>????? > I also run the test with -XComp at Mach5 linux-x64-debug >> ???? >> builds before and after the changes >> ???? >>????? > and for? the most of the tests the overhead is about 2 >> times >> ???? >> although for >> ???? >>????? > serviceability/sa/sadebugd/SADebugDTest.java it spikes >> up to 5 >> ???? >> times.? Probably at least for some tests >> ???? >>????? > it makes to filter out some properties (e.g. -Xcomp) >> before >> ???? >> forwarding them to j*-tools. >> ???? >>????? > >> ???? >>????? > serviceability/sa/sadebugd/SADebugDTest.java, before : >> 2m 23s >> ???? >> ,? after:11m 07s >> ???? >>????? > serviceability/sa/sadebugd/TestJmapCore.java,? before >> : 42s , >> ???? >> after:1m 09s >> ???? >>????? > serviceability/sa/TestSysProps.java, before : 36s , >> after: 1m 27s >> ???? >>????? > >> ???? >>????? > >> ???? >>????? > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 >> ???? >>????? > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 >> ???? >>????? > >> ???? >>????? > Thank you, >> ???? >>????? > Daniil >> ???? >>????? > >> ???? >>????? > >> ???? >> >> ???? >> >> ???? >> >> ???? >> >> ???? > >> ???? > >> >> >> > > From chris.plummer at oracle.com Mon May 11 20:52:07 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 11 May 2020 13:52:07 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> Message-ID: <05a1d029-85fc-8380-4c16-f9ea8a91d58b@oracle.com> ?228???????????? // If the app hangs, we don't want to wait for the to test timeout. Sorry, there was a typo in my suggestion. Should be "test to", not "to test". thanks, Chris On 5/11/20 12:53 PM, Alex Menkov wrote: > Hi Yasumasa, Serguei, Chris, > > Thank you for the feedback > updated webrev (all suggestions are applied): > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ > > > On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >> Hi Alex, >> >> It looks good in general. >> I have a couple of minor comments. >> >> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >> + // if for a reason the app hangs, we don't want to wait test timeout >> >> Nit: replace: wait test timeout => wait for test timeout >> >> I hope, you remember about copyright comments update. >> >> >> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >> >> >> Q1: How useful is this variable? : >> AixAttachListener::_shutdown = false; >> >> ???? Why is it needed on aix but not on other platforms? > > IFAIU AIX has issue with accept() - it hangs if the socket is closed. > I don't think this _shutdown flag helps a lot, but I don't want to > make significant changes in the AIX code as I cannot test it. > > --alex > >> >> Thanks, >> Serguei >> >> >> On 5/8/20 18:14, Alex Menkov wrote: >>> Hi all, >>> >>> please review the fix for >>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>> webrev: >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>> >>> >>> Test failures are caused by deadlock during attach listener restarting: >>> check_socket_file function aborts socket listening and waits while >>> attach listener state becomes AL_NOT_INITIALIZED (it happens when >>> AttachListener::dequeue returns NULL). >>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>> To solve it ThreadBlockInVM was added inside waiting cycle in >>> check_socket_file. >>> >>> Other changes: >>> - made _listener (and _shutdown for aix) volatile as they are used >>> by 2 threads (attach listener thread and signal handler thread) >>> without synchronization; >>> - added close() for listening socket on aix (before it had only >>> shutdown() for it); >>> - additional logging and some cleanup in the test; >>> - added handling of LingeredApp hang. >>> >>> --alex >> From alexey.menkov at oracle.com Mon May 11 21:12:18 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 11 May 2020 14:12:18 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <05a1d029-85fc-8380-4c16-f9ea8a91d58b@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <05a1d029-85fc-8380-4c16-f9ea8a91d58b@oracle.com> Message-ID: Please hold on with the review. webrev.2 causes LingeredApp crash. Looks like need to decrease ThreadBlockInVM scope. Will send new version after completing test run. On 05/11/2020 13:52, Chris Plummer wrote: > > ?228???????????? // If the app hangs, we don't want to wait for the to > test timeout. > > Sorry, there was a typo in my suggestion. Should be "test to", not "to > test". Oh, I didn't pay enough attention to comment updates :) --alex > > thanks, > > Chris > > On 5/11/20 12:53 PM, Alex Menkov wrote: >> Hi Yasumasa, Serguei, Chris, >> >> Thank you for the feedback >> updated webrev (all suggestions are applied): >> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >> >> >> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>> Hi Alex, >>> >>> It looks good in general. >>> I have a couple of minor comments. >>> >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>> + // if for a reason the app hangs, we don't want to wait test timeout >>> >>> Nit: replace: wait test timeout => wait for test timeout >>> >>> I hope, you remember about copyright comments update. >>> >>> >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>> >>> >>> Q1: How useful is this variable? : >>> AixAttachListener::_shutdown = false; >>> >>> ???? Why is it needed on aix but not on other platforms? >> >> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >> I don't think this _shutdown flag helps a lot, but I don't want to >> make significant changes in the AIX code as I cannot test it. >> >> --alex >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 5/8/20 18:14, Alex Menkov wrote: >>>> Hi all, >>>> >>>> please review the fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>> webrev: >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>> >>>> >>>> Test failures are caused by deadlock during attach listener restarting: >>>> check_socket_file function aborts socket listening and waits while >>>> attach listener state becomes AL_NOT_INITIALIZED (it happens when >>>> AttachListener::dequeue returns NULL). >>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>> check_socket_file. >>>> >>>> Other changes: >>>> - made _listener (and _shutdown for aix) volatile as they are used >>>> by 2 threads (attach listener thread and signal handler thread) >>>> without synchronization; >>>> - added close() for listening socket on aix (before it had only >>>> shutdown() for it); >>>> - additional logging and some cleanup in the test; >>>> - added handling of LingeredApp hang. >>>> >>>> --alex >>> > > From alexey.menkov at oracle.com Mon May 11 21:14:42 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 11 May 2020 14:14:42 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> Message-ID: <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> Hi Serguei, Alan, Updated webrev: http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.2/ --alex On 05/11/2020 11:52, Alan Bateman wrote: > On 11/05/2020 19:21, serguei.spitsyn at oracle.com wrote: >> Hi Alex, >> >> There is no need to repeat this: >> ? "deploy applications thatpackage an agent with the application, >> ?? or use tools that load agents into a running application" >> >> I'd suggest to rephrase it to something like: >> >> ? "Agents can transform classes in arbitrary ways at load time, transform >> ?? modules, or transform the bytecode of methods of already loaded >> classes. >> ?? Developers or administrators that deploy agents are responsible for >> their >> ?? trustworthiness and must therefore verify each agent including the >> content >> ?? and structure of its JAR file." >> >> >> Also, could you, please, replace: >> ?*

The three ways to start an agent is described below. >> >> with: >> ?*

The three ways to start an agent are described below. > Serguei's suggestions look good. > > -Alan From serguei.spitsyn at oracle.com Mon May 11 22:05:29 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 15:05:29 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> Message-ID: <1a1f2343-588c-2783-142f-e7c03e693240@oracle.com> Hi Alex, LGTM Thank you for the update! Serguei On 5/11/20 14:14, Alex Menkov wrote: > Hi Serguei, Alan, > > Updated webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.2/ > > --alex > > On 05/11/2020 11:52, Alan Bateman wrote: >> On 11/05/2020 19:21, serguei.spitsyn at oracle.com wrote: >>> Hi Alex, >>> >>> There is no need to repeat this: >>> ? "deploy applications thatpackage an agent with the application, >>> ?? or use tools that load agents into a running application" >>> >>> I'd suggest to rephrase it to something like: >>> >>> ? "Agents can transform classes in arbitrary ways at load time, >>> transform >>> ?? modules, or transform the bytecode of methods of already loaded >>> classes. >>> ?? Developers or administrators that deploy agents are responsible >>> for their >>> ?? trustworthiness and must therefore verify each agent including >>> the content >>> ?? and structure of its JAR file." >>> >>> >>> Also, could you, please, replace: >>> ?*

The three ways to start an agent is described below. >>> >>> with: >>> ?*

The three ways to start an agent are described below. >> Serguei's suggestions look good. >> >> -Alan From serguei.spitsyn at oracle.com Mon May 11 22:36:42 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 15:36:42 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> Message-ID: <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Tue May 12 00:01:00 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 11 May 2020 17:01:00 -0700 (PDT) Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> Message-ID: Hi Serguei, If I understand correctly, this also should work - when we mark a thread "_thread_blocked" VM does not need to suspend it at safepoint. --alex On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > I'm not sure this update suggested by Yasumasa is right: > > 536 // avoid deadlock if AttachListener thread is blocked at safepoint > 537 ThreadBlockInVM tbivm(JavaThread::current()); > 538 while (AttachListener::transit_state(AL_INITIALIZING, > 539 AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { > 540 os::naked_yield(); > 541 } > > > The synchronization window becomes smaller with adding ThreadBlockInVM, > so we probably do not see the deadlock anymore. > However, I think, we still need it inside the loop to work at each > iteration. > Also, we do not care about performance here as it is extremely rare case. > > Thanks, > Serguei > > > On 5/11/20 12:53, Alex Menkov wrote: >> Hi Yasumasa, Serguei, Chris, >> >> Thank you for the feedback >> updated webrev (all suggestions are applied): >> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >> >> >> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>> Hi Alex, >>> >>> It looks good in general. >>> I have a couple of minor comments. >>> >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>> + // if for a reason the app hangs, we don't want to wait test timeout >>> >>> Nit: replace: wait test timeout => wait for test timeout >>> >>> I hope, you remember about copyright comments update. >>> >>> >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>> >>> >>> Q1: How useful is this variable? : >>> AixAttachListener::_shutdown = false; >>> >>> ???? Why is it needed on aix but not on other platforms? >> >> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >> I don't think this _shutdown flag helps a lot, but I don't want to >> make significant changes in the AIX code as I cannot test it. >> >> --alex >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 5/8/20 18:14, Alex Menkov wrote: >>>> Hi all, >>>> >>>> please review the fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>> webrev: >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>> >>>> >>>> Test failures are caused by deadlock during attach listener restarting: >>>> check_socket_file function aborts socket listening and waits while >>>> attach listener state becomes AL_NOT_INITIALIZED (it happens when >>>> AttachListener::dequeue returns NULL). >>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>> check_socket_file. >>>> >>>> Other changes: >>>> - made _listener (and _shutdown for aix) volatile as they are used >>>> by 2 threads (attach listener thread and signal handler thread) >>>> without synchronization; >>>> - added close() for listening socket on aix (before it had only >>>> shutdown() for it); >>>> - additional logging and some cleanup in the test; >>>> - added handling of LingeredApp hang. >>>> >>>> --alex >>> > From serguei.spitsyn at oracle.com Tue May 12 00:04:11 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 17:04:11 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> Message-ID: <8394a835-2115-39e2-df77-d76863ddd24e@oracle.com> Hi Alex, I see the point, thanks. Serguei On 5/11/20 17:01, Alex Menkov wrote: > Hi Serguei, > > If I understand correctly, this also should work - when we mark a > thread "_thread_blocked" VM does not need to suspend it at safepoint. > > --alex > > On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >> Hi Alex, >> >> I'm not sure this update suggested by Yasumasa is right: >> >> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >> 537 ThreadBlockInVM tbivm(JavaThread::current()); >> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >> ? 539 AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >> ? 540?????? os::naked_yield(); >> ? 541???? } >> >> >> The synchronization window becomes smaller with adding ThreadBlockInVM, >> so we probably do not see the deadlock anymore. >> However, I think, we still need it inside the loop to work at each >> iteration. >> Also, we do not care about performance here as it is extremely rare >> case. >> >> Thanks, >> Serguei >> >> >> On 5/11/20 12:53, Alex Menkov wrote: >>> Hi Yasumasa, Serguei, Chris, >>> >>> Thank you for the feedback >>> updated webrev (all suggestions are applied): >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>> >>> >>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>> Hi Alex, >>>> >>>> It looks good in general. >>>> I have a couple of minor comments. >>>> >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>>> + // if for a reason the app hangs, we don't want to wait test timeout >>>> >>>> Nit: replace: wait test timeout => wait for test timeout >>>> >>>> I hope, you remember about copyright comments update. >>>> >>>> >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>> >>>> >>>> Q1: How useful is this variable? : >>>> AixAttachListener::_shutdown = false; >>>> >>>> ???? Why is it needed on aix but not on other platforms? >>> >>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>> I don't think this _shutdown flag helps a lot, but I don't want to >>> make significant changes in the AIX code as I cannot test it. >>> >>> --alex >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>> Hi all, >>>>> >>>>> please review the fix for >>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>> >>>>> >>>>> Test failures are caused by deadlock during attach listener >>>>> restarting: >>>>> check_socket_file function aborts socket listening and waits while >>>>> attach listener state becomes AL_NOT_INITIALIZED (it happens when >>>>> AttachListener::dequeue returns NULL). >>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>>> check_socket_file. >>>>> >>>>> Other changes: >>>>> - made _listener (and _shutdown for aix) volatile as they are used >>>>> by 2 threads (attach listener thread and signal handler thread) >>>>> without synchronization; >>>>> - added close() for listening socket on aix (before it had only >>>>> shutdown() for it); >>>>> - additional logging and some cleanup in the test; >>>>> - added handling of LingeredApp hang. >>>>> >>>>> --alex >>>> >> From suenaga at oss.nttdata.com Tue May 12 00:08:48 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 12 May 2020 09:08:48 +0900 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> Message-ID: On 2020/05/12 9:01, Alex Menkov wrote: > Hi Serguei, > > If I understand correctly, this also should work - when we mark a thread "_thread_blocked" VM does not need to suspend it at safepoint. I think so, but if state would be transit to the other in is_init_trigger(), it might be affect to incorrect behavior. So I suggested to wrap ThreadInVM and while loop with code block ({}). Thanks, Yasumasa > --alex > > On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >> Hi Alex, >> >> I'm not sure this update suggested by Yasumasa is right: >> >> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >> 537 ThreadBlockInVM tbivm(JavaThread::current()); >> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >> ? 539????????????????????????????????????????? AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >> ? 540?????? os::naked_yield(); >> ? 541???? } >> >> >> The synchronization window becomes smaller with adding ThreadBlockInVM, >> so we probably do not see the deadlock anymore. >> However, I think, we still need it inside the loop to work at each iteration. >> Also, we do not care about performance here as it is extremely rare case. >> >> Thanks, >> Serguei >> >> >> On 5/11/20 12:53, Alex Menkov wrote: >>> Hi Yasumasa, Serguei, Chris, >>> >>> Thank you for the feedback >>> updated webrev (all suggestions are applied): >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>> >>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>> Hi Alex, >>>> >>>> It looks good in general. >>>> I have a couple of minor comments. >>>> >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html + // if for a reason the app hangs, we don't want to wait test timeout >>>> >>>> Nit: replace: wait test timeout => wait for test timeout >>>> >>>> I hope, you remember about copyright comments update. >>>> >>>> >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>> >>>> Q1: How useful is this variable? : >>>> AixAttachListener::_shutdown = false; >>>> >>>> ???? Why is it needed on aix but not on other platforms? >>> >>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>> I don't think this _shutdown flag helps a lot, but I don't want to make significant changes in the AIX code as I cannot test it. >>> >>> --alex >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>> Hi all, >>>>> >>>>> please review the fix for >>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>> >>>>> Test failures are caused by deadlock during attach listener restarting: >>>>> check_socket_file function aborts socket listening and waits while attach listener state becomes AL_NOT_INITIALIZED (it happens when AttachListener::dequeue returns NULL). >>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>> To solve it ThreadBlockInVM was added inside waiting cycle in check_socket_file. >>>>> >>>>> Other changes: >>>>> - made _listener (and _shutdown for aix) volatile as they are used by 2 threads (attach listener thread and signal handler thread) without synchronization; >>>>> - added close() for listening socket on aix (before it had only shutdown() for it); >>>>> - additional logging and some cleanup in the test; >>>>> - added handling of LingeredApp hang. >>>>> >>>>> --alex >>>> >> From alexey.menkov at oracle.com Tue May 12 00:10:26 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 11 May 2020 17:10:26 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <05a1d029-85fc-8380-4c16-f9ea8a91d58b@oracle.com> Message-ID: Updated webrev: http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.3/ --alex On 05/11/2020 14:12, Alex Menkov wrote: > Please hold on with the review. > > webrev.2 causes LingeredApp crash. > Looks like need to decrease ThreadBlockInVM scope. > Will send new version after completing test run. > > On 05/11/2020 13:52, Chris Plummer wrote: >> >> ??228???????????? // If the app hangs, we don't want to wait for the >> to test timeout. >> >> Sorry, there was a typo in my suggestion. Should be "test to", not "to >> test". > > Oh, I didn't pay enough attention to comment updates :) > > --alex > >> >> thanks, >> >> Chris >> >> On 5/11/20 12:53 PM, Alex Menkov wrote: >>> Hi Yasumasa, Serguei, Chris, >>> >>> Thank you for the feedback >>> updated webrev (all suggestions are applied): >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>> >>> >>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>> Hi Alex, >>>> >>>> It looks good in general. >>>> I have a couple of minor comments. >>>> >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>>> + // if for a reason the app hangs, we don't want to wait test timeout >>>> >>>> Nit: replace: wait test timeout => wait for test timeout >>>> >>>> I hope, you remember about copyright comments update. >>>> >>>> >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>> >>>> >>>> Q1: How useful is this variable? : >>>> AixAttachListener::_shutdown = false; >>>> >>>> ???? Why is it needed on aix but not on other platforms? >>> >>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>> I don't think this _shutdown flag helps a lot, but I don't want to >>> make significant changes in the AIX code as I cannot test it. >>> >>> --alex >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>> Hi all, >>>>> >>>>> please review the fix for >>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>> >>>>> >>>>> Test failures are caused by deadlock during attach listener >>>>> restarting: >>>>> check_socket_file function aborts socket listening and waits while >>>>> attach listener state becomes AL_NOT_INITIALIZED (it happens when >>>>> AttachListener::dequeue returns NULL). >>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>>> check_socket_file. >>>>> >>>>> Other changes: >>>>> - made _listener (and _shutdown for aix) volatile as they are used >>>>> by 2 threads (attach listener thread and signal handler thread) >>>>> without synchronization; >>>>> - added close() for listening socket on aix (before it had only >>>>> shutdown() for it); >>>>> - additional logging and some cleanup in the test; >>>>> - added handling of LingeredApp hang. >>>>> >>>>> --alex >>>> >> >> From serguei.spitsyn at oracle.com Tue May 12 00:17:14 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 17:17:14 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> Message-ID: <864adb1c-b3ac-de6c-f85d-45268f39ac0b@oracle.com> Hi Yasumasa, On 5/11/20 17:08, Yasumasa Suenaga wrote: > On 2020/05/12 9:01, Alex Menkov wrote: >> Hi Serguei, >> >> If I understand correctly, this also should work - when we mark a >> thread "_thread_blocked" VM does not need to suspend it at safepoint. > > I think so, but if state would be transit to the other in > is_init_trigger(), it might be affect to incorrect behavior. > So I suggested to wrap ThreadInVM and while loop with code block ({}). Then I do not see it a problem to put helper inside the loop. In normal case, just one iteration is expected. I do not see any potential performance problem here. Thanks, Serguei > Thanks, > > Yasumasa > > >> --alex >> >> On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >>> Hi Alex, >>> >>> I'm not sure this update suggested by Yasumasa is right: >>> >>> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >>> 537 ThreadBlockInVM tbivm(JavaThread::current()); >>> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >>> ? 539 AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >>> ? 540?????? os::naked_yield(); >>> ? 541???? } >>> >>> >>> The synchronization window becomes smaller with adding ThreadBlockInVM, >>> so we probably do not see the deadlock anymore. >>> However, I think, we still need it inside the loop to work at each >>> iteration. >>> Also, we do not care about performance here as it is extremely rare >>> case. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 5/11/20 12:53, Alex Menkov wrote: >>>> Hi Yasumasa, Serguei, Chris, >>>> >>>> Thank you for the feedback >>>> updated webrev (all suggestions are applied): >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>>> >>>> >>>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>>> Hi Alex, >>>>> >>>>> It looks good in general. >>>>> I have a couple of minor comments. >>>>> >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>>>> + // if for a reason the app hangs, we don't want to wait test >>>>> timeout >>>>> >>>>> Nit: replace: wait test timeout => wait for test timeout >>>>> >>>>> I hope, you remember about copyright comments update. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>>> >>>>> >>>>> Q1: How useful is this variable? : >>>>> AixAttachListener::_shutdown = false; >>>>> >>>>> ???? Why is it needed on aix but not on other platforms? >>>> >>>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>>> I don't think this _shutdown flag helps a lot, but I don't want to >>>> make significant changes in the AIX code as I cannot test it. >>>> >>>> --alex >>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>>> Hi all, >>>>>> >>>>>> please review the fix for >>>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>>> >>>>>> >>>>>> Test failures are caused by deadlock during attach listener >>>>>> restarting: >>>>>> check_socket_file function aborts socket listening and waits >>>>>> while attach listener state becomes AL_NOT_INITIALIZED (it >>>>>> happens when AttachListener::dequeue returns NULL). >>>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>>>> check_socket_file. >>>>>> >>>>>> Other changes: >>>>>> - made _listener (and _shutdown for aix) volatile as they are >>>>>> used by 2 threads (attach listener thread and signal handler >>>>>> thread) without synchronization; >>>>>> - added close() for listening socket on aix (before it had only >>>>>> shutdown() for it); >>>>>> - additional logging and some cleanup in the test; >>>>>> - added handling of LingeredApp hang. >>>>>> >>>>>> --alex >>>>> >>> From alexey.menkov at oracle.com Tue May 12 00:17:38 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 11 May 2020 17:17:38 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> Message-ID: <3addb2fb-c797-ca53-b8a6-c9830bec4eb8@oracle.com> On 05/11/2020 17:08, Yasumasa Suenaga wrote: > On 2020/05/12 9:01, Alex Menkov wrote: >> Hi Serguei, >> >> If I understand correctly, this also should work - when we mark a >> thread "_thread_blocked" VM does not need to suspend it at safepoint. > > I think so, but if state would be transit to the other in > is_init_trigger(), it might be affect to incorrect behavior. > So I suggested to wrap ThreadInVM and while loop with code block ({}). I already did it :) http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.3/ --alex > > > Thanks, > > Yasumasa > > >> --alex >> >> On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >>> Hi Alex, >>> >>> I'm not sure this update suggested by Yasumasa is right: >>> >>> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >>> 537 ThreadBlockInVM tbivm(JavaThread::current()); >>> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >>> ? 539????????????????????????????????????????? AL_NOT_INITIALIZED) != >>> AL_NOT_INITIALIZED) { >>> ? 540?????? os::naked_yield(); >>> ? 541???? } >>> >>> >>> The synchronization window becomes smaller with adding ThreadBlockInVM, >>> so we probably do not see the deadlock anymore. >>> However, I think, we still need it inside the loop to work at each >>> iteration. >>> Also, we do not care about performance here as it is extremely rare >>> case. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 5/11/20 12:53, Alex Menkov wrote: >>>> Hi Yasumasa, Serguei, Chris, >>>> >>>> Thank you for the feedback >>>> updated webrev (all suggestions are applied): >>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>>> >>>> >>>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>>> Hi Alex, >>>>> >>>>> It looks good in general. >>>>> I have a couple of minor comments. >>>>> >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>>>> + // if for a reason the app hangs, we don't want to wait test timeout >>>>> >>>>> Nit: replace: wait test timeout => wait for test timeout >>>>> >>>>> I hope, you remember about copyright comments update. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>>> >>>>> >>>>> Q1: How useful is this variable? : >>>>> AixAttachListener::_shutdown = false; >>>>> >>>>> ???? Why is it needed on aix but not on other platforms? >>>> >>>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>>> I don't think this _shutdown flag helps a lot, but I don't want to >>>> make significant changes in the AIX code as I cannot test it. >>>> >>>> --alex >>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>>> Hi all, >>>>>> >>>>>> please review the fix for >>>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>>> >>>>>> >>>>>> Test failures are caused by deadlock during attach listener >>>>>> restarting: >>>>>> check_socket_file function aborts socket listening and waits while >>>>>> attach listener state becomes AL_NOT_INITIALIZED (it happens when >>>>>> AttachListener::dequeue returns NULL). >>>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>>>> check_socket_file. >>>>>> >>>>>> Other changes: >>>>>> - made _listener (and _shutdown for aix) volatile as they are used >>>>>> by 2 threads (attach listener thread and signal handler thread) >>>>>> without synchronization; >>>>>> - added close() for listening socket on aix (before it had only >>>>>> shutdown() for it); >>>>>> - additional logging and some cleanup in the test; >>>>>> - added handling of LingeredApp hang. >>>>>> >>>>>> --alex >>>>> >>> From serguei.spitsyn at oracle.com Tue May 12 00:21:31 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 17:21:31 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <3addb2fb-c797-ca53-b8a6-c9830bec4eb8@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> <3addb2fb-c797-ca53-b8a6-c9830bec4eb8@oracle.com> Message-ID: <8190c8ce-e4ba-0525-3737-3571a8e93f1c@oracle.com> On 5/11/20 17:17, Alex Menkov wrote: > > > On 05/11/2020 17:08, Yasumasa Suenaga wrote: >> On 2020/05/12 9:01, Alex Menkov wrote: >>> Hi Serguei, >>> >>> If I understand correctly, this also should work - when we mark a >>> thread "_thread_blocked" VM does not need to suspend it at safepoint. >> >> I think so, but if state would be transit to the other in >> is_init_trigger(), it might be affect to incorrect behavior. >> So I suggested to wrap ThreadInVM and while loop with code block ({}). > > I already did it :) > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.3/ It is okay with me in general. But as a side opinion: placing the ThreadInVM inside the loop would be simpler and without any extra problems. Thanks, Serguei > --alex > >> >> >> Thanks, >> >> Yasumasa >> >> >>> --alex >>> >>> On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >>>> Hi Alex, >>>> >>>> I'm not sure this update suggested by Yasumasa is right: >>>> >>>> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >>>> 537 ThreadBlockInVM tbivm(JavaThread::current()); >>>> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >>>> ? 539 AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >>>> ? 540?????? os::naked_yield(); >>>> ? 541???? } >>>> >>>> >>>> The synchronization window becomes smaller with adding >>>> ThreadBlockInVM, >>>> so we probably do not see the deadlock anymore. >>>> However, I think, we still need it inside the loop to work at each >>>> iteration. >>>> Also, we do not care about performance here as it is extremely rare >>>> case. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/11/20 12:53, Alex Menkov wrote: >>>>> Hi Yasumasa, Serguei, Chris, >>>>> >>>>> Thank you for the feedback >>>>> updated webrev (all suggestions are applied): >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>>>> >>>>> >>>>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Alex, >>>>>> >>>>>> It looks good in general. >>>>>> I have a couple of minor comments. >>>>>> >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>>>>> + // if for a reason the app hangs, we don't want to wait test >>>>>> timeout >>>>>> >>>>>> Nit: replace: wait test timeout => wait for test timeout >>>>>> >>>>>> I hope, you remember about copyright comments update. >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>>>> >>>>>> >>>>>> Q1: How useful is this variable? : >>>>>> AixAttachListener::_shutdown = false; >>>>>> >>>>>> ???? Why is it needed on aix but not on other platforms? >>>>> >>>>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>>>> I don't think this _shutdown flag helps a lot, but I don't want to >>>>> make significant changes in the AIX code as I cannot test it. >>>>> >>>>> --alex >>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> please review the fix for >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>>>> >>>>>>> >>>>>>> Test failures are caused by deadlock during attach listener >>>>>>> restarting: >>>>>>> check_socket_file function aborts socket listening and waits >>>>>>> while attach listener state becomes AL_NOT_INITIALIZED (it >>>>>>> happens when AttachListener::dequeue returns NULL). >>>>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>>>>> check_socket_file. >>>>>>> >>>>>>> Other changes: >>>>>>> - made _listener (and _shutdown for aix) volatile as they are >>>>>>> used by 2 threads (attach listener thread and signal handler >>>>>>> thread) without synchronization; >>>>>>> - added close() for listening socket on aix (before it had only >>>>>>> shutdown() for it); >>>>>>> - additional logging and some cleanup in the test; >>>>>>> - added handling of LingeredApp hang. >>>>>>> >>>>>>> --alex >>>>>> >>>> From suenaga at oss.nttdata.com Tue May 12 00:33:04 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 12 May 2020 09:33:04 +0900 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <864adb1c-b3ac-de6c-f85d-45268f39ac0b@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> <864adb1c-b3ac-de6c-f85d-45268f39ac0b@oracle.com> Message-ID: On 2020/05/12 9:17, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > > On 5/11/20 17:08, Yasumasa Suenaga wrote: >> On 2020/05/12 9:01, Alex Menkov wrote: >>> Hi Serguei, >>> >>> If I understand correctly, this also should work - when we mark a thread "_thread_blocked" VM does not need to suspend it at safepoint. >> >> I think so, but if state would be transit to the other in is_init_trigger(), it might be affect to incorrect behavior. >> So I suggested to wrap ThreadInVM and while loop with code block ({}). > > Then I do not see it a problem to put helper inside the loop. > In normal case, just one iteration is expected. I'm not sure that. The attach listener state would be transit to AL_NOT_INITIALIZED when AttachListener::dequeue() returns NULL. listener_cleanup() would shoutdown the socket, then accept() would be return with error at LinuxAttachListener::dequeue() in case of Linux. If they performs on high-spec (multi-core, and high-frequency) machine, while loop might not be finished with one iteration. It might be long way between shutdown() call and state transition. I agree with you this problem is rare case, so it is not a big problem ThreadInVM is located to inside the loop. But I prefer to locate it to outside the loop. Thanks, Yasumasa > I do not see any potential performance problem here. > > Thanks, > Serguei > >> Thanks, >> >> Yasumasa >> >> >>> --alex >>> >>> On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >>>> Hi Alex, >>>> >>>> I'm not sure this update suggested by Yasumasa is right: >>>> >>>> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >>>> 537 ThreadBlockInVM tbivm(JavaThread::current()); >>>> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >>>> ? 539 AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >>>> ? 540?????? os::naked_yield(); >>>> ? 541???? } >>>> >>>> >>>> The synchronization window becomes smaller with adding ThreadBlockInVM, >>>> so we probably do not see the deadlock anymore. >>>> However, I think, we still need it inside the loop to work at each iteration. >>>> Also, we do not care about performance here as it is extremely rare case. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/11/20 12:53, Alex Menkov wrote: >>>>> Hi Yasumasa, Serguei, Chris, >>>>> >>>>> Thank you for the feedback >>>>> updated webrev (all suggestions are applied): >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>>>> >>>>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Alex, >>>>>> >>>>>> It looks good in general. >>>>>> I have a couple of minor comments. >>>>>> >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html + // if for a reason the app hangs, we don't want to wait test timeout >>>>>> >>>>>> Nit: replace: wait test timeout => wait for test timeout >>>>>> >>>>>> I hope, you remember about copyright comments update. >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>>>> >>>>>> Q1: How useful is this variable? : >>>>>> AixAttachListener::_shutdown = false; >>>>>> >>>>>> ???? Why is it needed on aix but not on other platforms? >>>>> >>>>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>>>> I don't think this _shutdown flag helps a lot, but I don't want to make significant changes in the AIX code as I cannot test it. >>>>> >>>>> --alex >>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> please review the fix for >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>>>> >>>>>>> Test failures are caused by deadlock during attach listener restarting: >>>>>>> check_socket_file function aborts socket listening and waits while attach listener state becomes AL_NOT_INITIALIZED (it happens when AttachListener::dequeue returns NULL). >>>>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>>>> To solve it ThreadBlockInVM was added inside waiting cycle in check_socket_file. >>>>>>> >>>>>>> Other changes: >>>>>>> - made _listener (and _shutdown for aix) volatile as they are used by 2 threads (attach listener thread and signal handler thread) without synchronization; >>>>>>> - added close() for listening socket on aix (before it had only shutdown() for it); >>>>>>> - additional logging and some cleanup in the test; >>>>>>> - added handling of LingeredApp hang. >>>>>>> >>>>>>> --alex >>>>>> >>>> > From suenaga at oss.nttdata.com Tue May 12 00:34:08 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 12 May 2020 09:34:08 +0900 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <3addb2fb-c797-ca53-b8a6-c9830bec4eb8@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> <3addb2fb-c797-ca53-b8a6-c9830bec4eb8@oracle.com> Message-ID: Hi Alex, On 2020/05/12 9:17, Alex Menkov wrote: > > > On 05/11/2020 17:08, Yasumasa Suenaga wrote: >> On 2020/05/12 9:01, Alex Menkov wrote: >>> Hi Serguei, >>> >>> If I understand correctly, this also should work - when we mark a thread "_thread_blocked" VM does not need to suspend it at safepoint. >> >> I think so, but if state would be transit to the other in is_init_trigger(), it might be affect to incorrect behavior. >> So I suggested to wrap ThreadInVM and while loop with code block ({}). > > I already did it :) > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.3/ I sent the reply before reading yours :) Looks good to me. Thanks, Yasumasa > --alex > >> >> >> Thanks, >> >> Yasumasa >> >> >>> --alex >>> >>> On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >>>> Hi Alex, >>>> >>>> I'm not sure this update suggested by Yasumasa is right: >>>> >>>> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >>>> 537 ThreadBlockInVM tbivm(JavaThread::current()); >>>> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >>>> ? 539????????????????????????????????????????? AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >>>> ? 540?????? os::naked_yield(); >>>> ? 541???? } >>>> >>>> >>>> The synchronization window becomes smaller with adding ThreadBlockInVM, >>>> so we probably do not see the deadlock anymore. >>>> However, I think, we still need it inside the loop to work at each iteration. >>>> Also, we do not care about performance here as it is extremely rare case. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 5/11/20 12:53, Alex Menkov wrote: >>>>> Hi Yasumasa, Serguei, Chris, >>>>> >>>>> Thank you for the feedback >>>>> updated webrev (all suggestions are applied): >>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>>>> >>>>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Alex, >>>>>> >>>>>> It looks good in general. >>>>>> I have a couple of minor comments. >>>>>> >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html + // if for a reason the app hangs, we don't want to wait test timeout >>>>>> >>>>>> Nit: replace: wait test timeout => wait for test timeout >>>>>> >>>>>> I hope, you remember about copyright comments update. >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>>>> >>>>>> Q1: How useful is this variable? : >>>>>> AixAttachListener::_shutdown = false; >>>>>> >>>>>> ???? Why is it needed on aix but not on other platforms? >>>>> >>>>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>>>> I don't think this _shutdown flag helps a lot, but I don't want to make significant changes in the AIX code as I cannot test it. >>>>> >>>>> --alex >>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> please review the fix for >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>>>> >>>>>>> Test failures are caused by deadlock during attach listener restarting: >>>>>>> check_socket_file function aborts socket listening and waits while attach listener state becomes AL_NOT_INITIALIZED (it happens when AttachListener::dequeue returns NULL). >>>>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>>>> To solve it ThreadBlockInVM was added inside waiting cycle in check_socket_file. >>>>>>> >>>>>>> Other changes: >>>>>>> - made _listener (and _shutdown for aix) volatile as they are used by 2 threads (attach listener thread and signal handler thread) without synchronization; >>>>>>> - added close() for listening socket on aix (before it had only shutdown() for it); >>>>>>> - additional logging and some cleanup in the test; >>>>>>> - added handling of LingeredApp hang. >>>>>>> >>>>>>> --alex >>>>>> >>>> From serguei.spitsyn at oracle.com Tue May 12 00:42:09 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 17:42:09 -0700 Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> <864adb1c-b3ac-de6c-f85d-45268f39ac0b@oracle.com> Message-ID: <73b8abdf-0df1-0630-ab07-373977066f69@oracle.com> On 5/11/20 17:33, Yasumasa Suenaga wrote: > On 2020/05/12 9:17, serguei.spitsyn at oracle.com wrote: >> Hi Yasumasa, >> >> >> On 5/11/20 17:08, Yasumasa Suenaga wrote: >>> On 2020/05/12 9:01, Alex Menkov wrote: >>>> Hi Serguei, >>>> >>>> If I understand correctly, this also should work - when we mark a >>>> thread "_thread_blocked" VM does not need to suspend it at safepoint. >>> >>> I think so, but if state would be transit to the other in >>> is_init_trigger(), it might be affect to incorrect behavior. >>> So I suggested to wrap ThreadInVM and while loop with code block ({}). >> >> Then I do not see it a problem to put helper inside the loop. >> In normal case, just one iteration is expected. > > I'm not sure that. > > The attach listener state would be transit to AL_NOT_INITIALIZED when > AttachListener::dequeue() returns NULL. > listener_cleanup() would shoutdown the socket, then accept() would be > return with error at LinuxAttachListener::dequeue() in case of Linux. > > If they performs on high-spec (multi-core, and high-frequency) > machine, while loop might not be finished with one iteration. > It might be long way between shutdown() call and state transition. > > I agree with you this problem is rare case, so it is not a big problem > ThreadInVM is located to inside the loop. > But I prefer to locate it to outside the loop. I'm okay with both approaches as the differences are minor and potential performance issue (if any) is very minor. My normal preferences are in favor of simplicity. :) Thanks, Serguei > Thanks, > > Yasumasa > > >> I do not see any potential performance problem here. >> >> Thanks, >> Serguei >> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> --alex >>>> >>>> On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >>>>> Hi Alex, >>>>> >>>>> I'm not sure this update suggested by Yasumasa is right: >>>>> >>>>> 536 // avoid deadlock if AttachListener thread is blocked at >>>>> safepoint >>>>> 537 ThreadBlockInVM tbivm(JavaThread::current()); >>>>> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >>>>> ? 539 AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >>>>> ? 540?????? os::naked_yield(); >>>>> ? 541???? } >>>>> >>>>> >>>>> The synchronization window becomes smaller with adding >>>>> ThreadBlockInVM, >>>>> so we probably do not see the deadlock anymore. >>>>> However, I think, we still need it inside the loop to work at each >>>>> iteration. >>>>> Also, we do not care about performance here as it is extremely >>>>> rare case. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 5/11/20 12:53, Alex Menkov wrote: >>>>>> Hi Yasumasa, Serguei, Chris, >>>>>> >>>>>> Thank you for the feedback >>>>>> updated webrev (all suggestions are applied): >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>>>>> >>>>>> >>>>>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Alex, >>>>>>> >>>>>>> It looks good in general. >>>>>>> I have a couple of minor comments. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>>>>>> + // if for a reason the app hangs, we don't want to wait test >>>>>>> timeout >>>>>>> >>>>>>> Nit: replace: wait test timeout => wait for test timeout >>>>>>> >>>>>>> I hope, you remember about copyright comments update. >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>>>>> >>>>>>> >>>>>>> Q1: How useful is this variable? : >>>>>>> AixAttachListener::_shutdown = false; >>>>>>> >>>>>>> ???? Why is it needed on aix but not on other platforms? >>>>>> >>>>>> IFAIU AIX has issue with accept() - it hangs if the socket is >>>>>> closed. >>>>>> I don't think this _shutdown flag helps a lot, but I don't want >>>>>> to make significant changes in the AIX code as I cannot test it. >>>>>> >>>>>> --alex >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> please review the fix for >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>>>>> >>>>>>>> >>>>>>>> Test failures are caused by deadlock during attach listener >>>>>>>> restarting: >>>>>>>> check_socket_file function aborts socket listening and waits >>>>>>>> while attach listener state becomes AL_NOT_INITIALIZED (it >>>>>>>> happens when AttachListener::dequeue returns NULL). >>>>>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>>>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>>>>>> check_socket_file. >>>>>>>> >>>>>>>> Other changes: >>>>>>>> - made _listener (and _shutdown for aix) volatile as they are >>>>>>>> used by 2 threads (attach listener thread and signal handler >>>>>>>> thread) without synchronization; >>>>>>>> - added close() for listening socket on aix (before it had only >>>>>>>> shutdown() for it); >>>>>>>> - additional logging and some cleanup in the test; >>>>>>>> - added handling of LingeredApp hang. >>>>>>>> >>>>>>>> --alex >>>>>>> >>>>> >> From alexey.menkov at oracle.com Tue May 12 00:30:38 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 11 May 2020 17:30:38 -0700 (PDT) Subject: RFR: JDK-8235211: serviceability/attach/RemovingUnixDomainSocketTest.java fails with AttachNotSupportedException: Unable to open socket file In-Reply-To: <8190c8ce-e4ba-0525-3737-3571a8e93f1c@oracle.com> References: <9bfec051-2b5a-86bc-760e-e66bb9cab6d8@oracle.com> <73e7edeb-664b-f381-1c4f-19181deecfbe@oracle.com> <6a1c1c71-886b-147b-4eb7-020e90e5ecdf@oracle.com> <3addb2fb-c797-ca53-b8a6-c9830bec4eb8@oracle.com> <8190c8ce-e4ba-0525-3737-3571a8e93f1c@oracle.com> Message-ID: On 05/11/2020 17:21, serguei.spitsyn at oracle.com wrote: > On 5/11/20 17:17, Alex Menkov wrote: >> >> >> On 05/11/2020 17:08, Yasumasa Suenaga wrote: >>> On 2020/05/12 9:01, Alex Menkov wrote: >>>> Hi Serguei, >>>> >>>> If I understand correctly, this also should work - when we mark a >>>> thread "_thread_blocked" VM does not need to suspend it at safepoint. >>> >>> I think so, but if state would be transit to the other in >>> is_init_trigger(), it might be affect to incorrect behavior. >>> So I suggested to wrap ThreadInVM and while loop with code block ({}). >> >> I already did it :) >> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.3/ >> > > It is okay with me in general. > But as a side opinion: placing the ThreadInVM inside the loop would be > simpler and without any extra problems. I'm ok with both ways. Especially taking in mind that this case (when we need to restart attach listener) is quite artificial. --alex > > Thanks, > Serguei > > >> --alex >> >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> --alex >>>> >>>> On 05/11/2020 15:36, serguei.spitsyn at oracle.com wrote: >>>>> Hi Alex, >>>>> >>>>> I'm not sure this update suggested by Yasumasa is right: >>>>> >>>>> 536 // avoid deadlock if AttachListener thread is blocked at safepoint >>>>> 537 ThreadBlockInVM tbivm(JavaThread::current()); >>>>> ? 538???? while (AttachListener::transit_state(AL_INITIALIZING, >>>>> ? 539 AL_NOT_INITIALIZED) != AL_NOT_INITIALIZED) { >>>>> ? 540?????? os::naked_yield(); >>>>> ? 541???? } >>>>> >>>>> >>>>> The synchronization window becomes smaller with adding >>>>> ThreadBlockInVM, >>>>> so we probably do not see the deadlock anymore. >>>>> However, I think, we still need it inside the loop to work at each >>>>> iteration. >>>>> Also, we do not care about performance here as it is extremely rare >>>>> case. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 5/11/20 12:53, Alex Menkov wrote: >>>>>> Hi Yasumasa, Serguei, Chris, >>>>>> >>>>>> Thank you for the feedback >>>>>> updated webrev (all suggestions are applied): >>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev.2/ >>>>>> >>>>>> >>>>>> On 05/11/2020 00:31, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Alex, >>>>>>> >>>>>>> It looks good in general. >>>>>>> I have a couple of minor comments. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/test/lib/jdk/test/lib/apps/LingeredApp.java.udiff.html >>>>>>> + // if for a reason the app hangs, we don't want to wait test >>>>>>> timeout >>>>>>> >>>>>>> Nit: replace: wait test timeout => wait for test timeout >>>>>>> >>>>>>> I hope, you remember about copyright comments update. >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/src/hotspot/os/aix/attachListener_aix.cpp.udiff.html >>>>>>> >>>>>>> >>>>>>> Q1: How useful is this variable? : >>>>>>> AixAttachListener::_shutdown = false; >>>>>>> >>>>>>> ???? Why is it needed on aix but not on other platforms? >>>>>> >>>>>> IFAIU AIX has issue with accept() - it hangs if the socket is closed. >>>>>> I don't think this _shutdown flag helps a lot, but I don't want to >>>>>> make significant changes in the AIX code as I cannot test it. >>>>>> >>>>>> --alex >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 5/8/20 18:14, Alex Menkov wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> please review the fix for >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8235211 >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket/webrev/ >>>>>>>> >>>>>>>> >>>>>>>> Test failures are caused by deadlock during attach listener >>>>>>>> restarting: >>>>>>>> check_socket_file function aborts socket listening and waits >>>>>>>> while attach listener state becomes AL_NOT_INITIALIZED (it >>>>>>>> happens when AttachListener::dequeue returns NULL). >>>>>>>> AttachListener::dequeue method is blocked in ThreadBlockInVM dtor. >>>>>>>> To solve it ThreadBlockInVM was added inside waiting cycle in >>>>>>>> check_socket_file. >>>>>>>> >>>>>>>> Other changes: >>>>>>>> - made _listener (and _shutdown for aix) volatile as they are >>>>>>>> used by 2 threads (attach listener thread and signal handler >>>>>>>> thread) without synchronization; >>>>>>>> - added close() for listening socket on aix (before it had only >>>>>>>> shutdown() for it); >>>>>>>> - additional logging and some cleanup in the test; >>>>>>>> - added handling of LingeredApp hang. >>>>>>>> >>>>>>>> --alex >>>>>>> >>>>> > From david.holmes at oracle.com Tue May 12 04:28:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 May 2020 14:28:40 +1000 Subject: RFR (trivial) 8244779: ProblemList serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java pending JDK-8244571 Message-ID: <6ce31450-4a79-b8bb-e914-6c58d242807a@oracle.com> This test recently starting running with -Xcheck:jni applied and is failing, so we want to problem-list it until the issue can be investigated diff -r bbdcc6741915 test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt +++ b/test/hotspot/jtreg/ProblemList.txt @@ -108,6 +108,7 @@ serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java 8214032 generic-all serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java 8224150 generic-all +serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java 8244571 generic-all ############################################################################# Thanks, David From igor.ignatyev at oracle.com Tue May 12 04:38:21 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 11 May 2020 21:38:21 -0700 Subject: RFR (trivial) 8244779: ProblemList serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java pending JDK-8244571 In-Reply-To: <6ce31450-4a79-b8bb-e914-6c58d242807a@oracle.com> References: <6ce31450-4a79-b8bb-e914-6c58d242807a@oracle.com> Message-ID: LGTM -- Igor > On May 11, 2020, at 9:28 PM, David Holmes wrote: > > This test recently starting running with -Xcheck:jni applied and is failing, so we want to problem-list it until the issue can be investigated > > diff -r bbdcc6741915 test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt > +++ b/test/hotspot/jtreg/ProblemList.txt > @@ -108,6 +108,7 @@ > > serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java 8214032 generic-all > serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java 8224150 generic-all > +serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java 8244571 generic-all > > ############################################################################# > > Thanks, > David From david.holmes at oracle.com Tue May 12 04:46:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 May 2020 14:46:52 +1000 Subject: RFR (trivial) 8244779: ProblemList serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java pending JDK-8244571 In-Reply-To: References: <6ce31450-4a79-b8bb-e914-6c58d242807a@oracle.com> Message-ID: <7935a3ce-b73f-96c4-f836-596473329b7e@oracle.com> Thanks Igor! David On 12/05/2020 2:38 pm, Igor Ignatyev wrote: > LGTM > -- Igor > >> On May 11, 2020, at 9:28 PM, David Holmes wrote: >> >> This test recently starting running with -Xcheck:jni applied and is failing, so we want to problem-list it until the issue can be investigated >> >> diff -r bbdcc6741915 test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt >> +++ b/test/hotspot/jtreg/ProblemList.txt >> @@ -108,6 +108,7 @@ >> >> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java 8214032 generic-all >> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java 8224150 generic-all >> +serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java 8244571 generic-all >> >> ############################################################################# >> >> Thanks, >> David > From serguei.spitsyn at oracle.com Tue May 12 05:16:51 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 May 2020 22:16:51 -0700 Subject: RFR (trivial) 8244779: ProblemList serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java pending JDK-8244571 In-Reply-To: References: <6ce31450-4a79-b8bb-e914-6c58d242807a@oracle.com> Message-ID: <021f16ea-2adf-f0b4-675c-d3a267fafeaf@oracle.com> LGTM++ and trivial. Thanks, Serguei On 5/11/20 21:38, Igor Ignatyev wrote: > LGTM > -- Igor > >> On May 11, 2020, at 9:28 PM, David Holmes wrote: >> >> This test recently starting running with -Xcheck:jni applied and is failing, so we want to problem-list it until the issue can be investigated >> >> diff -r bbdcc6741915 test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt >> +++ b/test/hotspot/jtreg/ProblemList.txt >> @@ -108,6 +108,7 @@ >> >> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java 8214032 generic-all >> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java 8224150 generic-all >> +serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java 8244571 generic-all >> >> ############################################################################# >> >> Thanks, >> David From david.holmes at oracle.com Tue May 12 06:35:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 May 2020 16:35:49 +1000 Subject: RFR (trivial) 8244779: ProblemList serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java pending JDK-8244571 In-Reply-To: <021f16ea-2adf-f0b4-675c-d3a267fafeaf@oracle.com> References: <6ce31450-4a79-b8bb-e914-6c58d242807a@oracle.com> <021f16ea-2adf-f0b4-675c-d3a267fafeaf@oracle.com> Message-ID: Thanks Serguei. Already pushed :) David On 12/05/2020 3:16 pm, serguei.spitsyn at oracle.com wrote: > LGTM++ and trivial. > > Thanks, > Serguei > > On 5/11/20 21:38, Igor Ignatyev wrote: >> LGTM >> -- Igor >> >>> On May 11, 2020, at 9:28 PM, David Holmes >>> wrote: >>> >>> This test recently starting running with -Xcheck:jni applied and is >>> failing, so we want to problem-list it until the issue can be >>> investigated >>> >>> diff -r bbdcc6741915 test/hotspot/jtreg/ProblemList.txt >>> --- a/test/hotspot/jtreg/ProblemList.txt >>> +++ b/test/hotspot/jtreg/ProblemList.txt >>> @@ -108,6 +108,7 @@ >>> >>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatIntervalTest.java >>> 8214032 generic-all >>> serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java >>> 8224150 generic-all >>> +serviceability/jvmti/HiddenClass/P/Q/HiddenClassSigTest.java 8244571 >>> generic-all >>> >>> ############################################################################# >>> >>> >>> Thanks, >>> David > From david.holmes at oracle.com Tue May 12 07:32:27 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 May 2020 17:32:27 +1000 Subject: RFR 8244105: JDK 11.0.7 -Xlog gc issue with file path if not exist In-Reply-To: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> References: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> Message-ID: Hi Harold, On 9/05/2020 3:22 am, Harold Seigel wrote: > Hi, > > Please review this fix for JDK-8244105.? The fix continues program > execution even when specified logging options are invalid.? Previously, > invalid logging options terminated the program.? Now, a warning is > issued.? For example: > > > java -Xlog:"gc*:file=/dont/exist" -version > [0.001s][error][logging] Error opening log file '/dont/exist': No > such file or directory > [0.001s][error][logging] Initialization of output 'file=/dont/exist' > using options '(null)' failed. > Java HotSpot(TM) 64-Bit Server VM warning: Invalid -Xlog option > '-Xlog:gc*:file=/dont/exist', see error log for details. > > java version "15-internal" 2020-09-15 > Java(TM) SE Runtime Environment (fastdebug build > 15-internal+0-2020-05-08-1313404.hseigel.bug8244105) > Java HotSpot(TM) 64-Bit Server VM (fastdebug build > 15-internal+0-2020-05-08-1313404.hseigel.bug8244105, mixed mode, > sharing) But if I am reading this correctly you are now only issuing a warning and disabling logging if there are any errors of any kind in the -Xlog arguments! That is not desirable IMO and a significant change in behaviour. Even just changing the behaviour in relation to a non-existent log file will require a CSR request. Thanks, David ----- > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8244105/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8244105 > > The fix was regression tested by running Mach5 tiers 1 and 2 tests and > builds on Linux-x64, Solaris, Windows, and Mac OS X and by running Mach5 > tiers 3-5 tests on Linux-x64. > > Thanks, Harold > > From david.holmes at oracle.com Tue May 12 07:35:28 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 May 2020 17:35:28 +1000 Subject: RFR 8244105: JDK 11.0.7 -Xlog gc issue with file path if not exist In-Reply-To: References: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> Message-ID: On 12/05/2020 1:06 am, dmytro sheyko wrote: > Hi, > > I think it would be very convenient for app developers if JVM were able > to create intermediate directories to gc.log file if they do not exist. You can file a RFE for that, but personally I don't think it is something the VM should do. Cheers, David > I.e. > ? $ if [ -f logs/gc.log ]; then echo "log file exists"; else echo "log > file does not exist"; fi > ? log file does not exist > > ? $ if [ -d logs ]; then echo "log directory exists"; else echo "log > directory does not exist"; fi > ? log directory does not exist > > ? $ java -Xlog:"gc*:file=logs/gc.log" -version > ? ... > > ? $ if [ -d logs ]; then echo "log directory exists"; else echo "log > directory does not exist"; fi > ? log directory exists > > ? $ if [ -f logs/gc.log ]; then echo "log file exists"; else echo "log > file does not exist"; fi > ? log file exists > > Thanks, > Dmytro > > On Fri, May 8, 2020 at 8:31 PM Harold Seigel > wrote: > > Hi, > > Please review this fix for JDK-8244105.? The fix continues program > execution even when specified logging options are invalid. > Previously, invalid logging options terminated the program.? Now, a > warning is issued.? For example: > > > java -Xlog:"gc*:file=/dont/exist" -version > [0.001s][error][logging] Error opening log file '/dont/exist': > No such file or directory > [0.001s][error][logging] Initialization of output > 'file=/dont/exist' using options '(null)' failed. > Java HotSpot(TM) 64-Bit Server VM warning: Invalid -Xlog option > '-Xlog:gc*:file=/dont/exist', see error log for details. > > java version "15-internal" 2020-09-15 > Java(TM) SE Runtime Environment (fastdebug build > 15-internal+0-2020-05-08-1313404.hseigel.bug8244105) > Java HotSpot(TM) 64-Bit Server VM (fastdebug build > 15-internal+0-2020-05-08-1313404.hseigel.bug8244105, mixed mode, > sharing) > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8244105/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8244105 > > The fix was regression tested by running Mach5 tiers 1 and 2 tests > and builds on Linux-x64, Solaris, Windows, and Mac OS X and by > running Mach5 tiers 3-5 tests on Linux-x64. > > Thanks, Harold > > From Alan.Bateman at oracle.com Tue May 12 07:59:45 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 May 2020 08:59:45 +0100 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> Message-ID: <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> On 11/05/2020 22:14, Alex Menkov wrote: > > > Updated webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.2/ > > --alex This doesn't work for me because it drops the important point that the developer/admin is also responsible when deploying an agent that packages an agent with the application. Also anyone using a tool that loads agents into a running VM has responsibility too. So I think these points need to be included. -Alan. From martin.doerr at sap.com Tue May 12 08:42:31 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 12 May 2020 08:42:31 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> Message-ID: Hi Richard, I had already reviewed webrev.1. Looks good to me. Thanks for contributing it. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > Sent: Montag, 4. Mai 2020 12:33 > To: David Holmes ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > Subject: RE: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > > // Trimmed the list of recipients. If the list gets too long then the message > needs to be approved > // by a moderator. > > Hi David, > > > On 28/04/2020 12:09 am, Reingruber, Richard wrote: > > > Hi David, > > > > > >> Not a review but some general commentary ... > > > > > > That's welcome. > > > Having had to take an even closer look now I have a review comment too :) > > > src/hotspot/share/prims/jvmtiThreadState.cpp > > > void JvmtiThreadState::invalidate_cur_stack_depth() { > > ! assert(SafepointSynchronize::is_at_safepoint() || > > ! (Thread::current()->is_VM_thread() && > > get_thread()->is_vmthread_processing_handshake()) || > > (JavaThread *)Thread::current() == get_thread(), > > "must be current thread or at safepoint"); > > You're looking at an outdated webrev, I'm afraid. > > This would be the post with the current webrev.1 > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020- > April/031245.html > > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Montag, 4. Mai 2020 08:51 > To: Reingruber, Richard ; Yasumasa Suenaga > ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > > Hi Richard, > > On 28/04/2020 12:09 am, Reingruber, Richard wrote: > > Hi David, > > > >> Not a review but some general commentary ... > > > > That's welcome. > > Having had to take an even closer look now I have a review comment too :) > > src/hotspot/share/prims/jvmtiThreadState.cpp > > void JvmtiThreadState::invalidate_cur_stack_depth() { > ! assert(SafepointSynchronize::is_at_safepoint() || > ! (Thread::current()->is_VM_thread() && > get_thread()->is_vmthread_processing_handshake()) || > (JavaThread *)Thread::current() == get_thread(), > "must be current thread or at safepoint"); > > The message needs updating to include handshakes. > > More below ... > > >> On 25/04/2020 2:08 am, Reingruber, Richard wrote: > >>> Hi Yasumasa, Patricio, > >>> > >>>>>> I will send review request to replace VM_SetFramePop to > handshake in early next week in JDK-8242427. > >>>>>> Does it help you? I think it gives you to remove workaround. > >>>>> > >>>>> I think it would not help that much. Note that when replacing > VM_SetFramePop with a direct handshake > >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm > operation [1]. So you would have to > >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt > to these changes. > >>> > >>>> Thanks for your information. > >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and > vmTestbase/nsk/jvmti/NotifyFramePop. > >>>> I will modify and will test it after yours. > >>> > >>> Thanks :) > >>> > >>>>> Also my first impression was that it won't be that easy from a > synchronization point of view to > >>>>> replace VM_SetFramePop with a direct handshake. E.g. > VM_SetFramePop::doit() indirectly calls > >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, > JvmtiFramePop fpop) where > >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at > safepoint. It's not directly clear > >>>>> to me, how this has to be handled. > >>> > >>>> I think JvmtiEventController::set_frame_pop() should hold > JvmtiThreadState_lock because it affects other JVMTI operation especially > FramePop event. > >>> > >>> Yes. To me it is unclear what synchronization is necessary, if it is called > during a handshake. And > >>> also I'm unsure if a thread should do safepoint checks while executing a > handshake. > > > >> I'm growing increasingly concerned that use of direct handshakes to > >> replace VM operations needs a much greater examination for correctness > >> than might initially be thought. I see a number of issues: > > > > I agree. I'll address your concerns in the context of this review thread for > JDK-8238585 below. > > > > In addition I would suggest to take the general part of the discussion to a > dedicated thread or to > > the review thread for JDK-8242427. I would like to keep this thread closer > to its subject. > > I will focus on the issues in the context of this particular change > then, though the issues themselves are applicable to all handshake > situations (and more so with direct handshakes). This is mostly just > discussion. > > >> First, the VMThread executes (most) VM operations with a clean stack in > >> a clean state, so it has lots of room to work. If we now execute the > >> same logic in a JavaThread then we risk hitting stackoverflows if > >> nothing else. But we are also now executing code in a JavaThread and so > >> we have to be sure that code is not going to act differently (in a bad > >> way) if executed by a JavaThread rather than the VMThread. For > example, > >> may it be possible that if executing in the VMThread we defer some > >> activity that might require execution of Java code, or else hand it off > >> to one of the service threads? If we execute that code directly in the > >> current JavaThread instead we may not be in a valid state (e.g. consider > >> re-entrancy to various subsystems that is not allowed). > > > > It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is > doing. I already added a > > paragraph to the JBS-Item [1] explaining why the direct handshake is > sufficient from a > > synchronization point of view. > > Just to be clear, your proposed change is not using a direct handshake. > > > Furthermore the stack is walked and the return pc of compiled frames is > replaced with the address of > > the deopt handler. > > > > I can't see why this cannot be done with a direct handshake. Something > very similar is already done > > in JavaThread::deoptimize_marked_methods() which is executed as part > of an ordinary handshake. > > Note that existing non-direct handshakes may also have issues that not > have been fully investigated. > > > The demand on stack-space should be very modest. I would not expect a > higher risk for stackoverflow. > > For the target thread if you use more stack than would be used stopping > at a safepoint then you are at risk. For the thread initiating the > direct handshake if you use more stack than would be used enqueuing a VM > operation, then you are at risk. As we have not quantified these > numbers, nor have any easy way to establish the stack use of the actual > code to be executed, we're really just hoping for the best. This is a > general problem with handshakes that needs to be investigated more > deeply. As a simple, general, example just imagine if the code involves > logging that might utilise an on-stack buffer. > > >> Second, we have this question mark over what happens if the operation > >> hits further safepoint or handshake polls/checks? Are there constraints > >> on what is allowed here? How can we recognise this problem may exist > and > >> so deal with it? > > > > The thread in EnterInterpOnlyModeClosure::do_thread() can't become > safepoint/handshake safe. I > > tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a > NoSafepointVerifier. > > That's good to hear but such tests are not exhaustive, they will detect > if you do reach a safepoint/handshake but they can't prove that you > cannot reach one. What you have done is necessary but may not be > sufficient. Plus you didn't actually add the NSV to the code - is there > a reason we can't actually keep it in do_thread? (I'm not sure if the > NSV also acts as a NoHandshakeVerifier?) > > >> Third, while we are generally considering what appear to be > >> single-thread operations, which should be amenable to a direct > >> handshake, we also have to be careful that some of the code involved > >> doesn't already expect/assume we are at a safepoint - e.g. a VM op may > >> not need to take a lock where a direct handshake might! > > > > See again my arguments in the JBS item [1]. > > Yes I see the reasoning and that is good. My point is a general one as > it may not be obvious when such assumptions exist in the current code. > > Thanks, > David > > > Thanks, > > Richard. > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8238585 > > > > -----Original Message----- > > From: David Holmes > > Sent: Montag, 27. April 2020 07:16 > > To: Reingruber, Richard ; Yasumasa Suenaga > ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > > Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > > > > Hi all, > > > > Not a review but some general commentary ... > > > > On 25/04/2020 2:08 am, Reingruber, Richard wrote: > >> Hi Yasumasa, Patricio, > >> > >>>>> I will send review request to replace VM_SetFramePop to handshake > in early next week in JDK-8242427. > >>>>> Does it help you? I think it gives you to remove workaround. > >>>> > >>>> I think it would not help that much. Note that when replacing > VM_SetFramePop with a direct handshake > >>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm > operation [1]. So you would have to > >>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt > to these changes. > >> > >>> Thanks for your information. > >>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and > vmTestbase/nsk/jvmti/NotifyFramePop. > >>> I will modify and will test it after yours. > >> > >> Thanks :) > >> > >>>> Also my first impression was that it won't be that easy from a > synchronization point of view to > >>>> replace VM_SetFramePop with a direct handshake. E.g. > VM_SetFramePop::doit() indirectly calls > >>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, > JvmtiFramePop fpop) where > >>>> JvmtiThreadState_lock is acquired with safepoint check, if not at > safepoint. It's not directly clear > >>>> to me, how this has to be handled. > >> > >>> I think JvmtiEventController::set_frame_pop() should hold > JvmtiThreadState_lock because it affects other JVMTI operation especially > FramePop event. > >> > >> Yes. To me it is unclear what synchronization is necessary, if it is called > during a handshake. And > >> also I'm unsure if a thread should do safepoint checks while executing a > handshake. > > > > I'm growing increasingly concerned that use of direct handshakes to > > replace VM operations needs a much greater examination for correctness > > than might initially be thought. I see a number of issues: > > > > First, the VMThread executes (most) VM operations with a clean stack in > > a clean state, so it has lots of room to work. If we now execute the > > same logic in a JavaThread then we risk hitting stackoverflows if > > nothing else. But we are also now executing code in a JavaThread and so > > we have to be sure that code is not going to act differently (in a bad > > way) if executed by a JavaThread rather than the VMThread. For example, > > may it be possible that if executing in the VMThread we defer some > > activity that might require execution of Java code, or else hand it off > > to one of the service threads? If we execute that code directly in the > > current JavaThread instead we may not be in a valid state (e.g. consider > > re-entrancy to various subsystems that is not allowed). > > > > Second, we have this question mark over what happens if the operation > > hits further safepoint or handshake polls/checks? Are there constraints > > on what is allowed here? How can we recognise this problem may exist and > > so deal with it? > > > > Third, while we are generally considering what appear to be > > single-thread operations, which should be amenable to a direct > > handshake, we also have to be careful that some of the code involved > > doesn't already expect/assume we are at a safepoint - e.g. a VM op may > > not need to take a lock where a direct handshake might! > > > > Cheers, > > David > > ----- > > > >> @Patricio, coming back to my question [1]: > >> > >> In the example you gave in your answer [2]: the java thread would > execute a vm operation during a > >> direct handshake operation, while the VMThread is actually in the middle > of a VM_HandshakeAllThreads > >> operation, waiting to handshake the same handshakee: why can't the > VMThread just proceed? The > >> handshakee would be safepoint safe, wouldn't it? > >> > >> Thanks, Richard. > >> > >> [1] https://bugs.openjdk.java.net/browse/JDK- > 8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.syst > em.issuetabpanels:comment-tabpanel#comment-14301677 > >> > >> [2] https://bugs.openjdk.java.net/browse/JDK- > 8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.syst > em.issuetabpanels:comment-tabpanel#comment-14301763 > >> > >> -----Original Message----- > >> From: Yasumasa Suenaga > >> Sent: Freitag, 24. April 2020 17:23 > >> To: Reingruber, Richard ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >> > >> Hi Richard, > >> > >> On 2020/04/24 23:44, Reingruber, Richard wrote: > >>> Hi Yasumasa, > >>> > >>>> I will send review request to replace VM_SetFramePop to handshake > in early next week in JDK-8242427. > >>>> Does it help you? I think it gives you to remove workaround. > >>> > >>> I think it would not help that much. Note that when replacing > VM_SetFramePop with a direct handshake > >>> you could not just execute VM_EnterInterpOnlyMode as a nested vm > operation [1]. So you would have to > >>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to > these changes. > >> > >> Thanks for your information. > >> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and > vmTestbase/nsk/jvmti/NotifyFramePop. > >> I will modify and will test it after yours. > >> > >> > >>> Also my first impression was that it won't be that easy from a > synchronization point of view to > >>> replace VM_SetFramePop with a direct handshake. E.g. > VM_SetFramePop::doit() indirectly calls > >>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, > JvmtiFramePop fpop) where > >>> JvmtiThreadState_lock is acquired with safepoint check, if not at > safepoint. It's not directly clear > >>> to me, how this has to be handled. > >> > >> I think JvmtiEventController::set_frame_pop() should hold > JvmtiThreadState_lock because it affects other JVMTI operation especially > FramePop event. > >> > >> > >> Thanks, > >> > >> Yasumasa > >> > >> > >>> So it appears to me that it would be easier to push JDK-8242427 after this > (JDK-8238585). > >>> > >>>> (The patch is available, but I want to see the result of PIT in this > weekend whether JDK-8242425 works fine.) > >>> > >>> Would be interesting to see how you handled the issues above :) > >>> > >>> Thanks, Richard. > >>> > >>> [1] See question in comment > https://bugs.openjdk.java.net/browse/JDK- > 8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.syst > em.issuetabpanels:comment-tabpanel#comment-14302030 > >>> > >>> -----Original Message----- > >>> From: Yasumasa Suenaga > >>> Sent: Freitag, 24. April 2020 13:34 > >>> To: Reingruber, Richard ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >>> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>> > >>> Hi Richard, > >>> > >>> I will send review request to replace VM_SetFramePop to handshake in > early next week in JDK-8242427. > >>> Does it help you? I think it gives you to remove workaround. > >>> > >>> (The patch is available, but I want to see the result of PIT in this weekend > whether JDK-8242425 works fine.) > >>> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>> On 2020/04/24 17:18, Reingruber, Richard wrote: > >>>> Hi Patricio, Vladimir, and Serguei, > >>>> > >>>> now that direct handshakes are available, I've updated the patch to > make use of them. > >>>> > >>>> In addition I have done some clean-up changes I missed in the first > webrev. > >>>> > >>>> Finally I have implemented the workaround suggested by Patricio to > avoid nesting the handshake > >>>> into the vm operation VM_SetFramePop [1] > >>>> > >>>> Kindly review again: > >>>> > >>>> Webrev: > http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ > >>>> Webrev(delta): > http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ > >>>> > >>>> I updated the JBS item explaining why the vm operation > VM_EnterInterpOnlyMode can be replaced with a > >>>> direct handshake: > >>>> > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 > >>>> > >>>> Testing: > >>>> > >>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release > builds on all platforms. > >>>> > >>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 > >>>> > >>>> Thanks, > >>>> Richard. > >>>> > >>>> [1] An assertion in Handshake::execute_direct() fails, if called be > VMThread, because it is no JavaThread. > >>>> > >>>> -----Original Message----- > >>>> From: hotspot-dev On > Behalf Of Reingruber, Richard > >>>> Sent: Freitag, 14. Februar 2020 19:47 > >>>> To: Patricio Chilano ; > serviceability-dev at openjdk.java.net; hotspot-compiler- > dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > >>>> Subject: RE: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>>> > >>>> Hi Patricio, > >>>> > >>>> > > I'm really glad you noticed the problematic nesting. This seems > to be a general issue: currently a > >>>> > > handshake cannot be nested in a vm operation. Maybe it should > be asserted in the > >>>> > > Handshake::execute() methods that they are not called by the > vm thread evaluating a vm operation? > >>>> > > > >>>> > > > Alternatively I think you could do something similar to what > we do in > >>>> > > > Deoptimization::deoptimize_all_marked(): > >>>> > > > > >>>> > > > EnterInterpOnlyModeClosure hs; > >>>> > > > if (SafepointSynchronize::is_at_safepoint()) { > >>>> > > > hs.do_thread(state->get_thread()); > >>>> > > > } else { > >>>> > > > Handshake::execute(&hs, state->get_thread()); > >>>> > > > } > >>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to > the > >>>> > > > HandshakeClosure() constructor) > >>>> > > > >>>> > > Maybe this could be used also in the Handshake::execute() > methods as general solution? > >>>> > Right, we could also do that. Avoiding to clear the polling page in > >>>> > HandshakeState::clear_handshake() should be enough to fix this > issue and > >>>> > execute a handshake inside a safepoint, but adding that "if" > statement > >>>> > in Hanshake::execute() sounds good to avoid all the extra code > that we > >>>> > go through when executing a handshake. I filed 8239084 to make > that change. > >>>> > >>>> Thanks for taking care of this and creating the RFE. > >>>> > >>>> > > >>>> > > > I don?t know JVMTI code so I?m not sure if > VM_EnterInterpOnlyMode is > >>>> > > > always called in a nested operation or just sometimes. > >>>> > > > >>>> > > At least one execution path without vm operation exists: > >>>> > > > >>>> > > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) > : void > >>>> > > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState > *) : jlong > >>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void > >>>> > > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 > matches) > >>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, > bool) : void > >>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : > jvmtiError > >>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : > jvmtiError > >>>> > > > >>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't > my main intent to replace it with a > >>>> > > handshake, but to avoid making the compiled methods on stack > not_entrant.... unless I'm further > >>>> > > encouraged to do it with a handshake :) > >>>> > Ah! I think you can still do it with a handshake with the > >>>> > Deoptimization::deoptimize_all_marked() like solution. I can > change the > >>>> > if-else statement with just the Handshake::execute() call in > 8239084. > >>>> > But up to you. : ) > >>>> > >>>> Well, I think that's enough encouragement :) > >>>> I'll wait for 8239084 and try then again. > >>>> (no urgency and all) > >>>> > >>>> Thanks, > >>>> Richard. > >>>> > >>>> -----Original Message----- > >>>> From: Patricio Chilano > >>>> Sent: Freitag, 14. Februar 2020 15:54 > >>>> To: Reingruber, Richard ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >>>> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>>> > >>>> Hi Richard, > >>>> > >>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: > >>>>> Hi Patricio, > >>>>> > >>>>> thanks for having a look. > >>>>> > >>>>> > I?m only commenting on the handshake changes. > >>>>> > I see that operation VM_EnterInterpOnlyMode can be called > inside > >>>>> > operation VM_SetFramePop which also allows nested > operations. Here is a > >>>>> > comment in VM_SetFramePop definition: > >>>>> > > >>>>> > // Nested operation must be allowed for the > VM_EnterInterpOnlyMode that is > >>>>> > // called from the > JvmtiEventControllerPrivate::recompute_thread_enabled. > >>>>> > > >>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, > then now we > >>>>> > could have a handshake inside a safepoint operation. The issue I > see > >>>>> > there is that at the end of the handshake the polling page of the > target > >>>>> > thread could be disarmed. So if the target thread happens to be > in a > >>>>> > blocked state just transiently and wakes up then it will not stop > for > >>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the > >>>>> > polling page is armed at the beginning of disarm_safepoint(). > >>>>> > >>>>> I'm really glad you noticed the problematic nesting. This seems to be a > general issue: currently a > >>>>> handshake cannot be nested in a vm operation. Maybe it should be > asserted in the > >>>>> Handshake::execute() methods that they are not called by the vm > thread evaluating a vm operation? > >>>>> > >>>>> > Alternatively I think you could do something similar to what we > do in > >>>>> > Deoptimization::deoptimize_all_marked(): > >>>>> > > >>>>> > EnterInterpOnlyModeClosure hs; > >>>>> > if (SafepointSynchronize::is_at_safepoint()) { > >>>>> > hs.do_thread(state->get_thread()); > >>>>> > } else { > >>>>> > Handshake::execute(&hs, state->get_thread()); > >>>>> > } > >>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the > >>>>> > HandshakeClosure() constructor) > >>>>> > >>>>> Maybe this could be used also in the Handshake::execute() methods > as general solution? > >>>> Right, we could also do that. Avoiding to clear the polling page in > >>>> HandshakeState::clear_handshake() should be enough to fix this issue > and > >>>> execute a handshake inside a safepoint, but adding that "if" statement > >>>> in Hanshake::execute() sounds good to avoid all the extra code that we > >>>> go through when executing a handshake. I filed 8239084 to make that > change. > >>>> > >>>>> > I don?t know JVMTI code so I?m not sure if > VM_EnterInterpOnlyMode is > >>>>> > always called in a nested operation or just sometimes. > >>>>> > >>>>> At least one execution path without vm operation exists: > >>>>> > >>>>> > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) > : void > >>>>> > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState > *) : jlong > >>>>> JvmtiEventControllerPrivate::recompute_enabled() : void > >>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, > bool) : void (2 matches) > >>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : > void > >>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError > >>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : > jvmtiError > >>>>> > >>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my > main intent to replace it with a > >>>>> handshake, but to avoid making the compiled methods on stack > not_entrant.... unless I'm further > >>>>> encouraged to do it with a handshake :) > >>>> Ah! I think you can still do it with a handshake with the > >>>> Deoptimization::deoptimize_all_marked() like solution. I can change > the > >>>> if-else statement with just the Handshake::execute() call in 8239084. > >>>> But up to you.? : ) > >>>> > >>>> Thanks, > >>>> Patricio > >>>>> Thanks again, > >>>>> Richard. > >>>>> > >>>>> -----Original Message----- > >>>>> From: Patricio Chilano > >>>>> Sent: Donnerstag, 13. Februar 2020 18:47 > >>>>> To: Reingruber, Richard ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >>>>> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>>>> > >>>>> Hi Richard, > >>>>> > >>>>> I?m only commenting on the handshake changes. > >>>>> I see that operation VM_EnterInterpOnlyMode can be called inside > >>>>> operation VM_SetFramePop which also allows nested operations. > Here is a > >>>>> comment in VM_SetFramePop definition: > >>>>> > >>>>> // Nested operation must be allowed for the > VM_EnterInterpOnlyMode that is > >>>>> // called from the > JvmtiEventControllerPrivate::recompute_thread_enabled. > >>>>> > >>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then > now we > >>>>> could have a handshake inside a safepoint operation. The issue I see > >>>>> there is that at the end of the handshake the polling page of the > target > >>>>> thread could be disarmed. So if the target thread happens to be in a > >>>>> blocked state just transiently and wakes up then it will not stop for > >>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the > >>>>> polling page is armed at the beginning of disarm_safepoint(). > >>>>> > >>>>> I think one option could be to remove > >>>>> SafepointMechanism::disarm_if_needed() in > >>>>> HandshakeState::clear_handshake() and let each JavaThread disarm > itself > >>>>> for the handshake case. > >>>>> > >>>>> Alternatively I think you could do something similar to what we do in > >>>>> Deoptimization::deoptimize_all_marked(): > >>>>> > >>>>> ? EnterInterpOnlyModeClosure hs; > >>>>> ? if (SafepointSynchronize::is_at_safepoint()) { > >>>>> ??? hs.do_thread(state->get_thread()); > >>>>> ? } else { > >>>>> ??? Handshake::execute(&hs, state->get_thread()); > >>>>> ? } > >>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the > >>>>> HandshakeClosure() constructor) > >>>>> > >>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode > is > >>>>> always called in a nested operation or just sometimes. > >>>>> > >>>>> Thanks, > >>>>> Patricio > >>>>> > >>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: > >>>>>> // Repost including hotspot runtime and gc lists. > >>>>>> // Dean Long suggested to do so, because the enhancement > replaces a vm operation > >>>>>> // with a handshake. > >>>>>> // Original thread: > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020- > February/030359.html > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> could I please get reviews for this small enhancement in hotspot's > jvmti implementation: > >>>>>> > >>>>>> Webrev: > http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ > >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 > >>>>>> > >>>>>> The change avoids making all compiled methods on stack > not_entrant when switching a java thread to > >>>>>> interpreter only execution for jvmti purposes. It is sufficient to > deoptimize the compiled frames on stack. > >>>>>> > >>>>>> Additionally a handshake is used instead of a vm operation to walk > the stack and do the deoptimizations. > >>>>>> > >>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug > and release builds on all platforms. > >>>>>> > >>>>>> Thanks, Richard. > >>>>>> > >>>>>> See also my question if anyone knows a reason for making the > compiled methods not_entrant: > >>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020- > January/030339.html > >>>> From richard.reingruber at sap.com Tue May 12 09:42:39 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Tue, 12 May 2020 09:42:39 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> Message-ID: Thanks Martin. Cheers, Richard. -----Original Message----- From: Doerr, Martin Sent: Dienstag, 12. Mai 2020 10:43 To: Reingruber, Richard ; David Holmes ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant Hi Richard, I had already reviewed webrev.1. Looks good to me. Thanks for contributing it. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > Sent: Montag, 4. Mai 2020 12:33 > To: David Holmes ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > Subject: RE: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > > // Trimmed the list of recipients. If the list gets too long then the message > needs to be approved > // by a moderator. > > Hi David, > > > On 28/04/2020 12:09 am, Reingruber, Richard wrote: > > > Hi David, > > > > > >> Not a review but some general commentary ... > > > > > > That's welcome. > > > Having had to take an even closer look now I have a review comment too :) > > > src/hotspot/share/prims/jvmtiThreadState.cpp > > > void JvmtiThreadState::invalidate_cur_stack_depth() { > > ! assert(SafepointSynchronize::is_at_safepoint() || > > ! (Thread::current()->is_VM_thread() && > > get_thread()->is_vmthread_processing_handshake()) || > > (JavaThread *)Thread::current() == get_thread(), > > "must be current thread or at safepoint"); > > You're looking at an outdated webrev, I'm afraid. > > This would be the post with the current webrev.1 > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020- > April/031245.html > > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Montag, 4. Mai 2020 08:51 > To: Reingruber, Richard ; Yasumasa Suenaga > ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > > Hi Richard, > > On 28/04/2020 12:09 am, Reingruber, Richard wrote: > > Hi David, > > > >> Not a review but some general commentary ... > > > > That's welcome. > > Having had to take an even closer look now I have a review comment too :) > > src/hotspot/share/prims/jvmtiThreadState.cpp > > void JvmtiThreadState::invalidate_cur_stack_depth() { > ! assert(SafepointSynchronize::is_at_safepoint() || > ! (Thread::current()->is_VM_thread() && > get_thread()->is_vmthread_processing_handshake()) || > (JavaThread *)Thread::current() == get_thread(), > "must be current thread or at safepoint"); > > The message needs updating to include handshakes. > > More below ... > > >> On 25/04/2020 2:08 am, Reingruber, Richard wrote: > >>> Hi Yasumasa, Patricio, > >>> > >>>>>> I will send review request to replace VM_SetFramePop to > handshake in early next week in JDK-8242427. > >>>>>> Does it help you? I think it gives you to remove workaround. > >>>>> > >>>>> I think it would not help that much. Note that when replacing > VM_SetFramePop with a direct handshake > >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm > operation [1]. So you would have to > >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt > to these changes. > >>> > >>>> Thanks for your information. > >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and > vmTestbase/nsk/jvmti/NotifyFramePop. > >>>> I will modify and will test it after yours. > >>> > >>> Thanks :) > >>> > >>>>> Also my first impression was that it won't be that easy from a > synchronization point of view to > >>>>> replace VM_SetFramePop with a direct handshake. E.g. > VM_SetFramePop::doit() indirectly calls > >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, > JvmtiFramePop fpop) where > >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at > safepoint. It's not directly clear > >>>>> to me, how this has to be handled. > >>> > >>>> I think JvmtiEventController::set_frame_pop() should hold > JvmtiThreadState_lock because it affects other JVMTI operation especially > FramePop event. > >>> > >>> Yes. To me it is unclear what synchronization is necessary, if it is called > during a handshake. And > >>> also I'm unsure if a thread should do safepoint checks while executing a > handshake. > > > >> I'm growing increasingly concerned that use of direct handshakes to > >> replace VM operations needs a much greater examination for correctness > >> than might initially be thought. I see a number of issues: > > > > I agree. I'll address your concerns in the context of this review thread for > JDK-8238585 below. > > > > In addition I would suggest to take the general part of the discussion to a > dedicated thread or to > > the review thread for JDK-8242427. I would like to keep this thread closer > to its subject. > > I will focus on the issues in the context of this particular change > then, though the issues themselves are applicable to all handshake > situations (and more so with direct handshakes). This is mostly just > discussion. > > >> First, the VMThread executes (most) VM operations with a clean stack in > >> a clean state, so it has lots of room to work. If we now execute the > >> same logic in a JavaThread then we risk hitting stackoverflows if > >> nothing else. But we are also now executing code in a JavaThread and so > >> we have to be sure that code is not going to act differently (in a bad > >> way) if executed by a JavaThread rather than the VMThread. For > example, > >> may it be possible that if executing in the VMThread we defer some > >> activity that might require execution of Java code, or else hand it off > >> to one of the service threads? If we execute that code directly in the > >> current JavaThread instead we may not be in a valid state (e.g. consider > >> re-entrancy to various subsystems that is not allowed). > > > > It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is > doing. I already added a > > paragraph to the JBS-Item [1] explaining why the direct handshake is > sufficient from a > > synchronization point of view. > > Just to be clear, your proposed change is not using a direct handshake. > > > Furthermore the stack is walked and the return pc of compiled frames is > replaced with the address of > > the deopt handler. > > > > I can't see why this cannot be done with a direct handshake. Something > very similar is already done > > in JavaThread::deoptimize_marked_methods() which is executed as part > of an ordinary handshake. > > Note that existing non-direct handshakes may also have issues that not > have been fully investigated. > > > The demand on stack-space should be very modest. I would not expect a > higher risk for stackoverflow. > > For the target thread if you use more stack than would be used stopping > at a safepoint then you are at risk. For the thread initiating the > direct handshake if you use more stack than would be used enqueuing a VM > operation, then you are at risk. As we have not quantified these > numbers, nor have any easy way to establish the stack use of the actual > code to be executed, we're really just hoping for the best. This is a > general problem with handshakes that needs to be investigated more > deeply. As a simple, general, example just imagine if the code involves > logging that might utilise an on-stack buffer. > > >> Second, we have this question mark over what happens if the operation > >> hits further safepoint or handshake polls/checks? Are there constraints > >> on what is allowed here? How can we recognise this problem may exist > and > >> so deal with it? > > > > The thread in EnterInterpOnlyModeClosure::do_thread() can't become > safepoint/handshake safe. I > > tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a > NoSafepointVerifier. > > That's good to hear but such tests are not exhaustive, they will detect > if you do reach a safepoint/handshake but they can't prove that you > cannot reach one. What you have done is necessary but may not be > sufficient. Plus you didn't actually add the NSV to the code - is there > a reason we can't actually keep it in do_thread? (I'm not sure if the > NSV also acts as a NoHandshakeVerifier?) > > >> Third, while we are generally considering what appear to be > >> single-thread operations, which should be amenable to a direct > >> handshake, we also have to be careful that some of the code involved > >> doesn't already expect/assume we are at a safepoint - e.g. a VM op may > >> not need to take a lock where a direct handshake might! > > > > See again my arguments in the JBS item [1]. > > Yes I see the reasoning and that is good. My point is a general one as > it may not be obvious when such assumptions exist in the current code. > > Thanks, > David > > > Thanks, > > Richard. > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8238585 > > > > -----Original Message----- > > From: David Holmes > > Sent: Montag, 27. April 2020 07:16 > > To: Reingruber, Richard ; Yasumasa Suenaga > ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > > Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > > > > Hi all, > > > > Not a review but some general commentary ... > > > > On 25/04/2020 2:08 am, Reingruber, Richard wrote: > >> Hi Yasumasa, Patricio, > >> > >>>>> I will send review request to replace VM_SetFramePop to handshake > in early next week in JDK-8242427. > >>>>> Does it help you? I think it gives you to remove workaround. > >>>> > >>>> I think it would not help that much. Note that when replacing > VM_SetFramePop with a direct handshake > >>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm > operation [1]. So you would have to > >>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt > to these changes. > >> > >>> Thanks for your information. > >>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and > vmTestbase/nsk/jvmti/NotifyFramePop. > >>> I will modify and will test it after yours. > >> > >> Thanks :) > >> > >>>> Also my first impression was that it won't be that easy from a > synchronization point of view to > >>>> replace VM_SetFramePop with a direct handshake. E.g. > VM_SetFramePop::doit() indirectly calls > >>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, > JvmtiFramePop fpop) where > >>>> JvmtiThreadState_lock is acquired with safepoint check, if not at > safepoint. It's not directly clear > >>>> to me, how this has to be handled. > >> > >>> I think JvmtiEventController::set_frame_pop() should hold > JvmtiThreadState_lock because it affects other JVMTI operation especially > FramePop event. > >> > >> Yes. To me it is unclear what synchronization is necessary, if it is called > during a handshake. And > >> also I'm unsure if a thread should do safepoint checks while executing a > handshake. > > > > I'm growing increasingly concerned that use of direct handshakes to > > replace VM operations needs a much greater examination for correctness > > than might initially be thought. I see a number of issues: > > > > First, the VMThread executes (most) VM operations with a clean stack in > > a clean state, so it has lots of room to work. If we now execute the > > same logic in a JavaThread then we risk hitting stackoverflows if > > nothing else. But we are also now executing code in a JavaThread and so > > we have to be sure that code is not going to act differently (in a bad > > way) if executed by a JavaThread rather than the VMThread. For example, > > may it be possible that if executing in the VMThread we defer some > > activity that might require execution of Java code, or else hand it off > > to one of the service threads? If we execute that code directly in the > > current JavaThread instead we may not be in a valid state (e.g. consider > > re-entrancy to various subsystems that is not allowed). > > > > Second, we have this question mark over what happens if the operation > > hits further safepoint or handshake polls/checks? Are there constraints > > on what is allowed here? How can we recognise this problem may exist and > > so deal with it? > > > > Third, while we are generally considering what appear to be > > single-thread operations, which should be amenable to a direct > > handshake, we also have to be careful that some of the code involved > > doesn't already expect/assume we are at a safepoint - e.g. a VM op may > > not need to take a lock where a direct handshake might! > > > > Cheers, > > David > > ----- > > > >> @Patricio, coming back to my question [1]: > >> > >> In the example you gave in your answer [2]: the java thread would > execute a vm operation during a > >> direct handshake operation, while the VMThread is actually in the middle > of a VM_HandshakeAllThreads > >> operation, waiting to handshake the same handshakee: why can't the > VMThread just proceed? The > >> handshakee would be safepoint safe, wouldn't it? > >> > >> Thanks, Richard. > >> > >> [1] https://bugs.openjdk.java.net/browse/JDK- > 8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.syst > em.issuetabpanels:comment-tabpanel#comment-14301677 > >> > >> [2] https://bugs.openjdk.java.net/browse/JDK- > 8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.syst > em.issuetabpanels:comment-tabpanel#comment-14301763 > >> > >> -----Original Message----- > >> From: Yasumasa Suenaga > >> Sent: Freitag, 24. April 2020 17:23 > >> To: Reingruber, Richard ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >> > >> Hi Richard, > >> > >> On 2020/04/24 23:44, Reingruber, Richard wrote: > >>> Hi Yasumasa, > >>> > >>>> I will send review request to replace VM_SetFramePop to handshake > in early next week in JDK-8242427. > >>>> Does it help you? I think it gives you to remove workaround. > >>> > >>> I think it would not help that much. Note that when replacing > VM_SetFramePop with a direct handshake > >>> you could not just execute VM_EnterInterpOnlyMode as a nested vm > operation [1]. So you would have to > >>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to > these changes. > >> > >> Thanks for your information. > >> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and > vmTestbase/nsk/jvmti/NotifyFramePop. > >> I will modify and will test it after yours. > >> > >> > >>> Also my first impression was that it won't be that easy from a > synchronization point of view to > >>> replace VM_SetFramePop with a direct handshake. E.g. > VM_SetFramePop::doit() indirectly calls > >>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, > JvmtiFramePop fpop) where > >>> JvmtiThreadState_lock is acquired with safepoint check, if not at > safepoint. It's not directly clear > >>> to me, how this has to be handled. > >> > >> I think JvmtiEventController::set_frame_pop() should hold > JvmtiThreadState_lock because it affects other JVMTI operation especially > FramePop event. > >> > >> > >> Thanks, > >> > >> Yasumasa > >> > >> > >>> So it appears to me that it would be easier to push JDK-8242427 after this > (JDK-8238585). > >>> > >>>> (The patch is available, but I want to see the result of PIT in this > weekend whether JDK-8242425 works fine.) > >>> > >>> Would be interesting to see how you handled the issues above :) > >>> > >>> Thanks, Richard. > >>> > >>> [1] See question in comment > https://bugs.openjdk.java.net/browse/JDK- > 8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.syst > em.issuetabpanels:comment-tabpanel#comment-14302030 > >>> > >>> -----Original Message----- > >>> From: Yasumasa Suenaga > >>> Sent: Freitag, 24. April 2020 13:34 > >>> To: Reingruber, Richard ; Patricio Chilano > ; serguei.spitsyn at oracle.com; Vladimir > Ivanov ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >>> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>> > >>> Hi Richard, > >>> > >>> I will send review request to replace VM_SetFramePop to handshake in > early next week in JDK-8242427. > >>> Does it help you? I think it gives you to remove workaround. > >>> > >>> (The patch is available, but I want to see the result of PIT in this weekend > whether JDK-8242425 works fine.) > >>> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>> On 2020/04/24 17:18, Reingruber, Richard wrote: > >>>> Hi Patricio, Vladimir, and Serguei, > >>>> > >>>> now that direct handshakes are available, I've updated the patch to > make use of them. > >>>> > >>>> In addition I have done some clean-up changes I missed in the first > webrev. > >>>> > >>>> Finally I have implemented the workaround suggested by Patricio to > avoid nesting the handshake > >>>> into the vm operation VM_SetFramePop [1] > >>>> > >>>> Kindly review again: > >>>> > >>>> Webrev: > http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ > >>>> Webrev(delta): > http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ > >>>> > >>>> I updated the JBS item explaining why the vm operation > VM_EnterInterpOnlyMode can be replaced with a > >>>> direct handshake: > >>>> > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 > >>>> > >>>> Testing: > >>>> > >>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release > builds on all platforms. > >>>> > >>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 > >>>> > >>>> Thanks, > >>>> Richard. > >>>> > >>>> [1] An assertion in Handshake::execute_direct() fails, if called be > VMThread, because it is no JavaThread. > >>>> > >>>> -----Original Message----- > >>>> From: hotspot-dev On > Behalf Of Reingruber, Richard > >>>> Sent: Freitag, 14. Februar 2020 19:47 > >>>> To: Patricio Chilano ; > serviceability-dev at openjdk.java.net; hotspot-compiler- > dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > >>>> Subject: RE: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>>> > >>>> Hi Patricio, > >>>> > >>>> > > I'm really glad you noticed the problematic nesting. This seems > to be a general issue: currently a > >>>> > > handshake cannot be nested in a vm operation. Maybe it should > be asserted in the > >>>> > > Handshake::execute() methods that they are not called by the > vm thread evaluating a vm operation? > >>>> > > > >>>> > > > Alternatively I think you could do something similar to what > we do in > >>>> > > > Deoptimization::deoptimize_all_marked(): > >>>> > > > > >>>> > > > EnterInterpOnlyModeClosure hs; > >>>> > > > if (SafepointSynchronize::is_at_safepoint()) { > >>>> > > > hs.do_thread(state->get_thread()); > >>>> > > > } else { > >>>> > > > Handshake::execute(&hs, state->get_thread()); > >>>> > > > } > >>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to > the > >>>> > > > HandshakeClosure() constructor) > >>>> > > > >>>> > > Maybe this could be used also in the Handshake::execute() > methods as general solution? > >>>> > Right, we could also do that. Avoiding to clear the polling page in > >>>> > HandshakeState::clear_handshake() should be enough to fix this > issue and > >>>> > execute a handshake inside a safepoint, but adding that "if" > statement > >>>> > in Hanshake::execute() sounds good to avoid all the extra code > that we > >>>> > go through when executing a handshake. I filed 8239084 to make > that change. > >>>> > >>>> Thanks for taking care of this and creating the RFE. > >>>> > >>>> > > >>>> > > > I don?t know JVMTI code so I?m not sure if > VM_EnterInterpOnlyMode is > >>>> > > > always called in a nested operation or just sometimes. > >>>> > > > >>>> > > At least one execution path without vm operation exists: > >>>> > > > >>>> > > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) > : void > >>>> > > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState > *) : jlong > >>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void > >>>> > > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 > matches) > >>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, > bool) : void > >>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : > jvmtiError > >>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : > jvmtiError > >>>> > > > >>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't > my main intent to replace it with a > >>>> > > handshake, but to avoid making the compiled methods on stack > not_entrant.... unless I'm further > >>>> > > encouraged to do it with a handshake :) > >>>> > Ah! I think you can still do it with a handshake with the > >>>> > Deoptimization::deoptimize_all_marked() like solution. I can > change the > >>>> > if-else statement with just the Handshake::execute() call in > 8239084. > >>>> > But up to you. : ) > >>>> > >>>> Well, I think that's enough encouragement :) > >>>> I'll wait for 8239084 and try then again. > >>>> (no urgency and all) > >>>> > >>>> Thanks, > >>>> Richard. > >>>> > >>>> -----Original Message----- > >>>> From: Patricio Chilano > >>>> Sent: Freitag, 14. Februar 2020 15:54 > >>>> To: Reingruber, Richard ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >>>> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>>> > >>>> Hi Richard, > >>>> > >>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: > >>>>> Hi Patricio, > >>>>> > >>>>> thanks for having a look. > >>>>> > >>>>> > I?m only commenting on the handshake changes. > >>>>> > I see that operation VM_EnterInterpOnlyMode can be called > inside > >>>>> > operation VM_SetFramePop which also allows nested > operations. Here is a > >>>>> > comment in VM_SetFramePop definition: > >>>>> > > >>>>> > // Nested operation must be allowed for the > VM_EnterInterpOnlyMode that is > >>>>> > // called from the > JvmtiEventControllerPrivate::recompute_thread_enabled. > >>>>> > > >>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, > then now we > >>>>> > could have a handshake inside a safepoint operation. The issue I > see > >>>>> > there is that at the end of the handshake the polling page of the > target > >>>>> > thread could be disarmed. So if the target thread happens to be > in a > >>>>> > blocked state just transiently and wakes up then it will not stop > for > >>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the > >>>>> > polling page is armed at the beginning of disarm_safepoint(). > >>>>> > >>>>> I'm really glad you noticed the problematic nesting. This seems to be a > general issue: currently a > >>>>> handshake cannot be nested in a vm operation. Maybe it should be > asserted in the > >>>>> Handshake::execute() methods that they are not called by the vm > thread evaluating a vm operation? > >>>>> > >>>>> > Alternatively I think you could do something similar to what we > do in > >>>>> > Deoptimization::deoptimize_all_marked(): > >>>>> > > >>>>> > EnterInterpOnlyModeClosure hs; > >>>>> > if (SafepointSynchronize::is_at_safepoint()) { > >>>>> > hs.do_thread(state->get_thread()); > >>>>> > } else { > >>>>> > Handshake::execute(&hs, state->get_thread()); > >>>>> > } > >>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the > >>>>> > HandshakeClosure() constructor) > >>>>> > >>>>> Maybe this could be used also in the Handshake::execute() methods > as general solution? > >>>> Right, we could also do that. Avoiding to clear the polling page in > >>>> HandshakeState::clear_handshake() should be enough to fix this issue > and > >>>> execute a handshake inside a safepoint, but adding that "if" statement > >>>> in Hanshake::execute() sounds good to avoid all the extra code that we > >>>> go through when executing a handshake. I filed 8239084 to make that > change. > >>>> > >>>>> > I don?t know JVMTI code so I?m not sure if > VM_EnterInterpOnlyMode is > >>>>> > always called in a nested operation or just sometimes. > >>>>> > >>>>> At least one execution path without vm operation exists: > >>>>> > >>>>> > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) > : void > >>>>> > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState > *) : jlong > >>>>> JvmtiEventControllerPrivate::recompute_enabled() : void > >>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, > bool) : void (2 matches) > >>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : > void > >>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError > >>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : > jvmtiError > >>>>> > >>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my > main intent to replace it with a > >>>>> handshake, but to avoid making the compiled methods on stack > not_entrant.... unless I'm further > >>>>> encouraged to do it with a handshake :) > >>>> Ah! I think you can still do it with a handshake with the > >>>> Deoptimization::deoptimize_all_marked() like solution. I can change > the > >>>> if-else statement with just the Handshake::execute() call in 8239084. > >>>> But up to you.? : ) > >>>> > >>>> Thanks, > >>>> Patricio > >>>>> Thanks again, > >>>>> Richard. > >>>>> > >>>>> -----Original Message----- > >>>>> From: Patricio Chilano > >>>>> Sent: Donnerstag, 13. Februar 2020 18:47 > >>>>> To: Reingruber, Richard ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot- > gc-dev at openjdk.java.net > >>>>> Subject: Re: RFR(S) 8238585: Use handshake for > JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make > compiled methods on stack not_entrant > >>>>> > >>>>> Hi Richard, > >>>>> > >>>>> I?m only commenting on the handshake changes. > >>>>> I see that operation VM_EnterInterpOnlyMode can be called inside > >>>>> operation VM_SetFramePop which also allows nested operations. > Here is a > >>>>> comment in VM_SetFramePop definition: > >>>>> > >>>>> // Nested operation must be allowed for the > VM_EnterInterpOnlyMode that is > >>>>> // called from the > JvmtiEventControllerPrivate::recompute_thread_enabled. > >>>>> > >>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then > now we > >>>>> could have a handshake inside a safepoint operation. The issue I see > >>>>> there is that at the end of the handshake the polling page of the > target > >>>>> thread could be disarmed. So if the target thread happens to be in a > >>>>> blocked state just transiently and wakes up then it will not stop for > >>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the > >>>>> polling page is armed at the beginning of disarm_safepoint(). > >>>>> > >>>>> I think one option could be to remove > >>>>> SafepointMechanism::disarm_if_needed() in > >>>>> HandshakeState::clear_handshake() and let each JavaThread disarm > itself > >>>>> for the handshake case. > >>>>> > >>>>> Alternatively I think you could do something similar to what we do in > >>>>> Deoptimization::deoptimize_all_marked(): > >>>>> > >>>>> ? EnterInterpOnlyModeClosure hs; > >>>>> ? if (SafepointSynchronize::is_at_safepoint()) { > >>>>> ??? hs.do_thread(state->get_thread()); > >>>>> ? } else { > >>>>> ??? Handshake::execute(&hs, state->get_thread()); > >>>>> ? } > >>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the > >>>>> HandshakeClosure() constructor) > >>>>> > >>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode > is > >>>>> always called in a nested operation or just sometimes. > >>>>> > >>>>> Thanks, > >>>>> Patricio > >>>>> > >>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: > >>>>>> // Repost including hotspot runtime and gc lists. > >>>>>> // Dean Long suggested to do so, because the enhancement > replaces a vm operation > >>>>>> // with a handshake. > >>>>>> // Original thread: > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020- > February/030359.html > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> could I please get reviews for this small enhancement in hotspot's > jvmti implementation: > >>>>>> > >>>>>> Webrev: > http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ > >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 > >>>>>> > >>>>>> The change avoids making all compiled methods on stack > not_entrant when switching a java thread to > >>>>>> interpreter only execution for jvmti purposes. It is sufficient to > deoptimize the compiled frames on stack. > >>>>>> > >>>>>> Additionally a handshake is used instead of a vm operation to walk > the stack and do the deoptimizations. > >>>>>> > >>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug > and release builds on all platforms. > >>>>>> > >>>>>> Thanks, Richard. > >>>>>> > >>>>>> See also my question if anyone knows a reason for making the > compiled methods not_entrant: > >>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020- > January/030339.html > >>>> From harold.seigel at oracle.com Tue May 12 12:49:50 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 12 May 2020 08:49:50 -0400 Subject: RFR 8244105: JDK 11.0.7 -Xlog gc issue with file path if not exist In-Reply-To: References: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> Message-ID: <3b81573b-bbb5-f237-d18f-6cc91ba4579b@oracle.com> Hi David, Thanks for looking at this. This change disables logging and issuing a warning for any errors in the -Xlog arguments that currently cause the JVM to terminate. For which particular bad -Xlog arguments do you think this is a bad idea? Are there some bad -Xlog arguments where you think this would be a good thing to do? I agree that if we decide to do this, a CSR is needed. Thanks, Harold On 5/12/2020 3:32 AM, David Holmes wrote: > Hi Harold, > > On 9/05/2020 3:22 am, Harold Seigel wrote: >> Hi, >> >> Please review this fix for JDK-8244105.? The fix continues program >> execution even when specified logging options are invalid.? >> Previously, invalid logging options terminated the program.? Now, a >> warning is issued.? For example: >> >> ???? > java -Xlog:"gc*:file=/dont/exist" -version >> ??? [0.001s][error][logging] Error opening log file '/dont/exist': No >> ??? such file or directory >> ??? [0.001s][error][logging] Initialization of output 'file=/dont/exist' >> ??? using options '(null)' failed. >> ??? Java HotSpot(TM) 64-Bit Server VM warning: Invalid -Xlog option >> ??? '-Xlog:gc*:file=/dont/exist', see error log for details. >> >> ??? java version "15-internal" 2020-09-15 >> ??? Java(TM) SE Runtime Environment (fastdebug build >> ??? 15-internal+0-2020-05-08-1313404.hseigel.bug8244105) >> ??? Java HotSpot(TM) 64-Bit Server VM (fastdebug build >> ??? 15-internal+0-2020-05-08-1313404.hseigel.bug8244105, mixed mode, >> ??? sharing) > > But if I am reading this correctly you are now only issuing a warning > and disabling logging if there are any errors of any kind in the -Xlog > arguments! That is not desirable IMO and a significant change in > behaviour. > > Even just changing the behaviour in relation to a non-existent log > file will require a CSR request. > > Thanks, > David > ----- > >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8244105/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8244105 >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests >> and builds on Linux-x64, Solaris, Windows, and Mac OS X and by >> running Mach5 tiers 3-5 tests on Linux-x64. >> >> Thanks, Harold >> >> From daniil.x.titov at oracle.com Tue May 12 16:44:37 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 12 May 2020 09:44:37 -0700 Subject: RFR: 8242009: Review setting test.java/vm.opts in jcmd/jhsdb and debugger in serviceability tests In-Reply-To: References: <8A7DC761-66F8-4AB2-AE46-B8C41C964859@oracle.com> <4d934419-e427-96f9-dda9-f0b7bffba981@oracle.com> <0E6DBBFE-4395-45F5-9CB3-C023ADD7DD2F@oracle.com> Message-ID: <057802C3-B431-476F-A318-A18555A95A0E@oracle.com> Hi Chris, Thank you for reviewing this change. Best regards, Daniil ?On 5/11/20, 10:55 AM, "Chris Plummer" wrote: Hi Daniil, Looks good. thanks, Chris On 5/11/20 9:43 AM, Daniil Titov wrote: > Hi Chris, > > Please review a new version of the fix[1] that also updates jdk/sun/tools tests. > > Testing: Mach5 tier1-tier7 tests successfully passed. > > > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.03/ > [2] ] https://bugs.openjdk.java.net/browse/JDK-8242009 > > Thank you, > Daniil > > ?On 5/8/20, 12:21 AM, "Chris Plummer" wrote: > > Hi Daniil, > > I just noticed you didn't update the tests in jdk/sun/tools/jhsdb. Do > you think these should be done also? > > Chris > > On 5/7/20 11:44 PM, Chris Plummer wrote: > > Hi Daniil, > > > > The changes look good. > > > > thanks, > > > > Chris > > > > On 5/4/20 1:05 PM, Daniil Titov wrote: > >> Hi Chris, > >> > >> Please review a new version of webrev [1] that addresses your comments. > >> > >> For the following 3 tests that showed the increase of the execution > >> time with -Xcomp > >> more than 5 minutes this version of the change strips -Xcomp option > >> when > >> forwarding VM arguments to j*-tools : > >> > >> -- serviceability/sa/sadebugd/SADebugDTest.java, > >> -- serviceability/sa/sadebugd/DebugdConnectTest.java, > >> -- serviceability/sa/ClhsdbJstackXcompStress.java > >> > >> The execution time for the rest of the tests when running with -Xcomp > >> was increased > >> within 1 and half minute. > >> > >> > >> [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.02/ > >> [2] https://bugs.openjdk.java.net/browse/JDK-8242009 > >> > >> Thank you, > >> Daniil > >> > >> > >> ?On 4/27/20, 12:55 PM, "Chris Plummer" wrote: > >> > >> Hi Daniil, > >> > >> Overall it looks good. A few comments below. > >> > >> Can you add a comment to TestSysProps.java indicating the reason > >> for > >> checking if the line starts with "[". > >> > >> In JDKToolLauncher you have an extra empty line: > >> > >> 112 * Any platform specific arguments required for > >> running the > >> tool are > >> 113 * automatically added. > >> 114 * > >> 115 * > >> 116 * @param args > >> > >> In OutputAnalyzer.java, I wonder if all these matching APIs you > >> updated > >> should by default just include the version output in their > >> filtering. > >> > >> As for the added time to execute, I would suggest possibly > >> stripping > >> -Xcomp from the few outliers, and I would mostly focus on how much > >> longer it takes, not how many times longer. For example, > >> increasing from > >> 10 seconds to 40 seconds is not a big deal. Increasing from 10 > >> minutes > >> to 20 minutes is. > >> > >> SADebugDTest creates 8 tool processes so, that's probably the > >> reason for > >> the big increase, although I'm not sure why it is kind of slow > >> in the > >> first place. It does nothing more on each iteration than launch > >> "jhsdb > >> debugd", which will connect to the debuggee, and then is killed > >> off. > >> Maybe there is something slow with connecting, especial with RMI. > >> > >> thanks, > >> > >> Chris > >> > >> On 4/27/20 12:07 PM, Daniil Titov wrote: > >> > Please review the change [1] that ensures that VM and test > >> options are forwarded to > >> > j*-tools when they are launched from serviceability/sa tests. > >> > > >> > The tests that expect an empty output were corrected to > >> ignore the product version printed > >> > in the error stream since in some tiers the tests are run > >> with ' -showversion' VM option (tier3). > >> > > >> > In test serviceability/sa/TestSysProps.java the code that > >> counts the system properties was corrected > >> > to ignore the debug output when the test is run with " > >> -Xlog:cds=debug" option (tier4). > >> > > >> > Testing: Mach5 tests for tier1 - tier7 passed. > >> > > >> > I also run the test with -XComp at Mach5 linux-x64-debug > >> builds before and after the changes > >> > and for the most of the tests the overhead is about 2 times > >> although for > >> > serviceability/sa/sadebugd/SADebugDTest.java it spikes up to 5 > >> times. Probably at least for some tests > >> > it makes to filter out some properties (e.g. -Xcomp) before > >> forwarding them to j*-tools. > >> > > >> > serviceability/sa/sadebugd/SADebugDTest.java, before : 2m 23s > >> , after:11m 07s > >> > serviceability/sa/sadebugd/TestJmapCore.java, before : 42s , > >> after:1m 09s > >> > serviceability/sa/TestSysProps.java, before : 36s , after: 1m 27s > >> > > >> > > >> > [1] http://cr.openjdk.java.net/~dtitov/8242009/webrev.01 > >> > [2] https://bugs.openjdk.java.net/browse/JDK-8242009 > >> > > >> > Thank you, > >> > Daniil > >> > > >> > > >> > >> > >> > >> > > > > > > > From alexey.menkov at oracle.com Tue May 12 19:57:17 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 12 May 2020 12:57:17 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> Message-ID: Hi Alan, Serguei, lets try one more time :) What about: Agents can transform classes in arbitrary ways at load time, transform modules, or transform the bytecode of methods of already loaded classes. Developers or administrators that deploy agents, deploy applications that package an agent with the application, or use tools that load agents into a running application, are responsible for verifying the trustworthiness of each agent including the content and structure of the agent JAR file. please let me know what do you thinks, I'll prepare & publish new webrev as soon as we get agreement about the paragraph. --alex On 05/12/2020 00:59, Alan Bateman wrote: > On 11/05/2020 22:14, Alex Menkov wrote: >> >> >> Updated webrev: >> http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.2/ >> >> --alex > This doesn't work for me because it drops the important point that the > developer/admin is also responsible when deploying an agent that > packages an agent with the application. Also anyone using a tool that > loads agents into a running VM has responsibility too. So I think these > points need to be included. > > -Alan. From serguei.spitsyn at oracle.com Tue May 12 20:40:02 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 May 2020 13:40:02 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> Message-ID: <031ceb3e-90a6-ec66-bc3a-f9778b1d08d3@oracle.com> Hi Alex, This seems to resolve most of the Alan's concerns. Though, I'm not sure if we can treat users that deploy and use agents as developers. Otherwise, we may want to tweak the last sentence a little bit: ?"Developers or administrators that deploy agents, deploy applications that package an agent with the application, or anyone using a tools that loads agents into a running application, are responsible for verifying the trustworthiness of each agent including the content and structure of the agent JAR file. But let's wait for Alan's opinion. Thanks, Serguei On 5/12/20 12:57, Alex Menkov wrote: > Hi Alan, Serguei, > > lets try one more time :) > > What about: > > Agents can transform classes in arbitrary ways at load time, transform > modules, or transform the bytecode of methods of already loaded classes. > Developers or administrators that deploy agents, deploy applications that > package an agent with the application, or use tools that load agents > into a > running application, are responsible for verifying the trustworthiness > of each > agent including the content and structure of the agent JAR file. > > > please let me know what do you thinks, I'll prepare & publish new > webrev as soon as we get agreement about the paragraph. > > > --alex > > On 05/12/2020 00:59, Alan Bateman wrote: >> On 11/05/2020 22:14, Alex Menkov wrote: >>> >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.2/ >>> >>> >>> --alex >> This doesn't work for me because it drops the important point that >> the developer/admin is also responsible when deploying an agent that >> packages an agent with the application. Also anyone using a tool that >> loads agents into a running VM has responsibility too. So I think >> these points need to be included. >> >> -Alan. From david.holmes at oracle.com Wed May 13 04:13:38 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 13 May 2020 14:13:38 +1000 Subject: RFR 8244105: JDK 11.0.7 -Xlog gc issue with file path if not exist In-Reply-To: <3b81573b-bbb5-f237-d18f-6cc91ba4579b@oracle.com> References: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> <3b81573b-bbb5-f237-d18f-6cc91ba4579b@oracle.com> Message-ID: <00ce1fa5-3b22-a6ba-30b2-3399b63567e8@oracle.com> Hi Harold, On 12/05/2020 10:49 pm, Harold Seigel wrote: > Hi David, > > Thanks for looking at this. > > This change disables logging and issuing a warning for any errors in the > -Xlog arguments that currently cause the JVM to terminate. For which > particular bad -Xlog arguments do you think this is a bad idea? All of them! Unified Logging has always been fail-fast when it comes to the log configuration settings. I think that is a good design and certainly not something we should do a complete about-face on just because of one reported issue! My basic reaction to this issue now is that it is a "won't fix". Yes UL behaves differently compared to the old GC logging flags - so be it. At the very most we might consider making the inability to open the log file a non-fatal error, but even then I'm more inclined to fail-fast so that people realize they have set things up incorrectly rather than allowing mistakes to go undetected. This is also something that would need discussing at hotspot level rather than just runtime & serviceability. Cheers, David ----- > Are there some bad -Xlog arguments where you think this would be a good > thing to do? > > I agree that if we decide to do this, a CSR is needed. > > Thanks, Harold > > On 5/12/2020 3:32 AM, David Holmes wrote: >> Hi Harold, >> >> On 9/05/2020 3:22 am, Harold Seigel wrote: >>> Hi, >>> >>> Please review this fix for JDK-8244105.? The fix continues program >>> execution even when specified logging options are invalid. >>> Previously, invalid logging options terminated the program.? Now, a >>> warning is issued.? For example: >>> >>> ???? > java -Xlog:"gc*:file=/dont/exist" -version >>> ??? [0.001s][error][logging] Error opening log file '/dont/exist': No >>> ??? such file or directory >>> ??? [0.001s][error][logging] Initialization of output 'file=/dont/exist' >>> ??? using options '(null)' failed. >>> ??? Java HotSpot(TM) 64-Bit Server VM warning: Invalid -Xlog option >>> ??? '-Xlog:gc*:file=/dont/exist', see error log for details. >>> >>> ??? java version "15-internal" 2020-09-15 >>> ??? Java(TM) SE Runtime Environment (fastdebug build >>> ??? 15-internal+0-2020-05-08-1313404.hseigel.bug8244105) >>> ??? Java HotSpot(TM) 64-Bit Server VM (fastdebug build >>> ??? 15-internal+0-2020-05-08-1313404.hseigel.bug8244105, mixed mode, >>> ??? sharing) >> >> But if I am reading this correctly you are now only issuing a warning >> and disabling logging if there are any errors of any kind in the -Xlog >> arguments! That is not desirable IMO and a significant change in >> behaviour. >> >> Even just changing the behaviour in relation to a non-existent log >> file will require a CSR request. >> >> Thanks, >> David >> ----- >> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8244105/webrev/index.html >>> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8244105 >>> >>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>> and builds on Linux-x64, Solaris, Windows, and Mac OS X and by >>> running Mach5 tiers 3-5 tests on Linux-x64. >>> >>> Thanks, Harold >>> >>> From david.holmes at oracle.com Wed May 13 05:42:50 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 13 May 2020 15:42:50 +1000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> Message-ID: <36d5e2c0-c724-7ff7-d37e-decb5cc0005b@oracle.com> On 4/05/2020 8:33 pm, Reingruber, Richard wrote: > // Trimmed the list of recipients. If the list gets too long then the message needs to be approved > // by a moderator. Yes I noticed that too :) In general if you send to hotspot-dev you shouldn't need to also send to hotspot-X-dev. > Hi David, Hi Richard, >> On 28/04/2020 12:09 am, Reingruber, Richard wrote: >>> Hi David, >>> >>>> Not a review but some general commentary ... >>> >>> That's welcome. > >> Having had to take an even closer look now I have a review comment too :) > >> src/hotspot/share/prims/jvmtiThreadState.cpp > >> void JvmtiThreadState::invalidate_cur_stack_depth() { >> ! assert(SafepointSynchronize::is_at_safepoint() || >> ! (Thread::current()->is_VM_thread() && >> get_thread()->is_vmthread_processing_handshake()) || >> (JavaThread *)Thread::current() == get_thread(), >> "must be current thread or at safepoint"); > > You're looking at an outdated webrev, I'm afraid. > > This would be the post with the current webrev.1 > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html Sorry I missed that update. Okay so this is working with direct handshakes now. One style nit in jvmtiThreadState.cpp: assert(SafepointSynchronize::is_at_safepoint() || ! (JavaThread *)Thread::current() == get_thread() || ! Thread::current() == get_thread()->active_handshaker(), ! "bad synchronization with owner thread"); the ! lines should ident as follows assert(SafepointSynchronize::is_at_safepoint() || (JavaThread *)Thread::current() == get_thread() || Thread::current() == get_thread()->active_handshaker(), ! "bad synchronization with owner thread"); Lets see how this plays out. Cheers, David > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Montag, 4. Mai 2020 08:51 > To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Richard, > > On 28/04/2020 12:09 am, Reingruber, Richard wrote: >> Hi David, >> >>> Not a review but some general commentary ... >> >> That's welcome. > > Having had to take an even closer look now I have a review comment too :) > > src/hotspot/share/prims/jvmtiThreadState.cpp > > void JvmtiThreadState::invalidate_cur_stack_depth() { > ! assert(SafepointSynchronize::is_at_safepoint() || > ! (Thread::current()->is_VM_thread() && > get_thread()->is_vmthread_processing_handshake()) || > (JavaThread *)Thread::current() == get_thread(), > "must be current thread or at safepoint"); > > The message needs updating to include handshakes. > > More below ... > >>> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>>> Hi Yasumasa, Patricio, >>>> >>>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>>> Does it help you? I think it gives you to remove workaround. >>>>>> >>>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>> >>>>> Thanks for your information. >>>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>>> I will modify and will test it after yours. >>>> >>>> Thanks :) >>>> >>>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>>> to me, how this has to be handled. >>>> >>>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>> >>>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >> >>> I'm growing increasingly concerned that use of direct handshakes to >>> replace VM operations needs a much greater examination for correctness >>> than might initially be thought. I see a number of issues: >> >> I agree. I'll address your concerns in the context of this review thread for JDK-8238585 below. >> >> In addition I would suggest to take the general part of the discussion to a dedicated thread or to >> the review thread for JDK-8242427. I would like to keep this thread closer to its subject. > > I will focus on the issues in the context of this particular change > then, though the issues themselves are applicable to all handshake > situations (and more so with direct handshakes). This is mostly just > discussion. > >>> First, the VMThread executes (most) VM operations with a clean stack in >>> a clean state, so it has lots of room to work. If we now execute the >>> same logic in a JavaThread then we risk hitting stackoverflows if >>> nothing else. But we are also now executing code in a JavaThread and so >>> we have to be sure that code is not going to act differently (in a bad >>> way) if executed by a JavaThread rather than the VMThread. For example, >>> may it be possible that if executing in the VMThread we defer some >>> activity that might require execution of Java code, or else hand it off >>> to one of the service threads? If we execute that code directly in the >>> current JavaThread instead we may not be in a valid state (e.g. consider >>> re-entrancy to various subsystems that is not allowed). >> >> It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is doing. I already added a >> paragraph to the JBS-Item [1] explaining why the direct handshake is sufficient from a >> synchronization point of view. > > Just to be clear, your proposed change is not using a direct handshake. > >> Furthermore the stack is walked and the return pc of compiled frames is replaced with the address of >> the deopt handler. >> >> I can't see why this cannot be done with a direct handshake. Something very similar is already done >> in JavaThread::deoptimize_marked_methods() which is executed as part of an ordinary handshake. > > Note that existing non-direct handshakes may also have issues that not > have been fully investigated. > >> The demand on stack-space should be very modest. I would not expect a higher risk for stackoverflow. > > For the target thread if you use more stack than would be used stopping > at a safepoint then you are at risk. For the thread initiating the > direct handshake if you use more stack than would be used enqueuing a VM > operation, then you are at risk. As we have not quantified these > numbers, nor have any easy way to establish the stack use of the actual > code to be executed, we're really just hoping for the best. This is a > general problem with handshakes that needs to be investigated more > deeply. As a simple, general, example just imagine if the code involves > logging that might utilise an on-stack buffer. > >>> Second, we have this question mark over what happens if the operation >>> hits further safepoint or handshake polls/checks? Are there constraints >>> on what is allowed here? How can we recognise this problem may exist and >>> so deal with it? >> >> The thread in EnterInterpOnlyModeClosure::do_thread() can't become safepoint/handshake safe. I >> tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a NoSafepointVerifier. > > That's good to hear but such tests are not exhaustive, they will detect > if you do reach a safepoint/handshake but they can't prove that you > cannot reach one. What you have done is necessary but may not be > sufficient. Plus you didn't actually add the NSV to the code - is there > a reason we can't actually keep it in do_thread? (I'm not sure if the > NSV also acts as a NoHandshakeVerifier?) > >>> Third, while we are generally considering what appear to be >>> single-thread operations, which should be amenable to a direct >>> handshake, we also have to be careful that some of the code involved >>> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >>> not need to take a lock where a direct handshake might! >> >> See again my arguments in the JBS item [1]. > > Yes I see the reasoning and that is good. My point is a general one as > it may not be obvious when such assumptions exist in the current code. > > Thanks, > David > >> Thanks, >> Richard. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8238585 >> >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 27. April 2020 07:16 >> To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi all, >> >> Not a review but some general commentary ... >> >> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>> Hi Yasumasa, Patricio, >>> >>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>> >>>> Thanks for your information. >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>> I will modify and will test it after yours. >>> >>> Thanks :) >>> >>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>> to me, how this has to be handled. >>> >>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>> >>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >> >> I'm growing increasingly concerned that use of direct handshakes to >> replace VM operations needs a much greater examination for correctness >> than might initially be thought. I see a number of issues: >> >> First, the VMThread executes (most) VM operations with a clean stack in >> a clean state, so it has lots of room to work. If we now execute the >> same logic in a JavaThread then we risk hitting stackoverflows if >> nothing else. But we are also now executing code in a JavaThread and so >> we have to be sure that code is not going to act differently (in a bad >> way) if executed by a JavaThread rather than the VMThread. For example, >> may it be possible that if executing in the VMThread we defer some >> activity that might require execution of Java code, or else hand it off >> to one of the service threads? If we execute that code directly in the >> current JavaThread instead we may not be in a valid state (e.g. consider >> re-entrancy to various subsystems that is not allowed). >> >> Second, we have this question mark over what happens if the operation >> hits further safepoint or handshake polls/checks? Are there constraints >> on what is allowed here? How can we recognise this problem may exist and >> so deal with it? >> >> Third, while we are generally considering what appear to be >> single-thread operations, which should be amenable to a direct >> handshake, we also have to be careful that some of the code involved >> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >> not need to take a lock where a direct handshake might! >> >> Cheers, >> David >> ----- >> >>> @Patricio, coming back to my question [1]: >>> >>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a >>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads >>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The >>> handshakee would be safepoint safe, wouldn't it? >>> >>> Thanks, Richard. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677 >>> >>> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763 >>> >>> -----Original Message----- >>> From: Yasumasa Suenaga >>> Sent: Freitag, 24. April 2020 17:23 >>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>> >>> Hi Richard, >>> >>> On 2020/04/24 23:44, Reingruber, Richard wrote: >>>> Hi Yasumasa, >>>> >>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>> Does it help you? I think it gives you to remove workaround. >>>> >>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>> >>> Thanks for your information. >>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>> I will modify and will test it after yours. >>> >>> >>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>> to me, how this has to be handled. >>> >>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585). >>>> >>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>> >>>> Would be interesting to see how you handled the issues above :) >>>> >>>> Thanks, Richard. >>>> >>>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030 >>>> >>>> -----Original Message----- >>>> From: Yasumasa Suenaga >>>> Sent: Freitag, 24. April 2020 13:34 >>>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Richard, >>>> >>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>> Does it help you? I think it gives you to remove workaround. >>>> >>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/04/24 17:18, Reingruber, Richard wrote: >>>>> Hi Patricio, Vladimir, and Serguei, >>>>> >>>>> now that direct handshakes are available, I've updated the patch to make use of them. >>>>> >>>>> In addition I have done some clean-up changes I missed in the first webrev. >>>>> >>>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake >>>>> into the vm operation VM_SetFramePop [1] >>>>> >>>>> Kindly review again: >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ >>>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ >>>>> >>>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a >>>>> direct handshake: >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>> >>>>> Testing: >>>>> >>>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>> >>>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 >>>>> >>>>> Thanks, >>>>> Richard. >>>>> >>>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-dev On Behalf Of Reingruber, Richard >>>>> Sent: Freitag, 14. Februar 2020 19:47 >>>>> To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Patricio, >>>>> >>>>> > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>> > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>> > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>> > > >>>>> > > > Alternatively I think you could do something similar to what we do in >>>>> > > > Deoptimization::deoptimize_all_marked(): >>>>> > > > >>>>> > > > EnterInterpOnlyModeClosure hs; >>>>> > > > if (SafepointSynchronize::is_at_safepoint()) { >>>>> > > > hs.do_thread(state->get_thread()); >>>>> > > > } else { >>>>> > > > Handshake::execute(&hs, state->get_thread()); >>>>> > > > } >>>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>> > > > HandshakeClosure() constructor) >>>>> > > >>>>> > > Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>> > Right, we could also do that. Avoiding to clear the polling page in >>>>> > HandshakeState::clear_handshake() should be enough to fix this issue and >>>>> > execute a handshake inside a safepoint, but adding that "if" statement >>>>> > in Hanshake::execute() sounds good to avoid all the extra code that we >>>>> > go through when executing a handshake. I filed 8239084 to make that change. >>>>> >>>>> Thanks for taking care of this and creating the RFE. >>>>> >>>>> > >>>>> > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>> > > > always called in a nested operation or just sometimes. >>>>> > > >>>>> > > At least one execution path without vm operation exists: >>>>> > > >>>>> > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>> > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void >>>>> > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>> > > >>>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>> > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>> > > encouraged to do it with a handshake :) >>>>> > Ah! I think you can still do it with a handshake with the >>>>> > Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>> > if-else statement with just the Handshake::execute() call in 8239084. >>>>> > But up to you. : ) >>>>> >>>>> Well, I think that's enough encouragement :) >>>>> I'll wait for 8239084 and try then again. >>>>> (no urgency and all) >>>>> >>>>> Thanks, >>>>> Richard. >>>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Freitag, 14. Februar 2020 15:54 >>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Richard, >>>>> >>>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: >>>>>> Hi Patricio, >>>>>> >>>>>> thanks for having a look. >>>>>> >>>>>> > I?m only commenting on the handshake changes. >>>>>> > I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>> > operation VM_SetFramePop which also allows nested operations. Here is a >>>>>> > comment in VM_SetFramePop definition: >>>>>> > >>>>>> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>> > >>>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>> > could have a handshake inside a safepoint operation. The issue I see >>>>>> > there is that at the end of the handshake the polling page of the target >>>>>> > thread could be disarmed. So if the target thread happens to be in a >>>>>> > blocked state just transiently and wakes up then it will not stop for >>>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>> > polling page is armed at the beginning of disarm_safepoint(). >>>>>> >>>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>>> >>>>>> > Alternatively I think you could do something similar to what we do in >>>>>> > Deoptimization::deoptimize_all_marked(): >>>>>> > >>>>>> > EnterInterpOnlyModeClosure hs; >>>>>> > if (SafepointSynchronize::is_at_safepoint()) { >>>>>> > hs.do_thread(state->get_thread()); >>>>>> > } else { >>>>>> > Handshake::execute(&hs, state->get_thread()); >>>>>> > } >>>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>> > HandshakeClosure() constructor) >>>>>> >>>>>> Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>> Right, we could also do that. Avoiding to clear the polling page in >>>>> HandshakeState::clear_handshake() should be enough to fix this issue and >>>>> execute a handshake inside a safepoint, but adding that "if" statement >>>>> in Hanshake::execute() sounds good to avoid all the extra code that we >>>>> go through when executing a handshake. I filed 8239084 to make that change. >>>>> >>>>>> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>> > always called in a nested operation or just sometimes. >>>>>> >>>>>> At least one execution path without vm operation exists: >>>>>> >>>>>> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>>> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>>> JvmtiEventControllerPrivate::recompute_enabled() : void >>>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>>> >>>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>>> encouraged to do it with a handshake :) >>>>> Ah! I think you can still do it with a handshake with the >>>>> Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>> if-else statement with just the Handshake::execute() call in 8239084. >>>>> But up to you.? : ) >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Thanks again, >>>>>> Richard. >>>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Donnerstag, 13. Februar 2020 18:47 >>>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>> >>>>>> Hi Richard, >>>>>> >>>>>> I?m only commenting on the handshake changes. >>>>>> I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>> operation VM_SetFramePop which also allows nested operations. Here is a >>>>>> comment in VM_SetFramePop definition: >>>>>> >>>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>> >>>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>> could have a handshake inside a safepoint operation. The issue I see >>>>>> there is that at the end of the handshake the polling page of the target >>>>>> thread could be disarmed. So if the target thread happens to be in a >>>>>> blocked state just transiently and wakes up then it will not stop for >>>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>> polling page is armed at the beginning of disarm_safepoint(). >>>>>> >>>>>> I think one option could be to remove >>>>>> SafepointMechanism::disarm_if_needed() in >>>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself >>>>>> for the handshake case. >>>>>> >>>>>> Alternatively I think you could do something similar to what we do in >>>>>> Deoptimization::deoptimize_all_marked(): >>>>>> >>>>>> ? EnterInterpOnlyModeClosure hs; >>>>>> ? if (SafepointSynchronize::is_at_safepoint()) { >>>>>> ??? hs.do_thread(state->get_thread()); >>>>>> ? } else { >>>>>> ??? Handshake::execute(&hs, state->get_thread()); >>>>>> ? } >>>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>> HandshakeClosure() constructor) >>>>>> >>>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>> always called in a nested operation or just sometimes. >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>> >>>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>>>>>> // Repost including hotspot runtime and gc lists. >>>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>>>>>> // with a handshake. >>>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>>> >>>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>>>>>> >>>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>>>>>> >>>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>>> >>>>>>> Thanks, Richard. >>>>>>> >>>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html >>>>> From harold.seigel at oracle.com Wed May 13 12:50:48 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 13 May 2020 08:50:48 -0400 Subject: RFR 8244105: JDK 11.0.7 -Xlog gc issue with file path if not exist In-Reply-To: <00ce1fa5-3b22-a6ba-30b2-3399b63567e8@oracle.com> References: <7115570f-e71e-df62-12c1-626fdf11bd26@oracle.com> <3b81573b-bbb5-f237-d18f-6cc91ba4579b@oracle.com> <00ce1fa5-3b22-a6ba-30b2-3399b63567e8@oracle.com> Message-ID: Thanks David. I closed the issue as 'will not fix'. Harold On 5/13/2020 12:13 AM, David Holmes wrote: > Hi Harold, > > On 12/05/2020 10:49 pm, Harold Seigel wrote: >> Hi David, >> >> Thanks for looking at this. >> >> This change disables logging and issuing a warning for any errors in >> the -Xlog arguments that currently cause the JVM to terminate. For >> which particular bad -Xlog arguments do you think this is a bad idea? > > All of them! Unified Logging has always been fail-fast when it comes > to the log configuration settings. I think that is a good design and > certainly not something we should do a complete about-face on just > because of one reported issue! > > My basic reaction to this issue now is that it is a "won't fix". Yes > UL behaves differently compared to the old GC logging flags - so be it. > > At the very most we might consider making the inability to open the > log file a non-fatal error, but even then I'm more inclined to > fail-fast so that people realize they have set things up incorrectly > rather than allowing mistakes to go undetected. > > This is also something that would need discussing at hotspot level > rather than just runtime & serviceability. > > Cheers, > David > ----- > > >> Are there some bad -Xlog arguments where you think this would be a >> good thing to do? >> >> I agree that if we decide to do this, a CSR is needed. >> >> Thanks, Harold >> >> On 5/12/2020 3:32 AM, David Holmes wrote: >>> Hi Harold, >>> >>> On 9/05/2020 3:22 am, Harold Seigel wrote: >>>> Hi, >>>> >>>> Please review this fix for JDK-8244105.? The fix continues program >>>> execution even when specified logging options are invalid. >>>> Previously, invalid logging options terminated the program.? Now, a >>>> warning is issued.? For example: >>>> >>>> ???? > java -Xlog:"gc*:file=/dont/exist" -version >>>> ??? [0.001s][error][logging] Error opening log file '/dont/exist': No >>>> ??? such file or directory >>>> ??? [0.001s][error][logging] Initialization of output >>>> 'file=/dont/exist' >>>> ??? using options '(null)' failed. >>>> ??? Java HotSpot(TM) 64-Bit Server VM warning: Invalid -Xlog option >>>> ??? '-Xlog:gc*:file=/dont/exist', see error log for details. >>>> >>>> ??? java version "15-internal" 2020-09-15 >>>> ??? Java(TM) SE Runtime Environment (fastdebug build >>>> ??? 15-internal+0-2020-05-08-1313404.hseigel.bug8244105) >>>> ??? Java HotSpot(TM) 64-Bit Server VM (fastdebug build >>>> ??? 15-internal+0-2020-05-08-1313404.hseigel.bug8244105, mixed mode, >>>> ??? sharing) >>> >>> But if I am reading this correctly you are now only issuing a >>> warning and disabling logging if there are any errors of any kind in >>> the -Xlog arguments! That is not desirable IMO and a significant >>> change in behaviour. >>> >>> Even just changing the behaviour in relation to a non-existent log >>> file will require a CSR request. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8244105/webrev/index.html >>>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8244105 >>>> >>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>>> and builds on Linux-x64, Solaris, Windows, and Mac OS X and by >>>> running Mach5 tiers 3-5 tests on Linux-x64. >>>> >>>> Thanks, Harold >>>> >>>> From richard.reingruber at sap.com Wed May 13 15:37:59 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 13 May 2020 15:37:59 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: <36d5e2c0-c724-7ff7-d37e-decb5cc0005b@oracle.com> References: <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> <36d5e2c0-c724-7ff7-d37e-decb5cc0005b@oracle.com> Message-ID: Hi David, > On 4/05/2020 8:33 pm, Reingruber, Richard wrote: > > // Trimmed the list of recipients. If the list gets too long then the message needs to be approved > > // by a moderator. > Yes I noticed that too :) In general if you send to hotspot-dev you > shouldn't need to also send to hotspot-X-dev. Makes sense. Will do so next time. > > > > This would be the post with the current webrev.1 > > > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html > Sorry I missed that update. Okay so this is working with direct > handshakes now. > One style nit in jvmtiThreadState.cpp: > assert(SafepointSynchronize::is_at_safepoint() || > ! (JavaThread *)Thread::current() == get_thread() || > ! Thread::current() == get_thread()->active_handshaker(), > ! "bad synchronization with owner thread"); > the ! lines should ident as follows > assert(SafepointSynchronize::is_at_safepoint() || > (JavaThread *)Thread::current() == get_thread() || > Thread::current() == get_thread()->active_handshaker(), > ! "bad synchronization with owner thread"); Sure. > Lets see how this plays out. Hopefully not too bad... :) >> Not a review but some general commentary ... Still not a review, or is it now? Thanks, Richard. -----Original Message----- From: David Holmes Sent: Mittwoch, 13. Mai 2020 07:43 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant On 4/05/2020 8:33 pm, Reingruber, Richard wrote: > // Trimmed the list of recipients. If the list gets too long then the message needs to be approved > // by a moderator. Yes I noticed that too :) In general if you send to hotspot-dev you shouldn't need to also send to hotspot-X-dev. > Hi David, Hi Richard, >> On 28/04/2020 12:09 am, Reingruber, Richard wrote: >>> Hi David, >>> >>>> Not a review but some general commentary ... >>> >>> That's welcome. > >> Having had to take an even closer look now I have a review comment too :) > >> src/hotspot/share/prims/jvmtiThreadState.cpp > >> void JvmtiThreadState::invalidate_cur_stack_depth() { >> ! assert(SafepointSynchronize::is_at_safepoint() || >> ! (Thread::current()->is_VM_thread() && >> get_thread()->is_vmthread_processing_handshake()) || >> (JavaThread *)Thread::current() == get_thread(), >> "must be current thread or at safepoint"); > > You're looking at an outdated webrev, I'm afraid. > > This would be the post with the current webrev.1 > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html Sorry I missed that update. Okay so this is working with direct handshakes now. One style nit in jvmtiThreadState.cpp: assert(SafepointSynchronize::is_at_safepoint() || ! (JavaThread *)Thread::current() == get_thread() || ! Thread::current() == get_thread()->active_handshaker(), ! "bad synchronization with owner thread"); the ! lines should ident as follows assert(SafepointSynchronize::is_at_safepoint() || (JavaThread *)Thread::current() == get_thread() || Thread::current() == get_thread()->active_handshaker(), ! "bad synchronization with owner thread"); Lets see how this plays out. Cheers, David > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Montag, 4. Mai 2020 08:51 > To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Richard, > > On 28/04/2020 12:09 am, Reingruber, Richard wrote: >> Hi David, >> >>> Not a review but some general commentary ... >> >> That's welcome. > > Having had to take an even closer look now I have a review comment too :) > > src/hotspot/share/prims/jvmtiThreadState.cpp > > void JvmtiThreadState::invalidate_cur_stack_depth() { > ! assert(SafepointSynchronize::is_at_safepoint() || > ! (Thread::current()->is_VM_thread() && > get_thread()->is_vmthread_processing_handshake()) || > (JavaThread *)Thread::current() == get_thread(), > "must be current thread or at safepoint"); > > The message needs updating to include handshakes. > > More below ... > >>> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>>> Hi Yasumasa, Patricio, >>>> >>>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>>> Does it help you? I think it gives you to remove workaround. >>>>>> >>>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>> >>>>> Thanks for your information. >>>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>>> I will modify and will test it after yours. >>>> >>>> Thanks :) >>>> >>>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>>> to me, how this has to be handled. >>>> >>>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>> >>>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >> >>> I'm growing increasingly concerned that use of direct handshakes to >>> replace VM operations needs a much greater examination for correctness >>> than might initially be thought. I see a number of issues: >> >> I agree. I'll address your concerns in the context of this review thread for JDK-8238585 below. >> >> In addition I would suggest to take the general part of the discussion to a dedicated thread or to >> the review thread for JDK-8242427. I would like to keep this thread closer to its subject. > > I will focus on the issues in the context of this particular change > then, though the issues themselves are applicable to all handshake > situations (and more so with direct handshakes). This is mostly just > discussion. > >>> First, the VMThread executes (most) VM operations with a clean stack in >>> a clean state, so it has lots of room to work. If we now execute the >>> same logic in a JavaThread then we risk hitting stackoverflows if >>> nothing else. But we are also now executing code in a JavaThread and so >>> we have to be sure that code is not going to act differently (in a bad >>> way) if executed by a JavaThread rather than the VMThread. For example, >>> may it be possible that if executing in the VMThread we defer some >>> activity that might require execution of Java code, or else hand it off >>> to one of the service threads? If we execute that code directly in the >>> current JavaThread instead we may not be in a valid state (e.g. consider >>> re-entrancy to various subsystems that is not allowed). >> >> It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is doing. I already added a >> paragraph to the JBS-Item [1] explaining why the direct handshake is sufficient from a >> synchronization point of view. > > Just to be clear, your proposed change is not using a direct handshake. > >> Furthermore the stack is walked and the return pc of compiled frames is replaced with the address of >> the deopt handler. >> >> I can't see why this cannot be done with a direct handshake. Something very similar is already done >> in JavaThread::deoptimize_marked_methods() which is executed as part of an ordinary handshake. > > Note that existing non-direct handshakes may also have issues that not > have been fully investigated. > >> The demand on stack-space should be very modest. I would not expect a higher risk for stackoverflow. > > For the target thread if you use more stack than would be used stopping > at a safepoint then you are at risk. For the thread initiating the > direct handshake if you use more stack than would be used enqueuing a VM > operation, then you are at risk. As we have not quantified these > numbers, nor have any easy way to establish the stack use of the actual > code to be executed, we're really just hoping for the best. This is a > general problem with handshakes that needs to be investigated more > deeply. As a simple, general, example just imagine if the code involves > logging that might utilise an on-stack buffer. > >>> Second, we have this question mark over what happens if the operation >>> hits further safepoint or handshake polls/checks? Are there constraints >>> on what is allowed here? How can we recognise this problem may exist and >>> so deal with it? >> >> The thread in EnterInterpOnlyModeClosure::do_thread() can't become safepoint/handshake safe. I >> tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a NoSafepointVerifier. > > That's good to hear but such tests are not exhaustive, they will detect > if you do reach a safepoint/handshake but they can't prove that you > cannot reach one. What you have done is necessary but may not be > sufficient. Plus you didn't actually add the NSV to the code - is there > a reason we can't actually keep it in do_thread? (I'm not sure if the > NSV also acts as a NoHandshakeVerifier?) > >>> Third, while we are generally considering what appear to be >>> single-thread operations, which should be amenable to a direct >>> handshake, we also have to be careful that some of the code involved >>> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >>> not need to take a lock where a direct handshake might! >> >> See again my arguments in the JBS item [1]. > > Yes I see the reasoning and that is good. My point is a general one as > it may not be obvious when such assumptions exist in the current code. > > Thanks, > David > >> Thanks, >> Richard. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8238585 >> >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 27. April 2020 07:16 >> To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi all, >> >> Not a review but some general commentary ... >> >> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>> Hi Yasumasa, Patricio, >>> >>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>> >>>> Thanks for your information. >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>> I will modify and will test it after yours. >>> >>> Thanks :) >>> >>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>> to me, how this has to be handled. >>> >>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>> >>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >> >> I'm growing increasingly concerned that use of direct handshakes to >> replace VM operations needs a much greater examination for correctness >> than might initially be thought. I see a number of issues: >> >> First, the VMThread executes (most) VM operations with a clean stack in >> a clean state, so it has lots of room to work. If we now execute the >> same logic in a JavaThread then we risk hitting stackoverflows if >> nothing else. But we are also now executing code in a JavaThread and so >> we have to be sure that code is not going to act differently (in a bad >> way) if executed by a JavaThread rather than the VMThread. For example, >> may it be possible that if executing in the VMThread we defer some >> activity that might require execution of Java code, or else hand it off >> to one of the service threads? If we execute that code directly in the >> current JavaThread instead we may not be in a valid state (e.g. consider >> re-entrancy to various subsystems that is not allowed). >> >> Second, we have this question mark over what happens if the operation >> hits further safepoint or handshake polls/checks? Are there constraints >> on what is allowed here? How can we recognise this problem may exist and >> so deal with it? >> >> Third, while we are generally considering what appear to be >> single-thread operations, which should be amenable to a direct >> handshake, we also have to be careful that some of the code involved >> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >> not need to take a lock where a direct handshake might! >> >> Cheers, >> David >> ----- >> >>> @Patricio, coming back to my question [1]: >>> >>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a >>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads >>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The >>> handshakee would be safepoint safe, wouldn't it? >>> >>> Thanks, Richard. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677 >>> >>> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763 >>> >>> -----Original Message----- >>> From: Yasumasa Suenaga >>> Sent: Freitag, 24. April 2020 17:23 >>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>> >>> Hi Richard, >>> >>> On 2020/04/24 23:44, Reingruber, Richard wrote: >>>> Hi Yasumasa, >>>> >>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>> Does it help you? I think it gives you to remove workaround. >>>> >>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>> >>> Thanks for your information. >>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>> I will modify and will test it after yours. >>> >>> >>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>> to me, how this has to be handled. >>> >>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585). >>>> >>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>> >>>> Would be interesting to see how you handled the issues above :) >>>> >>>> Thanks, Richard. >>>> >>>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030 >>>> >>>> -----Original Message----- >>>> From: Yasumasa Suenaga >>>> Sent: Freitag, 24. April 2020 13:34 >>>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Richard, >>>> >>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>> Does it help you? I think it gives you to remove workaround. >>>> >>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/04/24 17:18, Reingruber, Richard wrote: >>>>> Hi Patricio, Vladimir, and Serguei, >>>>> >>>>> now that direct handshakes are available, I've updated the patch to make use of them. >>>>> >>>>> In addition I have done some clean-up changes I missed in the first webrev. >>>>> >>>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake >>>>> into the vm operation VM_SetFramePop [1] >>>>> >>>>> Kindly review again: >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ >>>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ >>>>> >>>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a >>>>> direct handshake: >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>> >>>>> Testing: >>>>> >>>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>> >>>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 >>>>> >>>>> Thanks, >>>>> Richard. >>>>> >>>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-dev On Behalf Of Reingruber, Richard >>>>> Sent: Freitag, 14. Februar 2020 19:47 >>>>> To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Patricio, >>>>> >>>>> > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>> > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>> > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>> > > >>>>> > > > Alternatively I think you could do something similar to what we do in >>>>> > > > Deoptimization::deoptimize_all_marked(): >>>>> > > > >>>>> > > > EnterInterpOnlyModeClosure hs; >>>>> > > > if (SafepointSynchronize::is_at_safepoint()) { >>>>> > > > hs.do_thread(state->get_thread()); >>>>> > > > } else { >>>>> > > > Handshake::execute(&hs, state->get_thread()); >>>>> > > > } >>>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>> > > > HandshakeClosure() constructor) >>>>> > > >>>>> > > Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>> > Right, we could also do that. Avoiding to clear the polling page in >>>>> > HandshakeState::clear_handshake() should be enough to fix this issue and >>>>> > execute a handshake inside a safepoint, but adding that "if" statement >>>>> > in Hanshake::execute() sounds good to avoid all the extra code that we >>>>> > go through when executing a handshake. I filed 8239084 to make that change. >>>>> >>>>> Thanks for taking care of this and creating the RFE. >>>>> >>>>> > >>>>> > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>> > > > always called in a nested operation or just sometimes. >>>>> > > >>>>> > > At least one execution path without vm operation exists: >>>>> > > >>>>> > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>> > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void >>>>> > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>> > > >>>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>> > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>> > > encouraged to do it with a handshake :) >>>>> > Ah! I think you can still do it with a handshake with the >>>>> > Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>> > if-else statement with just the Handshake::execute() call in 8239084. >>>>> > But up to you. : ) >>>>> >>>>> Well, I think that's enough encouragement :) >>>>> I'll wait for 8239084 and try then again. >>>>> (no urgency and all) >>>>> >>>>> Thanks, >>>>> Richard. >>>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Freitag, 14. Februar 2020 15:54 >>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Richard, >>>>> >>>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: >>>>>> Hi Patricio, >>>>>> >>>>>> thanks for having a look. >>>>>> >>>>>> > I?m only commenting on the handshake changes. >>>>>> > I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>> > operation VM_SetFramePop which also allows nested operations. Here is a >>>>>> > comment in VM_SetFramePop definition: >>>>>> > >>>>>> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>> > >>>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>> > could have a handshake inside a safepoint operation. The issue I see >>>>>> > there is that at the end of the handshake the polling page of the target >>>>>> > thread could be disarmed. So if the target thread happens to be in a >>>>>> > blocked state just transiently and wakes up then it will not stop for >>>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>> > polling page is armed at the beginning of disarm_safepoint(). >>>>>> >>>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>>> >>>>>> > Alternatively I think you could do something similar to what we do in >>>>>> > Deoptimization::deoptimize_all_marked(): >>>>>> > >>>>>> > EnterInterpOnlyModeClosure hs; >>>>>> > if (SafepointSynchronize::is_at_safepoint()) { >>>>>> > hs.do_thread(state->get_thread()); >>>>>> > } else { >>>>>> > Handshake::execute(&hs, state->get_thread()); >>>>>> > } >>>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>> > HandshakeClosure() constructor) >>>>>> >>>>>> Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>> Right, we could also do that. Avoiding to clear the polling page in >>>>> HandshakeState::clear_handshake() should be enough to fix this issue and >>>>> execute a handshake inside a safepoint, but adding that "if" statement >>>>> in Hanshake::execute() sounds good to avoid all the extra code that we >>>>> go through when executing a handshake. I filed 8239084 to make that change. >>>>> >>>>>> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>> > always called in a nested operation or just sometimes. >>>>>> >>>>>> At least one execution path without vm operation exists: >>>>>> >>>>>> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>>> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>>> JvmtiEventControllerPrivate::recompute_enabled() : void >>>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>>> >>>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>>> encouraged to do it with a handshake :) >>>>> Ah! I think you can still do it with a handshake with the >>>>> Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>> if-else statement with just the Handshake::execute() call in 8239084. >>>>> But up to you.? : ) >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Thanks again, >>>>>> Richard. >>>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Donnerstag, 13. Februar 2020 18:47 >>>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>> >>>>>> Hi Richard, >>>>>> >>>>>> I?m only commenting on the handshake changes. >>>>>> I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>> operation VM_SetFramePop which also allows nested operations. Here is a >>>>>> comment in VM_SetFramePop definition: >>>>>> >>>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>> >>>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>> could have a handshake inside a safepoint operation. The issue I see >>>>>> there is that at the end of the handshake the polling page of the target >>>>>> thread could be disarmed. So if the target thread happens to be in a >>>>>> blocked state just transiently and wakes up then it will not stop for >>>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>> polling page is armed at the beginning of disarm_safepoint(). >>>>>> >>>>>> I think one option could be to remove >>>>>> SafepointMechanism::disarm_if_needed() in >>>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself >>>>>> for the handshake case. >>>>>> >>>>>> Alternatively I think you could do something similar to what we do in >>>>>> Deoptimization::deoptimize_all_marked(): >>>>>> >>>>>> ? EnterInterpOnlyModeClosure hs; >>>>>> ? if (SafepointSynchronize::is_at_safepoint()) { >>>>>> ??? hs.do_thread(state->get_thread()); >>>>>> ? } else { >>>>>> ??? Handshake::execute(&hs, state->get_thread()); >>>>>> ? } >>>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>> HandshakeClosure() constructor) >>>>>> >>>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>> always called in a nested operation or just sometimes. >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>> >>>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>>>>>> // Repost including hotspot runtime and gc lists. >>>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>>>>>> // with a handshake. >>>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>>> >>>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>>>>>> >>>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>>>>>> >>>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>>> >>>>>>> Thanks, Richard. >>>>>>> >>>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html >>>>> From alexey.menkov at oracle.com Wed May 13 19:30:50 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 13 May 2020 12:30:50 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <031ceb3e-90a6-ec66-bc3a-f9778b1d08d3@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> <031ceb3e-90a6-ec66-bc3a-f9778b1d08d3@oracle.com> Message-ID: On 05/12/2020 13:40, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > This seems to resolve most of the Alan's concerns. > Though, I'm not sure if we can treat users that deploy and use agents as > developers. I think users that deploy agent or use tools to load agents can be called administrators :) --alex > > Otherwise, we may want to tweak the last sentence a little bit: > ?"Developers or administrators that deploy agents, deploy applications > that > package an agent with the application, or anyone using a tools that > loads agents into a > running application, are responsible for verifying the trustworthiness > of each > agent including the content and structure of the agent JAR file. > > > But let's wait for Alan's opinion. > > Thanks, > Serguei > > > On 5/12/20 12:57, Alex Menkov wrote: >> Hi Alan, Serguei, >> >> lets try one more time :) >> >> What about: >> >> Agents can transform classes in arbitrary ways at load time, transform >> modules, or transform the bytecode of methods of already loaded classes. >> Developers or administrators that deploy agents, deploy applications that >> package an agent with the application, or use tools that load agents >> into a >> running application, are responsible for verifying the trustworthiness >> of each >> agent including the content and structure of the agent JAR file. >> >> >> please let me know what do you thinks, I'll prepare & publish new >> webrev as soon as we get agreement about the paragraph. >> >> >> --alex >> >> On 05/12/2020 00:59, Alan Bateman wrote: >>> On 11/05/2020 22:14, Alex Menkov wrote: >>>> >>>> >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.2/ >>>> >>>> >>>> --alex >>> This doesn't work for me because it drops the important point that >>> the developer/admin is also responsible when deploying an agent that >>> packages an agent with the application. Also anyone using a tool that >>> loads agents into a running VM has responsibility too. So I think >>> these points need to be included. >>> >>> -Alan. > From serguei.spitsyn at oracle.com Wed May 13 22:31:47 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 13 May 2020 15:31:47 -0700 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> Message-ID: <03c9a0ce-8f78-00e7-9db3-70d6f6cb8156@oracle.com> Hi Richard, Thank you for the bug report update - it is helpful. The fix/update looks good in general but I need more time to check some points. I'm thinking it would be more safe to run full tier5. I can do it after you get all thumbs ups. Thanks, Serguei On 4/24/20 01:18, Reingruber, Richard wrote: > Hi Patricio, Vladimir, and Serguei, > > now that direct handshakes are available, I've updated the patch to make use of them. > > In addition I have done some clean-up changes I missed in the first webrev. > > Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake > into the vm operation VM_SetFramePop [1] > > Kindly review again: > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ > Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ > > I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a > direct handshake: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 > > Testing: > > * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 > > Thanks, > Richard. > > [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. > > -----Original Message----- > From: hotspot-dev On Behalf Of Reingruber, Richard > Sent: Freitag, 14. Februar 2020 19:47 > To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Patricio, > > > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a > > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the > > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? > > > > > > > Alternatively I think you could do something similar to what we do in > > > > Deoptimization::deoptimize_all_marked(): > > > > > > > > EnterInterpOnlyModeClosure hs; > > > > if (SafepointSynchronize::is_at_safepoint()) { > > > > hs.do_thread(state->get_thread()); > > > > } else { > > > > Handshake::execute(&hs, state->get_thread()); > > > > } > > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the > > > > HandshakeClosure() constructor) > > > > > > Maybe this could be used also in the Handshake::execute() methods as general solution? > > Right, we could also do that. Avoiding to clear the polling page in > > HandshakeState::clear_handshake() should be enough to fix this issue and > > execute a handshake inside a safepoint, but adding that "if" statement > > in Hanshake::execute() sounds good to avoid all the extra code that we > > go through when executing a handshake. I filed 8239084 to make that change. > > Thanks for taking care of this and creating the RFE. > > > > > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > > > > always called in a nested operation or just sometimes. > > > > > > At least one execution path without vm operation exists: > > > > > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void > > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong > > > JvmtiEventControllerPrivate::recompute_enabled() : void > > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) > > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void > > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError > > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError > > > > > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a > > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further > > > encouraged to do it with a handshake :) > > Ah! I think you can still do it with a handshake with the > > Deoptimization::deoptimize_all_marked() like solution. I can change the > > if-else statement with just the Handshake::execute() call in 8239084. > > But up to you. : ) > > Well, I think that's enough encouragement :) > I'll wait for 8239084 and try then again. > (no urgency and all) > > Thanks, > Richard. > > -----Original Message----- > From: Patricio Chilano > Sent: Freitag, 14. Februar 2020 15:54 > To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Richard, > > On 2/14/20 9:58 AM, Reingruber, Richard wrote: >> Hi Patricio, >> >> thanks for having a look. >> >> > I?m only commenting on the handshake changes. >> > I see that operation VM_EnterInterpOnlyMode can be called inside >> > operation VM_SetFramePop which also allows nested operations. Here is a >> > comment in VM_SetFramePop definition: >> > >> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >> > >> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >> > could have a handshake inside a safepoint operation. The issue I see >> > there is that at the end of the handshake the polling page of the target >> > thread could be disarmed. So if the target thread happens to be in a >> > blocked state just transiently and wakes up then it will not stop for >> > the ongoing safepoint. Maybe I can file an RFE to assert that the >> > polling page is armed at the beginning of disarm_safepoint(). >> >> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >> >> > Alternatively I think you could do something similar to what we do in >> > Deoptimization::deoptimize_all_marked(): >> > >> > EnterInterpOnlyModeClosure hs; >> > if (SafepointSynchronize::is_at_safepoint()) { >> > hs.do_thread(state->get_thread()); >> > } else { >> > Handshake::execute(&hs, state->get_thread()); >> > } >> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >> > HandshakeClosure() constructor) >> >> Maybe this could be used also in the Handshake::execute() methods as general solution? > Right, we could also do that. Avoiding to clear the polling page in > HandshakeState::clear_handshake() should be enough to fix this issue and > execute a handshake inside a safepoint, but adding that "if" statement > in Hanshake::execute() sounds good to avoid all the extra code that we > go through when executing a handshake. I filed 8239084 to make that change. > >> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >> > always called in a nested operation or just sometimes. >> >> At least one execution path without vm operation exists: >> >> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >> JvmtiEventControllerPrivate::recompute_enabled() : void >> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >> >> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >> encouraged to do it with a handshake :) > Ah! I think you can still do it with a handshake with the > Deoptimization::deoptimize_all_marked() like solution. I can change the > if-else statement with just the Handshake::execute() call in 8239084. > But up to you.? : ) > > Thanks, > Patricio >> Thanks again, >> Richard. >> >> -----Original Message----- >> From: Patricio Chilano >> Sent: Donnerstag, 13. Februar 2020 18:47 >> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi Richard, >> >> I?m only commenting on the handshake changes. >> I see that operation VM_EnterInterpOnlyMode can be called inside >> operation VM_SetFramePop which also allows nested operations. Here is a >> comment in VM_SetFramePop definition: >> >> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >> >> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >> could have a handshake inside a safepoint operation. The issue I see >> there is that at the end of the handshake the polling page of the target >> thread could be disarmed. So if the target thread happens to be in a >> blocked state just transiently and wakes up then it will not stop for >> the ongoing safepoint. Maybe I can file an RFE to assert that the >> polling page is armed at the beginning of disarm_safepoint(). >> >> I think one option could be to remove >> SafepointMechanism::disarm_if_needed() in >> HandshakeState::clear_handshake() and let each JavaThread disarm itself >> for the handshake case. >> >> Alternatively I think you could do something similar to what we do in >> Deoptimization::deoptimize_all_marked(): >> >> ? EnterInterpOnlyModeClosure hs; >> ? if (SafepointSynchronize::is_at_safepoint()) { >> ??? hs.do_thread(state->get_thread()); >> ? } else { >> ??? Handshake::execute(&hs, state->get_thread()); >> ? } >> (you could pass ?EnterInterpOnlyModeClosure? directly to the >> HandshakeClosure() constructor) >> >> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >> always called in a nested operation or just sometimes. >> >> Thanks, >> Patricio >> >> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>> // Repost including hotspot runtime and gc lists. >>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>> // with a handshake. >>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>> >>> Hi, >>> >>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>> >>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>> >>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>> >>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>> >>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>> >>> Thanks, Richard. >>> >>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html From alexey.menkov at oracle.com Thu May 14 00:55:03 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 13 May 2020 17:55:03 -0700 Subject: RFR: JDK-8229829: java/lang/management/ThreadMXBean/Locks.java fails with java.lang.RuntimeException: Thread WaitingThread is at WAITING state but is expected to be in Thread.State = WAITING Message-ID: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> Hi all, Please review the fix for https://bugs.openjdk.java.net/browse/JDK-8229829 webrev: http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev/ The fix adds handling for WAITING (and for consistency TIMED_WAITING which is not used in the test) states as it has for BLOCKED state. --alex From martinrb at google.com Thu May 14 00:58:48 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 13 May 2020 17:58:48 -0700 Subject: RFR: JDK-8229829: java/lang/management/ThreadMXBean/Locks.java fails with java.lang.RuntimeException: Thread WaitingThread is at WAITING state but is expected to be in Thread.State = WAITING In-Reply-To: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> References: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> Message-ID: Looks good to me. On Wed, May 13, 2020 at 5:55 PM Alex Menkov wrote: > > Hi all, > > Please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8229829 > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev/ > > The fix adds handling for WAITING (and for consistency TIMED_WAITING > which is not used in the test) states as it has for BLOCKED state. > > --alex From alexey.menkov at oracle.com Thu May 14 02:20:14 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 13 May 2020 19:20:14 -0700 Subject: RFR(S): JDK-8244973: serviceability/attach/RemovingUnixDomainSocketTest.java fails "stderr was not empty" Message-ID: <3de61b65-bc50-2d61-75a8-110b7d3c8f0f@oracle.com> Hi all, please review tiny (and I suppose trivial) fix for https://bugs.openjdk.java.net/browse/JDK-8244973 webrev: http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket_merge/webrev/ This bug is a result of bad merge: base: out.stderrShouldBeEmpty(); fix for JDK-8242009: - out.stderrShouldBeEmpty(); + out.stderrShouldBeEmptyIgnoreVMWarnings(); fix for JDK-8235211: - out.stderrShouldBeEmpty(); + out.shouldHaveExitValue(0) + .stderrShouldBeEmpty(); Test run is in progress. --alex From david.holmes at oracle.com Thu May 14 02:20:48 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 May 2020 12:20:48 +1000 Subject: RFR: JDK-8229829: java/lang/management/ThreadMXBean/Locks.java fails with java.lang.RuntimeException: Thread WaitingThread is at WAITING state but is expected to be in Thread.State = WAITING In-Reply-To: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> References: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> Message-ID: Hi Alex, On 14/05/2020 10:55 am, Alex Menkov wrote: > Hi all, > > Please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8229829 > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev/ > > The fix adds handling for WAITING That part is good. > (and for consistency TIMED_WAITING But this could be a problem. I know what your intent is but we are waiting to observe a thread transition to a state that it can't escape from, but with TIMED_WAITING the thread can escape - when the timeout elapses. So it is possible that the target becomes TIMED_WAITING before you check the state, but then leaves that state due to timeout, and then you check the state and will loop until you trigger a failure. Cheers, David ----- > which is not used in the test) states as it has for BLOCKED state. > > --alex From david.holmes at oracle.com Thu May 14 02:27:38 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 May 2020 12:27:38 +1000 Subject: RFR(S): JDK-8244973: serviceability/attach/RemovingUnixDomainSocketTest.java fails "stderr was not empty" In-Reply-To: <3de61b65-bc50-2d61-75a8-110b7d3c8f0f@oracle.com> References: <3de61b65-bc50-2d61-75a8-110b7d3c8f0f@oracle.com> Message-ID: <6e13060e-3151-fef2-68ec-3c510ebbd332@oracle.com> Hi Alex, On 14/05/2020 12:20 pm, Alex Menkov wrote: > Hi all, > > please review tiny (and I suppose trivial) fix for > https://bugs.openjdk.java.net/browse/JDK-8244973 > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket_merge/webrev/ > > > This bug is a result of bad merge: > base: > ??????? out.stderrShouldBeEmpty(); > > fix for JDK-8242009: > -??????? out.stderrShouldBeEmpty(); > +??????? out.stderrShouldBeEmptyIgnoreVMWarnings(); > > fix for JDK-8235211: > -??????? out.stderrShouldBeEmpty(); > +??????? out.shouldHaveExitValue(0) > +??????????????? .stderrShouldBeEmpty(); Merge fix looks good but see below ... Nit: for this style of chained call you should align the dots (as we do for stream operations): out.shouldHaveExitValue(0) .stderrShouldBeEmptyIgnoreVMWarnings(); > Test run is in progress. ... I'm puzzled by the failure mode as I see: ----------System.err:(23/1191)---------- [jcmd] java version "15-ea" 2020-09-15 stdout: [Java HotSpot(TM) 64-Bit Server VM version 15-ea+23-1102 JDK 15.0.0 ]; stderr: [Java(TM) SE Runtime Environment (fastdebug build 15-ea+23-1102) Java HotSpot(TM) 64-Bit Server VM (fastdebug build 15-ea+23-1102, mixed mode) ] and these are not VM warnings. Thanks, David > --alex From alexey.menkov at oracle.com Thu May 14 02:38:43 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 13 May 2020 19:38:43 -0700 Subject: RFR(S): JDK-8244973: serviceability/attach/RemovingUnixDomainSocketTest.java fails "stderr was not empty" In-Reply-To: <6e13060e-3151-fef2-68ec-3c510ebbd332@oracle.com> References: <3de61b65-bc50-2d61-75a8-110b7d3c8f0f@oracle.com> <6e13060e-3151-fef2-68ec-3c510ebbd332@oracle.com> Message-ID: <22326b23-7b6c-b36d-3a8c-b32a97467467@oracle.com> Hi David, On 05/13/2020 19:27, David Holmes wrote: > Hi Alex, > > On 14/05/2020 12:20 pm, Alex Menkov wrote: >> Hi all, >> >> please review tiny (and I suppose trivial) fix for >> https://bugs.openjdk.java.net/browse/JDK-8244973 >> webrev: >> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket_merge/webrev/ >> >> >> This bug is a result of bad merge: >> base: >> ???????? out.stderrShouldBeEmpty(); >> >> fix for JDK-8242009: >> -??????? out.stderrShouldBeEmpty(); >> +??????? out.stderrShouldBeEmptyIgnoreVMWarnings(); >> >> fix for JDK-8235211: >> -??????? out.stderrShouldBeEmpty(); >> +??????? out.shouldHaveExitValue(0) >> +??????????????? .stderrShouldBeEmpty(); > > Merge fix looks good but see below ... > > Nit: for this style of chained call you should align the dots (as we do > for stream operations): > > ?? out.shouldHaveExitValue(0) > ????? .stderrShouldBeEmptyIgnoreVMWarnings(); Will fix the indent before push. > >> Test run is in progress. > > ... I'm puzzled by the failure mode as I see: > > ----------System.err:(23/1191)---------- > [jcmd] java version "15-ea" 2020-09-15 > ?stdout: [Java HotSpot(TM) 64-Bit Server VM version 15-ea+23-1102 > JDK 15.0.0 > ]; > ?stderr: [Java(TM) SE Runtime Environment (fastdebug build 15-ea+23-1102) > Java HotSpot(TM) 64-Bit Server VM (fastdebug build 15-ea+23-1102, mixed > mode) > ] > and these are not VM warnings. Yes, I also was surprised seeing this version-related output in stderr. As far as I understand, this was introduced by fix for https://bugs.openjdk.java.net/browse/JDK-8242009 It excludes JAVA_WARNINGS_AND_VERSION_PATTERN which is private static final String JVM_WARNING_MSG = ".* VM warning:.*"; private static final String JAVA_VERSION_MSG = "^java version .*|^Java\\(TM\\).*|^Java HotSpot\\(TM\\).*|" + "^openjdk version .*|^OpenJDK .*"; private static final String JAVA_WARNINGS_AND_VERSION = JVM_WARNING_MSG + "|" + JAVA_VERSION_MSG; private static final Pattern JAVA_WARNINGS_AND_VERSION_PATTERN = Pattern.compile(JAVA_WARNINGS_AND_VERSION.replaceAll("\\|", "\\\\R|") + "\\R", Pattern.MULTILINE); --alex > > Thanks, > David > > >> --alex From david.holmes at oracle.com Thu May 14 03:02:03 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 May 2020 13:02:03 +1000 Subject: RFR(S): JDK-8244973: serviceability/attach/RemovingUnixDomainSocketTest.java fails "stderr was not empty" In-Reply-To: <22326b23-7b6c-b36d-3a8c-b32a97467467@oracle.com> References: <3de61b65-bc50-2d61-75a8-110b7d3c8f0f@oracle.com> <6e13060e-3151-fef2-68ec-3c510ebbd332@oracle.com> <22326b23-7b6c-b36d-3a8c-b32a97467467@oracle.com> Message-ID: <5f108965-6d14-3711-9b25-79fbf5147192@oracle.com> On 14/05/2020 12:38 pm, Alex Menkov wrote: > Hi David, > > On 05/13/2020 19:27, David Holmes wrote: >> Hi Alex, >> >> On 14/05/2020 12:20 pm, Alex Menkov wrote: >>> Hi all, >>> >>> please review tiny (and I suppose trivial) fix for >>> https://bugs.openjdk.java.net/browse/JDK-8244973 >>> webrev: >>> http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket_merge/webrev/ >>> >>> >>> This bug is a result of bad merge: >>> base: >>> ???????? out.stderrShouldBeEmpty(); >>> >>> fix for JDK-8242009: >>> -??????? out.stderrShouldBeEmpty(); >>> +??????? out.stderrShouldBeEmptyIgnoreVMWarnings(); >>> >>> fix for JDK-8235211: >>> -??????? out.stderrShouldBeEmpty(); >>> +??????? out.shouldHaveExitValue(0) >>> +??????????????? .stderrShouldBeEmpty(); >> >> Merge fix looks good but see below ... >> >> Nit: for this style of chained call you should align the dots (as we >> do for stream operations): >> >> ??? out.shouldHaveExitValue(0) >> ?????? .stderrShouldBeEmptyIgnoreVMWarnings(); > > Will fix the indent before push. > >> >>> Test run is in progress. >> >> ... I'm puzzled by the failure mode as I see: >> >> ----------System.err:(23/1191)---------- >> [jcmd] java version "15-ea" 2020-09-15 >> ??stdout: [Java HotSpot(TM) 64-Bit Server VM version 15-ea+23-1102 >> JDK 15.0.0 >> ]; >> ??stderr: [Java(TM) SE Runtime Environment (fastdebug build >> 15-ea+23-1102) >> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 15-ea+23-1102, >> mixed mode) >> ] >> and these are not VM warnings. > > Yes, I also was surprised seeing this version-related output in stderr. > > As far as I understand, this was introduced by fix for > https://bugs.openjdk.java.net/browse/JDK-8242009 > It excludes JAVA_WARNINGS_AND_VERSION_PATTERN which is Whoa! That fix should have been more widely reviewed before it changed the test library code that all groups are using! I don't think I agree with that change in that form. If it is desirable to ignore version strings then something more explicit should have been introduced IMO. That version output seems somewhat spurious. I'd be looking at where it originates from and may be addressing it at the source. I will file a bug to re-examine the OutputAnalyzer changes. Thanks, David > private static final String JVM_WARNING_MSG = ".* VM warning:.*"; > private static final String JAVA_VERSION_MSG = "^java version > .*|^Java\\(TM\\).*|^Java HotSpot\\(TM\\).*|" + > ??????????? "^openjdk version .*|^OpenJDK .*"; > private static final String JAVA_WARNINGS_AND_VERSION = JVM_WARNING_MSG > + "|" + JAVA_VERSION_MSG; > private static final Pattern JAVA_WARNINGS_AND_VERSION_PATTERN = > ??????????? Pattern.compile(JAVA_WARNINGS_AND_VERSION.replaceAll("\\|", > "\\\\R|") + "\\R", > ??????????????????? Pattern.MULTILINE); > > --alex > >> >> Thanks, >> David >> >> >>> --alex From serguei.spitsyn at oracle.com Thu May 14 03:12:39 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 13 May 2020 20:12:39 -0700 Subject: RFR(S): JDK-8244973: serviceability/attach/RemovingUnixDomainSocketTest.java fails "stderr was not empty" In-Reply-To: <3de61b65-bc50-2d61-75a8-110b7d3c8f0f@oracle.com> References: <3de61b65-bc50-2d61-75a8-110b7d3c8f0f@oracle.com> Message-ID: <7eee4587-8f6e-756c-eb3c-bb3cfafe3859@oracle.com> Hi Alex, The fix looks good. Thanks, Serguei On 5/13/20 19:20, Alex Menkov wrote: > Hi all, > > please review tiny (and I suppose trivial) fix for > https://bugs.openjdk.java.net/browse/JDK-8244973 > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/RemovingUnixDomainSocket_merge/webrev/ > > > This bug is a result of bad merge: > base: > ??????? out.stderrShouldBeEmpty(); > > fix for JDK-8242009: > -??????? out.stderrShouldBeEmpty(); > +??????? out.stderrShouldBeEmptyIgnoreVMWarnings(); > > fix for JDK-8235211: > -??????? out.stderrShouldBeEmpty(); > +??????? out.shouldHaveExitValue(0) > +??????????????? .stderrShouldBeEmpty(); > > Test run is in progress. > > --alex From serguei.spitsyn at oracle.com Thu May 14 03:46:25 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 13 May 2020 20:46:25 -0700 Subject: RFR: JDK-8229829: java/lang/management/ThreadMXBean/Locks.java fails with java.lang.RuntimeException: Thread WaitingThread is at WAITING state but is expected to be in Thread.State = WAITING In-Reply-To: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> References: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> Message-ID: <79934953-c0a9-8794-f446-4c135c47e396@oracle.com> Hi Alex, It looks good in general. But I agree with David, the TIMED_WAITING thread state can be escaped by timeout. More sophisticated logic is require to track it precisely. I'd suggest to get rid of the TIMED_WAITING in this condition as it is never passed to the assertThreadState() as an expectedState parameter . Hopefully, I do not miss anything here. Thanks, Serguei On 5/13/20 17:55, Alex Menkov wrote: > Hi all, > > Please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8229829 > webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev/ > > The fix adds handling for WAITING (and for consistency TIMED_WAITING > which is not used in the test) states as it has for BLOCKED state. > > --alex From david.holmes at oracle.com Thu May 14 05:29:22 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 May 2020 15:29:22 +1000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> <36d5e2c0-c724-7ff7-d37e-decb5cc0005b@oracle.com> Message-ID: <9e64b51d-8ac8-9e9a-1f89-52ca897932a4@oracle.com> > Still not a review, or is it now? I'd say still not a review as I'm only looking at the general structure. Cheers, David On 14/05/2020 1:37 am, Reingruber, Richard wrote: > Hi David, > >> On 4/05/2020 8:33 pm, Reingruber, Richard wrote: >>> // Trimmed the list of recipients. If the list gets too long then the message needs to be approved >>> // by a moderator. > >> Yes I noticed that too :) In general if you send to hotspot-dev you >> shouldn't need to also send to hotspot-X-dev. > > Makes sense. Will do so next time. > >>> >>> This would be the post with the current webrev.1 >>> >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html > >> Sorry I missed that update. Okay so this is working with direct >> handshakes now. > >> One style nit in jvmtiThreadState.cpp: > >> assert(SafepointSynchronize::is_at_safepoint() || >> ! (JavaThread *)Thread::current() == get_thread() || >> ! Thread::current() == get_thread()->active_handshaker(), >> ! "bad synchronization with owner thread"); > >> the ! lines should ident as follows > >> assert(SafepointSynchronize::is_at_safepoint() || >> (JavaThread *)Thread::current() == get_thread() || >> Thread::current() == get_thread()->active_handshaker(), >> ! "bad synchronization with owner thread"); > > Sure. > >> Lets see how this plays out. > > Hopefully not too bad... :) > >>> Not a review but some general commentary ... > > Still not a review, or is it now? > > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Mittwoch, 13. Mai 2020 07:43 > To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > On 4/05/2020 8:33 pm, Reingruber, Richard wrote: >> // Trimmed the list of recipients. If the list gets too long then the message needs to be approved >> // by a moderator. > > Yes I noticed that too :) In general if you send to hotspot-dev you > shouldn't need to also send to hotspot-X-dev. > >> Hi David, > > Hi Richard, > >>> On 28/04/2020 12:09 am, Reingruber, Richard wrote: >>>> Hi David, >>>> >>>>> Not a review but some general commentary ... >>>> >>>> That's welcome. >> >>> Having had to take an even closer look now I have a review comment too :) >> >>> src/hotspot/share/prims/jvmtiThreadState.cpp >> >>> void JvmtiThreadState::invalidate_cur_stack_depth() { >>> ! assert(SafepointSynchronize::is_at_safepoint() || >>> ! (Thread::current()->is_VM_thread() && >>> get_thread()->is_vmthread_processing_handshake()) || >>> (JavaThread *)Thread::current() == get_thread(), >>> "must be current thread or at safepoint"); >> >> You're looking at an outdated webrev, I'm afraid. >> >> This would be the post with the current webrev.1 >> >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html > > Sorry I missed that update. Okay so this is working with direct > handshakes now. > > One style nit in jvmtiThreadState.cpp: > > assert(SafepointSynchronize::is_at_safepoint() || > ! (JavaThread *)Thread::current() == get_thread() || > ! Thread::current() == get_thread()->active_handshaker(), > ! "bad synchronization with owner thread"); > > the ! lines should ident as follows > > assert(SafepointSynchronize::is_at_safepoint() || > (JavaThread *)Thread::current() == get_thread() || > Thread::current() == get_thread()->active_handshaker(), > ! "bad synchronization with owner thread"); > > Lets see how this plays out. > > Cheers, > David > >> Thanks, Richard. >> >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 4. Mai 2020 08:51 >> To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi Richard, >> >> On 28/04/2020 12:09 am, Reingruber, Richard wrote: >>> Hi David, >>> >>>> Not a review but some general commentary ... >>> >>> That's welcome. >> >> Having had to take an even closer look now I have a review comment too :) >> >> src/hotspot/share/prims/jvmtiThreadState.cpp >> >> void JvmtiThreadState::invalidate_cur_stack_depth() { >> ! assert(SafepointSynchronize::is_at_safepoint() || >> ! (Thread::current()->is_VM_thread() && >> get_thread()->is_vmthread_processing_handshake()) || >> (JavaThread *)Thread::current() == get_thread(), >> "must be current thread or at safepoint"); >> >> The message needs updating to include handshakes. >> >> More below ... >> >>>> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>>>> Hi Yasumasa, Patricio, >>>>> >>>>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>>>> Does it help you? I think it gives you to remove workaround. >>>>>>> >>>>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>>> >>>>>> Thanks for your information. >>>>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>>>> I will modify and will test it after yours. >>>>> >>>>> Thanks :) >>>>> >>>>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>>>> to me, how this has to be handled. >>>>> >>>>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>>> >>>>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>>>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >>> >>>> I'm growing increasingly concerned that use of direct handshakes to >>>> replace VM operations needs a much greater examination for correctness >>>> than might initially be thought. I see a number of issues: >>> >>> I agree. I'll address your concerns in the context of this review thread for JDK-8238585 below. >>> >>> In addition I would suggest to take the general part of the discussion to a dedicated thread or to >>> the review thread for JDK-8242427. I would like to keep this thread closer to its subject. >> >> I will focus on the issues in the context of this particular change >> then, though the issues themselves are applicable to all handshake >> situations (and more so with direct handshakes). This is mostly just >> discussion. >> >>>> First, the VMThread executes (most) VM operations with a clean stack in >>>> a clean state, so it has lots of room to work. If we now execute the >>>> same logic in a JavaThread then we risk hitting stackoverflows if >>>> nothing else. But we are also now executing code in a JavaThread and so >>>> we have to be sure that code is not going to act differently (in a bad >>>> way) if executed by a JavaThread rather than the VMThread. For example, >>>> may it be possible that if executing in the VMThread we defer some >>>> activity that might require execution of Java code, or else hand it off >>>> to one of the service threads? If we execute that code directly in the >>>> current JavaThread instead we may not be in a valid state (e.g. consider >>>> re-entrancy to various subsystems that is not allowed). >>> >>> It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is doing. I already added a >>> paragraph to the JBS-Item [1] explaining why the direct handshake is sufficient from a >>> synchronization point of view. >> >> Just to be clear, your proposed change is not using a direct handshake. >> >>> Furthermore the stack is walked and the return pc of compiled frames is replaced with the address of >>> the deopt handler. >>> >>> I can't see why this cannot be done with a direct handshake. Something very similar is already done >>> in JavaThread::deoptimize_marked_methods() which is executed as part of an ordinary handshake. >> >> Note that existing non-direct handshakes may also have issues that not >> have been fully investigated. >> >>> The demand on stack-space should be very modest. I would not expect a higher risk for stackoverflow. >> >> For the target thread if you use more stack than would be used stopping >> at a safepoint then you are at risk. For the thread initiating the >> direct handshake if you use more stack than would be used enqueuing a VM >> operation, then you are at risk. As we have not quantified these >> numbers, nor have any easy way to establish the stack use of the actual >> code to be executed, we're really just hoping for the best. This is a >> general problem with handshakes that needs to be investigated more >> deeply. As a simple, general, example just imagine if the code involves >> logging that might utilise an on-stack buffer. >> >>>> Second, we have this question mark over what happens if the operation >>>> hits further safepoint or handshake polls/checks? Are there constraints >>>> on what is allowed here? How can we recognise this problem may exist and >>>> so deal with it? >>> >>> The thread in EnterInterpOnlyModeClosure::do_thread() can't become safepoint/handshake safe. I >>> tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a NoSafepointVerifier. >> >> That's good to hear but such tests are not exhaustive, they will detect >> if you do reach a safepoint/handshake but they can't prove that you >> cannot reach one. What you have done is necessary but may not be >> sufficient. Plus you didn't actually add the NSV to the code - is there >> a reason we can't actually keep it in do_thread? (I'm not sure if the >> NSV also acts as a NoHandshakeVerifier?) >> >>>> Third, while we are generally considering what appear to be >>>> single-thread operations, which should be amenable to a direct >>>> handshake, we also have to be careful that some of the code involved >>>> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >>>> not need to take a lock where a direct handshake might! >>> >>> See again my arguments in the JBS item [1]. >> >> Yes I see the reasoning and that is good. My point is a general one as >> it may not be obvious when such assumptions exist in the current code. >> >> Thanks, >> David >> >>> Thanks, >>> Richard. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8238585 >>> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Montag, 27. April 2020 07:16 >>> To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>> >>> Hi all, >>> >>> Not a review but some general commentary ... >>> >>> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>>> Hi Yasumasa, Patricio, >>>> >>>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>>> Does it help you? I think it gives you to remove workaround. >>>>>> >>>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>> >>>>> Thanks for your information. >>>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>>> I will modify and will test it after yours. >>>> >>>> Thanks :) >>>> >>>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>>> to me, how this has to be handled. >>>> >>>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>> >>>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >>> >>> I'm growing increasingly concerned that use of direct handshakes to >>> replace VM operations needs a much greater examination for correctness >>> than might initially be thought. I see a number of issues: >>> >>> First, the VMThread executes (most) VM operations with a clean stack in >>> a clean state, so it has lots of room to work. If we now execute the >>> same logic in a JavaThread then we risk hitting stackoverflows if >>> nothing else. But we are also now executing code in a JavaThread and so >>> we have to be sure that code is not going to act differently (in a bad >>> way) if executed by a JavaThread rather than the VMThread. For example, >>> may it be possible that if executing in the VMThread we defer some >>> activity that might require execution of Java code, or else hand it off >>> to one of the service threads? If we execute that code directly in the >>> current JavaThread instead we may not be in a valid state (e.g. consider >>> re-entrancy to various subsystems that is not allowed). >>> >>> Second, we have this question mark over what happens if the operation >>> hits further safepoint or handshake polls/checks? Are there constraints >>> on what is allowed here? How can we recognise this problem may exist and >>> so deal with it? >>> >>> Third, while we are generally considering what appear to be >>> single-thread operations, which should be amenable to a direct >>> handshake, we also have to be careful that some of the code involved >>> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >>> not need to take a lock where a direct handshake might! >>> >>> Cheers, >>> David >>> ----- >>> >>>> @Patricio, coming back to my question [1]: >>>> >>>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a >>>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads >>>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The >>>> handshakee would be safepoint safe, wouldn't it? >>>> >>>> Thanks, Richard. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677 >>>> >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763 >>>> >>>> -----Original Message----- >>>> From: Yasumasa Suenaga >>>> Sent: Freitag, 24. April 2020 17:23 >>>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Richard, >>>> >>>> On 2020/04/24 23:44, Reingruber, Richard wrote: >>>>> Hi Yasumasa, >>>>> >>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>> >>>> Thanks for your information. >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>> I will modify and will test it after yours. >>>> >>>> >>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>> to me, how this has to be handled. >>>> >>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585). >>>>> >>>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>>> >>>>> Would be interesting to see how you handled the issues above :) >>>>> >>>>> Thanks, Richard. >>>>> >>>>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030 >>>>> >>>>> -----Original Message----- >>>>> From: Yasumasa Suenaga >>>>> Sent: Freitag, 24. April 2020 13:34 >>>>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Richard, >>>>> >>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/04/24 17:18, Reingruber, Richard wrote: >>>>>> Hi Patricio, Vladimir, and Serguei, >>>>>> >>>>>> now that direct handshakes are available, I've updated the patch to make use of them. >>>>>> >>>>>> In addition I have done some clean-up changes I missed in the first webrev. >>>>>> >>>>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake >>>>>> into the vm operation VM_SetFramePop [1] >>>>>> >>>>>> Kindly review again: >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ >>>>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ >>>>>> >>>>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a >>>>>> direct handshake: >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>> >>>>>> Testing: >>>>>> >>>>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>> >>>>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 >>>>>> >>>>>> Thanks, >>>>>> Richard. >>>>>> >>>>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. >>>>>> >>>>>> -----Original Message----- >>>>>> From: hotspot-dev On Behalf Of Reingruber, Richard >>>>>> Sent: Freitag, 14. Februar 2020 19:47 >>>>>> To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>> >>>>>> Hi Patricio, >>>>>> >>>>>> > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>>> > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>>> > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>>> > > >>>>>> > > > Alternatively I think you could do something similar to what we do in >>>>>> > > > Deoptimization::deoptimize_all_marked(): >>>>>> > > > >>>>>> > > > EnterInterpOnlyModeClosure hs; >>>>>> > > > if (SafepointSynchronize::is_at_safepoint()) { >>>>>> > > > hs.do_thread(state->get_thread()); >>>>>> > > > } else { >>>>>> > > > Handshake::execute(&hs, state->get_thread()); >>>>>> > > > } >>>>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>> > > > HandshakeClosure() constructor) >>>>>> > > >>>>>> > > Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>>> > Right, we could also do that. Avoiding to clear the polling page in >>>>>> > HandshakeState::clear_handshake() should be enough to fix this issue and >>>>>> > execute a handshake inside a safepoint, but adding that "if" statement >>>>>> > in Hanshake::execute() sounds good to avoid all the extra code that we >>>>>> > go through when executing a handshake. I filed 8239084 to make that change. >>>>>> >>>>>> Thanks for taking care of this and creating the RFE. >>>>>> >>>>>> > >>>>>> > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>> > > > always called in a nested operation or just sometimes. >>>>>> > > >>>>>> > > At least one execution path without vm operation exists: >>>>>> > > >>>>>> > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>>> > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void >>>>>> > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>>> > > >>>>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>>> > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>>> > > encouraged to do it with a handshake :) >>>>>> > Ah! I think you can still do it with a handshake with the >>>>>> > Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>>> > if-else statement with just the Handshake::execute() call in 8239084. >>>>>> > But up to you. : ) >>>>>> >>>>>> Well, I think that's enough encouragement :) >>>>>> I'll wait for 8239084 and try then again. >>>>>> (no urgency and all) >>>>>> >>>>>> Thanks, >>>>>> Richard. >>>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Freitag, 14. Februar 2020 15:54 >>>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>> >>>>>> Hi Richard, >>>>>> >>>>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>> thanks for having a look. >>>>>>> >>>>>>> > I?m only commenting on the handshake changes. >>>>>>> > I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>>> > operation VM_SetFramePop which also allows nested operations. Here is a >>>>>>> > comment in VM_SetFramePop definition: >>>>>>> > >>>>>>> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>>> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>>> > >>>>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>>> > could have a handshake inside a safepoint operation. The issue I see >>>>>>> > there is that at the end of the handshake the polling page of the target >>>>>>> > thread could be disarmed. So if the target thread happens to be in a >>>>>>> > blocked state just transiently and wakes up then it will not stop for >>>>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>>> > polling page is armed at the beginning of disarm_safepoint(). >>>>>>> >>>>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>>>> >>>>>>> > Alternatively I think you could do something similar to what we do in >>>>>>> > Deoptimization::deoptimize_all_marked(): >>>>>>> > >>>>>>> > EnterInterpOnlyModeClosure hs; >>>>>>> > if (SafepointSynchronize::is_at_safepoint()) { >>>>>>> > hs.do_thread(state->get_thread()); >>>>>>> > } else { >>>>>>> > Handshake::execute(&hs, state->get_thread()); >>>>>>> > } >>>>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>>> > HandshakeClosure() constructor) >>>>>>> >>>>>>> Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>>> Right, we could also do that. Avoiding to clear the polling page in >>>>>> HandshakeState::clear_handshake() should be enough to fix this issue and >>>>>> execute a handshake inside a safepoint, but adding that "if" statement >>>>>> in Hanshake::execute() sounds good to avoid all the extra code that we >>>>>> go through when executing a handshake. I filed 8239084 to make that change. >>>>>> >>>>>>> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>>> > always called in a nested operation or just sometimes. >>>>>>> >>>>>>> At least one execution path without vm operation exists: >>>>>>> >>>>>>> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>>>> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>>>> JvmtiEventControllerPrivate::recompute_enabled() : void >>>>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>>>> >>>>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>>>> encouraged to do it with a handshake :) >>>>>> Ah! I think you can still do it with a handshake with the >>>>>> Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>>> if-else statement with just the Handshake::execute() call in 8239084. >>>>>> But up to you.? : ) >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Thanks again, >>>>>>> Richard. >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Patricio Chilano >>>>>>> Sent: Donnerstag, 13. Februar 2020 18:47 >>>>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>>> >>>>>>> Hi Richard, >>>>>>> >>>>>>> I?m only commenting on the handshake changes. >>>>>>> I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>>> operation VM_SetFramePop which also allows nested operations. Here is a >>>>>>> comment in VM_SetFramePop definition: >>>>>>> >>>>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>>> >>>>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>>> could have a handshake inside a safepoint operation. The issue I see >>>>>>> there is that at the end of the handshake the polling page of the target >>>>>>> thread could be disarmed. So if the target thread happens to be in a >>>>>>> blocked state just transiently and wakes up then it will not stop for >>>>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>>> polling page is armed at the beginning of disarm_safepoint(). >>>>>>> >>>>>>> I think one option could be to remove >>>>>>> SafepointMechanism::disarm_if_needed() in >>>>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself >>>>>>> for the handshake case. >>>>>>> >>>>>>> Alternatively I think you could do something similar to what we do in >>>>>>> Deoptimization::deoptimize_all_marked(): >>>>>>> >>>>>>> ? EnterInterpOnlyModeClosure hs; >>>>>>> ? if (SafepointSynchronize::is_at_safepoint()) { >>>>>>> ??? hs.do_thread(state->get_thread()); >>>>>>> ? } else { >>>>>>> ??? Handshake::execute(&hs, state->get_thread()); >>>>>>> ? } >>>>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>>> HandshakeClosure() constructor) >>>>>>> >>>>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>>> always called in a nested operation or just sometimes. >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>> >>>>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>>>>>>> // Repost including hotspot runtime and gc lists. >>>>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>>>>>>> // with a handshake. >>>>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>>>> >>>>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>>>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>>>>>>> >>>>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>>>>>>> >>>>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>>>> >>>>>>>> Thanks, Richard. >>>>>>>> >>>>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>>>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html >>>>>> From richard.reingruber at sap.com Thu May 14 07:38:30 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 14 May 2020 07:38:30 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: <9e64b51d-8ac8-9e9a-1f89-52ca897932a4@oracle.com> References: <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com> <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com> <9c49ea2d-e3b8-b576-1d17-d18ad87cd6ed@oracle.com> <36d5e2c0-c724-7ff7-d37e-decb5cc0005b@oracle.com> <9e64b51d-8ac8-9e9a-1f89-52ca897932a4@oracle.com> Message-ID: Ok. Thanks for the feedback anyways. Cheers, Richard. -----Original Message----- From: David Holmes Sent: Donnerstag, 14. Mai 2020 07:29 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > Still not a review, or is it now? I'd say still not a review as I'm only looking at the general structure. Cheers, David On 14/05/2020 1:37 am, Reingruber, Richard wrote: > Hi David, > >> On 4/05/2020 8:33 pm, Reingruber, Richard wrote: >>> // Trimmed the list of recipients. If the list gets too long then the message needs to be approved >>> // by a moderator. > >> Yes I noticed that too :) In general if you send to hotspot-dev you >> shouldn't need to also send to hotspot-X-dev. > > Makes sense. Will do so next time. > >>> >>> This would be the post with the current webrev.1 >>> >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html > >> Sorry I missed that update. Okay so this is working with direct >> handshakes now. > >> One style nit in jvmtiThreadState.cpp: > >> assert(SafepointSynchronize::is_at_safepoint() || >> ! (JavaThread *)Thread::current() == get_thread() || >> ! Thread::current() == get_thread()->active_handshaker(), >> ! "bad synchronization with owner thread"); > >> the ! lines should ident as follows > >> assert(SafepointSynchronize::is_at_safepoint() || >> (JavaThread *)Thread::current() == get_thread() || >> Thread::current() == get_thread()->active_handshaker(), >> ! "bad synchronization with owner thread"); > > Sure. > >> Lets see how this plays out. > > Hopefully not too bad... :) > >>> Not a review but some general commentary ... > > Still not a review, or is it now? > > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Mittwoch, 13. Mai 2020 07:43 > To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > On 4/05/2020 8:33 pm, Reingruber, Richard wrote: >> // Trimmed the list of recipients. If the list gets too long then the message needs to be approved >> // by a moderator. > > Yes I noticed that too :) In general if you send to hotspot-dev you > shouldn't need to also send to hotspot-X-dev. > >> Hi David, > > Hi Richard, > >>> On 28/04/2020 12:09 am, Reingruber, Richard wrote: >>>> Hi David, >>>> >>>>> Not a review but some general commentary ... >>>> >>>> That's welcome. >> >>> Having had to take an even closer look now I have a review comment too :) >> >>> src/hotspot/share/prims/jvmtiThreadState.cpp >> >>> void JvmtiThreadState::invalidate_cur_stack_depth() { >>> ! assert(SafepointSynchronize::is_at_safepoint() || >>> ! (Thread::current()->is_VM_thread() && >>> get_thread()->is_vmthread_processing_handshake()) || >>> (JavaThread *)Thread::current() == get_thread(), >>> "must be current thread or at safepoint"); >> >> You're looking at an outdated webrev, I'm afraid. >> >> This would be the post with the current webrev.1 >> >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-April/031245.html > > Sorry I missed that update. Okay so this is working with direct > handshakes now. > > One style nit in jvmtiThreadState.cpp: > > assert(SafepointSynchronize::is_at_safepoint() || > ! (JavaThread *)Thread::current() == get_thread() || > ! Thread::current() == get_thread()->active_handshaker(), > ! "bad synchronization with owner thread"); > > the ! lines should ident as follows > > assert(SafepointSynchronize::is_at_safepoint() || > (JavaThread *)Thread::current() == get_thread() || > Thread::current() == get_thread()->active_handshaker(), > ! "bad synchronization with owner thread"); > > Lets see how this plays out. > > Cheers, > David > >> Thanks, Richard. >> >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 4. Mai 2020 08:51 >> To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi Richard, >> >> On 28/04/2020 12:09 am, Reingruber, Richard wrote: >>> Hi David, >>> >>>> Not a review but some general commentary ... >>> >>> That's welcome. >> >> Having had to take an even closer look now I have a review comment too :) >> >> src/hotspot/share/prims/jvmtiThreadState.cpp >> >> void JvmtiThreadState::invalidate_cur_stack_depth() { >> ! assert(SafepointSynchronize::is_at_safepoint() || >> ! (Thread::current()->is_VM_thread() && >> get_thread()->is_vmthread_processing_handshake()) || >> (JavaThread *)Thread::current() == get_thread(), >> "must be current thread or at safepoint"); >> >> The message needs updating to include handshakes. >> >> More below ... >> >>>> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>>>> Hi Yasumasa, Patricio, >>>>> >>>>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>>>> Does it help you? I think it gives you to remove workaround. >>>>>>> >>>>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>>> >>>>>> Thanks for your information. >>>>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>>>> I will modify and will test it after yours. >>>>> >>>>> Thanks :) >>>>> >>>>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>>>> to me, how this has to be handled. >>>>> >>>>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>>> >>>>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>>>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >>> >>>> I'm growing increasingly concerned that use of direct handshakes to >>>> replace VM operations needs a much greater examination for correctness >>>> than might initially be thought. I see a number of issues: >>> >>> I agree. I'll address your concerns in the context of this review thread for JDK-8238585 below. >>> >>> In addition I would suggest to take the general part of the discussion to a dedicated thread or to >>> the review thread for JDK-8242427. I would like to keep this thread closer to its subject. >> >> I will focus on the issues in the context of this particular change >> then, though the issues themselves are applicable to all handshake >> situations (and more so with direct handshakes). This is mostly just >> discussion. >> >>>> First, the VMThread executes (most) VM operations with a clean stack in >>>> a clean state, so it has lots of room to work. If we now execute the >>>> same logic in a JavaThread then we risk hitting stackoverflows if >>>> nothing else. But we are also now executing code in a JavaThread and so >>>> we have to be sure that code is not going to act differently (in a bad >>>> way) if executed by a JavaThread rather than the VMThread. For example, >>>> may it be possible that if executing in the VMThread we defer some >>>> activity that might require execution of Java code, or else hand it off >>>> to one of the service threads? If we execute that code directly in the >>>> current JavaThread instead we may not be in a valid state (e.g. consider >>>> re-entrancy to various subsystems that is not allowed). >>> >>> It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is doing. I already added a >>> paragraph to the JBS-Item [1] explaining why the direct handshake is sufficient from a >>> synchronization point of view. >> >> Just to be clear, your proposed change is not using a direct handshake. >> >>> Furthermore the stack is walked and the return pc of compiled frames is replaced with the address of >>> the deopt handler. >>> >>> I can't see why this cannot be done with a direct handshake. Something very similar is already done >>> in JavaThread::deoptimize_marked_methods() which is executed as part of an ordinary handshake. >> >> Note that existing non-direct handshakes may also have issues that not >> have been fully investigated. >> >>> The demand on stack-space should be very modest. I would not expect a higher risk for stackoverflow. >> >> For the target thread if you use more stack than would be used stopping >> at a safepoint then you are at risk. For the thread initiating the >> direct handshake if you use more stack than would be used enqueuing a VM >> operation, then you are at risk. As we have not quantified these >> numbers, nor have any easy way to establish the stack use of the actual >> code to be executed, we're really just hoping for the best. This is a >> general problem with handshakes that needs to be investigated more >> deeply. As a simple, general, example just imagine if the code involves >> logging that might utilise an on-stack buffer. >> >>>> Second, we have this question mark over what happens if the operation >>>> hits further safepoint or handshake polls/checks? Are there constraints >>>> on what is allowed here? How can we recognise this problem may exist and >>>> so deal with it? >>> >>> The thread in EnterInterpOnlyModeClosure::do_thread() can't become safepoint/handshake safe. I >>> tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a NoSafepointVerifier. >> >> That's good to hear but such tests are not exhaustive, they will detect >> if you do reach a safepoint/handshake but they can't prove that you >> cannot reach one. What you have done is necessary but may not be >> sufficient. Plus you didn't actually add the NSV to the code - is there >> a reason we can't actually keep it in do_thread? (I'm not sure if the >> NSV also acts as a NoHandshakeVerifier?) >> >>>> Third, while we are generally considering what appear to be >>>> single-thread operations, which should be amenable to a direct >>>> handshake, we also have to be careful that some of the code involved >>>> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >>>> not need to take a lock where a direct handshake might! >>> >>> See again my arguments in the JBS item [1]. >> >> Yes I see the reasoning and that is good. My point is a general one as >> it may not be obvious when such assumptions exist in the current code. >> >> Thanks, >> David >> >>> Thanks, >>> Richard. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8238585 >>> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Montag, 27. April 2020 07:16 >>> To: Reingruber, Richard ; Yasumasa Suenaga ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>> >>> Hi all, >>> >>> Not a review but some general commentary ... >>> >>> On 25/04/2020 2:08 am, Reingruber, Richard wrote: >>>> Hi Yasumasa, Patricio, >>>> >>>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>>> Does it help you? I think it gives you to remove workaround. >>>>>> >>>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>> >>>>> Thanks for your information. >>>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>>> I will modify and will test it after yours. >>>> >>>> Thanks :) >>>> >>>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>>> to me, how this has to be handled. >>>> >>>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>> >>>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And >>>> also I'm unsure if a thread should do safepoint checks while executing a handshake. >>> >>> I'm growing increasingly concerned that use of direct handshakes to >>> replace VM operations needs a much greater examination for correctness >>> than might initially be thought. I see a number of issues: >>> >>> First, the VMThread executes (most) VM operations with a clean stack in >>> a clean state, so it has lots of room to work. If we now execute the >>> same logic in a JavaThread then we risk hitting stackoverflows if >>> nothing else. But we are also now executing code in a JavaThread and so >>> we have to be sure that code is not going to act differently (in a bad >>> way) if executed by a JavaThread rather than the VMThread. For example, >>> may it be possible that if executing in the VMThread we defer some >>> activity that might require execution of Java code, or else hand it off >>> to one of the service threads? If we execute that code directly in the >>> current JavaThread instead we may not be in a valid state (e.g. consider >>> re-entrancy to various subsystems that is not allowed). >>> >>> Second, we have this question mark over what happens if the operation >>> hits further safepoint or handshake polls/checks? Are there constraints >>> on what is allowed here? How can we recognise this problem may exist and >>> so deal with it? >>> >>> Third, while we are generally considering what appear to be >>> single-thread operations, which should be amenable to a direct >>> handshake, we also have to be careful that some of the code involved >>> doesn't already expect/assume we are at a safepoint - e.g. a VM op may >>> not need to take a lock where a direct handshake might! >>> >>> Cheers, >>> David >>> ----- >>> >>>> @Patricio, coming back to my question [1]: >>>> >>>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a >>>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads >>>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The >>>> handshakee would be safepoint safe, wouldn't it? >>>> >>>> Thanks, Richard. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677 >>>> >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763 >>>> >>>> -----Original Message----- >>>> From: Yasumasa Suenaga >>>> Sent: Freitag, 24. April 2020 17:23 >>>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>> >>>> Hi Richard, >>>> >>>> On 2020/04/24 23:44, Reingruber, Richard wrote: >>>>> Hi Yasumasa, >>>>> >>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake >>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to >>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes. >>>> >>>> Thanks for your information. >>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop. >>>> I will modify and will test it after yours. >>>> >>>> >>>>> Also my first impression was that it won't be that easy from a synchronization point of view to >>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls >>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where >>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear >>>>> to me, how this has to be handled. >>>> >>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585). >>>>> >>>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>>> >>>>> Would be interesting to see how you handled the issues above :) >>>>> >>>>> Thanks, Richard. >>>>> >>>>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030 >>>>> >>>>> -----Original Message----- >>>>> From: Yasumasa Suenaga >>>>> Sent: Freitag, 24. April 2020 13:34 >>>>> To: Reingruber, Richard ; Patricio Chilano ; serguei.spitsyn at oracle.com; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>> >>>>> Hi Richard, >>>>> >>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427. >>>>> Does it help you? I think it gives you to remove workaround. >>>>> >>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.) >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/04/24 17:18, Reingruber, Richard wrote: >>>>>> Hi Patricio, Vladimir, and Serguei, >>>>>> >>>>>> now that direct handshakes are available, I've updated the patch to make use of them. >>>>>> >>>>>> In addition I have done some clean-up changes I missed in the first webrev. >>>>>> >>>>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake >>>>>> into the vm operation VM_SetFramePop [1] >>>>>> >>>>>> Kindly review again: >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ >>>>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ >>>>>> >>>>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a >>>>>> direct handshake: >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>> >>>>>> Testing: >>>>>> >>>>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>> >>>>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 >>>>>> >>>>>> Thanks, >>>>>> Richard. >>>>>> >>>>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. >>>>>> >>>>>> -----Original Message----- >>>>>> From: hotspot-dev On Behalf Of Reingruber, Richard >>>>>> Sent: Freitag, 14. Februar 2020 19:47 >>>>>> To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>> >>>>>> Hi Patricio, >>>>>> >>>>>> > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>>> > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>>> > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>>> > > >>>>>> > > > Alternatively I think you could do something similar to what we do in >>>>>> > > > Deoptimization::deoptimize_all_marked(): >>>>>> > > > >>>>>> > > > EnterInterpOnlyModeClosure hs; >>>>>> > > > if (SafepointSynchronize::is_at_safepoint()) { >>>>>> > > > hs.do_thread(state->get_thread()); >>>>>> > > > } else { >>>>>> > > > Handshake::execute(&hs, state->get_thread()); >>>>>> > > > } >>>>>> > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>> > > > HandshakeClosure() constructor) >>>>>> > > >>>>>> > > Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>>> > Right, we could also do that. Avoiding to clear the polling page in >>>>>> > HandshakeState::clear_handshake() should be enough to fix this issue and >>>>>> > execute a handshake inside a safepoint, but adding that "if" statement >>>>>> > in Hanshake::execute() sounds good to avoid all the extra code that we >>>>>> > go through when executing a handshake. I filed 8239084 to make that change. >>>>>> >>>>>> Thanks for taking care of this and creating the RFE. >>>>>> >>>>>> > >>>>>> > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>> > > > always called in a nested operation or just sometimes. >>>>>> > > >>>>>> > > At least one execution path without vm operation exists: >>>>>> > > >>>>>> > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>>> > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>>> > > JvmtiEventControllerPrivate::recompute_enabled() : void >>>>>> > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>>> > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>>> > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>>> > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>>> > > >>>>>> > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>>> > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>>> > > encouraged to do it with a handshake :) >>>>>> > Ah! I think you can still do it with a handshake with the >>>>>> > Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>>> > if-else statement with just the Handshake::execute() call in 8239084. >>>>>> > But up to you. : ) >>>>>> >>>>>> Well, I think that's enough encouragement :) >>>>>> I'll wait for 8239084 and try then again. >>>>>> (no urgency and all) >>>>>> >>>>>> Thanks, >>>>>> Richard. >>>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Freitag, 14. Februar 2020 15:54 >>>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>> >>>>>> Hi Richard, >>>>>> >>>>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>> thanks for having a look. >>>>>>> >>>>>>> > I?m only commenting on the handshake changes. >>>>>>> > I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>>> > operation VM_SetFramePop which also allows nested operations. Here is a >>>>>>> > comment in VM_SetFramePop definition: >>>>>>> > >>>>>>> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>>> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>>> > >>>>>>> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>>> > could have a handshake inside a safepoint operation. The issue I see >>>>>>> > there is that at the end of the handshake the polling page of the target >>>>>>> > thread could be disarmed. So if the target thread happens to be in a >>>>>>> > blocked state just transiently and wakes up then it will not stop for >>>>>>> > the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>>> > polling page is armed at the beginning of disarm_safepoint(). >>>>>>> >>>>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >>>>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >>>>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >>>>>>> >>>>>>> > Alternatively I think you could do something similar to what we do in >>>>>>> > Deoptimization::deoptimize_all_marked(): >>>>>>> > >>>>>>> > EnterInterpOnlyModeClosure hs; >>>>>>> > if (SafepointSynchronize::is_at_safepoint()) { >>>>>>> > hs.do_thread(state->get_thread()); >>>>>>> > } else { >>>>>>> > Handshake::execute(&hs, state->get_thread()); >>>>>>> > } >>>>>>> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>>> > HandshakeClosure() constructor) >>>>>>> >>>>>>> Maybe this could be used also in the Handshake::execute() methods as general solution? >>>>>> Right, we could also do that. Avoiding to clear the polling page in >>>>>> HandshakeState::clear_handshake() should be enough to fix this issue and >>>>>> execute a handshake inside a safepoint, but adding that "if" statement >>>>>> in Hanshake::execute() sounds good to avoid all the extra code that we >>>>>> go through when executing a handshake. I filed 8239084 to make that change. >>>>>> >>>>>>> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>>> > always called in a nested operation or just sometimes. >>>>>>> >>>>>>> At least one execution path without vm operation exists: >>>>>>> >>>>>>> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >>>>>>> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >>>>>>> JvmtiEventControllerPrivate::recompute_enabled() : void >>>>>>> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >>>>>>> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >>>>>>> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >>>>>>> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >>>>>>> >>>>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >>>>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >>>>>>> encouraged to do it with a handshake :) >>>>>> Ah! I think you can still do it with a handshake with the >>>>>> Deoptimization::deoptimize_all_marked() like solution. I can change the >>>>>> if-else statement with just the Handshake::execute() call in 8239084. >>>>>> But up to you.? : ) >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Thanks again, >>>>>>> Richard. >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Patricio Chilano >>>>>>> Sent: Donnerstag, 13. Februar 2020 18:47 >>>>>>> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >>>>>>> >>>>>>> Hi Richard, >>>>>>> >>>>>>> I?m only commenting on the handshake changes. >>>>>>> I see that operation VM_EnterInterpOnlyMode can be called inside >>>>>>> operation VM_SetFramePop which also allows nested operations. Here is a >>>>>>> comment in VM_SetFramePop definition: >>>>>>> >>>>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >>>>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >>>>>>> >>>>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >>>>>>> could have a handshake inside a safepoint operation. The issue I see >>>>>>> there is that at the end of the handshake the polling page of the target >>>>>>> thread could be disarmed. So if the target thread happens to be in a >>>>>>> blocked state just transiently and wakes up then it will not stop for >>>>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the >>>>>>> polling page is armed at the beginning of disarm_safepoint(). >>>>>>> >>>>>>> I think one option could be to remove >>>>>>> SafepointMechanism::disarm_if_needed() in >>>>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself >>>>>>> for the handshake case. >>>>>>> >>>>>>> Alternatively I think you could do something similar to what we do in >>>>>>> Deoptimization::deoptimize_all_marked(): >>>>>>> >>>>>>> ? EnterInterpOnlyModeClosure hs; >>>>>>> ? if (SafepointSynchronize::is_at_safepoint()) { >>>>>>> ??? hs.do_thread(state->get_thread()); >>>>>>> ? } else { >>>>>>> ??? Handshake::execute(&hs, state->get_thread()); >>>>>>> ? } >>>>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the >>>>>>> HandshakeClosure() constructor) >>>>>>> >>>>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >>>>>>> always called in a nested operation or just sometimes. >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>> >>>>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>>>>>>> // Repost including hotspot runtime and gc lists. >>>>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>>>>>>> // with a handshake. >>>>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>>>>>>> >>>>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>>>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>>>>>>> >>>>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>>>>>>> >>>>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>>>>>>> >>>>>>>> Thanks, Richard. >>>>>>>> >>>>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>>>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html >>>>>> From richard.reingruber at sap.com Thu May 14 07:48:43 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 14 May 2020 07:48:43 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: <03c9a0ce-8f78-00e7-9db3-70d6f6cb8156@oracle.com> References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> <03c9a0ce-8f78-00e7-9db3-70d6f6cb8156@oracle.com> Message-ID: Hi Serguei, > Thank you for the bug report update - it is helpful. > The fix/update looks good in general but I need more time to check some > points. Sure. I'd be happy if you could look at it again. > I'm thinking it would be more safe to run full tier5. > I can do it after you get all thumbs ups. The patch goes through extensive testing here at SAP every night since many weeks. Still it would be great if you could run full tier5. I'll wait then for a view more thumbs... Thanks, Richard. -----Original Message----- From: serguei.spitsyn at oracle.com Sent: Donnerstag, 14. Mai 2020 00:32 To: Reingruber, Richard ; Patricio Chilano ; Vladimir Ivanov ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant Hi Richard, Thank you for the bug report update - it is helpful. The fix/update looks good in general but I need more time to check some points. I'm thinking it would be more safe to run full tier5. I can do it after you get all thumbs ups. Thanks, Serguei On 4/24/20 01:18, Reingruber, Richard wrote: > Hi Patricio, Vladimir, and Serguei, > > now that direct handshakes are available, I've updated the patch to make use of them. > > In addition I have done some clean-up changes I missed in the first webrev. > > Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake > into the vm operation VM_SetFramePop [1] > > Kindly review again: > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/ > Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/ > > I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a > direct handshake: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8238585 > > Testing: > > * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737 > > Thanks, > Richard. > > [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread. > > -----Original Message----- > From: hotspot-dev On Behalf Of Reingruber, Richard > Sent: Freitag, 14. Februar 2020 19:47 > To: Patricio Chilano ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Patricio, > > > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a > > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the > > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? > > > > > > > Alternatively I think you could do something similar to what we do in > > > > Deoptimization::deoptimize_all_marked(): > > > > > > > > EnterInterpOnlyModeClosure hs; > > > > if (SafepointSynchronize::is_at_safepoint()) { > > > > hs.do_thread(state->get_thread()); > > > > } else { > > > > Handshake::execute(&hs, state->get_thread()); > > > > } > > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the > > > > HandshakeClosure() constructor) > > > > > > Maybe this could be used also in the Handshake::execute() methods as general solution? > > Right, we could also do that. Avoiding to clear the polling page in > > HandshakeState::clear_handshake() should be enough to fix this issue and > > execute a handshake inside a safepoint, but adding that "if" statement > > in Hanshake::execute() sounds good to avoid all the extra code that we > > go through when executing a handshake. I filed 8239084 to make that change. > > Thanks for taking care of this and creating the RFE. > > > > > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > > > > always called in a nested operation or just sometimes. > > > > > > At least one execution path without vm operation exists: > > > > > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void > > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong > > > JvmtiEventControllerPrivate::recompute_enabled() : void > > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) > > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void > > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError > > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError > > > > > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a > > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further > > > encouraged to do it with a handshake :) > > Ah! I think you can still do it with a handshake with the > > Deoptimization::deoptimize_all_marked() like solution. I can change the > > if-else statement with just the Handshake::execute() call in 8239084. > > But up to you. : ) > > Well, I think that's enough encouragement :) > I'll wait for 8239084 and try then again. > (no urgency and all) > > Thanks, > Richard. > > -----Original Message----- > From: Patricio Chilano > Sent: Freitag, 14. Februar 2020 15:54 > To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Richard, > > On 2/14/20 9:58 AM, Reingruber, Richard wrote: >> Hi Patricio, >> >> thanks for having a look. >> >> > I?m only commenting on the handshake changes. >> > I see that operation VM_EnterInterpOnlyMode can be called inside >> > operation VM_SetFramePop which also allows nested operations. Here is a >> > comment in VM_SetFramePop definition: >> > >> > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >> > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >> > >> > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >> > could have a handshake inside a safepoint operation. The issue I see >> > there is that at the end of the handshake the polling page of the target >> > thread could be disarmed. So if the target thread happens to be in a >> > blocked state just transiently and wakes up then it will not stop for >> > the ongoing safepoint. Maybe I can file an RFE to assert that the >> > polling page is armed at the beginning of disarm_safepoint(). >> >> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a >> handshake cannot be nested in a vm operation. Maybe it should be asserted in the >> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? >> >> > Alternatively I think you could do something similar to what we do in >> > Deoptimization::deoptimize_all_marked(): >> > >> > EnterInterpOnlyModeClosure hs; >> > if (SafepointSynchronize::is_at_safepoint()) { >> > hs.do_thread(state->get_thread()); >> > } else { >> > Handshake::execute(&hs, state->get_thread()); >> > } >> > (you could pass ?EnterInterpOnlyModeClosure? directly to the >> > HandshakeClosure() constructor) >> >> Maybe this could be used also in the Handshake::execute() methods as general solution? > Right, we could also do that. Avoiding to clear the polling page in > HandshakeState::clear_handshake() should be enough to fix this issue and > execute a handshake inside a safepoint, but adding that "if" statement > in Hanshake::execute() sounds good to avoid all the extra code that we > go through when executing a handshake. I filed 8239084 to make that change. > >> > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >> > always called in a nested operation or just sometimes. >> >> At least one execution path without vm operation exists: >> >> JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void >> JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong >> JvmtiEventControllerPrivate::recompute_enabled() : void >> JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) >> JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void >> JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError >> jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError >> >> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a >> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further >> encouraged to do it with a handshake :) > Ah! I think you can still do it with a handshake with the > Deoptimization::deoptimize_all_marked() like solution. I can change the > if-else statement with just the Handshake::execute() call in 8239084. > But up to you.? : ) > > Thanks, > Patricio >> Thanks again, >> Richard. >> >> -----Original Message----- >> From: Patricio Chilano >> Sent: Donnerstag, 13. Februar 2020 18:47 >> To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net >> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant >> >> Hi Richard, >> >> I?m only commenting on the handshake changes. >> I see that operation VM_EnterInterpOnlyMode can be called inside >> operation VM_SetFramePop which also allows nested operations. Here is a >> comment in VM_SetFramePop definition: >> >> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is >> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. >> >> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we >> could have a handshake inside a safepoint operation. The issue I see >> there is that at the end of the handshake the polling page of the target >> thread could be disarmed. So if the target thread happens to be in a >> blocked state just transiently and wakes up then it will not stop for >> the ongoing safepoint. Maybe I can file an RFE to assert that the >> polling page is armed at the beginning of disarm_safepoint(). >> >> I think one option could be to remove >> SafepointMechanism::disarm_if_needed() in >> HandshakeState::clear_handshake() and let each JavaThread disarm itself >> for the handshake case. >> >> Alternatively I think you could do something similar to what we do in >> Deoptimization::deoptimize_all_marked(): >> >> ? EnterInterpOnlyModeClosure hs; >> ? if (SafepointSynchronize::is_at_safepoint()) { >> ??? hs.do_thread(state->get_thread()); >> ? } else { >> ??? Handshake::execute(&hs, state->get_thread()); >> ? } >> (you could pass ?EnterInterpOnlyModeClosure? directly to the >> HandshakeClosure() constructor) >> >> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is >> always called in a nested operation or just sometimes. >> >> Thanks, >> Patricio >> >> On 2/12/20 7:23 AM, Reingruber, Richard wrote: >>> // Repost including hotspot runtime and gc lists. >>> // Dean Long suggested to do so, because the enhancement replaces a vm operation >>> // with a handshake. >>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >>> >>> Hi, >>> >>> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >>> >>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >>> >>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >>> >>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >>> >>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >>> >>> Thanks, Richard. >>> >>> See also my question if anyone knows a reason for making the compiled methods not_entrant: >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html From Alan.Bateman at oracle.com Thu May 14 11:25:34 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 May 2020 12:25:34 +0100 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> Message-ID: On 12/05/2020 20:57, Alex Menkov wrote: > Hi Alan, Serguei, > > lets try one more time :) > > What about: > > Agents can transform classes in arbitrary ways at load time, transform > modules, or transform the bytecode of methods of already loaded classes. > Developers or administrators that deploy agents, deploy applications that > package an agent with the application, or use tools that load agents > into a > running application, are responsible for verifying the trustworthiness > of each > agent including the content and structure of the agent JAR file. > > > please let me know what do you thinks, I'll prepare & publish new > webrev as soon as we get agreement about the paragraph. This version looks okay to me. -Alan From serguei.spitsyn at oracle.com Thu May 14 16:26:21 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 14 May 2020 09:26:21 -0700 Subject: RFR(XS): 8222005: ClassRedefinition crashes with: guarantee(false) failed: OLD and/or OBSOLETE method(s) found Message-ID: <5942b42c-b9b3-f1d4-6c13-774649fca32b@oracle.com> An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Thu May 14 18:04:56 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 14 May 2020 11:04:56 -0700 Subject: RFR: JDK-8229829: java/lang/management/ThreadMXBean/Locks.java fails with java.lang.RuntimeException: Thread WaitingThread is at WAITING state but is expected to be in Thread.State = WAITING In-Reply-To: References: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> Message-ID: I agree with the point. updated webrev (only WAITING handling is added): http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev.2/ --alex On 05/13/2020 19:20, David Holmes wrote: > Hi Alex, > > On 14/05/2020 10:55 am, Alex Menkov wrote: >> Hi all, >> >> Please review the fix for >> https://bugs.openjdk.java.net/browse/JDK-8229829 >> webrev: >> http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev/ >> >> The fix adds handling for WAITING > > That part is good. > >> (and for consistency TIMED_WAITING > > But this could be a problem. I know what your intent is but we are > waiting to observe a thread transition to a state that it can't escape > from, but with TIMED_WAITING the thread can escape - when the timeout > elapses. So it is possible that the target becomes TIMED_WAITING before > you check the state, but then leaves that state due to timeout, and then > you check the state and will loop until you trigger a failure. > > Cheers, > David > ----- > >> which is not used in the test) states as it has for BLOCKED state. >> >> --alex From alexey.menkov at oracle.com Thu May 14 18:30:06 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 14 May 2020 11:30:06 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> Message-ID: <051c9704-51b6-214e-faa4-4489b723c509@oracle.com> Hi Alan, Serguei, updated webrev: http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.3/ --alex On 05/14/2020 04:25, Alan Bateman wrote: > On 12/05/2020 20:57, Alex Menkov wrote: >> Hi Alan, Serguei, >> >> lets try one more time :) >> >> What about: >> >> Agents can transform classes in arbitrary ways at load time, transform >> modules, or transform the bytecode of methods of already loaded classes. >> Developers or administrators that deploy agents, deploy applications that >> package an agent with the application, or use tools that load agents >> into a >> running application, are responsible for verifying the trustworthiness >> of each >> agent including the content and structure of the agent JAR file. >> >> >> please let me know what do you thinks, I'll prepare & publish new >> webrev as soon as we get agreement about the paragraph. > This version looks okay to me. > > -Alan From Alan.Bateman at oracle.com Thu May 14 18:54:06 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 May 2020 19:54:06 +0100 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <051c9704-51b6-214e-faa4-4489b723c509@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> <051c9704-51b6-214e-faa4-4489b723c509@oracle.com> Message-ID: <11f98eb1-2653-8ce6-6565-a8db4afca26e@oracle.com> On 14/05/2020 19:30, Alex Menkov wrote: > Hi Alan, Serguei, > > updated webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.3/ Thanks. -Alan From serguei.spitsyn at oracle.com Thu May 14 20:18:49 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 14 May 2020 13:18:49 -0700 Subject: RFR: JDK-8229829: java/lang/management/ThreadMXBean/Locks.java fails with java.lang.RuntimeException: Thread WaitingThread is at WAITING state but is expected to be in Thread.State = WAITING In-Reply-To: References: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> Message-ID: <4fa7280e-844e-50e2-7de0-6f9bd056aaf4@oracle.com> Hi Alex, LGTM. Thanks, Serguei On 5/14/20 11:04, Alex Menkov wrote: > I agree with the point. > > updated webrev (only WAITING handling is added): > http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev.2/ > > --alex > > On 05/13/2020 19:20, David Holmes wrote: >> Hi Alex, >> >> On 14/05/2020 10:55 am, Alex Menkov wrote: >>> Hi all, >>> >>> Please review the fix for >>> https://bugs.openjdk.java.net/browse/JDK-8229829 >>> webrev: >>> http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev/ >>> >>> The fix adds handling for WAITING >> >> That part is good. >> >>> (and for consistency TIMED_WAITING >> >> But this could be a problem. I know what your intent is but we are >> waiting to observe a thread transition to a state that it can't >> escape from, but with TIMED_WAITING the thread can escape - when the >> timeout elapses. So it is possible that the target becomes >> TIMED_WAITING before you check the state, but then leaves that state >> due to timeout, and then you check the state and will loop until you >> trigger a failure. >> >> Cheers, >> David >> ----- >> >>> which is not used in the test) states as it has for BLOCKED state. >>> >>> --alex From serguei.spitsyn at oracle.com Thu May 14 20:21:15 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 14 May 2020 13:21:15 -0700 Subject: Ping: RFR: JDK-8243012: Fix issues in j.l.i package info In-Reply-To: <051c9704-51b6-214e-faa4-4489b723c509@oracle.com> References: <02291fdc-0e09-416f-7789-d8e8a9027468@oracle.com> <43559a98-6e46-56b0-2d57-d07da308cd1c@oracle.com> <5a199a5e-5675-aff2-b1dd-a2d8594ec4a1@oracle.com> <7e8c09b3-90cc-c445-ccb0-d485f3b97836@oracle.com> <20cfdb76-67d4-b011-b5fd-714df6ec78d6@oracle.com> <17350703-8ec1-6301-3a77-510c05ceb8c8@oracle.com> <051c9704-51b6-214e-faa4-4489b723c509@oracle.com> Message-ID: <1e09f758-99e8-c9ce-b00f-2b5bbb94d808@oracle.com> Hi Alex, LGTM. Thanks, Serguei On 5/14/20 11:30, Alex Menkov wrote: > Hi Alan, Serguei, > > updated webrev: > http://cr.openjdk.java.net/~amenkov/jdk15/java_instrument_spec/webrev.3/ > > --alex > > On 05/14/2020 04:25, Alan Bateman wrote: >> On 12/05/2020 20:57, Alex Menkov wrote: >>> Hi Alan, Serguei, >>> >>> lets try one more time :) >>> >>> What about: >>> >>> Agents can transform classes in arbitrary ways at load time, transform >>> modules, or transform the bytecode of methods of already loaded >>> classes. >>> Developers or administrators that deploy agents, deploy applications >>> that >>> package an agent with the application, or use tools that load agents >>> into a >>> running application, are responsible for verifying the >>> trustworthiness of each >>> agent including the content and structure of the agent JAR file. >>> >>> >>> please let me know what do you thinks, I'll prepare & publish new >>> webrev as soon as we get agreement about the paragraph. >> This version looks okay to me. >> >> -Alan From daniil.x.titov at oracle.com Thu May 14 20:33:31 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Thu, 14 May 2020 13:33:31 -0700 Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools In-Reply-To: References: Message-ID: <7B3DCB8E-B61E-4D9B-BEDC-F69A414818CA@oracle.com> Hi Serguei and Chris, Please review a new version of the change [1] that addresses your comments. Testing: Mach5 tier1-tier5 tests successfully passed. Regarding CR for the JDWP spec issues related to missing type information in some of the protocol packet descriptions [3], as Chris has just noticed we really don't need it, so I withdrew this CR. [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.02 [2] https://bugs.openjdk.java.net/browse/JDK-8241080 [3] https://bugs.openjdk.java.net/browse/JDK-8245057 Thank you, Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, May 11, 2020 at 11:53 AM To: Daniil Titov , serviceability-dev Subject: Re: RFR: 8241080: Consolidate signature parsing code in serviceability tools Hi Daniil, It looks pretty good in general. A couple of nits below. http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/invoker.c.udiff.html + void *cursor; + jbyte argumentTag; + jint argIndex = 0; + jvalue *argument = request->arguments;; . . . void *cursor; jint argIndex = 0; + jbyte argumentTag; jvalue *argument = request->arguments; It is better if the local variables 'cursor' and 'argumentTag' get initialized. There is double semicolon: "arguments;;" http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/signature.h.html 43 static inline jbyte basicType(const char *signature) { It'd be nice to rename it to basicTypeTag() to get it unified with other functions below. It is more safe to run tier5 as well. Thanks, Serguei On 5/9/20 09:29, Daniil Titov wrote: Please review a change[1] that centralizes the signature processing in serviceability tools to make it capable of being easily extensible in the future. Testing: Mach5 tier1-tier3 tests successfully passed. [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.01 [2] https://bugs.openjdk.java.net/browse/JDK-8241080 Thank you, Daniil From serguei.spitsyn at oracle.com Thu May 14 20:56:44 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 14 May 2020 13:56:44 -0700 Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools In-Reply-To: <7B3DCB8E-B61E-4D9B-BEDC-F69A414818CA@oracle.com> References: <7B3DCB8E-B61E-4D9B-BEDC-F69A414818CA@oracle.com> Message-ID: <01ad44b4-baf3-53be-eb34-6b0686a028fe@oracle.com> Hi Daniil, Thank you for the update! It looks good to me. > Regarding CR for the JDWP spec issues I guess, you wanted to say "CSR", not CR. :) Thanks, Serguei On 5/14/20 13:33, Daniil Titov wrote: > Hi Serguei and Chris, > > Please review a new version of the change [1] that addresses your comments. > > Testing: Mach5 tier1-tier5 tests successfully passed. > > Regarding CR for the JDWP spec issues related to missing type information in some of the protocol packet descriptions [3], as Chris has just noticed > we really don't need it, so I withdrew this CR. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.02 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > [3] https://bugs.openjdk.java.net/browse/JDK-8245057 > > Thank you, > Daniil > > > From: "serguei.spitsyn at oracle.com" > Date: Monday, May 11, 2020 at 11:53 AM > To: Daniil Titov , serviceability-dev > Subject: Re: RFR: 8241080: Consolidate signature parsing code in serviceability tools > > Hi Daniil, > > It looks pretty good in general. > A couple of nits below. > > http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/invoker.c.udiff.html > + void *cursor; > + jbyte argumentTag; > + jint argIndex = 0; > + jvalue *argument = request->arguments;; > . . . > void *cursor; > jint argIndex = 0; > + jbyte argumentTag; > jvalue *argument = request->arguments; > > It is better if the local variables 'cursor' and 'argumentTag' get initialized. > There is double semicolon: "arguments;;" > > > http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/signature.h.html > 43 static inline jbyte basicType(const char *signature) { > > It'd be nice to rename it to basicTypeTag() to get it unified with other functions below. > > It is more safe to run tier5 as well. > > Thanks, > Serguei > > > On 5/9/20 09:29, Daniil Titov wrote: > Please review a change[1] that centralizes the signature processing in serviceability tools to make it capable of being easily extensible in the future. > > Testing: Mach5 tier1-tier3 tests successfully passed. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.01 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > > Thank you, > Daniil > > > > > > From david.holmes at oracle.com Thu May 14 22:56:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 15 May 2020 08:56:58 +1000 Subject: RFR: JDK-8229829: java/lang/management/ThreadMXBean/Locks.java fails with java.lang.RuntimeException: Thread WaitingThread is at WAITING state but is expected to be in Thread.State = WAITING In-Reply-To: <4fa7280e-844e-50e2-7de0-6f9bd056aaf4@oracle.com> References: <2c7663b8-ba54-b6ee-f1e5-1c7388d50735@oracle.com> <4fa7280e-844e-50e2-7de0-6f9bd056aaf4@oracle.com> Message-ID: +1 Thanks, David On 15/05/2020 6:18 am, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > LGTM. > > Thanks, > Serguei > > > On 5/14/20 11:04, Alex Menkov wrote: >> I agree with the point. >> >> updated webrev (only WAITING handling is added): >> http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev.2/ >> >> --alex >> >> On 05/13/2020 19:20, David Holmes wrote: >>> Hi Alex, >>> >>> On 14/05/2020 10:55 am, Alex Menkov wrote: >>>> Hi all, >>>> >>>> Please review the fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8229829 >>>> webrev: >>>> http://cr.openjdk.java.net/~amenkov/jdk15/Locks_waiting/webrev/ >>>> >>>> The fix adds handling for WAITING >>> >>> That part is good. >>> >>>> (and for consistency TIMED_WAITING >>> >>> But this could be a problem. I know what your intent is but we are >>> waiting to observe a thread transition to a state that it can't >>> escape from, but with TIMED_WAITING the thread can escape - when the >>> timeout elapses. So it is possible that the target becomes >>> TIMED_WAITING before you check the state, but then leaves that state >>> due to timeout, and then you check the state and will loop until you >>> trigger a failure. >>> >>> Cheers, >>> David >>> ----- >>> >>>> which is not used in the test) states as it has for BLOCKED state. >>>> >>>> --alex > From coleen.phillimore at oracle.com Fri May 15 12:12:21 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 15 May 2020 08:12:21 -0400 Subject: RFR(XS): 8222005: ClassRedefinition crashes with: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <5942b42c-b9b3-f1d4-6c13-774649fca32b@oracle.com> References: <5942b42c-b9b3-f1d4-6c13-774649fca32b@oracle.com> Message-ID: <2f9aa92c-18f5-1203-1523-3c1fd9ba9ad1@oracle.com> Serguei, Good find!!? The fix looks good.? I'm sure the optimization wasn't noticeable and thank you for the additional comments. There is a Method::external_name() function that I believe prints all the things you want in the logging here: http://cr.openjdk.java.net/~sspitsyn/webrevs/2020/jvmti-redef.1/src/hotspot/share/oops/cpCache.cpp.udiff.html I don't need to see another webrev if you make this change. Thanks, Coleen On 5/14/20 12:26 PM, serguei.spitsyn at oracle.com wrote: > Please, review a fix for The Kitchensink bug: > https://bugs.openjdk.java.net/browse/JDK-8222005 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2020/jvmti-redef.1/ > > Summary: > ? The VM_RedefineClasses::doit()uses two helper classes to walk all VM > classes. > ? First is AdjustAndCleanMetadata to adjust method entries in the > vtables/itables/cpcaches. > ? Second is CheckClass to check that adjustments for all method > entries are correct. > ? The Kitchensink test is failing with two modes: > ??? - guarantee(false) failed: OLD and/or OBSOLETE method(s) found in the > VM_RedefineClasses::CheckClass::do_klass() > ??? - SIGSEGV in the > ConstantPoolCacheEntry::get_interesting_method_entry() in context > ????? of VM_RedefineClasses::CheckClass::do_klass() execution > > ? The second failure mode is rare. In is before the first one in the > code path. > ? The root cause of both is that the > VM_RedefineClasses::AdjustAndCleanMetadata::do_klass() > ? is skipping the cpcache update for classes that are being redefined > assuming they are > ? being redefined by the current VM_RedefineClasses operation. In such > cases, the adjustment > ? is not needed as the cpcache is empty. The problem is that the > assumption above is wrong. > ? The class can also be redefined by another VM_RedefineClasses > operation which has already > ? executed its doit_prologue. The cpcache djustment for such class is > necessary. > ? The fix is to always call the cp_cache->adjust_method_entries() even > if the class is > ? being redefined by the current VM_RedefineClasses operation. It is > possible to skip it > ? but it will add extra complexity to the code. > ? The fix also includes minor tweak in the cpCache.cpp to include > method's class name to > ? the redefinition cpcache log. > > Testing: > ? Ran Kitchensink test locally on a Linux server with the > Instrumentation module enabled. > ? The test does not fail anymore. > ? In progress, a mach5 tiers 1-5 and runs and separate mach5 > Kitchensink run. > > Thanks, > Serguei -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri May 15 22:14:02 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 15 May 2020 15:14:02 -0700 Subject: RFR(XS): 8222005: ClassRedefinition crashes with: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <2f9aa92c-18f5-1203-1523-3c1fd9ba9ad1@oracle.com> References: <5942b42c-b9b3-f1d4-6c13-774649fca32b@oracle.com> <2f9aa92c-18f5-1203-1523-3c1fd9ba9ad1@oracle.com> Message-ID: <52ba0f0f-a705-2043-1c1d-15ba4a441aba@oracle.com> An HTML attachment was scrubbed... URL: From ralf.schmelter at sap.com Mon May 18 07:22:54 2020 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Mon, 18 May 2020 07:22:54 +0000 Subject: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump In-Reply-To: References: <0343dfac-61f7-1b1c-ee96-bdee130578ad@oracle.com> <2363c58d-38c1-ae19-ed34-c82af6304780@oracle.com> Message-ID: Hi Christoph, I've updated the webrev: http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.3/ The significant changes are moving most of the new compression code to its own file, changing to use a single option (see CSR) called -gz with a mandatory compression level and to load the zlib only once (analog to the new class loader code). Additionally I've removed some long lines. Best regards, Ralf -----Original Message----- From: Langer, Christoph Sent: Friday, 1 May 2020 18:46 To: Schmelter, Ralf Cc: serviceability-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net runtime ; coleen.phillimore at oracle.com Subject: RE: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump Hi Ralf, while I'm reviewing your change I think extracting the compression coding to an own file would be a good idea. Maybe you could name it heapDumpCompression.cpp? When looking at the webrev I also figured that there are some very long lines (beyond 90 chars or so). Maybe you could have a look if you could shorten some of them and break a few of these long lines? More detailed review to follow. Best regards Christoph > -----Original Message----- > From: coleen.phillimore at oracle.com > Sent: Montag, 20. April 2020 14:13 > To: Reingruber, Richard ; Schmelter, Ralf > ; Ioi Lam ; Langer, Christoph > ; Yasumasa Suenaga > ; serguei.spitsyn at oracle.com; hotspot- > runtime-dev at openjdk.java.net runtime dev at openjdk.java.net> > Cc: serviceability-dev at openjdk.java.net > Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped heap > dump > > Hi, I don't want to review this but could you put this new code in its > own file?? heapDumper only needs CompressionBackend to be exported, > from > what I can tell. > > Thanks, > Coleen > > On 4/20/20 6:12 AM, Reingruber, Richard wrote: > > Hi Ralf, > > > >>> 767: I think _current->in_used doesn't take the final 9 bytes into account > that are written in > >>> DumperSupport::end_of_dump() after the last dump segment has been > finished. > >>> You could call get_new_buffer() instead of the if clause. > >> Wow, how did you found this? I've fixed it by making sure we flush the > DumpWriter before calling the deactivate method. > > Spending long hours on the review ;) > > Ok with the fix. > > > >>> ### src/java.base/share/native/libzip/zip_util.c > >>> 1610: Will be hard to beat zlib_block_alloc() and zlib_block_free() > performance wise. But have you > >>> measured the performance gain? In other words: is it worth it? :) > >> This is not done for performance, but to make sure the allocation will not > fail midway during writing the dump. Maybe it is not worth it, though. > > Understood. The heap dump will succeed if you can allocate at least one > WriteWork instance. Without > > that you could get out of memory errors in the zlib which would make the > dump fail. Ok! > > > >> http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.2/ > > Thanks for the clarifications and the changes in the new webrev. > > Webrev.2 looks good to me. > > > > Cheers, Richard. > > > > -----Original Message----- > > From: Schmelter, Ralf > > Sent: Montag, 20. April 2020 10:14 > > To: Reingruber, Richard ; Ioi Lam > ; Langer, Christoph ; > Yasumasa Suenaga ; > serguei.spitsyn at oracle.com; hotspot-runtime-dev at openjdk.java.net > runtime > > Cc: serviceability-dev at openjdk.java.net > > Subject: RE: RFR(L) 8237354: Add option to jcmd to write a gzipped heap > dump > > > > Hi Richard, > > > > thanks for the review. I have incorporated your remarks into a new > webrev: > > http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.2/ > > > > Some remarks to specific points: > > > >> ### src/hotspot/share/services/heapDumper.cpp > >> 762: assert(_active, "Must be active"); > >> > >> It appears to me that the assertion would fail, if an error occurred creating > the CompressionBackend. > > You are supposed to check for errors after creating the DumpWriter (which > creates the CompressionBackend). And in case of an error, you directly > destruct the object. I've added a comment to make that clear. > > > >> 767: I think _current->in_used doesn't take the final 9 bytes into account > that are written in > >> DumperSupport::end_of_dump() after the last dump segment has been > finished. > >> You could call get_new_buffer() instead of the if clause. > > Wow, how did you found this? I've fixed it by making sure we flush the > DumpWriter before calling the deactivate method. > > > >> 1064: DumpWriter::DumpWriter() > >> > >> There doesn't seem to be enough error handling if _buffer cannot be > allocated. > >> E.g. DumpWriter::write_raw() at line 1091 will enter an endless loop. > > As described above, this will not happen if we check for error after > constructing the DumpWriter. > > > >> ### src/java.base/share/native/libzip/zip_util.c > >> 1610: Will be hard to beat zlib_block_alloc() and zlib_block_free() > performance wise. But have you > >> measured the performance gain? In other words: is it worth it? :) > > This is not done for performance, but to make sure the allocation will not > fail midway during writing the dump. Maybe it is not worth it, though. > > > >> 1655: The result of deflateBound() seems to depend on the header > comment, which is not given > >> here. Could this be an issue, because ZIP_GZip_Fully() can take a > comment? > > I've added a 1024 byte additional bytes to avoid the problem. > > > >> ### test/lib/jdk/test/lib/hprof/parser/Reader.java > >> > >> 93: is the created GzipRandomAccess instance closed somewhere? > > The object is not closed since it is still used by the Snapshot returned. > > > > Best regard, > > Ralf > > > > > > -----Original Message----- > > From: Reingruber, Richard > > Sent: Tuesday, 14 April 2020 10:30 > > To: Schmelter, Ralf ; Ioi Lam > ; Langer, Christoph ; > Yasumasa Suenaga ; > serguei.spitsyn at oracle.com; hotspot-runtime-dev at openjdk.java.net > runtime > > Cc: serviceability-dev at openjdk.java.net > > Subject: RE: RFR(L) 8237354: Add option to jcmd to write a gzipped heap > dump > > > > Hi Ralf, > > > > thanks for providing this enhancement to parallel gzip-compress heap > dumps! > > > > I reckon it's safe to say that the coding is sophisticated. It would be > awesome if you could sketch > > the idea of how HeapDumper, DumpWriter and CompressionBackend work > together to produce the gzipped > > dump in a source code comment. Just enough to get started if somebody > should ever have to track down > > a bug -- an unlikely event, I know ;) > > > > Please find the details of my review below. > > > > Thanks, Richard. > > // Not Reviewer > > > > -- > > > > ### src/hotspot/share/services/diagnosticCommand.cpp > > > > 510 _gzip_level("-gz-level", "The compression level from 0 (store) to 9 > (best) when writing in gzipped format.", > > 511 "INT", "FALSE", "1") { > > > > "FALSE" should be probably false. > > > > ### src/hotspot/share/services/diagnosticCommand.hpp > > Ok. > > > > ### src/hotspot/share/services/heapDumper.cpp > > > > 390: Typo: initized > > > > 415: Typo: GZipComressor > > > > 477: Could you please add a comment, how the "HPROF BLOCKSIZE" > comment is helpful? > > > > 539: Member variables of WriteWork are missing the '_' prefix. > > > > 546: Just a comment: WriteWork::in_max is actually a compile time > constant. Would be nice if it could be > > declared so. One could use templates for this, but then my favourite ide > (eclipse cdt) doesn't > > show me references and call hierarchies anymore. So I don't think it is > worth it. > > > > 591: Typo: Removes the first element. Returns NULL is empty. > > > > 663: _writer, _compressor, _lock could be const. > > > > 762: assert(_active, "Must be active"); > > > > It appears to me that the assertion would fail, if an error occurred > creating the CompressionBackend. > > > > 767: I think _current->in_used doesn't take the final 9 bytes into account > that are written in > > DumperSupport::end_of_dump() after the last dump segment has > been finished. > > You could call get_new_buffer() instead of the if clause. > > > > 903: Typo: Check if we don not waste more than _max_waste > > > > 1064: DumpWriter::DumpWriter() > > > > There doesn't seem to be enough error handling if _buffer cannot be > allocated. > > E.g. DumpWriter::write_raw() at line 1091 will enter an endless loop. > > > > 2409: A comment, why Shenandoah is not supported, would be good. > > In general I'd say it is good and natural to use the GC work threads. > > > > ### src/hotspot/share/services/heapDumper.hpp > > Ok. > > > > ### src/java.base/share/native/libzip/zip_util.c > > > > I'm not familiar with zlib, but here are my .02? :) > > > > 1610: Will be hard to beat zlib_block_alloc() and zlib_block_free() > performance wise. But have you > > measured the performance gain? In other words: is it worth it? :) > > > > 1655: The result of deflateBound() seems to depend on the header > comment, which is not given > > here. Could this be an issue, because ZIP_GZip_Fully() can take a > comment? > > > > 1658: deflateEnd() should not be called if deflateInit2Wrapper() failed. I > think this can lead > > otherwise to a double free() call. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTest.java > > > > 66: Maybe additionally check the exit value? > > > > 73: It's unclear to me, why this fails. Because the dump already exists? > Because the level is > > invalid? Reading the comment I'd expect success, not failure. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTestEpsilo > n.java > > Ok. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTestShen > andoah.java > > Ok. > > > > ### > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpCompressedTestZ.jav > a > > Ok. > > > > ### test/lib/jdk/test/lib/hprof/parser/GzipRandomAccess.java > > Ok. > > > > ### test/lib/jdk/test/lib/hprof/parser/HprofReader.java > > Ok. > > > > ### test/lib/jdk/test/lib/hprof/parser/Reader.java > > > > 93: is the created GzipRandomAccess instance closed somewhere? From serguei.spitsyn at oracle.com Mon May 18 07:30:17 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 18 May 2020 00:30:17 -0700 Subject: RFR(XS): 8222005: ClassRedefinition crashes with: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <52ba0f0f-a705-2043-1c1d-15ba4a441aba@oracle.com> References: <5942b42c-b9b3-f1d4-6c13-774649fca32b@oracle.com> <2f9aa92c-18f5-1203-1523-3c1fd9ba9ad1@oracle.com> <52ba0f0f-a705-2043-1c1d-15ba4a441aba@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon May 18 07:34:58 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 18 May 2020 00:34:58 -0700 Subject: RFR(XS): 8222005: ClassRedefinition crashes with: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: References: <5942b42c-b9b3-f1d4-6c13-774649fca32b@oracle.com> <2f9aa92c-18f5-1203-1523-3c1fd9ba9ad1@oracle.com> <52ba0f0f-a705-2043-1c1d-15ba4a441aba@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From sgehwolf at redhat.com Mon May 18 09:08:18 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 18 May 2020 11:08:18 +0200 Subject: [8u] RFR: 8150986: serviceability/sa/jmap-hprof/JMapHProfLargeHeapTest.java failing because expects HPROF JAVA PROFILE 1.0.1 file format Message-ID: <7afbfa4c0a30f76a424f0e40c6c80d015fb7e91b.camel@redhat.com> Hi, Please review this 8u backport of JDK-8150986. It's a clean backport except for the copyright change. In the 8u252 cycle JDK-8144732 got backported which now makes the test fail. Bug: https://bugs.openjdk.java.net/browse/JDK-8150986 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8150986/01/webrev/ Testing: Test fails prior patch, passes after. Thoughts? Thanks, Severin From hohensee at amazon.com Mon May 18 15:23:55 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 18 May 2020 15:23:55 +0000 Subject: RFR (S): 8245129: Enhance jstat gc option output and tests Message-ID: <79438EAF-531A-488F-BED7-A619C3E227D5@amazon.com> Please review an enhancement to the jstat gc option output to make the columns wider (for up to a 2TB heap) so one can read the output without going cross-eyed. Bug: https://bugs.openjdk.java.net/browse/JDK-8245129 Webrev: http://cr.openjdk.java.net/~phh/8245129/webrev.00/ I added tests using ParallelGC since the output can differ for non-G1 collectors. Successfully ran the test/hotspot/jtreg/serviceability/tmtools/jstat and test/jdk/sun/tools/jstat tests. A submit repo run had one failure runtime/MemberName/MemberNameLeak.java tier1 macosx-x64-debug but rerunning it on my laptop succeeded, and there?s no connection between this test and my patch. Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Mon May 18 17:31:18 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Mon, 18 May 2020 10:31:18 -0700 Subject: RFR: 8131745: java/lang/management/ThreadMXBean/AllThreadIds.java still fails intermittently Message-ID: <35299A9D-FBE7-443F-AFA4-765CA247931E@oracle.com> Please review the change [1] that fixes an intermittent failure of the test. This test creates and destroys a given number of daemon/user threads and validates the count of those started/stopped threads against values returned from ThreadMXBean thread counts. The problem here is that if some internal threads is started ( e.g. " HotSpotGraalManagement Bean Registration"), or destroyed (e.g. "JVMCI CompilerThread ") the test hangs waiting for expected number of live threads. The fix limits the time the test is waiting for desired number of live threads and in case if this limit is exceeded the test repeats itself. Testing. Test with Graal on and Mach5 tier1-tier7 test passed. [1] http://cr.openjdk.java.net/~dtitov/8131745/webrev.01 [2] https://bugs.openjdk.java.net/browse/JDK-8131745 Thank you, Daniil From chris.plummer at oracle.com Mon May 18 19:41:12 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 18 May 2020 12:41:12 -0700 Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools In-Reply-To: <7B3DCB8E-B61E-4D9B-BEDC-F69A414818CA@oracle.com> References: <7B3DCB8E-B61E-4D9B-BEDC-F69A414818CA@oracle.com> Message-ID: Looks good. thanks, Chris On 5/14/20 1:33 PM, Daniil Titov wrote: > Hi Serguei and Chris, > > Please review a new version of the change [1] that addresses your comments. > > Testing: Mach5 tier1-tier5 tests successfully passed. > > Regarding CR for the JDWP spec issues related to missing type information in some of the protocol packet descriptions [3], as Chris has just noticed > we really don't need it, so I withdrew this CR. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.02 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > [3] https://bugs.openjdk.java.net/browse/JDK-8245057 > > Thank you, > Daniil > > > From: "serguei.spitsyn at oracle.com" > Date: Monday, May 11, 2020 at 11:53 AM > To: Daniil Titov , serviceability-dev > Subject: Re: RFR: 8241080: Consolidate signature parsing code in serviceability tools > > Hi Daniil, > > It looks pretty good in general. > A couple of nits below. > > http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/invoker.c.udiff.html > + void *cursor; > + jbyte argumentTag; > + jint argIndex = 0; > + jvalue *argument = request->arguments;; > . . . > void *cursor; > jint argIndex = 0; > + jbyte argumentTag; > jvalue *argument = request->arguments; > > It is better if the local variables 'cursor' and 'argumentTag' get initialized. > There is double semicolon: "arguments;;" > > > http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/signature.h.html > 43 static inline jbyte basicType(const char *signature) { > > It'd be nice to rename it to basicTypeTag() to get it unified with other functions below. > > It is more safe to run tier5 as well. > > Thanks, > Serguei > > > On 5/9/20 09:29, Daniil Titov wrote: > Please review a change[1] that centralizes the signature processing in serviceability tools to make it capable of being easily extensible in the future. > > Testing: Mach5 tier1-tier3 tests successfully passed. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.01 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > > Thank you, > Daniil > > > > > > From gnu.andrew at redhat.com Tue May 19 01:26:37 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Tue, 19 May 2020 02:26:37 +0100 Subject: [8u] RFR: 8150986: serviceability/sa/jmap-hprof/JMapHProfLargeHeapTest.java failing because expects HPROF JAVA PROFILE 1.0.1 file format In-Reply-To: <7afbfa4c0a30f76a424f0e40c6c80d015fb7e91b.camel@redhat.com> References: <7afbfa4c0a30f76a424f0e40c6c80d015fb7e91b.camel@redhat.com> Message-ID: <20a1d199-eb7d-1451-f222-3513b30d0639@redhat.com> On 18/05/2020 10:08, Severin Gehwolf wrote: > Hi, > > Please review this 8u backport of JDK-8150986. It's a clean backport > except for the copyright change. In the 8u252 cycle JDK-8144732 got > backported which now makes the test fail. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8150986 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8150986/01/webrev/ > > Testing: Test fails prior patch, passes after. > > Thoughts? > > Thanks, > Severin > Looks fine. Thanks. -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: From david.holmes at oracle.com Tue May 19 05:26:38 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 May 2020 15:26:38 +1000 Subject: RFR: JDK-8225056 VM support for sealed classes In-Reply-To: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> References: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> Message-ID: Hi Harold, Adding serviceability-dev for the serviceability related changes. Nit: "VM support for sealed classes" This RFR covers the VM, core-libs, serviceability and even some langtools tests. AFAICS only the javac changes are not included here so when and where will they be reviewed and under what bug id? Ideally there will be a single JBS issue for "Implementation of JEP 360: Sealed types". It's okay to break up the RFRs across different areas, but it should be done under one bug id. Overall this looks good. I've looked at all files and mainly have some style nits in various places. But there are some more significant items below. On 14/05/2020 7:09 am, Harold Seigel wrote: > Hi, > > Please review this patch for JVM and Core-libs support for the JEP 360 > Sealed Classes preview feature.? Code changes include the following: > > ?* Processing of the new PermittedSubclasses attribute to enforce that > ?? a class or interface, whose super is a sealed class or interface, > ?? must be listed in the super's PermittedSubclasses attribute. > ?* Disallow redefinition of a sealed class or interface if its > ?? redefinition would change its PermittedSubclasses attribute. > ?* Support API's to determine if a class or interface is sealed and, if > ?? it's sealed, return a list of its permitted subclasses. > ?* asm support for the PermittedSubclasses attribute I assume Remi is providing the upstream support in ASM? :) But also see below ... > > Open Webrev: > http://cr.openjdk.java.net/~vromero/8225056/webrev.00/index.html make/data/jdwp/jdwp.spec There needs to be a sub-task and associated CSR request for this JDWP spec update. I couldn't see this covered anywhere. --- src/hotspot/share/classfile/classFileParser.cpp 3215 u2 ClassFileParser::parse_classfile_permitted_subclasses_attribute(const ClassFileStream* const cfs, 3216 const u1* const permitted_subclasses_attribute_start, 3217 TRAPS) { Indention on L3216/17 needs fixing after the copy'n'edit. 3515 return _major_version == JVM_CLASSFILE_MAJOR_VERSION && 3516 _minor_version == JAVA_PREVIEW_MINOR_VERSION && 3517 Arguments::enable_preview(); Too much indentation on L3516/17 3790 // Check for PermittedSubclasses tag That comment (copied from my nestmates code :) is in the wrong place. It needs to be before 3788 if (tag == vmSymbols::tag_permitted_subclasses()) { Minor nit: I would suggest checking parsed_permitted_subclasses_attribute before checking ACC_FINAL. 3876 if (parsed_permitted_subclasses_attribute) { 3877 const u2 num_of_subclasses = parse_classfile_permitted_subclasses_attribute( 3878 cfs, 3879 permitted_subclasses_attribute_start, 3880 CHECK); Although it looks odd the preceding, similarly shaped, sections all indent to the same absolute position. Can you make L3878/78/80 match please. 3882 guarantee_property( 3883 permitted_subclasses_attribute_length == 3884 sizeof(num_of_subclasses) + sizeof(u2) * num_of_subclasses, 3885 "Wrong PermittedSubclasses attribute length in class file %s", CHECK); Nits: please reformat as: 3882 guarantee_property( 3883 permitted_subclasses_attribute_length == sizeof(num_of_subclasses) + sizeof(u2) * num_of_subclasses, 3885 "Wrong PermittedSubclasses attribute length in class file %s", CHECK); It would also look slightly better if you shortened the name of the num_of_subclasses variable. --- src/hotspot/share/classfile/classFileParser.hpp + u2 parse_classfile_permitted_subclasses_attribute(const ClassFileStream* const cfs, + const u1* const permitted_subclasses_attribute_start, + TRAPS); Please fix indentation after copy'n'edit. --- src/hotspot/share/oops/instanceKlass.cpp 247 if (classloader1 != classloader2) { I'm not clear what rule this is verifying. The same module check follows this one. The rule is that the subclass must be accessible to the superclass implying: 1. same named module (regardless of class access modifiers); or 2. (implicitly in un-named module) same package if subclass not public; or 3. public subclass Having the same classloader implies same package, but that alone doesn't address 2 or 3. So this doesn't conform to proposed JVMS rules. 264 if (_constants->tag_at(cp_index).is_klass()) { 265 Klass* k2 = _constants->klass_at(cp_index, CHECK_false); You've copied this code from the nestmember checks but your changes don't quite make sense to me. If we have already checked is_klass() then klass_at() cannot lead to any exceptions. 272 if (name == k->name()) { 273 log_trace(class, sealed)("- Found it at permitted_subclasses[%d] => cp[%d]", i, cp_index); 274 return true; I was wondering why you don't resolve the cp entry when you find the name matches, as we do for nest members, but realized that unlike the nest membership check, which can happen many times for a given class, this permitted subclass check can only happen once per class. As you don't actually resolve here, and given that the earlier check cannot throw exceptions, it follows that the entire method never results in any exceptions and so callers should not be using the CHECK macro. --- src/hotspot/share/oops/method.cpp I don't understand how knowing the class is sealed allows you to infer that a non-final method is actually final ?? --- src/hotspot/share/prims/jvm.cpp It would be simpler (and cheaper) if the Java side of this ensures it doesn't call into the VM with an array or primitive class. --- src/hotspot/share/prims/jvmti.xml The JVM TI spec changes also need to be covered by a CSR request. --- src/hotspot/share/prims/jvmtiRedefineClasses.cpp We should file a RFE to refactor the logic that checks that an attribute consisting of a list of classes has not changed. :) Aside: I spotted a bug in the nest member code (missing NULL check!) thanks to your change :) --- src/java.base/share/classes/java/lang/Class.java There needs to be a CSR request for these changes. + * Returns an array containing {@code ClassDesc} objects representing all the + * permitted subclasses of this {@linkplain Class} if it is sealed. Returns an empty array if this + * {@linkplain Class} is not sealed. should add "or this class represents an array or primitive type" (using the standard wording for such cases). + * @throws IllegalArgumentException if a class descriptor is not in the correct format IllegalArgumentException is not an appropriate exception to use as this method takes no arguments. If the class descriptor is not valid and it comes from the VM then I think we have a problem with how the VM validates class descriptors. Any IAE from ClassDesc.of should be caught and converted to a more suitable exception type - preferably InternalError if the VM should always return valid strings. + public ClassDesc[] getPermittedSubclasses() { As mentioned for jvm.cpp this Java code should do the isArray() and isPrimitive() check before calling the VM. + String[] descriptors = getPermittedSubclasses0(); Nit: what you get from the VM are not descriptors, just name strings in internal form. This wouldn't really matter except it then looks strange to call ClassDesc.of(...) instead of ClassDesc.ofDescriptor(...). + if (descriptors == null The VM never returns null. + return getPermittedSubclasses().length != 0; It's grossly inefficient to create the ClassDesc array and then throw it away IMO. The result should be cached either in a field of Class or in the ReflectionData of the class. --- src/java.base/share/classes/jdk/internal/org/objectweb/asm/ClassReader.java ! // - The offset of the PermittedSubclasses attribute, or 0 int permittedSubtypesOffset = 0; Obviously ASM already has some prelim support for sealed classes, but now that the attribute has been renamed that should also flow through to the ASM code ie the variable, not just the comment. Ditto for ClassWriter.java and its fields. --- src/java.base/share/native/libjava/Class.c {"isRecord0", "()Z", (void *)&JVM_IsRecord}, + {"getPermittedSubclasses0", "()[" STR, (void *)&JVM_GetPermittedSubclasses}, please align (void --- src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java There needs to be a CSR for these changes too. --- test/langtools/tools/javac/processing/model/TestSourceVersion.java ! // Assume "record" and "sealed" will be restricted keywords. ! "record", "sealed"); What about the non-sealed keyword defined in the JEP? --- In the tests you don't need to explicitly include sun.hotspot.WhiteBox$WhiteBoxPermission on the ClassFileInstaller invocation. (previous RFE's have been removing existing occurrences after the CFI was fixed to handle it internally). Please ensure all new tests have an @bug 8225056 (or whatever the actual JBS issue will be) All test classes (and thus files) should be named in camel-case i.e. C1 not c1, C2 not c2, SuperClass not superClass etc. test/hotspot/jtreg/runtime/modules/sealedP1/superClass.jcod test/hotspot/jtreg/runtime/sealedClasses/Pkg/sealedInterface.jcod Please add comments clarifying why these must be jcod files. --- That's it from me. Thanks, David ----- > JBS bug: https://bugs.openjdk.java.net/browse/JDK-8225056 > > Java Language Spec changes: > http://cr.openjdk.java.net/~gbierman/jep360/jep360-20200513/specs/sealed-classes-jls.html > > > JVM Spec changes: > http://cr.openjdk.java.net/~gbierman/jep360/jep360-20200513/specs/sealed-classes-jvms.html > > > JEP 360: https://bugs.openjdk.java.net/browse/JDK-8227043 > > JVM CSR: https://bugs.openjdk.java.net/browse/JDK-8242578 > > Changes to javac and other language tools will be reviewed separately. > > Thanks, Harold > > From forax at univ-mlv.fr Tue May 19 08:41:54 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 19 May 2020 10:41:54 +0200 (CEST) Subject: RFR: JDK-8225056 VM support for sealed classes In-Reply-To: References: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> Message-ID: <1968935181.766173.1589877714107.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "David Holmes" > ?: "Harold David Seigel" , "hotspot-runtime-dev" , > "amber-dev" , "core-libs-dev" , "serviceability-dev" > > Envoy?: Mardi 19 Mai 2020 07:26:38 > Objet: Re: RFR: JDK-8225056 VM support for sealed classes > Hi Harold, > > Adding serviceability-dev for the serviceability related changes. > > Nit: "VM support for sealed classes" > > This RFR covers the VM, core-libs, serviceability and even some > langtools tests. AFAICS only the javac changes are not included here so > when and where will they be reviewed and under what bug id? Ideally > there will be a single JBS issue for "Implementation of JEP 360: Sealed > types". It's okay to break up the RFRs across different areas, but it > should be done under one bug id. > > Overall this looks good. I've looked at all files and mainly have some > style nits in various places. But there are some more significant items > below. > > On 14/05/2020 7:09 am, Harold Seigel wrote: >> Hi, >> >> Please review this patch for JVM and Core-libs support for the JEP 360 >> Sealed Classes preview feature.? Code changes include the following: >> >> ?* Processing of the new PermittedSubclasses attribute to enforce that >> ?? a class or interface, whose super is a sealed class or interface, >> ?? must be listed in the super's PermittedSubclasses attribute. >> ?* Disallow redefinition of a sealed class or interface if its >> ?? redefinition would change its PermittedSubclasses attribute. >> ?* Support API's to determine if a class or interface is sealed and, if >> ?? it's sealed, return a list of its permitted subclasses. >> ?* asm support for the PermittedSubclasses attribute > > I assume Remi is providing the upstream support in ASM? :) But also see > below ... Currently the version of ASM used JDK already supports sealed classes but with the attribute named PermittedSubtypes instead of PermittedSubclasses. This patch renames the attribute which is the right thing to do. Then when JDK 15 will be released, we will release ASM 9 with full support of PermittedSubclasses, with the right method names, docs, etc, that will be then integrated in JDK 16. So with my ASM hat, the changes to the 3 ASM classes are good. R?mi From serguei.spitsyn at oracle.com Tue May 19 15:49:22 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 May 2020 08:49:22 -0700 Subject: RFR: JDK-8225056 VM support for sealed classes In-Reply-To: References: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue May 19 15:53:54 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 May 2020 08:53:54 -0700 Subject: RFR: JDK-8225056 VM support for sealed classes In-Reply-To: References: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> Message-ID: Hi Harold, The Serviceability part including the tests looks good to me. I can file a JVMTI enhancement on the jvmtiRedefineClasses.cpp refactoring if you are okay with it. Thanks, Serguei On 5/18/20 22:26, David Holmes wrote: > Hi Harold, > > Adding serviceability-dev for the serviceability related changes. > > Nit: "VM support for sealed classes" > > This RFR covers the VM, core-libs, serviceability and even some > langtools tests. AFAICS only the javac changes are not included here > so when and where will they be reviewed and under what bug id? Ideally > there will be a single JBS issue for "Implementation of JEP 360: > Sealed types". It's okay to break up the RFRs across different areas, > but it should be done under one bug id. > > Overall this looks good. I've looked at all files and mainly have some > style nits in various places. But there are some more significant > items below. > > On 14/05/2020 7:09 am, Harold Seigel wrote: >> Hi, >> >> Please review this patch for JVM and Core-libs support for the JEP >> 360 Sealed Classes preview feature.? Code changes include the following: >> >> ??* Processing of the new PermittedSubclasses attribute to enforce that >> ??? a class or interface, whose super is a sealed class or interface, >> ??? must be listed in the super's PermittedSubclasses attribute. >> ??* Disallow redefinition of a sealed class or interface if its >> ??? redefinition would change its PermittedSubclasses attribute. >> ??* Support API's to determine if a class or interface is sealed and, if >> ??? it's sealed, return a list of its permitted subclasses. >> ??* asm support for the PermittedSubclasses attribute > > I assume Remi is providing the upstream support in ASM? :) But also > see below ... > >> >> Open Webrev: >> http://cr.openjdk.java.net/~vromero/8225056/webrev.00/index.html > > make/data/jdwp/jdwp.spec > > There needs to be a sub-task and associated CSR request for this JDWP > spec update. I couldn't see this covered anywhere. > > --- > > src/hotspot/share/classfile/classFileParser.cpp > > 3215 u2 > ClassFileParser::parse_classfile_permitted_subclasses_attribute(const > ClassFileStream* const cfs, > 3216 const u1* const permitted_subclasses_attribute_start, > 3217 TRAPS) { > > Indention on L3216/17 needs fixing after the copy'n'edit. > > 3515?? return _major_version == JVM_CLASSFILE_MAJOR_VERSION && > 3516????????????? _minor_version == JAVA_PREVIEW_MINOR_VERSION && > 3517????????????? Arguments::enable_preview(); > > Too much indentation on L3516/17 > > 3790???????????????? // Check for PermittedSubclasses tag > > That comment (copied from my nestmates code :) is in the wrong place. > It needs to be before > > 3788???????????? if (tag == vmSymbols::tag_permitted_subclasses()) { > > > Minor nit: I would suggest checking > parsed_permitted_subclasses_attribute before checking ACC_FINAL. > > 3876?? if (parsed_permitted_subclasses_attribute) { > 3877???? const u2 num_of_subclasses = > parse_classfile_permitted_subclasses_attribute( > 3878??????????????????????????????????? cfs, > 3879 permitted_subclasses_attribute_start, > 3880??????????????????????????????????? CHECK); > > Although it looks odd the preceding, similarly shaped, sections all > indent to the same absolute position. Can you make L3878/78/80 match > please. > > 3882?????? guarantee_property( > 3883???????? permitted_subclasses_attribute_length == > 3884?????????? sizeof(num_of_subclasses) + sizeof(u2) * > num_of_subclasses, > 3885???????? "Wrong PermittedSubclasses attribute length in class file > %s", CHECK); > > Nits: please reformat as: > > 3882?????? guarantee_property( > 3883???????? permitted_subclasses_attribute_length == > sizeof(num_of_subclasses) + sizeof(u2) * num_of_subclasses, > 3885???????? "Wrong PermittedSubclasses attribute length in class file > %s", CHECK); > > It would also look slightly better if you shortened the name of the > num_of_subclasses variable. > > --- > > src/hotspot/share/classfile/classFileParser.hpp > > +?? u2 parse_classfile_permitted_subclasses_attribute(const > ClassFileStream* const cfs, > +???????????????????????????????????????????? const u1* const > permitted_subclasses_attribute_start, > +???????????????????????????????????????????? TRAPS); > > Please fix indentation after copy'n'edit. > > --- > > src/hotspot/share/oops/instanceKlass.cpp > > ?247?? if (classloader1 != classloader2) { > > I'm not clear what rule this is verifying. The same module check > follows this one. The rule is that the subclass must be accessible to > the superclass implying: > 1. same named module (regardless of class access modifiers); or > 2. (implicitly in un-named module) same package if subclass not > public; or > 3. public subclass > > Having the same classloader implies same package, but that alone > doesn't address 2 or 3. So this doesn't conform to proposed JVMS rules. > > > ?264???? if (_constants->tag_at(cp_index).is_klass()) { > ?265?????? Klass* k2 = _constants->klass_at(cp_index, CHECK_false); > > You've copied this code from the nestmember checks but your changes > don't quite make sense to me. If we have already checked is_klass() > then klass_at() cannot lead to any exceptions. > > ?272?????? if (name == k->name()) { > ?273???????? log_trace(class, sealed)("- Found it at > permitted_subclasses[%d] => cp[%d]", i, cp_index); > ?274???????? return true; > > I was wondering why you don't resolve the cp entry when you find the > name matches, as we do for nest members, but realized that unlike the > nest membership check, which can happen many times for a given class, > this permitted subclass check can only happen once per class. As you > don't actually resolve here, and given that the earlier check cannot > throw exceptions, it follows that the entire method never results in > any exceptions and so callers should not be using the CHECK macro. > > --- > > src/hotspot/share/oops/method.cpp > > I don't understand how knowing the class is sealed allows you to infer > that a non-final method is actually final ?? > > --- > > ?src/hotspot/share/prims/jvm.cpp > > It would be simpler (and cheaper) if the Java side of this ensures it > doesn't call into the VM with an array or primitive class. > > --- > > ?src/hotspot/share/prims/jvmti.xml > > The JVM TI spec changes also need to be covered by a CSR request. > > --- > > src/hotspot/share/prims/jvmtiRedefineClasses.cpp > > We should file a RFE to refactor the logic that checks that an > attribute consisting of a list of classes has not changed. :) > > Aside: I spotted a bug in the nest member code (missing NULL check!) > thanks to your change :) > > --- > > src/java.base/share/classes/java/lang/Class.java > > There needs to be a CSR request for these changes. > > +????? * Returns an array containing {@code ClassDesc} objects > representing all the > +????? * permitted subclasses of this {@linkplain Class} if it is > sealed. Returns an empty array if this > +????? * {@linkplain Class} is not sealed. > > should add "or this class represents an array or primitive type" > (using the standard wording for such cases). > > +????? * @throws IllegalArgumentException if a class descriptor is not > in the correct format > > IllegalArgumentException is not an appropriate exception to use as > this method takes no arguments. If the class descriptor is not valid > and it comes from the VM then I think we have a problem with how the > VM validates class descriptors. Any IAE from ClassDesc.of should be > caught and converted to a more suitable exception type - preferably > InternalError if the VM should always return valid strings. > > +???? public ClassDesc[] getPermittedSubclasses() { > > As mentioned for jvm.cpp this Java code should do the isArray() and > isPrimitive() check before calling the VM. > > +???????? String[] descriptors = getPermittedSubclasses0(); > > Nit: what you get from the VM are not descriptors, just name strings > in internal form. This wouldn't really matter except it then looks > strange to call ClassDesc.of(...) instead of ClassDesc.ofDescriptor(...). > > +???????? if (descriptors == null > > The VM never returns null. > > +???????? return getPermittedSubclasses().length != 0; > > It's grossly inefficient to create the ClassDesc array and then throw > it away IMO. The result should be cached either in a field of Class or > in the ReflectionData of the class. > > --- > > src/java.base/share/classes/jdk/internal/org/objectweb/asm/ClassReader.java > > > !???????? // - The offset of the PermittedSubclasses attribute, or 0 > ????????? int permittedSubtypesOffset = 0; > > Obviously ASM already has some prelim support for sealed classes, but > now that the attribute has been renamed that should also flow through > to the ASM code ie the variable, not just the comment. > > Ditto for ClassWriter.java and its fields. > > --- > > src/java.base/share/native/libjava/Class.c > > ????? {"isRecord0",??????????? "()Z",???????? (void *)&JVM_IsRecord}, > +???? {"getPermittedSubclasses0", "()[" STR,??? (void > *)&JVM_GetPermittedSubclasses}, > > please align (void > > --- > > src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java > > src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java > > There needs to be a CSR for these changes too. > > --- > > test/langtools/tools/javac/processing/model/TestSourceVersion.java > > !??????????????????? // Assume "record" and "sealed" will be > restricted keywords. > !??????????????????? "record", "sealed"); > > What about the non-sealed keyword defined in the JEP? > > --- > > In the tests you don't need to explicitly include > sun.hotspot.WhiteBox$WhiteBoxPermission on the ClassFileInstaller > invocation. (previous RFE's have been removing existing occurrences > after the CFI was fixed to handle it internally). > > Please ensure all new tests have an @bug 8225056 (or whatever the > actual JBS issue will be) > > All test classes (and thus files) should be named in camel-case i.e. > C1 not c1, C2 not c2, SuperClass not superClass etc. > > > test/hotspot/jtreg/runtime/modules/sealedP1/superClass.jcod > test/hotspot/jtreg/runtime/sealedClasses/Pkg/sealedInterface.jcod > > Please add comments clarifying why these must be jcod files. > > --- > > That's it from me. > > Thanks, > David > ----- > > > >> JBS bug: https://bugs.openjdk.java.net/browse/JDK-8225056 >> >> Java Language Spec changes: >> http://cr.openjdk.java.net/~gbierman/jep360/jep360-20200513/specs/sealed-classes-jls.html >> >> >> JVM Spec changes: >> http://cr.openjdk.java.net/~gbierman/jep360/jep360-20200513/specs/sealed-classes-jvms.html >> >> >> JEP 360: https://bugs.openjdk.java.net/browse/JDK-8227043 >> >> JVM CSR: https://bugs.openjdk.java.net/browse/JDK-8242578 >> >> Changes to javac and other language tools will be reviewed separately. >> >> Thanks, Harold >> >> From serguei.spitsyn at oracle.com Tue May 19 16:28:59 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 May 2020 09:28:59 -0700 Subject: RFR (XS): 8244571: assert(!_thread->is_pending_jni_exception_check()) failed: Pending JNI Exception Check during class loading Message-ID: <994efe77-fad2-66ae-7546-4d4a184803a7@oracle.com> An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Tue May 19 16:32:27 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 19 May 2020 09:32:27 -0700 Subject: RFR: 8241080: Consolidate signature parsing code in serviceability tools In-Reply-To: References: <7B3DCB8E-B61E-4D9B-BEDC-F69A414818CA@oracle.com> Message-ID: Hi Chris and Serguei, Thank you for reviewing this change. Best regards, Daniil ?On 5/18/20, 12:41 PM, "Chris Plummer" wrote: Looks good. thanks, Chris On 5/14/20 1:33 PM, Daniil Titov wrote: > Hi Serguei and Chris, > > Please review a new version of the change [1] that addresses your comments. > > Testing: Mach5 tier1-tier5 tests successfully passed. > > Regarding CR for the JDWP spec issues related to missing type information in some of the protocol packet descriptions [3], as Chris has just noticed > we really don't need it, so I withdrew this CR. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.02 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > [3] https://bugs.openjdk.java.net/browse/JDK-8245057 > > Thank you, > Daniil > > > From: "serguei.spitsyn at oracle.com" > Date: Monday, May 11, 2020 at 11:53 AM > To: Daniil Titov , serviceability-dev > Subject: Re: RFR: 8241080: Consolidate signature parsing code in serviceability tools > > Hi Daniil, > > It looks pretty good in general. > A couple of nits below. > > http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/invoker.c.udiff.html > + void *cursor; > + jbyte argumentTag; > + jint argIndex = 0; > + jvalue *argument = request->arguments;; > . . . > void *cursor; > jint argIndex = 0; > + jbyte argumentTag; > jvalue *argument = request->arguments; > > It is better if the local variables 'cursor' and 'argumentTag' get initialized. > There is double semicolon: "arguments;;" > > > http://cr.openjdk.java.net/~dtitov/8241080/webrev.01/src/jdk.jdwp.agent/share/native/libjdwp/signature.h.html > 43 static inline jbyte basicType(const char *signature) { > > It'd be nice to rename it to basicTypeTag() to get it unified with other functions below. > > It is more safe to run tier5 as well. > > Thanks, > Serguei > > > On 5/9/20 09:29, Daniil Titov wrote: > Please review a change[1] that centralizes the signature processing in serviceability tools to make it capable of being easily extensible in the future. > > Testing: Mach5 tier1-tier3 tests successfully passed. > > [1] http://cr.openjdk.java.net/~dtitov/8241080/webrev.01 > [2] https://bugs.openjdk.java.net/browse/JDK-8241080 > > Thank you, > Daniil > > > > > > From harold.seigel at oracle.com Tue May 19 16:46:21 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 19 May 2020 12:46:21 -0400 Subject: RFR: JDK-8225056 VM support for sealed classes In-Reply-To: References: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> Message-ID: I think that's a good idea. Thanks, Harold On 5/19/2020 11:49 AM, serguei.spitsyn at oracle.com wrote: > Hi Harold and David, > > Just one comment. > We could save on the CSR's for: > ? make/data/jdwp/jdwp.spec > || src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java > || > src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java > > As we discussed with David, it could make sense to remove duplication > in the Serviceability class redefinition and retransformation specs. > I'd suggest something like this webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2020/redef-spec-dedup.1/ > > If the direction looks okay then I could file RFE+CSR (and post an RFR). > > Thanks, > Serguei > > > On 5/18/20 22:26, David Holmes wrote: >> Hi Harold, >> >> Adding serviceability-dev for the serviceability related changes. >> >> Nit: "VM support for sealed classes" >> >> This RFR covers the VM, core-libs, serviceability and even some >> langtools tests. AFAICS only the javac changes are not included here >> so when and where will they be reviewed and under what bug id? >> Ideally there will be a single JBS issue for "Implementation of JEP >> 360: Sealed types". It's okay to break up the RFRs across different >> areas, but it should be done under one bug id. >> >> Overall this looks good. I've looked at all files and mainly have >> some style nits in various places. But there are some more >> significant items below. >> >> On 14/05/2020 7:09 am, Harold Seigel wrote: >>> Hi, >>> >>> Please review this patch for JVM and Core-libs support for the JEP >>> 360 Sealed Classes preview feature.? Code changes include the >>> following: >>> >>> ??* Processing of the new PermittedSubclasses attribute to enforce that >>> ??? a class or interface, whose super is a sealed class or interface, >>> ??? must be listed in the super's PermittedSubclasses attribute. >>> ??* Disallow redefinition of a sealed class or interface if its >>> ??? redefinition would change its PermittedSubclasses attribute. >>> ??* Support API's to determine if a class or interface is sealed >>> and, if >>> ??? it's sealed, return a list of its permitted subclasses. >>> ??* asm support for the PermittedSubclasses attribute >> >> I assume Remi is providing the upstream support in ASM? :) But also >> see below ... >> >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~vromero/8225056/webrev.00/index.html >> >> make/data/jdwp/jdwp.spec >> >> There needs to be a sub-task and associated CSR request for this JDWP >> spec update. I couldn't see this covered anywhere. >> >> --- >> >> src/hotspot/share/classfile/classFileParser.cpp >> >> 3215 u2 >> ClassFileParser::parse_classfile_permitted_subclasses_attribute(const >> ClassFileStream* const cfs, >> 3216 const u1* const permitted_subclasses_attribute_start, >> 3217 TRAPS) { >> >> Indention on L3216/17 needs fixing after the copy'n'edit. >> >> 3515?? return _major_version == JVM_CLASSFILE_MAJOR_VERSION && >> 3516????????????? _minor_version == JAVA_PREVIEW_MINOR_VERSION && >> 3517????????????? Arguments::enable_preview(); >> >> Too much indentation on L3516/17 >> >> 3790???????????????? // Check for PermittedSubclasses tag >> >> That comment (copied from my nestmates code :) is in the wrong place. >> It needs to be before >> >> 3788???????????? if (tag == vmSymbols::tag_permitted_subclasses()) { >> >> >> Minor nit: I would suggest checking >> parsed_permitted_subclasses_attribute before checking ACC_FINAL. >> >> 3876?? if (parsed_permitted_subclasses_attribute) { >> 3877???? const u2 num_of_subclasses = >> parse_classfile_permitted_subclasses_attribute( >> 3878??????????????????????????????????? cfs, >> 3879 permitted_subclasses_attribute_start, >> 3880??????????????????????????????????? CHECK); >> >> Although it looks odd the preceding, similarly shaped, sections all >> indent to the same absolute position. Can you make L3878/78/80 match >> please. >> >> 3882?????? guarantee_property( >> 3883???????? permitted_subclasses_attribute_length == >> 3884?????????? sizeof(num_of_subclasses) + sizeof(u2) * >> num_of_subclasses, >> 3885???????? "Wrong PermittedSubclasses attribute length in class >> file %s", CHECK); >> >> Nits: please reformat as: >> >> 3882?????? guarantee_property( >> 3883???????? permitted_subclasses_attribute_length == >> sizeof(num_of_subclasses) + sizeof(u2) * num_of_subclasses, >> 3885???????? "Wrong PermittedSubclasses attribute length in class >> file %s", CHECK); >> >> It would also look slightly better if you shortened the name of the >> num_of_subclasses variable. >> >> --- >> >> src/hotspot/share/classfile/classFileParser.hpp >> >> +?? u2 parse_classfile_permitted_subclasses_attribute(const >> ClassFileStream* const cfs, >> +???????????????????????????????????????????? const u1* const >> permitted_subclasses_attribute_start, >> +???????????????????????????????????????????? TRAPS); >> >> Please fix indentation after copy'n'edit. >> >> --- >> >> src/hotspot/share/oops/instanceKlass.cpp >> >> ?247?? if (classloader1 != classloader2) { >> >> I'm not clear what rule this is verifying. The same module check >> follows this one. The rule is that the subclass must be accessible to >> the superclass implying: >> 1. same named module (regardless of class access modifiers); or >> 2. (implicitly in un-named module) same package if subclass not >> public; or >> 3. public subclass >> >> Having the same classloader implies same package, but that alone >> doesn't address 2 or 3. So this doesn't conform to proposed JVMS rules. >> >> >> ?264???? if (_constants->tag_at(cp_index).is_klass()) { >> ?265?????? Klass* k2 = _constants->klass_at(cp_index, CHECK_false); >> >> You've copied this code from the nestmember checks but your changes >> don't quite make sense to me. If we have already checked is_klass() >> then klass_at() cannot lead to any exceptions. >> >> ?272?????? if (name == k->name()) { >> ?273???????? log_trace(class, sealed)("- Found it at >> permitted_subclasses[%d] => cp[%d]", i, cp_index); >> ?274???????? return true; >> >> I was wondering why you don't resolve the cp entry when you find the >> name matches, as we do for nest members, but realized that unlike the >> nest membership check, which can happen many times for a given class, >> this permitted subclass check can only happen once per class. As you >> don't actually resolve here, and given that the earlier check cannot >> throw exceptions, it follows that the entire method never results in >> any exceptions and so callers should not be using the CHECK macro. >> >> --- >> >> src/hotspot/share/oops/method.cpp >> >> I don't understand how knowing the class is sealed allows you to >> infer that a non-final method is actually final ?? >> >> --- >> >> ?src/hotspot/share/prims/jvm.cpp >> >> It would be simpler (and cheaper) if the Java side of this ensures it >> doesn't call into the VM with an array or primitive class. >> >> --- >> >> ?src/hotspot/share/prims/jvmti.xml >> >> The JVM TI spec changes also need to be covered by a CSR request. >> >> --- >> >> src/hotspot/share/prims/jvmtiRedefineClasses.cpp >> >> We should file a RFE to refactor the logic that checks that an >> attribute consisting of a list of classes has not changed. :) >> >> Aside: I spotted a bug in the nest member code (missing NULL check!) >> thanks to your change :) >> >> --- >> >> src/java.base/share/classes/java/lang/Class.java >> >> There needs to be a CSR request for these changes. >> >> +????? * Returns an array containing {@code ClassDesc} objects >> representing all the >> +????? * permitted subclasses of this {@linkplain Class} if it is >> sealed. Returns an empty array if this >> +????? * {@linkplain Class} is not sealed. >> >> should add "or this class represents an array or primitive type" >> (using the standard wording for such cases). >> >> +????? * @throws IllegalArgumentException if a class descriptor is >> not in the correct format >> >> IllegalArgumentException is not an appropriate exception to use as >> this method takes no arguments. If the class descriptor is not valid >> and it comes from the VM then I think we have a problem with how the >> VM validates class descriptors. Any IAE from ClassDesc.of should be >> caught and converted to a more suitable exception type - preferably >> InternalError if the VM should always return valid strings. >> >> +???? public ClassDesc[] getPermittedSubclasses() { >> >> As mentioned for jvm.cpp this Java code should do the isArray() and >> isPrimitive() check before calling the VM. >> >> +???????? String[] descriptors = getPermittedSubclasses0(); >> >> Nit: what you get from the VM are not descriptors, just name strings >> in internal form. This wouldn't really matter except it then looks >> strange to call ClassDesc.of(...) instead of >> ClassDesc.ofDescriptor(...). >> >> +???????? if (descriptors == null >> >> The VM never returns null. >> >> +???????? return getPermittedSubclasses().length != 0; >> >> It's grossly inefficient to create the ClassDesc array and then throw >> it away IMO. The result should be cached either in a field of Class >> or in the ReflectionData of the class. >> >> --- >> >> src/java.base/share/classes/jdk/internal/org/objectweb/asm/ClassRea