From thomas.stuefe at gmail.com Sun Apr 2 11:26:49 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 2 Apr 2017 13:26:49 +0200 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: References: Message-ID: Ping... nobody? Kind Regards, Thomas On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe wrote: > Hi, > > https://bugs.openjdk.java.net/browse/JDK-8170520 > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2- > 8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/ > webrev/ > > may I please get a review of another small cleanup change to the > metaspace. Compared with the last one (JDK-8170933), this one is smaller. > > I posted this originally for jdk 9 last november, and it got reviewed > already: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 > 016-November/021946.html. Mikael found a small bug and by then it was too > late to bring this into jdk9, so I postponed the patch for jdk10. > > Now the patch got rebased to Jdk10, and it is also quite a bit simpler > because it meshes well with the cleanup work done on JDK-8170933. > > The change replaces the calls to Atomic::inc/dec for the ChunkManager > counters with simple +/-, because all code paths which modify the > ChunkManager counters are under lock protection (SpaceManager::expand_lock()). > This makes updating these counters cheep and thus removes the need to be > frugal with the number of updates. > > Before the patch - when a list of chunks was returned to a ChunkManager in > ~SpaceManager() - the increment values were updated once, with just one > call to ChunkManager::inc_free_chunks_total(). This left a time window in > which the counters did not reflect reality, so one had to be really careful > where to place asserts to check the ChunkManager state. That made modifying > and playing with the code error prone. > > Since JDK-8170933, chunks are returned to the ChunkManager via one common > path, which always ends in ChunkManager::return_single_chunk(). Because > of that and because updating the counters is now cheap, I moved the several > invocations of ChunkManager::inc_free_chunks_total() to > ChunkManager::return_single_chunk(). > > The rest of the changes is cosmetical: > - Moved ChunkManager::inc_free_chunks_total() up to the private section > of the class, because it is not called from outside anymore > - renamed arguments for ChunkManager::inc_free_chunks_total() and > ChunkManager::dec_free_chunks_total() to be clearer named, and gave both > of them the same arguments > - Added an assert to both function asserting that the increment/decrement > word size value should be a multiple of the smallest chunk size > > I ran gtests and jtreg tests on Linux and AIX, no issues popped up. > > Thank you for reviewing, > > Thomas > > > > > > > > > From igor.ignatyev at oracle.com Sun Apr 2 17:09:08 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sun, 2 Apr 2017 10:09:08 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: <58DBF832.2010902@oracle.com> Message-ID: <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> Hi, http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next iteration w/ copyright years fixed, LineNumberOnBraceTarg and LambdaBreakpointTest made more unified w/ the rest. Thanks, -- Igor > On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com wrote: > > This one also does not look unified: > http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html > On 3/29/17 11:08, Mikhailo Seledtsov wrote: > > One style nit: > > LineNumberOnBraceTarg: > All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) for line number variables, but this test uses 'stopLine'. > Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. > On Mar 28, 2017, at 6:37 PM, David Holmes wrote: > > Two nits: > - test/com/sun/jdi/FetchLocals.java > - test/com/sun/jdi/LambdaBreakpointTest.java > > Second copyright year should be 2017. > On Mar 24, 2017, at 1:56 PM, Igor Ignatyev wrote: > Hi all, > > could you please review this fix for 8177507? > > due to their nature, some of jdi tests are line number sensitive. unfortunately different tests indicate that differently, so it's quite easy to overlook that and incidentally break tests, for example by changing module dependency declaration or license modification. this fix unifies the way line number sensitivity is indicated and also improves readability/maintainability of some tests by using constant fields instead of magic numbers. > > some of line number sensitive tests have been unexpectedly removed from execution because they had @test/nodynamiccopyright/ instead of @test tag. this changeset fixes and returns them to regular execution. > > webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 > testing: test/com/sun/jdi From david.holmes at oracle.com Sun Apr 2 20:31:34 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Apr 2017 06:31:34 +1000 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> Message-ID: <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> Hi Igor, On 3/04/2017 3:09 AM, Igor Ignatyev wrote: > Hi, > > http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next > iteration w/ copyright years fixed, test/com/sun/jdi/BreakpointTest.java also needs fixing - says 2015. > LineNumberOnBraceTarg and test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. > LambdaBreakpointTest made more unified w/ the rest. I don't see any change here. Thanks, David ----- > Thanks, > -- Igor > >> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >> wrote: >> >> This one also does not look unified: >> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html > >> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >> >> One style nit: >> >> LineNumberOnBraceTarg: >> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >> for line number variables, but this test uses 'stopLine'. >> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. > >> On Mar 28, 2017, at 6:37 PM, David Holmes > > wrote: >> >> Two nits: >> - test/com/sun/jdi/FetchLocals.java >> - test/com/sun/jdi/LambdaBreakpointTest.java >> >> Second copyright year should be 2017. > >> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev > > wrote: >> Hi all, >> >> could you please review this fix for 8177507? >> >> due to their nature, some of jdi tests are line number sensitive. >> unfortunately different tests indicate that differently, so it's quite >> easy to overlook that and incidentally break tests, for example by >> changing module dependency declaration or license modification. this >> fix unifies the way line number sensitivity is indicated and also >> improves readability/maintainability of some tests by using constant >> fields instead of magic numbers. >> >> some of line number sensitive tests have been unexpectedly removed >> from execution because they had @test/nodynamiccopyright/ instead of >> @test tag. this changeset fixes and returns them to regular execution. >> >> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >> testing: test/com/sun/jdi From mikael.gerdin at oracle.com Mon Apr 3 08:38:21 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 3 Apr 2017 10:38:21 +0200 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: References: Message-ID: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> Hi Thomas, On 2017-04-02 13:26, Thomas St?fe wrote: > Ping... nobody? Sorry for delaying looking at this. I have two questions: It looks like all callers of inc and dec both only ever pass "1" as num_chunks, should we remove the parameter instead? To document that the inc and dec operations are properly synchronized maybe we should add assert_lock_strong(SpaceManager::expand_lock()); to inc and dec to show that? /Mikael > > Kind Regards, Thomas > > On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe > wrote: > > Hi, > > https://bugs.openjdk.java.net/browse/JDK-8170520 > > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ > > > may I please get a review of another small cleanup change to the > metaspace. Compared with the last one (JDK-8170933), this one is > smaller. > > I posted this originally for jdk 9 last november, and it got > reviewed already: > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-November/021946.html > . > Mikael found a small bug and by then it was too late to bring this > into jdk9, so I postponed the patch for jdk10. > > Now the patch got rebased to Jdk10, and it is also quite a bit > simpler because it meshes well with the cleanup work done on > JDK-8170933. > > The change replaces the calls to Atomic::inc/dec for the > ChunkManager counters with simple +/-, because all code paths which > modify the ChunkManager counters are under lock protection > (SpaceManager::expand_lock()). This makes updating these counters > cheep and thus removes the need to be frugal with the number of updates. > > Before the patch - when a list of chunks was returned to a > ChunkManager in ~SpaceManager() - the increment values were updated > once, with just one call to ChunkManager::inc_free_chunks_total(). > This left a time window in which the counters did not reflect > reality, so one had to be really careful where to place asserts to > check the ChunkManager state. That made modifying and playing with > the code error prone. > > Since JDK-8170933, chunks are returned to the ChunkManager via one > common path, which always ends in > ChunkManager::return_single_chunk(). Because of that and because > updating the counters is now cheap, I moved the several invocations > of ChunkManager::inc_free_chunks_total() to > ChunkManager::return_single_chunk(). > > The rest of the changes is cosmetical: > - Moved ChunkManager::inc_free_chunks_total() up to the private > section of the class, because it is not called from outside anymore > - renamed arguments for ChunkManager::inc_free_chunks_total() > and ChunkManager::dec_free_chunks_total() to be clearer named, and > gave both of them the same arguments > - Added an assert to both function asserting that the > increment/decrement word size value should be a multiple of the > smallest chunk size > > I ran gtests and jtreg tests on Linux and AIX, no issues popped up. > > Thank you for reviewing, > > Thomas > > > > > > > > > From thomas.stuefe at gmail.com Mon Apr 3 09:22:55 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 3 Apr 2017 11:22:55 +0200 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> References: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> Message-ID: Hi Mikael, On Mon, Apr 3, 2017 at 10:38 AM, Mikael Gerdin wrote: > Hi Thomas, > > On 2017-04-02 13:26, Thomas St?fe wrote: > >> Ping... nobody? >> > > Sorry for delaying looking at this. > > I have two questions: > It looks like all callers of inc and dec both only ever pass "1" as > num_chunks, should we remove the parameter instead? > > You are right, makes sense. Alternativly, one also could hand in the pointer to the Metachunk about to be added/removed from the ChunkManager as in void account_added_chunk(const Metachunk* c) // increment counters void account_removed_chunk(const Metachunk* c) // decrement counters which I would like a but more but it is only cosmetic. What do you think? > To document that the inc and dec operations are properly synchronized > maybe we should add > assert_lock_strong(SpaceManager::expand_lock()); > to inc and dec to show that? > Yes, definitely. ...Thomas > /Mikael > > >> Kind Regards, Thomas >> >> On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe > > wrote: >> >> Hi, >> >> https://bugs.openjdk.java.net/browse/JDK-8170520 >> >> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-81705 >> 20-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ >> > 520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10- >> webrev.00/webrev/> >> >> may I please get a review of another small cleanup change to the >> metaspace. Compared with the last one (JDK-8170933), this one is >> smaller. >> >> I posted this originally for jdk 9 last november, and it got >> reviewed already: >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >> 016-November/021946.html >> > 2016-November/021946.html>. >> >> Mikael found a small bug and by then it was too late to bring this >> into jdk9, so I postponed the patch for jdk10. >> >> Now the patch got rebased to Jdk10, and it is also quite a bit >> simpler because it meshes well with the cleanup work done on >> JDK-8170933. >> >> The change replaces the calls to Atomic::inc/dec for the >> ChunkManager counters with simple +/-, because all code paths which >> modify the ChunkManager counters are under lock protection >> (SpaceManager::expand_lock()). This makes updating these counters >> cheep and thus removes the need to be frugal with the number of >> updates. >> >> Before the patch - when a list of chunks was returned to a >> ChunkManager in ~SpaceManager() - the increment values were updated >> once, with just one call to ChunkManager::inc_free_chunks_total(). >> This left a time window in which the counters did not reflect >> reality, so one had to be really careful where to place asserts to >> check the ChunkManager state. That made modifying and playing with >> the code error prone. >> >> Since JDK-8170933, chunks are returned to the ChunkManager via one >> common path, which always ends in >> ChunkManager::return_single_chunk(). Because of that and because >> updating the counters is now cheap, I moved the several invocations >> of ChunkManager::inc_free_chunks_total() to >> ChunkManager::return_single_chunk(). >> >> The rest of the changes is cosmetical: >> - Moved ChunkManager::inc_free_chunks_total() up to the private >> section of the class, because it is not called from outside anymore >> - renamed arguments for ChunkManager::inc_free_chunks_total() >> and ChunkManager::dec_free_chunks_total() to be clearer named, and >> gave both of them the same arguments >> - Added an assert to both function asserting that the >> increment/decrement word size value should be a multiple of the >> smallest chunk size >> >> I ran gtests and jtreg tests on Linux and AIX, no issues popped up. >> >> Thank you for reviewing, >> >> Thomas >> >> >> >> >> >> >> >> >> >> From mikael.gerdin at oracle.com Mon Apr 3 11:59:54 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 3 Apr 2017 13:59:54 +0200 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: References: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> Message-ID: <4cc99043-6f5f-f512-7db3-05d7d7442087@oracle.com> Hi Thomas, On 2017-04-03 11:22, Thomas St?fe wrote: > Hi Mikael, > > On Mon, Apr 3, 2017 at 10:38 AM, Mikael Gerdin > wrote: > > Hi Thomas, > > On 2017-04-02 13:26, Thomas St?fe wrote: > > Ping... nobody? > > > Sorry for delaying looking at this. > > I have two questions: > It looks like all callers of inc and dec both only ever pass "1" as > num_chunks, should we remove the parameter instead? > > > You are right, makes sense. > > Alternativly, one also could hand in the pointer to the Metachunk about > to be added/removed from the ChunkManager as in > > void account_added_chunk(const Metachunk* c) // increment counters > void account_removed_chunk(const Metachunk* c) // decrement counters > > which I would like a but more but it is only cosmetic. What do you think? Since all callers just pass in chunk->word_size() I think that makes sense. /Mikael > > > To document that the inc and dec operations are properly > synchronized maybe we should add > assert_lock_strong(SpaceManager::expand_lock()); > to inc and dec to show that? > > > Yes, definitely. > > ...Thomas > > > /Mikael > > > Kind Regards, Thomas > > On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe > > >> wrote: > > Hi, > > https://bugs.openjdk.java.net/browse/JDK-8170520 > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ > > > > > > may I please get a review of another small cleanup change to the > metaspace. Compared with the last one (JDK-8170933), this one is > smaller. > > I posted this originally for jdk 9 last november, and it got > reviewed already: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-November/021946.html > > > >. > > Mikael found a small bug and by then it was too late to > bring this > into jdk9, so I postponed the patch for jdk10. > > Now the patch got rebased to Jdk10, and it is also quite a bit > simpler because it meshes well with the cleanup work done on > JDK-8170933. > > The change replaces the calls to Atomic::inc/dec for the > ChunkManager counters with simple +/-, because all code > paths which > modify the ChunkManager counters are under lock protection > (SpaceManager::expand_lock()). This makes updating these > counters > cheep and thus removes the need to be frugal with the number > of updates. > > Before the patch - when a list of chunks was returned to a > ChunkManager in ~SpaceManager() - the increment values were > updated > once, with just one call to > ChunkManager::inc_free_chunks_total(). > This left a time window in which the counters did not reflect > reality, so one had to be really careful where to place > asserts to > check the ChunkManager state. That made modifying and > playing with > the code error prone. > > Since JDK-8170933, chunks are returned to the ChunkManager > via one > common path, which always ends in > ChunkManager::return_single_chunk(). Because of that and because > updating the counters is now cheap, I moved the several > invocations > of ChunkManager::inc_free_chunks_total() to > ChunkManager::return_single_chunk(). > > The rest of the changes is cosmetical: > - Moved ChunkManager::inc_free_chunks_total() up to the private > section of the class, because it is not called from outside > anymore > - renamed arguments for ChunkManager::inc_free_chunks_total() > and ChunkManager::dec_free_chunks_total() to be clearer > named, and > gave both of them the same arguments > - Added an assert to both function asserting that the > increment/decrement word size value should be a multiple of the > smallest chunk size > > I ran gtests and jtreg tests on Linux and AIX, no issues > popped up. > > Thank you for reviewing, > > Thomas > > > > > > > > > > From thomas.stuefe at gmail.com Mon Apr 3 14:16:32 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 3 Apr 2017 16:16:32 +0200 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: <4cc99043-6f5f-f512-7db3-05d7d7442087@oracle.com> References: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> <4cc99043-6f5f-f512-7db3-05d7d7442087@oracle.com> Message-ID: Hi Mikael, thanks for the review. New Version: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.01/webrev/ Changes to before: - reworked ChunkManager::dec_free_chunks_total() and ChunkManager::inc_free_chunks_total() to ChunkManager::account_removed_chunk() and ChunkManager::account_added_chunk(), as discussed. Methods now take a const pointer to the Metachunk added/removed, after it has been added/removed. - added assert_lock_strong as suggested. Note that this made it necessary to move the methods out of the class body to be able to access SpaceManager::expand_lock(), which is defined after ChunkManager. Kind Regards, Thomas On Mon, Apr 3, 2017 at 1:59 PM, Mikael Gerdin wrote: > Hi Thomas, > > On 2017-04-03 11:22, Thomas St?fe wrote: > >> Hi Mikael, >> >> On Mon, Apr 3, 2017 at 10:38 AM, Mikael Gerdin > > wrote: >> >> Hi Thomas, >> >> On 2017-04-02 13:26, Thomas St?fe wrote: >> >> Ping... nobody? >> >> >> Sorry for delaying looking at this. >> >> I have two questions: >> It looks like all callers of inc and dec both only ever pass "1" as >> num_chunks, should we remove the parameter instead? >> >> >> You are right, makes sense. >> >> Alternativly, one also could hand in the pointer to the Metachunk about >> to be added/removed from the ChunkManager as in >> >> void account_added_chunk(const Metachunk* c) // increment counters >> void account_removed_chunk(const Metachunk* c) // decrement counters >> >> which I would like a but more but it is only cosmetic. What do you think? >> > > Since all callers just pass in chunk->word_size() I think that makes sense. > > /Mikael > > >> >> To document that the inc and dec operations are properly >> synchronized maybe we should add >> assert_lock_strong(SpaceManager::expand_lock()); >> to inc and dec to show that? >> >> >> Yes, definitely. >> >> ...Thomas >> >> >> /Mikael >> >> >> Kind Regards, Thomas >> >> On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe >> >> > >> >> wrote: >> >> Hi, >> >> https://bugs.openjdk.java.net/browse/JDK-8170520 >> >> > > >> >> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-81705 >> 20-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ >> > 520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10- >> webrev.00/webrev/> >> >> > 520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10- >> webrev.00/webrev/ >> > 520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10- >> webrev.00/webrev/>> >> >> may I please get a review of another small cleanup change to >> the >> metaspace. Compared with the last one (JDK-8170933), this one >> is >> smaller. >> >> I posted this originally for jdk 9 last november, and it got >> reviewed already: >> >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >> 016-November/021946.html >> > 2016-November/021946.html> >> >> > 2016-November/021946.html >> > 2016-November/021946.html>>. >> >> Mikael found a small bug and by then it was too late to >> bring this >> into jdk9, so I postponed the patch for jdk10. >> >> Now the patch got rebased to Jdk10, and it is also quite a bit >> simpler because it meshes well with the cleanup work done on >> JDK-8170933. >> >> The change replaces the calls to Atomic::inc/dec for the >> ChunkManager counters with simple +/-, because all code >> paths which >> modify the ChunkManager counters are under lock protection >> (SpaceManager::expand_lock()). This makes updating these >> counters >> cheep and thus removes the need to be frugal with the number >> of updates. >> >> Before the patch - when a list of chunks was returned to a >> ChunkManager in ~SpaceManager() - the increment values were >> updated >> once, with just one call to >> ChunkManager::inc_free_chunks_total(). >> This left a time window in which the counters did not reflect >> reality, so one had to be really careful where to place >> asserts to >> check the ChunkManager state. That made modifying and >> playing with >> the code error prone. >> >> Since JDK-8170933, chunks are returned to the ChunkManager >> via one >> common path, which always ends in >> ChunkManager::return_single_chunk(). Because of that and >> because >> updating the counters is now cheap, I moved the several >> invocations >> of ChunkManager::inc_free_chunks_total() to >> ChunkManager::return_single_chunk(). >> >> The rest of the changes is cosmetical: >> - Moved ChunkManager::inc_free_chunks_total() up to the >> private >> section of the class, because it is not called from outside >> anymore >> - renamed arguments for ChunkManager::inc_free_chunks_total() >> and ChunkManager::dec_free_chunks_total() to be clearer >> named, and >> gave both of them the same arguments >> - Added an assert to both function asserting that the >> increment/decrement word size value should be a multiple of >> the >> smallest chunk size >> >> I ran gtests and jtreg tests on Linux and AIX, no issues >> popped up. >> >> Thank you for reviewing, >> >> Thomas >> >> >> >> >> >> >> >> >> >> >> From igor.ignatyev at oracle.com Mon Apr 3 16:46:04 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 3 Apr 2017 09:46:04 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> Message-ID: looks like I have uploaded the webrev before I saved files... uploaded a new webrev w/ all mentioned changes: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 -- Igor > On Apr 2, 2017, at 1:31 PM, David Holmes > wrote: > > Hi Igor, > > On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >> Hi, >> >> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next >> iteration w/ copyright years fixed, > > test/com/sun/jdi/BreakpointTest.java > > also needs fixing - says 2015. > >> LineNumberOnBraceTarg and > > test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. > >> LambdaBreakpointTest made more unified w/ the rest. > > I don't see any change here. > > Thanks, > David > ----- > >> Thanks, >> -- Igor >> >>> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >>> > wrote: >>> >>> This one also does not look unified: >>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >> >>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>> >>> One style nit: >>> >>> LineNumberOnBraceTarg: >>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>> for line number variables, but this test uses 'stopLine'. >>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >> >>> On Mar 28, 2017, at 6:37 PM, David Holmes >>> >> wrote: >>> >>> Two nits: >>> - test/com/sun/jdi/FetchLocals.java >>> - test/com/sun/jdi/LambdaBreakpointTest.java >>> >>> Second copyright year should be 2017. >> >>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>> >> wrote: >>> Hi all, >>> >>> could you please review this fix for 8177507? >>> >>> due to their nature, some of jdi tests are line number sensitive. >>> unfortunately different tests indicate that differently, so it's quite >>> easy to overlook that and incidentally break tests, for example by >>> changing module dependency declaration or license modification. this >>> fix unifies the way line number sensitivity is indicated and also >>> improves readability/maintainability of some tests by using constant >>> fields instead of magic numbers. >>> >>> some of line number sensitive tests have been unexpectedly removed >>> from execution because they had @test/nodynamiccopyright/ instead of >>> @test tag. this changeset fixes and returns them to regular execution. >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>> testing: test/com/sun/jdi From coleen.phillimore at oracle.com Mon Apr 3 17:03:08 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 3 Apr 2017 13:03:08 -0400 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: References: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> <4cc99043-6f5f-f512-7db3-05d7d7442087@oracle.com> Message-ID: <61598117-d50b-528b-dc96-2dd398123fe3@oracle.com> This looks good to me. I have a small request. Can you change it to account_for_added_chunk() and account_for_removed_chunk() because these names read better to me? I can sponsor your change. thanks, Coleen On 4/3/17 10:16 AM, Thomas St?fe wrote: > Hi Mikael, > > thanks for the review. New Version: > > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.01/webrev/ > > > Changes to before: > > - reworked ChunkManager::dec_free_chunks_total() and > ChunkManager::inc_free_chunks_total() to > ChunkManager::account_removed_chunk() and > ChunkManager::account_added_chunk(), as discussed. Methods now take a > const pointer to the Metachunk added/removed, after it has been > added/removed. > - added assert_lock_strong as suggested. Note that this made it > necessary to move the methods out of the class body to be able to > access SpaceManager::expand_lock(), which is defined after ChunkManager. > > Kind Regards, Thomas > > > On Mon, Apr 3, 2017 at 1:59 PM, Mikael Gerdin > > wrote: > > Hi Thomas, > > On 2017-04-03 11:22, Thomas St?fe wrote: > > Hi Mikael, > > On Mon, Apr 3, 2017 at 10:38 AM, Mikael Gerdin > > >> wrote: > > Hi Thomas, > > On 2017-04-02 13:26, Thomas St?fe wrote: > > Ping... nobody? > > > Sorry for delaying looking at this. > > I have two questions: > It looks like all callers of inc and dec both only ever > pass "1" as > num_chunks, should we remove the parameter instead? > > > You are right, makes sense. > > Alternativly, one also could hand in the pointer to the > Metachunk about > to be added/removed from the ChunkManager as in > > void account_added_chunk(const Metachunk* c) // increment > counters > void account_removed_chunk(const Metachunk* c) // decrement > counters > > which I would like a but more but it is only cosmetic. What do > you think? > > > Since all callers just pass in chunk->word_size() I think that > makes sense. > > /Mikael > > > > To document that the inc and dec operations are properly > synchronized maybe we should add > assert_lock_strong(SpaceManager::expand_lock()); > to inc and dec to show that? > > > Yes, definitely. > > ...Thomas > > > /Mikael > > > Kind Regards, Thomas > > On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe > > > > > > >>> wrote: > > Hi, > > https://bugs.openjdk.java.net/browse/JDK-8170520 > > > > > >> > > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ > > > > > > > > > >> > > may I please get a review of another small cleanup > change to the > metaspace. Compared with the last one > (JDK-8170933), this one is > smaller. > > I posted this originally for jdk 9 last november, > and it got > reviewed already: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-November/021946.html > > > > > > > > > >>. > > Mikael found a small bug and by then it was too > late to > bring this > into jdk9, so I postponed the patch for jdk10. > > Now the patch got rebased to Jdk10, and it is also > quite a bit > simpler because it meshes well with the cleanup > work done on > JDK-8170933. > > The change replaces the calls to Atomic::inc/dec > for the > ChunkManager counters with simple +/-, because all > code > paths which > modify the ChunkManager counters are under lock > protection > (SpaceManager::expand_lock()). This makes updating > these > counters > cheep and thus removes the need to be frugal with > the number > of updates. > > Before the patch - when a list of chunks was > returned to a > ChunkManager in ~SpaceManager() - the increment > values were > updated > once, with just one call to > ChunkManager::inc_free_chunks_total(). > This left a time window in which the counters did > not reflect > reality, so one had to be really careful where to > place > asserts to > check the ChunkManager state. That made modifying and > playing with > the code error prone. > > Since JDK-8170933, chunks are returned to the > ChunkManager > via one > common path, which always ends in > ChunkManager::return_single_chunk(). Because of > that and because > updating the counters is now cheap, I moved the > several > invocations > of ChunkManager::inc_free_chunks_total() to > ChunkManager::return_single_chunk(). > > The rest of the changes is cosmetical: > - Moved ChunkManager::inc_free_chunks_total() up > to the private > section of the class, because it is not called > from outside > anymore > - renamed arguments for > ChunkManager::inc_free_chunks_total() > and ChunkManager::dec_free_chunks_total() to be > clearer > named, and > gave both of them the same arguments > - Added an assert to both function asserting that the > increment/decrement word size value should be a > multiple of the > smallest chunk size > > I ran gtests and jtreg tests on Linux and AIX, no > issues > popped up. > > Thank you for reviewing, > > Thomas > > > > > > > > > > > From thomas.stuefe at gmail.com Mon Apr 3 17:37:59 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 3 Apr 2017 19:37:59 +0200 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: <61598117-d50b-528b-dc96-2dd398123fe3@oracle.com> References: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> <4cc99043-6f5f-f512-7db3-05d7d7442087@oracle.com> <61598117-d50b-528b-dc96-2dd398123fe3@oracle.com> Message-ID: Hi Coleen, On Mon, Apr 3, 2017 at 7:03 PM, wrote: > This looks good to me. I have a small request. > Can you change it to account_for_added_chunk() and > account_for_removed_chunk() because these names read better to me? > > Certainly :) here you go: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.02/webrev/ Only the names of the functions changed as requested, the rest is unchanged. I can sponsor your change. > > Thank you, Coleen! ..Thomas > thanks, > Coleen > > > On 4/3/17 10:16 AM, Thomas St?fe wrote: > > Hi Mikael, > > thanks for the review. New Version: > > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2- > 8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.01/ > webrev/ > > Changes to before: > > - reworked ChunkManager::dec_free_chunks_total() and > ChunkManager::inc_free_chunks_total() to ChunkManager::account_removed_chunk() > and ChunkManager::account_added_chunk(), as discussed. Methods now take a > const pointer to the Metachunk added/removed, after it has been > added/removed. > - added assert_lock_strong as suggested. Note that this made it necessary > to move the methods out of the class body to be able to access > SpaceManager::expand_lock(), which is defined after ChunkManager. > > Kind Regards, Thomas > > > On Mon, Apr 3, 2017 at 1:59 PM, Mikael Gerdin > wrote: > >> Hi Thomas, >> >> On 2017-04-03 11:22, Thomas St?fe wrote: >> >>> Hi Mikael, >>> >>> On Mon, Apr 3, 2017 at 10:38 AM, Mikael Gerdin >> > wrote: >>> >>> Hi Thomas, >>> >>> On 2017-04-02 13:26, Thomas St?fe wrote: >>> >>> Ping... nobody? >>> >>> >>> Sorry for delaying looking at this. >>> >>> I have two questions: >>> It looks like all callers of inc and dec both only ever pass "1" as >>> num_chunks, should we remove the parameter instead? >>> >>> >>> You are right, makes sense. >>> >>> Alternativly, one also could hand in the pointer to the Metachunk about >>> to be added/removed from the ChunkManager as in >>> >>> void account_added_chunk(const Metachunk* c) // increment counters >>> void account_removed_chunk(const Metachunk* c) // decrement counters >>> >>> which I would like a but more but it is only cosmetic. What do you think? >>> >> >> Since all callers just pass in chunk->word_size() I think that makes >> sense. >> >> /Mikael >> >> >>> >>> To document that the inc and dec operations are properly >>> synchronized maybe we should add >>> assert_lock_strong(SpaceManager::expand_lock()); >>> to inc and dec to show that? >>> >>> >>> Yes, definitely. >>> >>> ...Thomas >>> >>> >>> /Mikael >>> >>> >>> Kind Regards, Thomas >>> >>> On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe >>> >>> >> >>> >> wrote: >>> >>> Hi, >>> >>> https://bugs.openjdk.java.net/browse/JDK-8170520 >>> >>> >> > >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-81705 >>> 20-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webr >>> ev.00/webrev/ >>> >> 520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-web >>> rev.00/webrev/> >>> >>> >> 520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-web >>> rev.00/webrev/ >>> >> 520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-web >>> rev.00/webrev/>> >>> >>> may I please get a review of another small cleanup change to >>> the >>> metaspace. Compared with the last one (JDK-8170933), this >>> one is >>> smaller. >>> >>> I posted this originally for jdk 9 last november, and it got >>> reviewed already: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>> 016-November/021946.html >>> >> 2016-November/021946.html> >>> >>> >> 2016-November/021946.html >>> >> 2016-November/021946.html>>. >>> >>> Mikael found a small bug and by then it was too late to >>> bring this >>> into jdk9, so I postponed the patch for jdk10. >>> >>> Now the patch got rebased to Jdk10, and it is also quite a >>> bit >>> simpler because it meshes well with the cleanup work done on >>> JDK-8170933. >>> >>> The change replaces the calls to Atomic::inc/dec for the >>> ChunkManager counters with simple +/-, because all code >>> paths which >>> modify the ChunkManager counters are under lock protection >>> (SpaceManager::expand_lock()). This makes updating these >>> counters >>> cheep and thus removes the need to be frugal with the number >>> of updates. >>> >>> Before the patch - when a list of chunks was returned to a >>> ChunkManager in ~SpaceManager() - the increment values were >>> updated >>> once, with just one call to >>> ChunkManager::inc_free_chunks_total(). >>> This left a time window in which the counters did not reflect >>> reality, so one had to be really careful where to place >>> asserts to >>> check the ChunkManager state. That made modifying and >>> playing with >>> the code error prone. >>> >>> Since JDK-8170933, chunks are returned to the ChunkManager >>> via one >>> common path, which always ends in >>> ChunkManager::return_single_chunk(). Because of that and >>> because >>> updating the counters is now cheap, I moved the several >>> invocations >>> of ChunkManager::inc_free_chunks_total() to >>> ChunkManager::return_single_chunk(). >>> >>> The rest of the changes is cosmetical: >>> - Moved ChunkManager::inc_free_chunks_total() up to the >>> private >>> section of the class, because it is not called from outside >>> anymore >>> - renamed arguments for ChunkManager::inc_free_chunks_ >>> total() >>> and ChunkManager::dec_free_chunks_total() to be clearer >>> named, and >>> gave both of them the same arguments >>> - Added an assert to both function asserting that the >>> increment/decrement word size value should be a multiple of >>> the >>> smallest chunk size >>> >>> I ran gtests and jtreg tests on Linux and AIX, no issues >>> popped up. >>> >>> Thank you for reviewing, >>> >>> Thomas >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> > > From coleen.phillimore at oracle.com Mon Apr 3 17:45:34 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 3 Apr 2017 13:45:34 -0400 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: References: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> <4cc99043-6f5f-f512-7db3-05d7d7442087@oracle.com> <61598117-d50b-528b-dc96-2dd398123fe3@oracle.com> Message-ID: <3aaab75e-752f-c19c-cf7f-7ade53e1234d@oracle.com> On 4/3/17 1:37 PM, Thomas St?fe wrote: > Hi Coleen, > > > On Mon, Apr 3, 2017 at 7:03 PM, > wrote: > > This looks good to me. I have a small request. > Can you change it to account_for_added_chunk() and > account_for_removed_chunk() because these names read better to me? > > > Certainly :) > > here you go: > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.02/webrev/ > > Only the names of the functions changed as requested, the rest is > unchanged. > > I can sponsor your change. > > > Thank you, Coleen! I'll wait for the final word from Mikael. Thanks, Coleen > ..Thomas > > thanks, > Coleen > > > On 4/3/17 10:16 AM, Thomas St?fe wrote: >> Hi Mikael, >> >> thanks for the review. New Version: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.01/webrev/ >> >> >> Changes to before: >> >> - reworked ChunkManager::dec_free_chunks_total() and >> ChunkManager::inc_free_chunks_total() to >> ChunkManager::account_removed_chunk() and >> ChunkManager::account_added_chunk(), as discussed. Methods now >> take a const pointer to the Metachunk added/removed, after it has >> been added/removed. >> - added assert_lock_strong as suggested. Note that this made it >> necessary to move the methods out of the class body to be able to >> access SpaceManager::expand_lock(), which is defined after >> ChunkManager. >> >> Kind Regards, Thomas >> >> >> On Mon, Apr 3, 2017 at 1:59 PM, Mikael Gerdin >> > wrote: >> >> Hi Thomas, >> >> On 2017-04-03 11:22, Thomas St?fe wrote: >> >> Hi Mikael, >> >> On Mon, Apr 3, 2017 at 10:38 AM, Mikael Gerdin >> >> > >> wrote: >> >> Hi Thomas, >> >> On 2017-04-02 13:26, Thomas St?fe wrote: >> >> Ping... nobody? >> >> >> Sorry for delaying looking at this. >> >> I have two questions: >> It looks like all callers of inc and dec both only >> ever pass "1" as >> num_chunks, should we remove the parameter instead? >> >> >> You are right, makes sense. >> >> Alternativly, one also could hand in the pointer to the >> Metachunk about >> to be added/removed from the ChunkManager as in >> >> void account_added_chunk(const Metachunk* c) // >> increment counters >> void account_removed_chunk(const Metachunk* c) // >> decrement counters >> >> which I would like a but more but it is only cosmetic. >> What do you think? >> >> >> Since all callers just pass in chunk->word_size() I think >> that makes sense. >> >> /Mikael >> >> >> >> To document that the inc and dec operations are properly >> synchronized maybe we should add >> assert_lock_strong(SpaceManager::expand_lock()); >> to inc and dec to show that? >> >> >> Yes, definitely. >> >> ...Thomas >> >> >> /Mikael >> >> >> Kind Regards, Thomas >> >> On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe >> > >> > > >> > >> >> > >>> wrote: >> >> Hi, >> >> https://bugs.openjdk.java.net/browse/JDK-8170520 >> >> > > >> >> > >> > >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ >> >> >> > > >> >> >> > >> >> > >> >> >> may I please get a review of another small >> cleanup change to the >> metaspace. Compared with the last one >> (JDK-8170933), this one is >> smaller. >> >> I posted this originally for jdk 9 last >> november, and it got >> reviewed already: >> >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-November/021946.html >> >> >> > > >> >> >> > >> >> > >>. >> >> Mikael found a small bug and by then it was >> too late to >> bring this >> into jdk9, so I postponed the patch for jdk10. >> >> Now the patch got rebased to Jdk10, and it is >> also quite a bit >> simpler because it meshes well with the >> cleanup work done on >> JDK-8170933. >> >> The change replaces the calls to >> Atomic::inc/dec for the >> ChunkManager counters with simple +/-, >> because all code >> paths which >> modify the ChunkManager counters are under >> lock protection >> (SpaceManager::expand_lock()). This makes updating these >> counters >> cheep and thus removes the need to be frugal >> with the number >> of updates. >> >> Before the patch - when a list of chunks was >> returned to a >> ChunkManager in ~SpaceManager() - the >> increment values were >> updated >> once, with just one call to >> ChunkManager::inc_free_chunks_total(). >> This left a time window in which the counters >> did not reflect >> reality, so one had to be really careful >> where to place >> asserts to >> check the ChunkManager state. That made >> modifying and >> playing with >> the code error prone. >> >> Since JDK-8170933, chunks are returned to the >> ChunkManager >> via one >> common path, which always ends in >> ChunkManager::return_single_chunk(). Because of that and >> because >> updating the counters is now cheap, I moved >> the several >> invocations >> of ChunkManager::inc_free_chunks_total() to >> ChunkManager::return_single_chunk(). >> >> The rest of the changes is cosmetical: >> - Moved ChunkManager::inc_free_chunks_total() >> up to the private >> section of the class, because it is not >> called from outside >> anymore >> - renamed arguments for >> ChunkManager::inc_free_chunks_total() >> and ChunkManager::dec_free_chunks_total() to >> be clearer >> named, and >> gave both of them the same arguments >> - Added an assert to both function asserting >> that the >> increment/decrement word size value should be >> a multiple of the >> smallest chunk size >> >> I ran gtests and jtreg tests on Linux and >> AIX, no issues >> popped up. >> >> Thank you for reviewing, >> >> Thomas >> >> >> >> >> >> >> >> >> >> >> > > From serguei.spitsyn at oracle.com Mon Apr 3 19:53:36 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Apr 2017 12:53:36 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> Message-ID: <30f9c884-9e75-1fe2-9233-dd9858e2ef5f@oracle.com> Hi Igor, It looks good. Thank you for the update! Thanks, Serguei On 4/3/17 09:46, Igor Ignatyev wrote: > looks like I have uploaded the webrev before I saved files... uploaded > a new webrev w/ all mentioned changes: > http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 > > > -- Igor > >> On Apr 2, 2017, at 1:31 PM, David Holmes > > wrote: >> >> Hi Igor, >> >> On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >>> Hi, >>> >>> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 >>> is the next >>> iteration w/ copyright years fixed, >> >> test/com/sun/jdi/BreakpointTest.java >> >> also needs fixing - says 2015. >> >>> LineNumberOnBraceTarg and >> >> test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. >> >>> LambdaBreakpointTest made more unified w/ the rest. >> >> I don't see any change here. >> >> Thanks, >> David >> ----- >> >>> Thanks, >>> -- Igor >>> >>>> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >>>> >>>> wrote: >>>> >>>> This one also does not look unified: >>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >>>> >>> >>>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>>> >>>> One style nit: >>>> >>>> LineNumberOnBraceTarg: >>>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>>> for line number variables, but this test uses 'stopLine'. >>>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >>> >>>> On Mar 28, 2017, at 6:37 PM, David Holmes >>> >>>> > wrote: >>>> >>>> Two nits: >>>> - test/com/sun/jdi/FetchLocals.java >>>> - test/com/sun/jdi/LambdaBreakpointTest.java >>>> >>>> Second copyright year should be 2017. >>> >>>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>>> >>>> > wrote: >>>> Hi all, >>>> >>>> could you please review this fix for 8177507? >>>> >>>> due to their nature, some of jdi tests are line number sensitive. >>>> unfortunately different tests indicate that differently, so it's quite >>>> easy to overlook that and incidentally break tests, for example by >>>> changing module dependency declaration or license modification. this >>>> fix unifies the way line number sensitivity is indicated and also >>>> improves readability/maintainability of some tests by using constant >>>> fields instead of magic numbers. >>>> >>>> some of line number sensitive tests have been unexpectedly removed >>>> from execution because they had @test/nodynamiccopyright/ instead of >>>> @test tag. this changeset fixes and returns them to regular execution. >>>> >>>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>>> testing: test/com/sun/jdi > From david.holmes at oracle.com Mon Apr 3 20:53:42 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Apr 2017 06:53:42 +1000 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> Message-ID: <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> On 4/04/2017 2:46 AM, Igor Ignatyev wrote: > looks like I have uploaded the webrev before I saved files... uploaded a > new webrev w/ all mentioned changes: > http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 I don't see how the change to LambdaBreakpointTest can possibly work. The new test steps to each line in source code order, but that is not the execution order! David ----- > -- Igor > >> On Apr 2, 2017, at 1:31 PM, David Holmes > > wrote: >> >> Hi Igor, >> >> On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >>> Hi, >>> >>> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next >>> iteration w/ copyright years fixed, >> >> test/com/sun/jdi/BreakpointTest.java >> >> also needs fixing - says 2015. >> >>> LineNumberOnBraceTarg and >> >> test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. >> >>> LambdaBreakpointTest made more unified w/ the rest. >> >> I don't see any change here. >> >> Thanks, >> David >> ----- >> >>> Thanks, >>> -- Igor >>> >>>> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >>>> >>>> wrote: >>>> >>>> This one also does not look unified: >>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >>> >>>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>>> >>>> One style nit: >>>> >>>> LineNumberOnBraceTarg: >>>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>>> for line number variables, but this test uses 'stopLine'. >>>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >>> >>>> On Mar 28, 2017, at 6:37 PM, David Holmes >>> >>>> > wrote: >>>> >>>> Two nits: >>>> - test/com/sun/jdi/FetchLocals.java >>>> - test/com/sun/jdi/LambdaBreakpointTest.java >>>> >>>> Second copyright year should be 2017. >>> >>>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>> >>>> > wrote: >>>> Hi all, >>>> >>>> could you please review this fix for 8177507? >>>> >>>> due to their nature, some of jdi tests are line number sensitive. >>>> unfortunately different tests indicate that differently, so it's quite >>>> easy to overlook that and incidentally break tests, for example by >>>> changing module dependency declaration or license modification. this >>>> fix unifies the way line number sensitivity is indicated and also >>>> improves readability/maintainability of some tests by using constant >>>> fields instead of magic numbers. >>>> >>>> some of line number sensitive tests have been unexpectedly removed >>>> from execution because they had @test/nodynamiccopyright/ instead of >>>> @test tag. this changeset fixes and returns them to regular execution. >>>> >>>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>>> testing: test/com/sun/jdi > From serguei.spitsyn at oracle.com Mon Apr 3 21:39:01 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Apr 2017 14:39:01 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> Message-ID: On 4/3/17 13:53, David Holmes wrote: > On 4/04/2017 2:46 AM, Igor Ignatyev wrote: >> looks like I have uploaded the webrev before I saved files... uploaded a >> new webrev w/ all mentioned changes: >> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 > > I don't see how the change to LambdaBreakpointTest can possibly work. > The new test steps to each line in source code order, but that is not > the execution order! Nice catch! 62 private static void test() { 63 Runnable r = () -> { // B1: L62 64 String from = "lambda"; // B3: L63 65 System.out.println("Hello from " + from); // B4: L64 66 }; // B5: L65 67 r.run(); // B2: L66 68 System.out.println("Goodbye."); // B6: L67 69 } The B2 breakpoint is at the L66, but B3 is at L63. Thanks, Serguei > > David > ----- > >> -- Igor >> >>> On Apr 2, 2017, at 1:31 PM, David Holmes >> > wrote: >>> >>> Hi Igor, >>> >>> On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >>>> Hi, >>>> >>>> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next >>>> iteration w/ copyright years fixed, >>> >>> test/com/sun/jdi/BreakpointTest.java >>> >>> also needs fixing - says 2015. >>> >>>> LineNumberOnBraceTarg and >>> >>> test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. >>> >>>> LambdaBreakpointTest made more unified w/ the rest. >>> >>> I don't see any change here. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Thanks, >>>> -- Igor >>>> >>>>> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >>>>> >>>>> wrote: >>>>> >>>>> This one also does not look unified: >>>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >>>>> >>>> >>>>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>>>> >>>>> One style nit: >>>>> >>>>> LineNumberOnBraceTarg: >>>>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>>>> for line number variables, but this test uses 'stopLine'. >>>>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >>>> >>>>> On Mar 28, 2017, at 6:37 PM, David Holmes >>>> >>>>> > wrote: >>>>> >>>>> Two nits: >>>>> - test/com/sun/jdi/FetchLocals.java >>>>> - test/com/sun/jdi/LambdaBreakpointTest.java >>>>> >>>>> Second copyright year should be 2017. >>>> >>>>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>>> >>>>> > wrote: >>>>> Hi all, >>>>> >>>>> could you please review this fix for 8177507? >>>>> >>>>> due to their nature, some of jdi tests are line number sensitive. >>>>> unfortunately different tests indicate that differently, so it's >>>>> quite >>>>> easy to overlook that and incidentally break tests, for example by >>>>> changing module dependency declaration or license modification. this >>>>> fix unifies the way line number sensitivity is indicated and also >>>>> improves readability/maintainability of some tests by using constant >>>>> fields instead of magic numbers. >>>>> >>>>> some of line number sensitive tests have been unexpectedly removed >>>>> from execution because they had @test/nodynamiccopyright/ instead of >>>>> @test tag. this changeset fixes and returns them to regular >>>>> execution. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>>>> testing: test/com/sun/jdi >> From igor.ignatyev at oracle.com Mon Apr 3 23:09:05 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 3 Apr 2017 16:09:05 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> Message-ID: yes, David you are right. it's so embarrassing ... besides uploading webrev w/o all changes, I have also tested it w/o all changes, I should have noticed that. Thank you for catching that. I have fixed that, tested and uploaded new webrev -- http://cr.openjdk.java.net/~iignatyev/8177507/webrev.03 Thanks, -- Igor > On Apr 3, 2017, at 1:53 PM, David Holmes wrote: > > On 4/04/2017 2:46 AM, Igor Ignatyev wrote: >> looks like I have uploaded the webrev before I saved files... uploaded a >> new webrev w/ all mentioned changes: >> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 > > I don't see how the change to LambdaBreakpointTest can possibly work. The new test steps to each line in source code order, but that is not the execution order! > > David > ----- > >> -- Igor >> >>> On Apr 2, 2017, at 1:31 PM, David Holmes >>> >> wrote: >>> >>> Hi Igor, >>> >>> On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >>>> Hi, >>>> >>>> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next >>>> iteration w/ copyright years fixed, >>> >>> test/com/sun/jdi/BreakpointTest.java >>> >>> also needs fixing - says 2015. >>> >>>> LineNumberOnBraceTarg and >>> >>> test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. >>> >>>> LambdaBreakpointTest made more unified w/ the rest. >>> >>> I don't see any change here. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Thanks, >>>> -- Igor >>>> >>>>> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >>>>> > >>>>> > wrote: >>>>> >>>>> This one also does not look unified: >>>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >>>> >>>>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>>>> >>>>> One style nit: >>>>> >>>>> LineNumberOnBraceTarg: >>>>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>>>> for line number variables, but this test uses 'stopLine'. >>>>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >>>> >>>>> On Mar 28, 2017, at 6:37 PM, David Holmes >>>>> > >>>>> >> wrote: >>>>> >>>>> Two nits: >>>>> - test/com/sun/jdi/FetchLocals.java >>>>> - test/com/sun/jdi/LambdaBreakpointTest.java >>>>> >>>>> Second copyright year should be 2017. >>>> >>>>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>>>> > >>>>> > wrote: >>>>> Hi all, >>>>> >>>>> could you please review this fix for 8177507? >>>>> >>>>> due to their nature, some of jdi tests are line number sensitive. >>>>> unfortunately different tests indicate that differently, so it's quite >>>>> easy to overlook that and incidentally break tests, for example by >>>>> changing module dependency declaration or license modification. this >>>>> fix unifies the way line number sensitivity is indicated and also >>>>> improves readability/maintainability of some tests by using constant >>>>> fields instead of magic numbers. >>>>> >>>>> some of line number sensitive tests have been unexpectedly removed >>>>> from execution because they had @test/nodynamiccopyright/ instead of >>>>> @test tag. this changeset fixes and returns them to regular execution. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>>>> testing: test/com/sun/jdi From serguei.spitsyn at oracle.com Mon Apr 3 23:28:29 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Apr 2017 16:28:29 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> Message-ID: <85455de5-205e-be05-7e5d-1b6a06edf1b5@oracle.com> It is good now. Thanks, Serguei On 4/3/17 16:09, Igor Ignatyev wrote: > yes, David you are right. it's so embarrassing ... besides uploading > webrev w/o all changes, I have also tested it w/o all changes, I > should have noticed that. Thank you for catching that. > > I have fixed that, tested and uploaded new webrev -- > _http://cr.openjdk.java.net/~iignatyev/8177507/webrev.03 > _ > > Thanks, > -- Igor >> On Apr 3, 2017, at 1:53 PM, David Holmes > > wrote: >> >> On 4/04/2017 2:46 AM, Igor Ignatyev wrote: >>> looks like I have uploaded the webrev before I saved files... uploaded a >>> new webrev w/ all mentioned changes: >>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 >>> >> >> I don't see how the change to LambdaBreakpointTest can possibly work. >> The new test steps to each line in source code order, but that is not >> the execution order! >> >> David >> ----- >> >>> -- Igor >>> >>>> On Apr 2, 2017, at 1:31 PM, David Holmes >>> >>>> > wrote: >>>> >>>> Hi Igor, >>>> >>>> On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >>>>> Hi, >>>>> >>>>> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 >>>>> is >>>>> the next >>>>> iteration w/ copyright years fixed, >>>> >>>> test/com/sun/jdi/BreakpointTest.java >>>> >>>> also needs fixing - says 2015. >>>> >>>>> LineNumberOnBraceTarg and >>>> >>>> test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. >>>> >>>>> LambdaBreakpointTest made more unified w/ the rest. >>>> >>>> I don't see any change here. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>>>>> On Mar 29, 2017, at 12:57 PM,serguei.spitsyn at oracle.com >>>>>> >>>>>> >>>>>> wrote: >>>>>> >>>>>> This one also does not look unified: >>>>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >>>>>> >>>>> >>>>>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>>>>> >>>>>> One style nit: >>>>>> >>>>>> LineNumberOnBraceTarg: >>>>>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>>>>> for line number variables, but this test uses 'stopLine'. >>>>>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >>>>> >>>>>> On Mar 28, 2017, at 6:37 PM, David Holmes >>>>>> >>>>>> >>>>>> > wrote: >>>>>> >>>>>> Two nits: >>>>>> - test/com/sun/jdi/FetchLocals.java >>>>>> - test/com/sun/jdi/LambdaBreakpointTest.java >>>>>> >>>>>> Second copyright year should be 2017. >>>>> >>>>>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>>>>> >>>>>> >>>>>> > wrote: >>>>>> Hi all, >>>>>> >>>>>> could you please review this fix for 8177507? >>>>>> >>>>>> due to their nature, some of jdi tests are line number sensitive. >>>>>> unfortunately different tests indicate that differently, so it's >>>>>> quite >>>>>> easy to overlook that and incidentally break tests, for example by >>>>>> changing module dependency declaration or license modification. this >>>>>> fix unifies the way line number sensitivity is indicated and also >>>>>> improves readability/maintainability of some tests by using constant >>>>>> fields instead of magic numbers. >>>>>> >>>>>> some of line number sensitive tests have been unexpectedly removed >>>>>> from execution because they had @test/nodynamiccopyright/ instead of >>>>>> @test tag. this changeset fixes and returns them to regular >>>>>> execution. >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>>>>> testing: test/com/sun/jdi > From david.holmes at oracle.com Mon Apr 3 23:54:12 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Apr 2017 09:54:12 +1000 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> Message-ID: On 4/04/2017 9:09 AM, Igor Ignatyev wrote: > yes, David you are right. it's so embarrassing ... besides uploading > webrev w/o all changes, I have also tested it w/o all changes, I should > have noticed that. Thank you for catching that. > > I have fixed that, tested and uploaded new webrev -- > _http://cr.openjdk.java.net/~iignatyev/8177507/webrev.03_ Looks fine now. Thanks, David > Thanks, > -- Igor >> On Apr 3, 2017, at 1:53 PM, David Holmes > > wrote: >> >> On 4/04/2017 2:46 AM, Igor Ignatyev wrote: >>> looks like I have uploaded the webrev before I saved files... uploaded a >>> new webrev w/ all mentioned changes: >>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 >> >> I don't see how the change to LambdaBreakpointTest can possibly work. >> The new test steps to each line in source code order, but that is not >> the execution order! >> >> David >> ----- >> >>> -- Igor >>> >>>> On Apr 2, 2017, at 1:31 PM, David Holmes >>> >>>> > wrote: >>>> >>>> Hi Igor, >>>> >>>> On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >>>>> Hi, >>>>> >>>>> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next >>>>> iteration w/ copyright years fixed, >>>> >>>> test/com/sun/jdi/BreakpointTest.java >>>> >>>> also needs fixing - says 2015. >>>> >>>>> LineNumberOnBraceTarg and >>>> >>>> test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. >>>> >>>>> LambdaBreakpointTest made more unified w/ the rest. >>>> >>>> I don't see any change here. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>>>>> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >>>>>> >>>>>> >>>>>> wrote: >>>>>> >>>>>> This one also does not look unified: >>>>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >>>>> >>>>>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>>>>> >>>>>> One style nit: >>>>>> >>>>>> LineNumberOnBraceTarg: >>>>>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>>>>> for line number variables, but this test uses 'stopLine'. >>>>>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >>>>> >>>>>> On Mar 28, 2017, at 6:37 PM, David Holmes >>>>> >>>>>> >>>>>> > wrote: >>>>>> >>>>>> Two nits: >>>>>> - test/com/sun/jdi/FetchLocals.java >>>>>> - test/com/sun/jdi/LambdaBreakpointTest.java >>>>>> >>>>>> Second copyright year should be 2017. >>>>> >>>>>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>>>>> >>>>>> >>>>>> > wrote: >>>>>> Hi all, >>>>>> >>>>>> could you please review this fix for 8177507? >>>>>> >>>>>> due to their nature, some of jdi tests are line number sensitive. >>>>>> unfortunately different tests indicate that differently, so it's quite >>>>>> easy to overlook that and incidentally break tests, for example by >>>>>> changing module dependency declaration or license modification. this >>>>>> fix unifies the way line number sensitivity is indicated and also >>>>>> improves readability/maintainability of some tests by using constant >>>>>> fields instead of magic numbers. >>>>>> >>>>>> some of line number sensitive tests have been unexpectedly removed >>>>>> from execution because they had @test/nodynamiccopyright/ instead of >>>>>> @test tag. this changeset fixes and returns them to regular execution. >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>>>>> testing: test/com/sun/jdi > From mikhailo.seledtsov at oracle.com Tue Apr 4 00:01:47 2017 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Mon, 03 Apr 2017 17:01:47 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> Message-ID: <58E2E26B.2020006@oracle.com> Looks good to me. Misha On 4/3/17, 4:54 PM, David Holmes wrote: > On 4/04/2017 9:09 AM, Igor Ignatyev wrote: >> yes, David you are right. it's so embarrassing ... besides uploading >> webrev w/o all changes, I have also tested it w/o all changes, I should >> have noticed that. Thank you for catching that. >> >> I have fixed that, tested and uploaded new webrev -- >> _http://cr.openjdk.java.net/~iignatyev/8177507/webrev.03_ > > Looks fine now. > > Thanks, > David > >> Thanks, >> -- Igor >>> On Apr 3, 2017, at 1:53 PM, David Holmes >> > wrote: >>> >>> On 4/04/2017 2:46 AM, Igor Ignatyev wrote: >>>> looks like I have uploaded the webrev before I saved files... >>>> uploaded a >>>> new webrev w/ all mentioned changes: >>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.02 >>> >>> I don't see how the change to LambdaBreakpointTest can possibly work. >>> The new test steps to each line in source code order, but that is not >>> the execution order! >>> >>> David >>> ----- >>> >>>> -- Igor >>>> >>>>> On Apr 2, 2017, at 1:31 PM, David Holmes >>>> >>>>> > wrote: >>>>> >>>>> Hi Igor, >>>>> >>>>> On 3/04/2017 3:09 AM, Igor Ignatyev wrote: >>>>>> Hi, >>>>>> >>>>>> http://cr.openjdk.java.net/~iignatyev//8177507/webrev.01 is the next >>>>>> iteration w/ copyright years fixed, >>>>> >>>>> test/com/sun/jdi/BreakpointTest.java >>>>> >>>>> also needs fixing - says 2015. >>>>> >>>>>> LineNumberOnBraceTarg and >>>>> >>>>> test/com/sun/jdi/LineNumberOnBraceTest.java - Ok. >>>>> >>>>>> LambdaBreakpointTest made more unified w/ the rest. >>>>> >>>>> I don't see any change here. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Thanks, >>>>>> -- Igor >>>>>> >>>>>>> On Mar 29, 2017, at 12:57 PM, serguei.spitsyn at oracle.com >>>>>>> >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> This one also does not look unified: >>>>>>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html >>>>>>> >>>>>> >>>>>>> On 3/29/17 11:08, Mikhailo Seledtsov wrote: >>>>>>> >>>>>>> One style nit: >>>>>>> >>>>>>> LineNumberOnBraceTarg: >>>>>>> All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) >>>>>>> for line number variables, but this test uses 'stopLine'. >>>>>>> Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. >>>>>> >>>>>>> On Mar 28, 2017, at 6:37 PM, David Holmes >>>>>> >>>>>>> >>>>>>> > wrote: >>>>>>> >>>>>>> Two nits: >>>>>>> - test/com/sun/jdi/FetchLocals.java >>>>>>> - test/com/sun/jdi/LambdaBreakpointTest.java >>>>>>> >>>>>>> Second copyright year should be 2017. >>>>>> >>>>>>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev >>>>>>> >>>>>>> >>>>>>> > wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> could you please review this fix for 8177507? >>>>>>> >>>>>>> due to their nature, some of jdi tests are line number sensitive. >>>>>>> unfortunately different tests indicate that differently, so it's >>>>>>> quite >>>>>>> easy to overlook that and incidentally break tests, for example by >>>>>>> changing module dependency declaration or license modification. >>>>>>> this >>>>>>> fix unifies the way line number sensitivity is indicated and also >>>>>>> improves readability/maintainability of some tests by using >>>>>>> constant >>>>>>> fields instead of magic numbers. >>>>>>> >>>>>>> some of line number sensitive tests have been unexpectedly removed >>>>>>> from execution because they had @test/nodynamiccopyright/ >>>>>>> instead of >>>>>>> @test tag. this changeset fixes and returns them to regular >>>>>>> execution. >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>>>>>> testing: test/com/sun/jdi >> From igor.ignatyev at oracle.com Tue Apr 4 00:04:11 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 3 Apr 2017 17:04:11 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: <58E2E26B.2020006@oracle.com> References: <58DBF832.2010902@oracle.com> <866DB157-A7DB-4CC2-B696-9A3EF3848365@oracle.com> <3d1c3744-c27a-6a9f-1296-d51e9745fa61@oracle.com> <5996e3be-e755-debf-384c-f30b7c8c6ceb@oracle.com> <58E2E26B.2020006@oracle.com> Message-ID: <8CD361AC-1AAB-4A3F-824D-FA566F10D7B1@oracle.com> Misha, David, Serguei, Thank you for review! -- Igor From michael.rasmussen at zeroturnaround.com Tue Apr 4 08:49:35 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Tue, 4 Apr 2017 11:49:35 +0300 Subject: Question regarding JVM crashes in GC Message-ID: Hi We are currently in the middle of making JRebel ready for Java 9, and in that process came across some issues with the concurrent GCs, and was wondering if anyone here could help shed some insight into what's going wrong? Basically, running with the current internal build of JRebel, I can reproducible crash the JVM by calling System.gc(), if using a concurrent GC. Output from 9-ea+160-jigsaw-nightly-h6207-20170316 build on Windows x64: -XX:+UseG1GC: EXCEPTION_ACCESS_VIOLATION # UpdateRSOopClosure::do_oop+0x47 -XX:+UseConcMarkSweepGC: Internal Error (.../synchronizer.cpp:1574), pid=12588, tid=6176 # guarantee(obj->mark() == markOopDesc::encode(mid)) failed: invariant -XX:+UseParallelOldGC: Internal Error (.../synchronizer.cpp:1574), pid=13792, tid=12284 # guarantee(obj->mark() == markOopDesc::encode(mid)) failed: invariant -XX:+UseParallelGC: EXCEPTION_ACCESS_VIOLATION # ServiceUtil::visible_oop+0x32 -XX:+UseSerialGC works With -XX:+VerifyBeforeGC -XX:+VerifyAfterGC -XX:+UseParallelOldGC: EXCEPTION_ACCESS_VIOLATION # oopDesc::verify+0x2b -XX:+UseParallelGC: EXCEPTION_ACCESS_VIOLATION # oopDesc::verify+0x38 Rest have same outcome as without the verification. Also ran on build 9-ea+163 (Windows x64), with similar results (though less symbol info): G1: EXCEPTION_ACCESS_VIOLATION [jvm.dll+0x2ac0b7] CMS: EXCEPTION_ACCESS_VIOLATION [jvm.dll+0x45f1d2] ParallelOld: Internal Error (.../synchronizer.cpp:1574), pid=10472, tid=13212 # guarantee(obj->mark() == markOopDesc::encode(mid)) failed: invariant Parallel: EXCEPTION_ACCESS_VIOLATION [jvm.dll+0x45f1d2] JRebel basically instruments every class in the system, including the classes in java.base. We also set native prefix, so it is likely that we trigger something somewhere. Though, at this point, I don't know what. Perhaps the GCs make some assumptions about some of the classes we touch (like hard-coded method or field offsets etc?) ? If anyone here has any idea where to start looking, it would be greatly appreciated! Kind regards Michael Rasmussen Product Manager, JRebel ZeroTurnaround Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> From mikael.gerdin at oracle.com Tue Apr 4 13:38:23 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 4 Apr 2017 15:38:23 +0200 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic In-Reply-To: <3aaab75e-752f-c19c-cf7f-7ade53e1234d@oracle.com> References: <99c725b3-d0c8-b927-7e3c-da182283e515@oracle.com> <4cc99043-6f5f-f512-7db3-05d7d7442087@oracle.com> <61598117-d50b-528b-dc96-2dd398123fe3@oracle.com> <3aaab75e-752f-c19c-cf7f-7ade53e1234d@oracle.com> Message-ID: On 2017-04-03 19:45, coleen.phillimore at oracle.com wrote: > > > On 4/3/17 1:37 PM, Thomas St?fe wrote: >> Hi Coleen, >> >> >> On Mon, Apr 3, 2017 at 7:03 PM, > > wrote: >> >> This looks good to me. I have a small request. >> Can you change it to account_for_added_chunk() and >> account_for_removed_chunk() because these names read better to me? >> >> >> Certainly :) >> >> here you go: >> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.02/webrev/ >> >> >> Only the names of the functions changed as requested, the rest is >> unchanged. >> >> I can sponsor your change. >> >> >> Thank you, Coleen! > > I'll wait for the final word from Mikael. Fine by me. /Mikael > Thanks, > Coleen > >> ..Thomas >> >> >> thanks, >> Coleen >> >> >> On 4/3/17 10:16 AM, Thomas St?fe wrote: >>> Hi Mikael, >>> >>> thanks for the review. New Version: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.01/webrev/ >>> >>> >>> Changes to before: >>> >>> - reworked ChunkManager::dec_free_chunks_total() and >>> ChunkManager::inc_free_chunks_total() to >>> ChunkManager::account_removed_chunk() and >>> ChunkManager::account_added_chunk(), as discussed. Methods now >>> take a const pointer to the Metachunk added/removed, after it has >>> been added/removed. >>> - added assert_lock_strong as suggested. Note that this made it >>> necessary to move the methods out of the class body to be able to >>> access SpaceManager::expand_lock(), which is defined after >>> ChunkManager. >>> >>> Kind Regards, Thomas >>> >>> >>> On Mon, Apr 3, 2017 at 1:59 PM, Mikael Gerdin >>> > wrote: >>> >>> Hi Thomas, >>> >>> On 2017-04-03 11:22, Thomas St?fe wrote: >>> >>> Hi Mikael, >>> >>> On Mon, Apr 3, 2017 at 10:38 AM, Mikael Gerdin >>> >>> >> >> wrote: >>> >>> Hi Thomas, >>> >>> On 2017-04-02 13:26, Thomas St?fe wrote: >>> >>> Ping... nobody? >>> >>> >>> Sorry for delaying looking at this. >>> >>> I have two questions: >>> It looks like all callers of inc and dec both only >>> ever pass "1" as >>> num_chunks, should we remove the parameter instead? >>> >>> >>> You are right, makes sense. >>> >>> Alternativly, one also could hand in the pointer to the >>> Metachunk about >>> to be added/removed from the ChunkManager as in >>> >>> void account_added_chunk(const Metachunk* c) // >>> increment counters >>> void account_removed_chunk(const Metachunk* c) // >>> decrement counters >>> >>> which I would like a but more but it is only cosmetic. >>> What do you think? >>> >>> >>> Since all callers just pass in chunk->word_size() I think >>> that makes sense. >>> >>> /Mikael >>> >>> >>> >>> To document that the inc and dec operations are properly >>> synchronized maybe we should add >>> assert_lock_strong(SpaceManager::expand_lock()); >>> to inc and dec to show that? >>> >>> >>> Yes, definitely. >>> >>> ...Thomas >>> >>> >>> /Mikael >>> >>> >>> Kind Regards, Thomas >>> >>> On Tue, Mar 21, 2017 at 9:59 AM, Thomas St?fe >>> >> >>> >> > >>> >> >>> >>> >> >>> wrote: >>> >>> Hi, >>> >>> >>> https://bugs.openjdk.java.net/browse/JDK-8170520 >>> >>> >> > >>> >>> >> >>> >> >> >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ >>> >>> >>> >> > >>> >>> >>> >> >>> >>> >> >> >>> >>> may I please get a review of another small >>> cleanup change to the >>> metaspace. Compared with the last one >>> (JDK-8170933), this one is >>> smaller. >>> >>> I posted this originally for jdk 9 last >>> november, and it got >>> reviewed already: >>> >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-November/021946.html >>> >>> >>> >> > >>> >>> >>> >> >>> >>> >> >>. >>> >>> Mikael found a small bug and by then it was >>> too late to >>> bring this >>> into jdk9, so I postponed the patch for jdk10. >>> >>> Now the patch got rebased to Jdk10, and it is >>> also quite a bit >>> simpler because it meshes well with the >>> cleanup work done on >>> JDK-8170933. >>> >>> The change replaces the calls to >>> Atomic::inc/dec for the >>> ChunkManager counters with simple +/-, >>> because all code >>> paths which >>> modify the ChunkManager counters are under >>> lock protection >>> (SpaceManager::expand_lock()). This makes >>> updating these >>> counters >>> cheep and thus removes the need to be frugal >>> with the number >>> of updates. >>> >>> Before the patch - when a list of chunks was >>> returned to a >>> ChunkManager in ~SpaceManager() - the >>> increment values were >>> updated >>> once, with just one call to >>> ChunkManager::inc_free_chunks_total(). >>> This left a time window in which the counters >>> did not reflect >>> reality, so one had to be really careful >>> where to place >>> asserts to >>> check the ChunkManager state. That made >>> modifying and >>> playing with >>> the code error prone. >>> >>> Since JDK-8170933, chunks are returned to the >>> ChunkManager >>> via one >>> common path, which always ends in >>> ChunkManager::return_single_chunk(). Because >>> of that and because >>> updating the counters is now cheap, I moved >>> the several >>> invocations >>> of ChunkManager::inc_free_chunks_total() to >>> ChunkManager::return_single_chunk(). >>> >>> The rest of the changes is cosmetical: >>> - Moved >>> ChunkManager::inc_free_chunks_total() up to the private >>> section of the class, because it is not >>> called from outside >>> anymore >>> - renamed arguments for >>> ChunkManager::inc_free_chunks_total() >>> and ChunkManager::dec_free_chunks_total() to >>> be clearer >>> named, and >>> gave both of them the same arguments >>> - Added an assert to both function asserting >>> that the >>> increment/decrement word size value should be >>> a multiple of the >>> smallest chunk size >>> >>> I ran gtests and jtreg tests on Linux and >>> AIX, no issues >>> popped up. >>> >>> Thank you for reviewing, >>> >>> Thomas >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> >> > From michael.rasmussen at zeroturnaround.com Wed Apr 5 07:11:00 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Wed, 5 Apr 2017 10:11:00 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: References: Message-ID: Ok Found Shipilev's fastdebug builds, and tried running with that (openjdk-jdk9-dev-fastdebug-2017-04-04.tar.gz). Using the fastdebug build and running with G1, the JVM crashes when calling System.gc(), with one of the following assertions: Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:61) assert(is_archive_object(obj) || new_obj != __null || obj->mark() == markOopDesc::prototype() || (UseBiasedLocking && obj->mark()->has_bias_pattern())) failed: should be forwarded Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:53) assert(Universe::heap()->is_in(obj)) failed: should be in heap Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:65) assert(Universe::heap()->is_in_reserved(new_obj)) failed: should be in object space /Michael From mikael.gerdin at oracle.com Wed Apr 5 08:00:20 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 5 Apr 2017 10:00:20 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: Message-ID: <110b0b8e-d2dd-98c9-16a8-cb263b710bc1@oracle.com> Hi Michael, On 2017-04-05 09:11, Michael Rasmussen wrote: > Ok > > Found Shipilev's fastdebug builds, and tried running with that > (openjdk-jdk9-dev-fastdebug-2017-04-04.tar.gz). > > Using the fastdebug build and running with G1, the JVM crashes when > calling System.gc(), with one of the following assertions: Did you try out -XX:+Verify{Before/After}GC with G1? G1 has the most complex verification code so that might provide some more information. /Mikael > > Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:61) > assert(is_archive_object(obj) || new_obj != __null || obj->mark() == > markOopDesc::prototype() || (UseBiasedLocking && > obj->mark()->has_bias_pattern())) failed: should be forwarded > > Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:53) > assert(Universe::heap()->is_in(obj)) failed: should be in heap > > Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:65) > assert(Universe::heap()->is_in_reserved(new_obj)) failed: should be in > object space > > /Michael > From michael.rasmussen at zeroturnaround.com Wed Apr 5 08:12:40 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Wed, 5 Apr 2017 11:12:40 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: <110b0b8e-d2dd-98c9-16a8-cb263b710bc1@oracle.com> References: <110b0b8e-d2dd-98c9-16a8-cb263b710bc1@oracle.com> Message-ID: On 5 April 2017 at 11:00, Mikael Gerdin wrote: > Did you try out -XX:+Verify{Before/After}GC with G1? G1 has the most complex > verification code so that might provide some more information. > Adding -XX:+VerifyBeforeGC -XX:+VerifyAfterGC didn't produce any different result, still get one of the 3 before mentioned assertions. Adding -Xlog:gc*=debug, I see that "Verifying Before GC" complete without anything, and the last log line I get is: [info ][gc,phases,start ] GC(16) Phase 3: Adjust pointers /Michael From stefan.johansson at oracle.com Wed Apr 5 12:41:51 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 5 Apr 2017 14:41:51 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: Message-ID: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Hi Michael, On 2017-04-05 09:11, Michael Rasmussen wrote: > Ok > > Found Shipilev's fastdebug builds, and tried running with that > (openjdk-jdk9-dev-fastdebug-2017-04-04.tar.gz). > > Using the fastdebug build and running with G1, the JVM crashes when > calling System.gc(), with one of the following assertions: > > Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:61) > assert(is_archive_object(obj) || new_obj != __null || obj->mark() == > markOopDesc::prototype() || (UseBiasedLocking && > obj->mark()->has_bias_pattern())) failed: should be forwarded > > Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:53) > assert(Universe::heap()->is_in(obj)) failed: should be in heap > > Internal Error (hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:65) > assert(Universe::heap()->is_in_reserved(new_obj)) failed: should be in > object space All these assertions are in MarkSweep::adjust_pointer and they suggest that you are visiting an object with bad references. What's causing this is really hard to say without more information. It sounds like this is easy for you to reproduce, could we run the same tests ourselves? Or could you construct a reproducer that we can run? Thanks, Stefan > /Michael From michael.rasmussen at zeroturnaround.com Wed Apr 5 14:34:10 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Wed, 5 Apr 2017 17:34:10 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Message-ID: On 5 April 2017 at 15:41, Stefan Johansson wrote: > All these assertions are in MarkSweep::adjust_pointer and they suggest that > you are visiting an object with bad references. What's causing this is > really hard to say without more information. It sounds like this is easy for > you to reproduce, could we run the same tests ourselves? Or could you > construct a reproducer that we can run? Yes, it reproduces every single time :) At the moment, I assume it has something to do with java.lang.ref.Reference, as I see InstanceRefKlass::oop_ms_adjust_pointers(oop)+0x281 in the stacktrace in the core file. And I know we use a lot of weak references. Unfortunately, currently I cannot share the test, but I'll try to see if can make a small standalone test case that I can share. /Michael From aph at redhat.com Wed Apr 5 15:13:09 2017 From: aph at redhat.com (Andrew Haley) Date: Wed, 5 Apr 2017 16:13:09 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters Message-ID: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> If sizeof (MethodCounters) is not a multiple of wordSize, memory allocator metadata is corrupted, causing the VM to become unstable and eventually crash. The fix is very simple: diff -r 85b6ca9458ed src/share/vm/oops/methodCounters.hpp --- a/src/share/vm/oops/methodCounters.hpp Wed Mar 29 15:44:34 2017 +0000 +++ b/src/share/vm/oops/methodCounters.hpp Wed Apr 05 15:42:18 2017 +0100 @@ -116,7 +116,7 @@ AOT_ONLY(Method* method() const { return _method; }) - static int size() { return sizeof(MethodCounters) / wordSize; } + static int size() { return align_size_up(sizeof(MethodCounters), wordSize) / wordSize; } bool is_klass() const { return false; } This is very low risk because if the size is already a multiple of wordSize, this patch will have no effect. If the size is not a multiple of wordSize, this patch will prevent an inevitable crash. I've applied for a JDK9 fix request. I'll need a sponsor. Andrew. From kim.barrett at oracle.com Thu Apr 6 01:31:24 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Apr 2017 21:31:24 -0400 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> Message-ID: <432392A3-2DB0-440B-8F9B-FE77D4C03CBD@oracle.com> > On Apr 5, 2017, at 11:13 AM, Andrew Haley wrote: > > If sizeof (MethodCounters) is not a multiple of wordSize, memory > allocator metadata is corrupted, causing the VM to become unstable and > eventually crash. > > The fix is very simple: > > diff -r 85b6ca9458ed src/share/vm/oops/methodCounters.hpp > --- a/src/share/vm/oops/methodCounters.hpp Wed Mar 29 15:44:34 2017 +0000 > +++ b/src/share/vm/oops/methodCounters.hpp Wed Apr 05 15:42:18 2017 +0100 > @@ -116,7 +116,7 @@ > > AOT_ONLY(Method* method() const { return _method; }) > > - static int size() { return sizeof(MethodCounters) / wordSize; } > + static int size() { return align_size_up(sizeof(MethodCounters), wordSize) / wordSize; } > > bool is_klass() const { return false; } > > This is very low risk because if the size is already a multiple of > wordSize, this patch will have no effect. If the size is not a > multiple of wordSize, this patch will prevent an inevitable crash. > > I've applied for a JDK9 fix request. I'll need a sponsor. > > Andrew. It looks like it might require a fairly specific set of build options, but yikes! Change looks good. From aph at redhat.com Thu Apr 6 08:31:39 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 6 Apr 2017 09:31:39 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <432392A3-2DB0-440B-8F9B-FE77D4C03CBD@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <432392A3-2DB0-440B-8F9B-FE77D4C03CBD@oracle.com> Message-ID: On 06/04/17 02:31, Kim Barrett wrote: >> On Apr 5, 2017, at 11:13 AM, Andrew Haley wrote: >> >> If sizeof (MethodCounters) is not a multiple of wordSize, memory >> allocator metadata is corrupted, causing the VM to become unstable and >> eventually crash. >> >> I've applied for a JDK9 fix request. I'll need a sponsor. >> >> Andrew. > > It looks like it might require a fairly specific set of build options, but yikes! > > Change looks good. Yikes indeed. :-) Webrev at http://cr.openjdk.java.net/~aph/8170812/. Thanks, Andrew. From thomas.stuefe at gmail.com Thu Apr 6 09:05:29 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Apr 2017 11:05:29 +0200 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> Message-ID: :( Don't the other childs of MetaspaceObj (e.g. Annotations, ConstMethod, MethodCounters) have the same problem? ..Thomas On Wed, Apr 5, 2017 at 5:13 PM, Andrew Haley wrote: > If sizeof (MethodCounters) is not a multiple of wordSize, memory > allocator metadata is corrupted, causing the VM to become unstable and > eventually crash. > > The fix is very simple: > > diff -r 85b6ca9458ed src/share/vm/oops/methodCounters.hpp > --- a/src/share/vm/oops/methodCounters.hpp Wed Mar 29 15:44:34 2017 > +0000 > +++ b/src/share/vm/oops/methodCounters.hpp Wed Apr 05 15:42:18 2017 > +0100 > @@ -116,7 +116,7 @@ > > AOT_ONLY(Method* method() const { return _method; }) > > - static int size() { return sizeof(MethodCounters) / wordSize; } > + static int size() { return align_size_up(sizeof(MethodCounters), > wordSize) / wordSize; } > > bool is_klass() const { return false; } > > This is very low risk because if the size is already a multiple of > wordSize, this patch will have no effect. If the size is not a > multiple of wordSize, this patch will prevent an inevitable crash. > > I've applied for a JDK9 fix request. I'll need a sponsor. > > Andrew. > From aph at redhat.com Thu Apr 6 09:09:09 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 6 Apr 2017 10:09:09 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> Message-ID: <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> On 06/04/17 10:05, Thomas St?fe wrote: > > Don't the other childs of MetaspaceObj (e.g. Annotations, ConstMethod, MethodCounters) have the same problem? Possibly. If anyone thinks I should fix them all, I will. Andrew. From stefan.johansson at oracle.com Thu Apr 6 09:58:17 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 6 Apr 2017 11:58:17 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Message-ID: On 2017-04-05 16:34, Michael Rasmussen wrote: > On 5 April 2017 at 15:41, Stefan Johansson wrote: >> All these assertions are in MarkSweep::adjust_pointer and they suggest that >> you are visiting an object with bad references. What's causing this is >> really hard to say without more information. It sounds like this is easy for >> you to reproduce, could we run the same tests ourselves? Or could you >> construct a reproducer that we can run? > Yes, it reproduces every single time :) > > At the moment, I assume it has something to do with > java.lang.ref.Reference, as I see > InstanceRefKlass::oop_ms_adjust_pointers(oop)+0x281 in the stacktrace > in the core file. And I know we use a lot of weak references. Interesting, the GC (or rather reference processing) actually do have some assumptions when it comes to the fields in reference objects. We expect the referent, discovered and next fields to be at certain offsets. You mentioned earlier that such assumptions might be problematic, are you doing any kind of instrumentation that could change those offsets? > Unfortunately, currently I cannot share the test, but I'll try to see > if can make a small standalone test case that I can share. That would be really helpful to understand what's going on. Stefan > /Michael From thomas.stuefe at gmail.com Thu Apr 6 10:35:09 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Apr 2017 12:35:09 +0200 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> Message-ID: On Thu, Apr 6, 2017 at 11:09 AM, Andrew Haley wrote: > On 06/04/17 10:05, Thomas St?fe wrote: > > > > Don't the other childs of MetaspaceObj (e.g. Annotations, ConstMethod, > MethodCounters) have the same problem? > > Possibly. If anyone thinks I should fix them all, I will. > > I think it would make sense. From what I can see, the following methods are affected: ConstantPool::header_size() ConstMethod::header_size() ConstantPoolCache::header_size() Method::header_size() MethodCounters::size() Worst thing the alignment is a noop because the class sizes are already multiples of MetaWord. Curiously, there is a function named "align_metadata_size" which gets called in many cases but does not really do much: inline intptr_t align_metadata_size(intptr_t size) { return align_size_up(size, 1); } ..Thomas > Andrew. > > From aph at redhat.com Thu Apr 6 10:57:25 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 6 Apr 2017 11:57:25 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> Message-ID: <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> On 06/04/17 11:35, Thomas St?fe wrote: > > I think it would make sense. From what I can see, the following methods are affected: > > ConstantPool::header_size() Not this one, because the declaration of ConstantPoolCache is int _length; ConstantPool* _constant_pool; // the corresponding constant pool The size of this declaration must be a multiple of wordSize because the fields must be allocated in order, they must be aligned, and _constant_pool comes last. It's very confusing code, but not actually a bug, IMO. Andrew. From michael.rasmussen at zeroturnaround.com Thu Apr 6 11:42:34 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Thu, 6 Apr 2017 14:42:34 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Message-ID: On 6 April 2017 at 12:58, Stefan Johansson wrote: > Interesting, the GC (or rather reference processing) actually do have some > assumptions when it comes to the fields in reference objects. We expect the > referent, discovered and next fields to be at certain offsets. You mentioned > earlier that such assumptions might be problematic, are you doing any kind > of instrumentation that could change those offsets? Generally, we are doing instrumentation that add fields to classes, so can change those offsets, but it doesn't seem to affect the Reference classes. I ran some tests with and without JRebel, printing out the results of unsafe::objectFieldOffset and unsafe::staticFieldOffset for all declared fields on PhantomReference, WeakReference, SoftReference and Reference. The offsets were identical. So assuming those unsafe methods return the equivalent of what's used internally, then we don't modify the offset. We are using a JVMTI agent for instrumenting the early classes (classes loaded before premain is called), I'm currently trying to investigate if this is causing some side effects, since when doing the equivalent using --patch-module, I cannot reproduce it with a simple main() { System.gc(); } test class. /Michael From stefan.johansson at oracle.com Thu Apr 6 12:26:53 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 6 Apr 2017 14:26:53 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Message-ID: On 2017-04-06 13:42, Michael Rasmussen wrote: > On 6 April 2017 at 12:58, Stefan Johansson wrote: >> Interesting, the GC (or rather reference processing) actually do have some >> assumptions when it comes to the fields in reference objects. We expect the >> referent, discovered and next fields to be at certain offsets. You mentioned >> earlier that such assumptions might be problematic, are you doing any kind >> of instrumentation that could change those offsets? > Generally, we are doing instrumentation that add fields to classes, so > can change those offsets, but it doesn't seem to affect the Reference > classes. > I ran some tests with and without JRebel, printing out the results of > unsafe::objectFieldOffset and unsafe::staticFieldOffset for all > declared fields on PhantomReference, WeakReference, SoftReference and > Reference. The offsets were identical. So assuming those unsafe > methods return the equivalent of what's used internally, then we don't > modify the offset. Sounds good, but still suspicious since we seem to be crashing on references. > We are using a JVMTI agent for instrumenting the early classes > (classes loaded before premain is called), I'm currently trying to > investigate if this is causing some side effects, since when doing the > equivalent using --patch-module, I cannot reproduce it with a simple > main() { System.gc(); } test class. One thing that is a bit strange is that you don't reproduce this with SerialGC, G1 and Serial currently share a lot of the Full GC code, but there is a optimization only used for serial that skips objects in the beginning of the heap, could you try running with -XX:+UseSerialGC -XX:MarkSweepDeadRatio=0 -XX:MarkSweepAlwaysCompactCount=1 and see if you run into the same issues with Serial as the other collectors? Thanks, Stefan > > /Michael From coleen.phillimore at oracle.com Thu Apr 6 12:33:49 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Apr 2017 08:33:49 -0400 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> Message-ID: <11eefddd-a7cb-2a8b-594f-9a9f9dedf260@oracle.com> I'm confused. Metaspace is allocated in granularity of 3 pointer sized words. Coleen On 4/5/17 11:13 AM, Andrew Haley wrote: > If sizeof (MethodCounters) is not a multiple of wordSize, memory > allocator metadata is corrupted, causing the VM to become unstable and > eventually crash. > > The fix is very simple: > > diff -r 85b6ca9458ed src/share/vm/oops/methodCounters.hpp > --- a/src/share/vm/oops/methodCounters.hpp Wed Mar 29 15:44:34 2017 +0000 > +++ b/src/share/vm/oops/methodCounters.hpp Wed Apr 05 15:42:18 2017 +0100 > @@ -116,7 +116,7 @@ > > AOT_ONLY(Method* method() const { return _method; }) > > - static int size() { return sizeof(MethodCounters) / wordSize; } > + static int size() { return align_size_up(sizeof(MethodCounters), wordSize) / wordSize; } > > bool is_klass() const { return false; } > > This is very low risk because if the size is already a multiple of > wordSize, this patch will have no effect. If the size is not a > multiple of wordSize, this patch will prevent an inevitable crash. > > I've applied for a JDK9 fix request. I'll need a sponsor. > > Andrew. From ioi.lam at oracle.com Thu Apr 6 12:50:09 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 06 Apr 2017 20:50:09 +0800 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <11eefddd-a7cb-2a8b-594f-9a9f9dedf260@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <11eefddd-a7cb-2a8b-594f-9a9f9dedf260@oracle.com> Message-ID: <58E63981.7090505@oracle.com> On 4/6/17 8:33 PM, coleen.phillimore at oracle.com wrote: > > I'm confused. Metaspace is allocated in granularity of 3 pointer > sized words. > But sizeof(MethodCounters) is pretty big (72 bytes on linux/x86/debug). If it becomes, say 76 bytes, due to some build config options, sizeof(MethodCounters) / wordSize will cause the last 4 bytes to be chopped. - Ioi > Coleen > > On 4/5/17 11:13 AM, Andrew Haley wrote: >> If sizeof (MethodCounters) is not a multiple of wordSize, memory >> allocator metadata is corrupted, causing the VM to become unstable and >> eventually crash. >> >> The fix is very simple: >> >> diff -r 85b6ca9458ed src/share/vm/oops/methodCounters.hpp >> --- a/src/share/vm/oops/methodCounters.hpp Wed Mar 29 15:44:34 >> 2017 +0000 >> +++ b/src/share/vm/oops/methodCounters.hpp Wed Apr 05 15:42:18 >> 2017 +0100 >> @@ -116,7 +116,7 @@ >> >> AOT_ONLY(Method* method() const { return _method; }) >> >> - static int size() { return sizeof(MethodCounters) / wordSize; } >> + static int size() { return align_size_up(sizeof(MethodCounters), >> wordSize) / wordSize; } >> >> bool is_klass() const { return false; } >> >> This is very low risk because if the size is already a multiple of >> wordSize, this patch will have no effect. If the size is not a >> multiple of wordSize, this patch will prevent an inevitable crash. >> >> I've applied for a JDK9 fix request. I'll need a sponsor. >> >> Andrew. > From coleen.phillimore at oracle.com Thu Apr 6 12:56:18 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Apr 2017 08:56:18 -0400 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <58E63981.7090505@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <11eefddd-a7cb-2a8b-594f-9a9f9dedf260@oracle.com> <58E63981.7090505@oracle.com> Message-ID: <3d170f5f-9695-54cb-bb07-e773a78f943c@oracle.com> On 4/6/17 8:50 AM, Ioi Lam wrote: > On 4/6/17 8:33 PM, coleen.phillimore at oracle.com wrote: >> >> I'm confused. Metaspace is allocated in granularity of 3 pointer >> sized words. >> > But sizeof(MethodCounters) is pretty big (72 bytes on > linux/x86/debug). If it becomes, say 76 bytes, due to some build > config options, > > sizeof(MethodCounters) / wordSize > > will cause the last 4 bytes to be chopped. I see. It isn't that it's too small. I think all of the allocation functions might have this, except the last thing in these classes is typically a pointer so they aren't unaligned. I will sponsor this, Andrew. Can you check the other ones? Thanks! Coleen > > - Ioi >> Coleen >> >> On 4/5/17 11:13 AM, Andrew Haley wrote: >>> If sizeof (MethodCounters) is not a multiple of wordSize, memory >>> allocator metadata is corrupted, causing the VM to become unstable and >>> eventually crash. >>> >>> The fix is very simple: >>> >>> diff -r 85b6ca9458ed src/share/vm/oops/methodCounters.hpp >>> --- a/src/share/vm/oops/methodCounters.hpp Wed Mar 29 15:44:34 >>> 2017 +0000 >>> +++ b/src/share/vm/oops/methodCounters.hpp Wed Apr 05 15:42:18 >>> 2017 +0100 >>> @@ -116,7 +116,7 @@ >>> >>> AOT_ONLY(Method* method() const { return _method; }) >>> >>> - static int size() { return sizeof(MethodCounters) / wordSize; } >>> + static int size() { return align_size_up(sizeof(MethodCounters), >>> wordSize) / wordSize; } >>> >>> bool is_klass() const { return false; } >>> >>> This is very low risk because if the size is already a multiple of >>> wordSize, this patch will have no effect. If the size is not a >>> multiple of wordSize, this patch will prevent an inevitable crash. >>> >>> I've applied for a JDK9 fix request. I'll need a sponsor. >>> >>> Andrew. >> > From aph at redhat.com Thu Apr 6 13:01:21 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 6 Apr 2017 14:01:21 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <3d170f5f-9695-54cb-bb07-e773a78f943c@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <11eefddd-a7cb-2a8b-594f-9a9f9dedf260@oracle.com> <58E63981.7090505@oracle.com> <3d170f5f-9695-54cb-bb07-e773a78f943c@oracle.com> Message-ID: <65e69107-f05b-4fc8-cf69-5a87497acd4c@redhat.com> On 06/04/17 13:56, coleen.phillimore at oracle.com wrote: > I see. It isn't that it's too small. I think all of the allocation > functions might have this, except the last thing in these classes is > typically a pointer so they aren't unaligned. > > I will sponsor this, Andrew. Can you check the other ones? Yes. Do you want me to leave the ones which always end with a pointer for JDK10, or fix the lot now? Andrew. From thomas.stuefe at gmail.com Thu Apr 6 13:02:38 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Apr 2017 15:02:38 +0200 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> Message-ID: Hi Andrew, On Thu, Apr 6, 2017 at 12:57 PM, Andrew Haley wrote: > On 06/04/17 11:35, Thomas St?fe wrote: > > > > I think it would make sense. From what I can see, the following methods > are affected: > > > > ConstantPool::header_size() > > Not this one, because the declaration of ConstantPoolCache is > > int _length; > ConstantPool* _constant_pool; // the corresponding constant > pool > > The size of this declaration must be a multiple of wordSize > because the fields must be allocated in order, they must be > aligned, and _constant_pool comes last. > > It's very confusing code, but not actually a bug, IMO. > > I am confused: ConstantPool::header_size() is used by ConstantPool::size(len), which in turn is used in ConstantPool::allocate() to allocate space for a ConstantPool. header_size() may be too small, and so the space allocated for the whole ConstantPool plus data may be too small, no? ..Thomas > Andrew. > > From coleen.phillimore at oracle.com Thu Apr 6 13:20:27 2017 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 6 Apr 2017 09:20:27 -0400 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> Message-ID: <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> Deciding... I say fix the lot now. I think it's low risk to do so and will save filing another bug etc. Thanks! Coleen > On Apr 6, 2017, at 9:02 AM, Thomas St?fe wrote: > > Hi Andrew, > >> On Thu, Apr 6, 2017 at 12:57 PM, Andrew Haley wrote: >> >>> On 06/04/17 11:35, Thomas St?fe wrote: >>> >>> I think it would make sense. From what I can see, the following methods >> are affected: >>> >>> ConstantPool::header_size() >> >> Not this one, because the declaration of ConstantPoolCache is >> >> int _length; >> ConstantPool* _constant_pool; // the corresponding constant >> pool >> >> The size of this declaration must be a multiple of wordSize >> because the fields must be allocated in order, they must be >> aligned, and _constant_pool comes last. >> >> It's very confusing code, but not actually a bug, IMO. >> >> > I am confused: > > ConstantPool::header_size() is used by ConstantPool::size(len), which in > turn is used in ConstantPool::allocate() to allocate space for a > ConstantPool. header_size() may be too small, and so the space allocated > for the whole ConstantPool plus data may be too small, no? > > ..Thomas > > > >> Andrew. >> >> From ioi.lam at oracle.com Thu Apr 6 13:29:12 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 06 Apr 2017 21:29:12 +0800 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> Message-ID: <58E642A8.70307@oracle.com> I think we should also fix align_metadata_size() and use that for the size() function of all MetaspaceObj classes. Thanks - Ioi On 4/6/17 9:20 PM, Coleen Phillimore wrote: > Deciding... I say fix the lot now. I think it's low risk to do so and will save filing another bug etc. > Thanks! > Coleen > >> On Apr 6, 2017, at 9:02 AM, Thomas St?fe wrote: >> >> Hi Andrew, >> >>> On Thu, Apr 6, 2017 at 12:57 PM, Andrew Haley wrote: >>> >>>> On 06/04/17 11:35, Thomas St?fe wrote: >>>> >>>> I think it would make sense. From what I can see, the following methods >>> are affected: >>>> ConstantPool::header_size() >>> Not this one, because the declaration of ConstantPoolCache is >>> >>> int _length; >>> ConstantPool* _constant_pool; // the corresponding constant >>> pool >>> >>> The size of this declaration must be a multiple of wordSize >>> because the fields must be allocated in order, they must be >>> aligned, and _constant_pool comes last. >>> >>> It's very confusing code, but not actually a bug, IMO. >>> >>> >> I am confused: >> >> ConstantPool::header_size() is used by ConstantPool::size(len), which in >> turn is used in ConstantPool::allocate() to allocate space for a >> ConstantPool. header_size() may be too small, and so the space allocated >> for the whole ConstantPool plus data may be too small, no? >> >> ..Thomas >> >> >> >>> Andrew. >>> >>> From aph at redhat.com Thu Apr 6 13:47:52 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 6 Apr 2017 14:47:52 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <58E642A8.70307@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> <58E642A8.70307@oracle.com> Message-ID: <53e3e760-38b8-45ad-4b29-5c9bb3174122@redhat.com> On 06/04/17 14:29, Ioi Lam wrote: > I think we should also fix align_metadata_size() and use that for the > size() function of all MetaspaceObj classes. IMO: in this phase let's do the minimum correct thing, which is calling align_size_up() in all of the size() methods. Andrew. From mikael.vidstedt at oracle.com Thu Apr 6 14:13:26 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 6 Apr 2017 07:13:26 -0700 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> Message-ID: <625BF222-B6F0-441C-A3AD-AB1189D0281F@oracle.com> Worth adding a few helpful asserts somewhere? I?m thinking something like: assert(MethodCounters::size() >= sizeof(MethodCounters)); Cheers, Mikael > On Apr 5, 2017, at 8:13 AM, Andrew Haley wrote: > > If sizeof (MethodCounters) is not a multiple of wordSize, memory > allocator metadata is corrupted, causing the VM to become unstable and > eventually crash. > > The fix is very simple: > > diff -r 85b6ca9458ed src/share/vm/oops/methodCounters.hpp > --- a/src/share/vm/oops/methodCounters.hpp Wed Mar 29 15:44:34 2017 +0000 > +++ b/src/share/vm/oops/methodCounters.hpp Wed Apr 05 15:42:18 2017 +0100 > @@ -116,7 +116,7 @@ > > AOT_ONLY(Method* method() const { return _method; }) > > - static int size() { return sizeof(MethodCounters) / wordSize; } > + static int size() { return align_size_up(sizeof(MethodCounters), wordSize) / wordSize; } > > bool is_klass() const { return false; } > > This is very low risk because if the size is already a multiple of > wordSize, this patch will have no effect. If the size is not a > multiple of wordSize, this patch will prevent an inevitable crash. > > I've applied for a JDK9 fix request. I'll need a sponsor. > > Andrew. From michael.rasmussen at zeroturnaround.com Thu Apr 6 15:36:27 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Thu, 6 Apr 2017 18:36:27 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Message-ID: On 6 April 2017 at 15:26, Stefan Johansson wrote: > could you try running with -XX:+UseSerialGC > -XX:MarkSweepDeadRatio=0 -XX:MarkSweepAlwaysCompactCount=1 and see if you > run into the same issues with Serial as the other collectors? With that, it fails on the very first GC: # Internal Error (genOopClosures.inline.hpp:110), pid=28567, tid=28569 # assert(!_g->to()->is_in_reserved(obj)) failed: Scanning field twice? With the following stack trace in hs_err (before report_vm_error is called): V [libjvm.so+0xcd37a6] void FastScanClosure::do_oop_work(unsigned int*)+0x1d6 V [libjvm.so+0xcd6fce] void InstanceRefKlass::oop_oop_iterate_ref_processing(oop, FastScanClosure*)+0x37e V [libjvm.so+0xcdade7] void InstanceRefKlass::oop_oop_iterate(oop, FastScanClosure*)+0x187 V [libjvm.so+0xcceff2] InstanceRefKlass::oop_oop_iterate_nv(oop, FastScanClosure*)+0x32 I'm still trying to debug if it's somewhere in our native code that triggers this, although I don't understand how/if we can touch something this deep in the GC. /Michael From michael.rasmussen at zeroturnaround.com Thu Apr 6 15:51:17 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Thu, 6 Apr 2017 18:51:17 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Message-ID: On 6 April 2017 at 18:36, Michael Rasmussen wrote: > I'm still trying to debug if it's somewhere in our native code that > triggers this, although I don't understand how/if we can touch > something this deep in the GC. Ok, think I've found the likely culprit. I think some of the instrumentation we're doing to these early classes are causing the Reference class to be loaded too soon! Would that explain this behavior? /Michael From HORIE at jp.ibm.com Fri Apr 7 05:49:55 2017 From: HORIE at jp.ibm.com (Michihiro Horie) Date: Fri, 7 Apr 2017 14:49:55 +0900 Subject: Optimizing byte reverse code for int value Message-ID: Dear all, Would you please review our change for JDK10 on ppc64? Issue: https://bugs.openjdk.java.net/browse/JDK-8178294 Webrev: http://cr.openjdk.java.net/~horii/8178294/webrev.00/ This change adds two conversion rules of reversing contiguous 4 bytes for int value. The first conversion rule finds a pattern below and emits a lwz instruction instead. Original: lbz r14,19(r12) lbz r11,17(r12) lbz r10,18(r12) lbz r9,16(r12) extsb r14,r14 rlwinm r10,r10,16,0,15 rlwinm r14,r14,24,0,7 add r14,r10,r14 rlwinm r11,r11,8,0,23 add r12,r11,r9 add r14,r12,r14 Optimization with first conversion rule: lwz r14,16(r12) The second conversion rule finds a pattern below and emits only lfs instruction. Original: lbz r14,19(r12) lbz r11,17(r12) lbz r10,18(r12) lbz r9,16(r12) extsb r14,r14 rlwinm r10,r10,16,0,15 rlwinm r14,r14,24,0,7 add r14,r10,r14 rlwinm r11,r11,8,0,23 add r12,r11,r9 add r14,r12,r14 stw r14,156(r1) lfs f12,156(r1) Optimization with first conversion rule: lfs f12,156(r1) Our motivation comes from the fact that a performance bottleneck exists in byte reversing code in Apache ORC on Tez framework as shown below. https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/impl/SerializationUtils.java We believe this kind of procedures is typical in Java. public float readFloat(InputStream in) throws IOException { readFully(in, readBuffer, 0, 4); int val = (((readBuffer[0] & 0xff) << 0) + ((readBuffer[1] & 0xff) << 8) + ((readBuffer[2] & 0xff) << 16) + ((readBuffer[3] & 0xff) << 24)); return Float.intBitsToFloat(val); } By using our change, we could observe 5% performance improvement in a micro benchmark. (See attached file: ReadFloatTest.java) Best regards, -- Michihiro, IBM Research - Tokyo From martin.doerr at sap.com Fri Apr 7 07:12:34 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 7 Apr 2017 07:12:34 +0000 Subject: Optimizing byte reverse code for int value In-Reply-To: References: Message-ID: <4d45658f669541379b167df2ca6d6e2b@sap.com> Hi Michihiro, thanks for providing the webrev. I appreciate improvements for this bottleneck. After taking a first look over it, it looks good to me for ppc64le. But I think it would break big endian platforms. I suggest replacing the use of loadI by endianness specific code (which could possibly use lwbrx on big endian). Best regards, Martin From: Michihiro Horie [mailto:HORIE at jp.ibm.com] Sent: Freitag, 7. April 2017 07:50 To: ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net Cc: Doerr, Martin ; Simonis, Volker ; volker.simonis at gmail.com; Hiroshi H Horii ; Lindenmaier, Goetz ; Gustavo Bueno Romero Subject: Optimizing byte reverse code for int value Dear all, Would you please review our change for JDK10 on ppc64? Issue: https://bugs.openjdk.java.net/browse/JDK-8178294 Webrev: http://cr.openjdk.java.net/~horii/8178294/webrev.00/ This change adds two conversion rules of reversing contiguous 4 bytes for int value. The first conversion rule finds a pattern below and emits a lwz instruction instead. Original: lbz r14,19(r12) lbz r11,17(r12) lbz r10,18(r12) lbz r9,16(r12) extsb r14,r14 rlwinm r10,r10,16,0,15 rlwinm r14,r14,24,0,7 add r14,r10,r14 rlwinm r11,r11,8,0,23 add r12,r11,r9 add r14,r12,r14 Optimization with first conversion rule: lwz r14,16(r12) The second conversion rule finds a pattern below and emits only lfs instruction. Original: lbz r14,19(r12) lbz r11,17(r12) lbz r10,18(r12) lbz r9,16(r12) extsb r14,r14 rlwinm r10,r10,16,0,15 rlwinm r14,r14,24,0,7 add r14,r10,r14 rlwinm r11,r11,8,0,23 add r12,r11,r9 add r14,r12,r14 stw r14,156(r1) lfs f12,156(r1) Optimization with first conversion rule: lfs f12,156(r1) Our motivation comes from the fact that a performance bottleneck exists in byte reversing code in Apache ORC on Tez framework as shown below. https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/impl/SerializationUtils.java We believe this kind of procedures is typical in Java. public float readFloat(InputStream in) throws IOException { readFully(in, readBuffer, 0, 4); int val = (((readBuffer[0] & 0xff) << 0) + ((readBuffer[1] & 0xff) << 8) + ((readBuffer[2] & 0xff) << 16) + ((readBuffer[3] & 0xff) << 24)); return Float.intBitsToFloat(val); } By using our change, we could observe 5% performance improvement in a micro benchmark. (See attached file: ReadFloatTest.java) Best regards, -- Michihiro, IBM Research - Tokyo From aph at redhat.com Fri Apr 7 08:36:13 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 7 Apr 2017 09:36:13 +0100 Subject: Optimizing byte reverse code for int value In-Reply-To: References: Message-ID: On 07/04/17 06:49, Michihiro Horie wrote: > > Would you please review our change for JDK10 on ppc64? > Issue: https://bugs.openjdk.java.net/browse/JDK-8178294 > Webrev: http://cr.openjdk.java.net/~horii/8178294/webrev.00/ > > This change adds two conversion rules of reversing contiguous 4 bytes for > int value. > The first conversion rule finds a pattern below and emits a lwz instruction > instead. Surely the source code needs fixing. It could be: public float readFloat(InputStream in) throws IOException { readFully(in, aByteBuffer, 0, 4); int val = aByteBuffer.getInt(0); return Float.intBitsToFloat(val); } Then there would be no need for a special ppc64 pattern. Andrew. From aph at redhat.com Fri Apr 7 12:48:11 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 7 Apr 2017 13:48:11 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> Message-ID: <318d8fe0-9cfa-2b4e-7238-c870f6d1235e@redhat.com> On 06/04/17 14:20, Coleen Phillimore wrote: > Deciding... I say fix the lot now. I think it's low risk to do so and will save filing another bug etc. > Thanks! http://cr.openjdk.java.net/~aph/8170812-2/ Andrew. From kim.barrett at oracle.com Fri Apr 7 14:10:58 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Apr 2017 10:10:58 -0400 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <318d8fe0-9cfa-2b4e-7238-c870f6d1235e@redhat.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> <318d8fe0-9cfa-2b4e-7238-c870f6d1235e@redhat.com> Message-ID: <2A1D167A-CBA6-4A33-97EE-A2288C826602@oracle.com> > On Apr 7, 2017, at 8:48 AM, Andrew Haley wrote: > > On 06/04/17 14:20, Coleen Phillimore wrote: >> Deciding... I say fix the lot now. I think it's low risk to do so and will save filing another bug etc. >> Thanks! > > > http://cr.openjdk.java.net/~aph/8170812-2/ > > Andrew. Looks good. From aph at redhat.com Fri Apr 7 14:20:43 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 7 Apr 2017 15:20:43 +0100 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: <2A1D167A-CBA6-4A33-97EE-A2288C826602@oracle.com> References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> <318d8fe0-9cfa-2b4e-7238-c870f6d1235e@redhat.com> <2A1D167A-CBA6-4A33-97EE-A2288C826602@oracle.com> Message-ID: On 07/04/17 15:10, Kim Barrett wrote: >> On Apr 7, 2017, at 8:48 AM, Andrew Haley wrote: >> >> On 06/04/17 14:20, Coleen Phillimore wrote: >>> Deciding... I say fix the lot now. I think it's low risk to do so and will save filing another bug etc. >>> Thanks! >> >> >> http://cr.openjdk.java.net/~aph/8170812-2/ >> >> Andrew. > > Looks good. OK. Will you please push this? Thanks, Andrew. From coleen.phillimore at oracle.com Fri Apr 7 14:24:55 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 7 Apr 2017 10:24:55 -0400 Subject: RFR: 8170812: Metaspace corruption caused by incorrect memory size for MethodCounters In-Reply-To: References: <4640eee7-938b-5dc1-8b1c-13f89ff3eabc@redhat.com> <7a7106f4-bbe2-b708-0277-ca2bec9751d3@redhat.com> <42a57b75-6570-9d0f-1460-28d11eeb0547@redhat.com> <8068E42D-89B0-4BF2-98C0-E17E7939B353@oracle.com> <318d8fe0-9cfa-2b4e-7238-c870f6d1235e@redhat.com> <2A1D167A-CBA6-4A33-97EE-A2288C826602@oracle.com> Message-ID: On 4/7/17 10:20 AM, Andrew Haley wrote: > On 07/04/17 15:10, Kim Barrett wrote: >>> On Apr 7, 2017, at 8:48 AM, Andrew Haley wrote: >>> >>> On 06/04/17 14:20, Coleen Phillimore wrote: >>>> Deciding... I say fix the lot now. I think it's low risk to do so and will save filing another bug etc. >>>> Thanks! >>> >>> http://cr.openjdk.java.net/~aph/8170812-2/ >>> >>> Andrew. >> Looks good. > OK. Will you please push this? yes, this looks good to me also. I'm in the process of pushing it. thanks for finding this and the fix. Coleen > > Thanks, > > Andrew. > > From coleen.phillimore at oracle.com Fri Apr 7 14:40:16 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 7 Apr 2017 10:40:16 -0400 Subject: RFR 8150388: Remove SPARC 32-bit support In-Reply-To: <2f1f97e9-8b33-c7ba-4eb7-3605ba82922a@oracle.com> References: <2f1f97e9-8b33-c7ba-4eb7-3605ba82922a@oracle.com> Message-ID: This looks great but I think we should have the compiler group review also, since lots of compiler code is changed. thanks! Coleen On 4/7/17 9:43 AM, George Triantafillou wrote: > Please review this fix to remove SPARC 32-bit support. Support for > solaris-sparc has been dropped from the list of supported platforms. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8150388 > webrev: > http://cr.openjdk.java.net/~gtriantafill/8150388-webrev/webrev/ > > > Built and tested on solaris-sparcv9-debug,solaris-x64-debug with the > nsk.jvmti, nsk.jdwp, and nsk.jdi testlists. > > Thanks. > > -George > From HORII at jp.ibm.com Fri Apr 7 17:51:09 2017 From: HORII at jp.ibm.com (Hiroshi H Horii) Date: Sat, 8 Apr 2017 02:51:09 +0900 Subject: Optimizing byte reverse code for int value In-Reply-To: References: Message-ID: > I suggest replacing the use of loadI by endianness specific code > (which could possibly use lwbrx on big endian). I believe that lwbrx is necessary for BE. > Surely the source code needs fixing. It could be: > > public float readFloat(InputStream in) throws IOException { > readFully(in, aByteBuffer, 0, 4); > int val = aByteBuffer.getInt(0); > > return Float.intBitsToFloat(val); > } In my understanding, ByteBuffer.getInt() does the similar thing. I guess that application does not use ByteBuffer only for calling getInt(). Heap-X-Buffer.java.template public int getInt() { return Bits.getInt(this, ix(nextGetIndex(4)), bigEndian); } Bits.java static int getInt(ByteBuffer bb, int bi, boolean bigEndian) { return bigEndian ? getIntB(bb, bi) : getIntL(bb, bi) ; } static int getIntL(ByteBuffer bb, int bi) { return makeInt(bb._get(bi + 3), bb._get(bi + 2), bb._get(bi + 1), bb._get(bi )); } static private int makeInt(byte b3, byte b2, byte b1, byte b0) { return (((b3 ) << 24) | ((b2 & 0xff) << 16) | ((b1 & 0xff) << 8) | ((b0 & 0xff) )); } Heap-X-Buffer.java.template byte _get(int i) { // package-private return hb[i]; } Direct-X-Buffer.java.template byte _get(int i) { // package-private return unsafe.getByte(address + i); } Regards, Hiroshi Andrew Haley wrote on 2017/04/07 17:36:13: > From: Andrew Haley > To: Michihiro Horie/Japan/IBM at IBMJP, ppc-aix-port- > dev at openjdk.java.net, hotspot-dev at openjdk.java.net > Cc: Hiroshi H Horii/Japan/IBM at IBMJP, volker.simonis at sap.com > Date: 2017/04/07 17:37 > Subject: Re: Optimizing byte reverse code for int value > > On 07/04/17 06:49, Michihiro Horie wrote: > > > > Would you please review our change for JDK10 on ppc64? > > Issue: https://bugs.openjdk.java.net/browse/JDK-8178294 > > Webrev: http://cr.openjdk.java.net/~horii/8178294/webrev.00/ > > > > This change adds two conversion rules of reversing contiguous 4 bytes for > > int value. > > The first conversion rule finds a pattern below and emits a lwz instruction > > instead. > > Surely the source code needs fixing. It could be: > > public float readFloat(InputStream in) throws IOException { > readFully(in, aByteBuffer, 0, 4); > int val = aByteBuffer.getInt(0); > > return Float.intBitsToFloat(val); > } > > Then there would be no need for a special ppc64 pattern. > > Andrew. > From aph at redhat.com Fri Apr 7 17:58:45 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 7 Apr 2017 18:58:45 +0100 Subject: Optimizing byte reverse code for int value In-Reply-To: References: Message-ID: <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> On 07/04/17 18:51, Hiroshi H Horii wrote: >> I suggest replacing the use of loadI by endianness specific code >> (which could possibly use lwbrx on big endian). > > I believe that lwbrx is necessary for BE. > >> Surely the source code needs fixing. It could be: >> >> public float readFloat(InputStream in) throws IOException { >> readFully(in, aByteBuffer, 0, 4); >> int val = aByteBuffer.getInt(0); >> >> return Float.intBitsToFloat(val); >> } > > In my understanding, ByteBuffer.getInt() does the similar thing. It doesn't. > I guess that application does not use ByteBuffer only for calling getInt(). > > Heap-X-Buffer.java.template > public int getInt() { > return Bits.getInt(this, ix(nextGetIndex(4)), bigEndian); > } This is old code. In JDK9 it looks like public int getInt() { return unsafe.getIntUnaligned(hb, byteOffset(nextGetIndex(4)), bigEndian); } Unsafe.getIntUnaligned is a HotSpot intrinsic. Bits.java is not used for this. Andrew. From HORII at jp.ibm.com Fri Apr 7 18:13:16 2017 From: HORII at jp.ibm.com (Hiroshi H Horii) Date: Sat, 8 Apr 2017 03:13:16 +0900 Subject: Optimizing byte reverse code for int value In-Reply-To: <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> References: <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> Message-ID: > > I guess that application does not use ByteBuffer only for calling getInt(). > > > > Heap-X-Buffer.java.template > > public int getInt() { > > return Bits.getInt(this, ix(nextGetIndex(4)), bigEndian); > > } > > This is old code. In JDK9 it looks like > > public int getInt() { > return unsafe.getIntUnaligned(hb, > byteOffset(nextGetIndex(4)), bigEndian); > } > > Unsafe.getIntUnaligned is a HotSpot intrinsic. Sorry. You are right. I checked jdk8u... jdk8u doesn't have Unsafe.getIntUnaligned. This change will improve jdk8u. Regards, Hiroshi Andrew Haley wrote on 2017/04/08 02:58:45: > From: Andrew Haley > To: Hiroshi H Horii/Japan/IBM at IBMJP > Cc: hotspot-dev at openjdk.java.net, Michihiro Horie/Japan/IBM at IBMJP, > ppc-aix-port-dev at openjdk.java.net, volker.simonis at sap.com > Date: 2017/04/08 02:59 > Subject: Re: Optimizing byte reverse code for int value > > On 07/04/17 18:51, Hiroshi H Horii wrote: > >> I suggest replacing the use of loadI by endianness specific code > >> (which could possibly use lwbrx on big endian). > > > > I believe that lwbrx is necessary for BE. > > > >> Surely the source code needs fixing. It could be: > >> > >> public float readFloat(InputStream in) throws IOException { > >> readFully(in, aByteBuffer, 0, 4); > >> int val = aByteBuffer.getInt(0); > >> > >> return Float.intBitsToFloat(val); > >> } > > > > In my understanding, ByteBuffer.getInt() does the similar thing. > > It doesn't. > > > I guess that application does not use ByteBuffer only for calling getInt(). > > > > Heap-X-Buffer.java.template > > public int getInt() { > > return Bits.getInt(this, ix(nextGetIndex(4)), bigEndian); > > } > > This is old code. In JDK9 it looks like > > public int getInt() { > return unsafe.getIntUnaligned(hb, > byteOffset(nextGetIndex(4)), bigEndian); > } > > Unsafe.getIntUnaligned is a HotSpot intrinsic. > > Bits.java is not used for this. > > Andrew. > From kim.barrett at oracle.com Sun Apr 9 22:00:16 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 9 Apr 2017 18:00:16 -0400 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac Message-ID: Please review this change to BitMap::get_next_zero_offset_inline to remove its use of left_n_bits, thereby avoiding https://bugs.openjdk.java.net/browse/JDK-8178348. CR: https://bugs.openjdk.java.net/browse/JDK-8178352 Webrev: http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.00/ Testing: JPRT, with the test from https://bugs.openjdk.java.net/browse/JDK-8169039 included in the tested repository. From stefan.karlsson at oracle.com Mon Apr 10 07:44:38 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 10 Apr 2017 09:44:38 +0200 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: References: Message-ID: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> Hi Kim, On 2017-04-10 00:00, Kim Barrett wrote: > Please review this change to BitMap::get_next_zero_offset_inline to > remove its use of left_n_bits, thereby avoiding > https://bugs.openjdk.java.net/browse/JDK-8178348. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8178352 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.00/ The bug fix looks good. You did a couple of stylistic changes that now make the code inconsistent: - for (; res & 1; res_offset++) { - res = res >> 1; + for (; (res & 1) == 0; ++res_offset) { + res >>= 1; But the surrounding code uses this format: 271 for (; !(res & 1); res_offset++) { 272 res = res >> 1; 273 } I think these changes should be kept out of this patch. Thanks, StefanK > > Testing: > JPRT, with the test from > https://bugs.openjdk.java.net/browse/JDK-8169039 > included in the tested repository. > From stefan.johansson at oracle.com Mon Apr 10 12:31:49 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 10 Apr 2017 14:31:49 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> Message-ID: <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> On 2017-04-06 17:51, Michael Rasmussen wrote: > On 6 April 2017 at 18:36, Michael Rasmussen > wrote: >> I'm still trying to debug if it's somewhere in our native code that >> triggers this, although I don't understand how/if we can touch >> something this deep in the GC. > Ok, think I've found the likely culprit. I think some of the > instrumentation we're doing to these early classes are causing the > Reference class to be loaded too soon! > Would that explain this behavior? I'm not sure I follow what too soon mean. For the GC we should be fine as long as the offsets are correct, and you seem to have verified this. I guess there could be an issue where loading the Reference class earlier than expected could cause us to write bad oops into the reference fields somehow and that would then crash in the GC. But I would have expected that to have been caught by running with -XX:+VerifyBeforeGC. Stefan > /Michael From michael.rasmussen at zeroturnaround.com Mon Apr 10 13:31:40 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Mon, 10 Apr 2017 16:31:40 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> Message-ID: On 10 April 2017 at 15:31, Stefan Johansson wrote: > I'm not sure I follow what too soon mean. For the GC we should be fine as > long as the offsets are correct, and you seem to have verified this. Here is a minimal test case, with a JVMTI agent that defines a subtype of WeakReference "too soon", causing the issue: https://gist.github.com/anonymous/d6fbe111082b0059cbeb36b90ba87558 /Michael From stefan.johansson at oracle.com Mon Apr 10 14:48:12 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 10 Apr 2017 16:48:12 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> Message-ID: On 2017-04-10 15:31, Michael Rasmussen wrote: > On 10 April 2017 at 15:31, Stefan Johansson wrote: >> I'm not sure I follow what too soon mean. For the GC we should be fine as >> long as the offsets are correct, and you seem to have verified this. > Here is a minimal test case, with a JVMTI agent that defines a subtype > of WeakReference "too soon", causing the issue: > https://gist.github.com/anonymous/d6fbe111082b0059cbeb36b90ba87558 Thanks Michael, This sounds great. Could you attach the test case to the mailing list in case we want to add this to the OpenJDK testing going forward? Thanks, Stefan > /Michael From michael.rasmussen at zeroturnaround.com Mon Apr 10 15:37:23 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Mon, 10 Apr 2017 18:37:23 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> Message-ID: On 10 April 2017 at 17:48, Stefan Johansson wrote: > This sounds great. Could you attach the test case to the mailing list in > case we want to add this to the OpenJDK testing going forward? Attached inline below /Michael --- agent.c --- #include #include "jni.h" #include "jvmti.h" /* public class java.lang.ref.MyWeakReference extends java.lang.ref.WeakReference {}; */ jbyte MyWeakReference_bytes[] = { -54, -2, -70, -66, 0, 0, 0, 49, 0, 5, 1, 0, 29, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 114, 101, 102, 47, 77, 121, 87, 101, 97, 107, 82, 101, 102, 101, 114, 101, 110, 99, 101, 7, 0, 1, 1, 0, 27, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 114, 101, 102, 47, 87, 101, 97, 107, 82, 101, 102, 101, 114, 101, 110, 99, 101, 7, 0, 3, 0, 1, 0, 2, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0 }; jsize MyWeakReference_len = 0x5C; void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, jclass class_being_redefined, jobject loader, const char* name, jobject protection_domain, jint class_data_len, const unsigned char* class_data, jint* new_class_data_len, unsigned char** new_class_data) { if (name && !strcmp(name, "java/lang/System")) { (*env)->DefineClass(env, "java/lang/ref/MyWeakReference", NULL, MyWeakReference_bytes, MyWeakReference_len); } } JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, char *options, void *reserved) { jvmtiEventCallbacks callbacks; jvmtiCapabilities caps; jvmtiEnv *jvmti; (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); memset(&caps, 0, sizeof(caps)); memset(&callbacks, 0, sizeof(callbacks)); caps.can_generate_early_vmstart = 1; caps.can_generate_all_class_hook_events = 1; caps.can_generate_early_class_hook_events = 1; callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; (*jvmti)->AddCapabilities(jvmti, &caps); (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks)); (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); return 0; } --- MiscTest9.java --- package app1; import java.lang.ref.WeakReference; public class MiscTest9 { public static void main(String[] args) throws Exception { WeakReference weak = new WeakReference<>(new MiscTest9()); boolean present = true; while (present) { System.out.println("Running GC"); System.gc(); present = weak.get() != null; } } } From kim.barrett at oracle.com Mon Apr 10 16:18:29 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Apr 2017 12:18:29 -0400 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> Message-ID: <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> > On Apr 10, 2017, at 3:44 AM, Stefan Karlsson wrote: > > Hi Kim, > > On 2017-04-10 00:00, Kim Barrett wrote: >> Please review this change to BitMap::get_next_zero_offset_inline to >> remove its use of left_n_bits, thereby avoiding >> https://bugs.openjdk.java.net/browse/JDK-8178348. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8178352 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.00/ > > > The bug fix looks good. Thanks. > You did a couple of stylistic changes that now make the code inconsistent: > > - for (; res & 1; res_offset++) { > - res = res >> 1; > > + for (; (res & 1) == 0; ++res_offset) { > + res >>= 1; > > > But the surrounding code uses this format: > 271 for (; !(res & 1); res_offset++) { > 272 res = res >> 1; > 273 } > > I think these changes should be kept out of this patch. Post-decrement in for loops still makes me twitch, even though it doesn?t matter in this code base. But okay. From kim.barrett at oracle.com Mon Apr 10 16:33:34 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Apr 2017 12:33:34 -0400 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> Message-ID: > On Apr 10, 2017, at 12:18 PM, Kim Barrett wrote: >> You did a couple of stylistic changes that now make the code inconsistent: >> >> [?] >> I think these changes should be kept out of this patch. > > Post-decrement in for loops still makes me twitch, even though it doesn?t matter > in this code base. But okay. Updated webrev: http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.01/ From stefan.karlsson at oracle.com Mon Apr 10 16:48:03 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 10 Apr 2017 18:48:03 +0200 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> Message-ID: On 10/04/17 18:33, Kim Barrett wrote: >> On Apr 10, 2017, at 12:18 PM, Kim Barrett wrote: >>> You did a couple of stylistic changes that now make the code inconsistent: >>> >>> [?] >>> I think these changes should be kept out of this patch. >> Post-decrement in for loops still makes me twitch, even though it doesn?t matter >> in this code base. But okay. > Updated webrev: > http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.01/ > To be consistent with the other code it in that file (res & 1) == 0 should be !(res & 1). Maybe create a small, trivial cleanup RFE to change the following in the BitMap files: * !(res & 1) => (res & 1) == 0 * res = res >> 1 => res >>= 1 I'll promise to do a quick review ;) StefanK From volker.simonis at gmail.com Mon Apr 10 18:14:17 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 10 Apr 2017 20:14:17 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> Message-ID: Hi Michael, I think the problem is the following: In normal operation (i.e. without your agent) classes get loaded in the following order: ... [0.046s][info][class,load] java.lang.ClassLoader source: jrt:/java.base [0.046s][info][class,load] java.lang.System source: jrt:/java.base ... [0.047s][info][class,load] java.lang.ref.Reference source: jrt:/java.base [0.047s][info][class,load] java.lang.ref.SoftReference source: jrt:/java.base [0.047s][info][class,load] java.lang.ref.WeakReference source: jrt:/java.base [0.047s][info][class,load] java.lang.ref.FinalReference source: jrt:/java.base [0.047s][info][class,load] java.lang.ref.PhantomReference source: jrt:/java.base This order is specified in systemDictionary.hpp: // The order of these definitions is significant; it is the order in which // preloading is actually performed by initialize_preloaded_classes. #define WK_KLASSES_DO(do_klass) \ /* well-known classes */ \ do_klass(Object_klass, java_lang_Object, Pre ) \ do_klass(String_klass, java_lang_String, Pre ) \ do_klass(Class_klass, java_lang_Class, Pre ) \ do_klass(Cloneable_klass, java_lang_Cloneable, Pre ) \ do_klass(ClassLoader_klass, java_lang_ClassLoader, Pre ) \ do_klass(Serializable_klass, java_io_Serializable, Pre ) \ do_klass(System_klass, java_lang_System, Pre ) \ ... do_klass(Reference_klass, java_lang_ref_Reference, Pre ) \ /* Preload ref klasses and set reference types */ \ do_klass(SoftReference_klass, java_lang_ref_SoftReference, Pre ) \ do_klass(WeakReference_klass, java_lang_ref_WeakReference, Pre ) \ do_klass(FinalReference_klass, java_lang_ref_FinalReference, Pre ) \ do_klass(PhantomReference_klass, java_lang_ref_PhantomReference, Pre ) \ The method 'SystemDictionary::initialize_preloaded_classes(TRAPS)' in systemDictionary.cpp not only loads the classes in this order, it also does some initialization stuff in between (i.e. specific initializations after some specific class have been loaded): initialize_wk_klasses_through(WK_KLASS_ENUM_NAME(Reference_klass), scan, CHECK); // Preload ref klasses and set reference types InstanceKlass::cast(WK_KLASS(Reference_klass))->set_reference_type(REF_OTHER); InstanceRefKlass::update_nonstatic_oop_maps(WK_KLASS(Reference_klass)); initialize_wk_klasses_through(WK_KLASS_ENUM_NAME(PhantomReference_klass), scan, CHECK); InstanceKlass::cast(WK_KLASS(SoftReference_klass))->set_reference_type(REF_SOFT); InstanceKlass::cast(WK_KLASS(WeakReference_klass))->set_reference_type(REF_WEAK); InstanceKlass::cast(WK_KLASS(FinalReference_klass))->set_reference_type(REF_FINAL); InstanceKlass::cast(WK_KLASS(PhantomReference_klass))->set_reference_type(REF_PHANTOM); As you can see, first all the classes up to java.lang.ref.Reference are loaded. Then the reference type of java.lang.ref.Reference is set to "REF_OTHER". After that the other reference class (from SoftReference up to PhantomReference) are loaded and their corresponding reference type is set. For all the other classes which get loaded later, the reference type will be set to the reference type of their parent class (see 'ClassFileParser::post_process_parsed_stream()' in classFileParser.cpp): // Compute reference typ _rt = (NULL ==_super_klass) ? REF_NONE : _super_klass->reference_type(); So when you derive a class from WeakReference, its reference type will be "REF_WEAK". Now with your agent, you change the loading of classes as follows: ... [0.046s][info][class,load] java.lang.ClassLoader source: jrt:/java.base [0.046s][info][class,load] java.lang.ref.Reference source: jrt:/java.base [0.046s][info][class,load] java.lang.ref.WeakReference source: jrt:/java.base [0.046s][info][class,load] java.lang.ref.MyWeakReference [0.047s][info][class,load] java.lang.System source: jrt:/java.base ... [0.048s][info][class,load] java.lang.ref.SoftReference source: jrt:/java.base [0.048s][info][class,load] java.lang.ref.FinalReference source: jrt:/java.base [0.048s][info][class,load] java.lang.ref.PhantomReference source: jrt:/java.base ... This means that your class 'java.lang.ref.MyWeakReference' will get reference type "REF_NONE", because you implicitly triggered the loading of WeakReference and Reference without updating their reference type accordingly. Later on in the game, the reference type of WeakReference and Reference will be corrected, but your custom reference class will remain with its initial reference type "REF_NONE". I think this confuses the GC in the end and leads to the crashes observed by you. Funny enough, I don't see any crash with a product build of the latest jdk9-dev sources on Linux, but I can reproduce "should be in heap" assertion observed by you: # Internal Error (/usr/work/d046063/OpenJDK/jdk9-dev/hotspot/src/share/vm/gc/serial/markSweep.inline.hpp:53), pid=18985, tid=19058 # assert(Universe::heap()->is_in(obj)) failed: should be in heap So to keep a long story short, I don't think it is a good idea to change the class loading and initialization order of the "well-known" classes as described in systemDictionary.hpp. That order is carefully handcrafted and extremely sensitive to changes and can have all kind of unforeseeable side effects (as you've painfully detected yourself :) Regards, Volker On Mon, Apr 10, 2017 at 5:37 PM, Michael Rasmussen < michael.rasmussen at zeroturnaround.com> wrote: > On 10 April 2017 at 17:48, Stefan Johansson > wrote: > > This sounds great. Could you attach the test case to the mailing list in > > case we want to add this to the OpenJDK testing going forward? > > Attached inline below > > /Michael > > --- agent.c --- > > #include > > #include "jni.h" > #include "jvmti.h" > > /* > public class java.lang.ref.MyWeakReference > extends java.lang.ref.WeakReference {}; > */ > jbyte MyWeakReference_bytes[] = { > -54, -2, -70, -66, 0, 0, 0, 49, > 0, 5, 1, 0, 29, 106, 97, 118, > 97, 47, 108, 97, 110, 103, 47, 114, > 101, 102, 47, 77, 121, 87, 101, 97, > 107, 82, 101, 102, 101, 114, 101, 110, > 99, 101, 7, 0, 1, 1, 0, 27, > 106, 97, 118, 97, 47, 108, 97, 110, > 103, 47, 114, 101, 102, 47, 87, 101, > 97, 107, 82, 101, 102, 101, 114, 101, > 110, 99, 101, 7, 0, 3, 0, 1, > 0, 2, 0, 4, 0, 0, 0, 0, > 0, 0, 0, 0 > }; > jsize MyWeakReference_len = 0x5C; > > void JNICALL callback_ClassFileLoadHook(jvmtiEnv *jvmti, JNIEnv* env, > jclass class_being_redefined, jobject loader, const char* name, > jobject protection_domain, > jint class_data_len, const unsigned char* class_data, > jint* new_class_data_len, unsigned char** new_class_data) { > > if (name && !strcmp(name, "java/lang/System")) { > (*env)->DefineClass(env, "java/lang/ref/MyWeakReference", NULL, > MyWeakReference_bytes, MyWeakReference_len); > } > } > > JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *vm, > char *options, void *reserved) { > > jvmtiEventCallbacks callbacks; > jvmtiCapabilities caps; > jvmtiEnv *jvmti; > > (*vm)->GetEnv(vm, (void **)&jvmti, JVMTI_VERSION_9); > > memset(&caps, 0, sizeof(caps)); > memset(&callbacks, 0, sizeof(callbacks)); > > caps.can_generate_early_vmstart = 1; > caps.can_generate_all_class_hook_events = 1; > caps.can_generate_early_class_hook_events = 1; > > callbacks.ClassFileLoadHook = &callback_ClassFileLoadHook; > > (*jvmti)->AddCapabilities(jvmti, &caps); > (*jvmti)->SetEventCallbacks(jvmti, &callbacks, sizeof(callbacks)); > (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE, > JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, NULL); > > return 0; > } > > --- MiscTest9.java --- > > package app1; > > import java.lang.ref.WeakReference; > > public class MiscTest9 { > public static void main(String[] args) throws Exception { > WeakReference weak = new WeakReference<>(new MiscTest9()); > > boolean present = true; > while (present) { > System.out.println("Running GC"); > System.gc(); > present = weak.get() != null; > } > } > } > From kim.barrett at oracle.com Mon Apr 10 18:21:32 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Apr 2017 14:21:32 -0400 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> Message-ID: <1213C931-FABD-4BF3-994C-3DA9E9F5D9EA@oracle.com> > On Apr 10, 2017, at 12:48 PM, Stefan Karlsson wrote: > > On 10/04/17 18:33, Kim Barrett wrote: >>> On Apr 10, 2017, at 12:18 PM, Kim Barrett >>> wrote: >>> >>>> You did a couple of stylistic changes that now make the code inconsistent: >>>> >>>> [?] >>>> I think these changes should be kept out of this patch. >>>> >>> Post-decrement in for loops still makes me twitch, even though it doesn?t matter >>> in this code base. But okay. >>> >> Updated webrev: >> >> http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.01/ >> >> >> > To be consistent with the other code it in that file (res & 1) == 0 should be !(res & 1). That would be contrary to hotspot style guide; I think that should take precedence. > Maybe create a small, trivial cleanup RFE to change the following in the BitMap files: > * !(res & 1) => (res & 1) == 0 > * res = res >> 1 => res >>= 1 I can do something like that, but would prefer it as a followup. There are other similar syntactic / style guide cleanups, such as consistent use of helpers like bit_in_word, word_index, and bit_index. Having unit tests make these kinds of changes easier. From igor.ignatyev at oracle.com Mon Apr 10 18:55:13 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 10 Apr 2017 11:55:13 -0700 Subject: Coming soon: second generation prototype of Hg forest consolidation In-Reply-To: <504a500e-f913-627d-b642-c02e88c22292@oracle.com> References: <732ba4ba-5538-9afb-8d29-24d7911d1baf@oracle.com> <504a500e-f913-627d-b642-c02e88c22292@oracle.com> Message-ID: (moved to hotspot-dev alias) Hi Erik, I have a question about test/hotspot layout. when we started jep 281[1], we deliberately selected not to use harness/framework names in directory and the proposed the following layout for hotspot tests was > For clarity the tests should not be mixed with existing jtreg tests; instead, they should be split in two directories. We propose to split the current jdk9/hotspot/test directory into two subdirectories: > ? jdk9/hotspot/test/java > ? jdk9/hotspot/test/native now the consolidated repo[2] has test/hotspot/jtreg and test/hotspot/gtest and directories. hence I'm wondering why you have renamed the directories, there were any problems w/ the originally proposed layout? Thanks, -- Igor [1] JEP 281: HotSpot C++ Unit-Test Framework http://openjdk.java.net/jeps/281 [2] http://hg.openjdk.java.net/jdk10/consol-proto/file/9ea8bfbbe49b/test/hotspot > On Apr 10, 2017, at 9:42 AM, joe darcy wrote: > > Hello, > > Second generation prototype now available from > > http://hg.openjdk.java.net/jdk10/consol-proto/ > > Sources equivalent to JDK 9 b161. > > Thanks, > > -Joe > > > On 4/5/2017 9:44 AM, joe darcy wrote: >> Hello, >> >> As previously discussed [1], we've been exploring consolidating the many repositories in an OpenJDK Hg forest (jdk, langtools, hotspot, ...) into a single repository for JDK 10. The initial prototype of a consolidated repo [2], equivalent to JDK 9 build 138, was published in October 2016. >> >> Erik Joelsson has been working on a refined version of the consolidation repo and we plan to publish this second generation prototype in the near future. The new prototype presents a more faithful view of the sources between tags for promoted builds and provides a cleaner overall file layout. >> >> The corresponding JEP [3] will be updated to reflect the details of the new prototype in due course. >> >> Cheers, >> >> -Joe >> >> [1] "Looking ahead: proposed Hg forest consolidation for JDK 10," >> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/004987.html >> >> [2] http://hg.openjdk.java.net/jdk9/consol-proto/ >> >> [3] JEP 296: Consolidate the JDK Forest into a Single Repository >> http://openjdk.java.net/jeps/296 >> > From michael.rasmussen at zeroturnaround.com Mon Apr 10 19:15:55 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Mon, 10 Apr 2017 22:15:55 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> Message-ID: On 10 April 2017 at 21:14, Volker Simonis wrote: > So to keep a long story short, I don't think it is a good idea to change the > class loading and initialization order of the "well-known" classes as > described in systemDictionary.hpp. That order is carefully handcrafted and > extremely sensitive to changes and can have all kind of unforeseeable side > effects (as you've painfully detected yourself :) Hi Volker Thank you very much for the in-depth explanation. I'm aware that the order is a bit fragile, and the approach we've been using up till now hasn't changed that order (we've only caused extra classes to be loaded, basically a class and an interface loaded very early, and then the rest later as needed) -- unfortunately that approach doesn't work with JDK9 and the restrictions added by the module system, where what we can load before the module system is up and running has been severely limited. Usually though, when we've tinkered with something like that, we've seen crashes immediately, which was why this one was kind of tricky, since basically all our integration tests were working, except a few here and there, which we narrowed down to those that did a full GC (and then with help from Stefan, I figured out that it was because of our Reference types being loaded). /Michael From coleen.phillimore at oracle.com Mon Apr 10 20:18:49 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 10 Apr 2017 16:18:49 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness Message-ID: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> Summary: remove system dictionary walk and pass strong closure for !ClassUnloading See bug for more details: open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8178336 Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese with -XX:-ClassUnloading. Thanks, Coleen From igor.ignatyev at oracle.com Mon Apr 10 22:42:04 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 10 Apr 2017 15:42:04 -0700 Subject: RFR(XS) : 8178340: remove unneeded "throws" from ProcessTools::createJavaProcessBuilder Message-ID: <30D206D1-8044-4205-A687-26AF48784FAC@oracle.com> http://cr.openjdk.java.net/~iignatyev//8178340/webrev.00/index.html > 8 lines changed: 0 ins; 2 del; 6 mod; Hi all, could you please review this tiny fix which clean up testlibary? ProcessTools::createJavaProcessBuilder methods can not throw any checked exceptions, but were declared with 'throws Exception', which forced all users of these methods to either have redundant try-catch or throws. the fix cleans up the method declaration in all three copies of testlibary. webrev: http://cr.openjdk.java.net/~iignatyev//8178340/webrev.00/index.html jbs: https://bugs.openjdk.java.net/browse/JDK-8178340 testing: hotspot, jdk, langtools and jaxp tests Thanks, -- Igor From jiangli.zhou at Oracle.COM Tue Apr 11 01:44:59 2017 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Mon, 10 Apr 2017 18:44:59 -0700 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> Message-ID: <65FA39B2-FA0E-4067-BD7C-2C9671CFC601@oracle.com> Hi Coleen, I noticed there are two comments in Dictionary::do_unloading() refer to always_strong_oops_do. They should be updated as well. 150 // The placeholder array has been handled in always_strong_oops_do. 160 // Non-unloadable classes were handled in always_strong_oops_do Thanks, Jiangli > On Apr 10, 2017, at 1:18 PM, coleen.phillimore at oracle.com wrote: > > Summary: remove system dictionary walk and pass strong closure for !ClassUnloading > > See bug for more details: > > open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8178336 > > Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese with -XX:-ClassUnloading. > > Thanks, > Coleen From david.holmes at oracle.com Tue Apr 11 05:26:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Apr 2017 15:26:17 +1000 Subject: Coming soon: second generation prototype of Hg forest consolidation In-Reply-To: References: <732ba4ba-5538-9afb-8d29-24d7911d1baf@oracle.com> <504a500e-f913-627d-b642-c02e88c22292@oracle.com> Message-ID: <8eb62a0d-e498-df71-43ad-64ae856a4a22@oracle.com> Hi Igor, I'm not Erik but this got my attention :) On 11/04/2017 4:55 AM, Igor Ignatyev wrote: > (moved to hotspot-dev alias) > > Hi Erik, > > I have a question about test/hotspot layout. when we started jep 281[1], we deliberately selected not to use harness/framework names in directory and the proposed the following layout for hotspot tests was >> For clarity the tests should not be mixed with existing jtreg tests; instead, they should be split in two directories. We propose to split the current jdk9/hotspot/test directory into two subdirectories: >> ? jdk9/hotspot/test/java >> ? jdk9/hotspot/test/native > > > now the consolidated repo[2] has test/hotspot/jtreg and test/hotspot/gtest and directories. hence I'm wondering why you have renamed the directories, there were any problems w/ the originally proposed layout? Mea culpa but I never paid much attention to JEP 281 so was unaware of the proposal for "java" and "native" subdirectories. If I had noticed it I would have objected as the names are not indicative of the contents. I think naming based on the testing framework makes much more sense: jtreg and gtest. My 2c. David > Thanks, > -- Igor > > > [1] JEP 281: HotSpot C++ Unit-Test Framework http://openjdk.java.net/jeps/281 > [2] http://hg.openjdk.java.net/jdk10/consol-proto/file/9ea8bfbbe49b/test/hotspot > >> On Apr 10, 2017, at 9:42 AM, joe darcy wrote: >> >> Hello, >> >> Second generation prototype now available from >> >> http://hg.openjdk.java.net/jdk10/consol-proto/ >> >> Sources equivalent to JDK 9 b161. >> >> Thanks, >> >> -Joe >> >> >> On 4/5/2017 9:44 AM, joe darcy wrote: >>> Hello, >>> >>> As previously discussed [1], we've been exploring consolidating the many repositories in an OpenJDK Hg forest (jdk, langtools, hotspot, ...) into a single repository for JDK 10. The initial prototype of a consolidated repo [2], equivalent to JDK 9 build 138, was published in October 2016. >>> >>> Erik Joelsson has been working on a refined version of the consolidation repo and we plan to publish this second generation prototype in the near future. The new prototype presents a more faithful view of the sources between tags for promoted builds and provides a cleaner overall file layout. >>> >>> The corresponding JEP [3] will be updated to reflect the details of the new prototype in due course. >>> >>> Cheers, >>> >>> -Joe >>> >>> [1] "Looking ahead: proposed Hg forest consolidation for JDK 10," >>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/004987.html >>> >>> [2] http://hg.openjdk.java.net/jdk9/consol-proto/ >>> >>> [3] JEP 296: Consolidate the JDK Forest into a Single Repository >>> http://openjdk.java.net/jeps/296 >>> >> > From david.holmes at oracle.com Tue Apr 11 06:05:27 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Apr 2017 16:05:27 +1000 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> Message-ID: <93fdc779-0c38-facc-9a66-8c4115e86264@oracle.com> Hi Coleen, On 11/04/2017 6:18 AM, coleen.phillimore at oracle.com wrote: > Summary: remove system dictionary walk and pass strong closure for > !ClassUnloading Can't really comment on functional change but noticed this comment: 241 // Then iterate over the protection domain cache to apply the closure on the 242 // previously marked ones. 243 _pd_cache_table->oops_do(blk); is no longer accurate: there is nothing "before" for the "then" to follow, and there is no previous marking done. Thanks, David > See bug for more details: > > open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8178336 > > Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese > with -XX:-ClassUnloading. > > Thanks, > Coleen From ioi.lam at oracle.com Tue Apr 11 06:26:35 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 11 Apr 2017 14:26:35 +0800 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only Message-ID: <58EC771B.9020202@oracle.com> Hi,please review the following change https://bugs.openjdk.java.net/browse/JDK-8171392 http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool is modified to store the Klass pointer. After: + ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + When a klass is resolved, _resolved_klasses->at(i) is modified. In addition: + I moved _resolved_references and _reference_map from ConstantPool to ConstantPoolCache. + Now _flags is no longer modified for shared ConstantPools. As a result, none of the fields in shared ConstantPools are modified at run time, so we can move them into the RO region in the CDS archive. *Testing:** * - Benchmark results show no performance regression, despite the extra level of indirection, which has a negligible overhead for the interpreter. - RBT testing for tier2 and tier3. *Ports:** * - I have tested only the Oracle-support ports. For the aarch64, ppc and s390 ports, I have added some code without testing (it's probably incomplete) - Port owners, please check if my patch work for you, and I can incorporate your changes in my push. Alternatively, you can wait for my push and provide fixes (if necessary) in a new changeset, and I will be happy to sponsor it. Thanks - Ioi From ioi.lam at oracle.com Tue Apr 11 06:35:13 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 11 Apr 2017 14:35:13 +0800 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> Message-ID: <58EC7921.3090107@oracle.com> Hi Coleen, Thanks for doing this clean up. I was guiltily of writing the original code :-( A few questions: Why is this block of code moved and the comments dropped? 328 void Dictionary::oops_do(OopClosure* f) { 329 // Only the protection domain oops contain references into the heap. Iterate 330 // over all of them. 331 _pd_cache_table->oops_do(f); 332 } 333 It would be better to make the changes in-place. Also, have you validated that (either with an explicit test, or inside the debugger) [1] live protection domains in _pd_cache_table are properly relocated during GC? [2] dead protection domains are removed after class unloading? Thanks - Ioi On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: > Summary: remove system dictionary walk and pass strong closure for > !ClassUnloading > > See bug for more details: > > open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8178336 > > Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese > with -XX:-ClassUnloading. > > Thanks, > Coleen From david.holmes at oracle.com Tue Apr 11 07:04:38 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Apr 2017 17:04:38 +1000 Subject: RFR(XS) : 8178340: remove unneeded "throws" from ProcessTools::createJavaProcessBuilder In-Reply-To: <30D206D1-8044-4205-A687-26AF48784FAC@oracle.com> References: <30D206D1-8044-4205-A687-26AF48784FAC@oracle.com> Message-ID: Hi Igor, This looks fine to me. I was a little concerned any tests with a try/catch around these calls would fail to compile due to exception checking, but class Exception is exempt from that checking. Thanks, David On 11/04/2017 8:42 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8178340/webrev.00/index.html >> 8 lines changed: 0 ins; 2 del; 6 mod; > Hi all, > > could you please review this tiny fix which clean up testlibary? > > ProcessTools::createJavaProcessBuilder methods can not throw any checked exceptions, but were declared with 'throws Exception', which forced all users of these methods to either have redundant try-catch or throws. the fix cleans up the method declaration in all three copies of testlibary. > > webrev: http://cr.openjdk.java.net/~iignatyev//8178340/webrev.00/index.html > jbs: https://bugs.openjdk.java.net/browse/JDK-8178340 > testing: hotspot, jdk, langtools and jaxp tests > > Thanks, > -- Igor > From aph at redhat.com Tue Apr 11 08:00:46 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 11 Apr 2017 09:00:46 +0100 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> Message-ID: <9f0e4cc9-dcd2-b15c-aba2-b394da0f69fc@redhat.com> On 10/04/17 20:15, Michael Rasmussen wrote: > I'm aware that the order is a bit fragile, and the approach we've been > using up till now hasn't changed that order (we've only caused extra > classes to be loaded, basically a class and an interface loaded very > early, and then the rest later as needed) -- unfortunately that > approach doesn't work with JDK9 and the restrictions added by the > module system, where what we can load before the module system is up > and running has been severely limited. Right, but if you initialize the classes in the correct order then your program won't be fragile at all. You've then got a solid and repeatable solution. Andrew. From michael.rasmussen at zeroturnaround.com Tue Apr 11 08:07:19 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Tue, 11 Apr 2017 11:07:19 +0300 Subject: Question regarding JVM crashes in GC In-Reply-To: <9f0e4cc9-dcd2-b15c-aba2-b394da0f69fc@redhat.com> References: <686c07c9-da2b-afc9-f1ca-2c0570fdf71e@oracle.com> <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> <9f0e4cc9-dcd2-b15c-aba2-b394da0f69fc@redhat.com> Message-ID: On 11 April 2017 at 11:00, Andrew Haley wrote: > Right, but if you initialize the classes in the correct order then > your program won't be fragile at all. You've then got a solid and > repeatable solution. Yeah, based on information from here we are working on a solution right now, trying to defer loading as many of our classes as possible until after VMStart. I feel confident we'll get it to work, without crashing the JVM :D /Michael From stefan.johansson at oracle.com Tue Apr 11 09:01:30 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 11 Apr 2017 11:01:30 +0200 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> <9f0e4cc9-dcd2-b15c-aba2-b394da0f69fc@redhat.com> Message-ID: <19fbad2a-8609-2d77-ec71-de6169d8fa7e@oracle.com> Thanks Volker for digging down and identifying the root cause. On 2017-04-11 10:07, Michael Rasmussen wrote: > On 11 April 2017 at 11:00, Andrew Haley wrote: >> Right, but if you initialize the classes in the correct order then >> your program won't be fragile at all. You've then got a solid and >> repeatable solution. > Yeah, based on information from here we are working on a solution > right now, trying to defer loading as many of our classes as possible > until after VMStart. > I feel confident we'll get it to work, without crashing the JVM :D Sounds like you have a way forward Michael. Good luck =) Stefan > /Michael From erik.joelsson at oracle.com Tue Apr 11 10:00:37 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 11 Apr 2017 12:00:37 +0200 Subject: Coming soon: second generation prototype of Hg forest consolidation In-Reply-To: <8eb62a0d-e498-df71-43ad-64ae856a4a22@oracle.com> References: <732ba4ba-5538-9afb-8d29-24d7911d1baf@oracle.com> <504a500e-f913-627d-b642-c02e88c22292@oracle.com> <8eb62a0d-e498-df71-43ad-64ae856a4a22@oracle.com> Message-ID: <652689ed-4c81-f5bb-89be-4c7353a93f8c@oracle.com> Hello, On 2017-04-11 07:26, David Holmes wrote: > Hi Igor, > > I'm not Erik but this got my attention :) > > On 11/04/2017 4:55 AM, Igor Ignatyev wrote: >> (moved to hotspot-dev alias) >> >> Hi Erik, >> >> I have a question about test/hotspot layout. when we started jep >> 281[1], we deliberately selected not to use harness/framework names >> in directory and the proposed the following layout for hotspot tests was >>> For clarity the tests should not be mixed with existing jtreg tests; >>> instead, they should be split in two directories. We propose to >>> split the current jdk9/hotspot/test directory into two subdirectories: >>> ? jdk9/hotspot/test/java >>> ? jdk9/hotspot/test/native >> >> >> now the consolidated repo[2] has test/hotspot/jtreg and >> test/hotspot/gtest and directories. hence I'm wondering why you have >> renamed the directories, there were any problems w/ the originally >> proposed layout? > > Mea culpa but I never paid much attention to JEP 281 so was unaware of > the proposal for "java" and "native" subdirectories. If I had noticed > it I would have objected as the names are not indicative of the > contents. I think naming based on the testing framework makes much > more sense: jtreg and gtest. > I was aware of the suggestion to use java and native, but I was never part of the discussion that lead to that decision. I used that naming for a long time in the prototype, but it didn't sit right with me however since, as David points out, it does not reflect the contents well. The jtreg tests consist of Java, C/C++ and shell script tests. The gtest tests are C++. The term "native" is not clearly denoting one or the other. Browsing this I wouldn't know if the native parts of jtreg tests are supposed to be in that directory or not for example. There are also a fair bit of framework specific parts of the implementations, like jtreg tags, which make the tests very specific to their frameworks. I just don't see the point in trying to hide that. As I see it the problem with using the framework name would be if we one day decided to change frameworks but didn't want to move the tests. I would find this a very unlikely scenario. If we were to change frameworks, it would be a lengthy process of converting test by test and we would most likely want to move each converted test to a new directory to clearly signify its conversion. So even in that case, a framework based directory makes more sense. /Erik > My 2c. > > David > >> Thanks, >> -- Igor >> >> >> [1] JEP 281: HotSpot C++ Unit-Test Framework >> http://openjdk.java.net/jeps/281 >> [2] >> http://hg.openjdk.java.net/jdk10/consol-proto/file/9ea8bfbbe49b/test/hotspot >> >>> On Apr 10, 2017, at 9:42 AM, joe darcy wrote: >>> >>> Hello, >>> >>> Second generation prototype now available from >>> >>> http://hg.openjdk.java.net/jdk10/consol-proto/ >>> >>> Sources equivalent to JDK 9 b161. >>> >>> Thanks, >>> >>> -Joe >>> >>> >>> On 4/5/2017 9:44 AM, joe darcy wrote: >>>> Hello, >>>> >>>> As previously discussed [1], we've been exploring consolidating the >>>> many repositories in an OpenJDK Hg forest (jdk, langtools, hotspot, >>>> ...) into a single repository for JDK 10. The initial prototype of >>>> a consolidated repo [2], equivalent to JDK 9 build 138, was >>>> published in October 2016. >>>> >>>> Erik Joelsson has been working on a refined version of the >>>> consolidation repo and we plan to publish this second generation >>>> prototype in the near future. The new prototype presents a more >>>> faithful view of the sources between tags for promoted builds and >>>> provides a cleaner overall file layout. >>>> >>>> The corresponding JEP [3] will be updated to reflect the details of >>>> the new prototype in due course. >>>> >>>> Cheers, >>>> >>>> -Joe >>>> >>>> [1] "Looking ahead: proposed Hg forest consolidation for JDK 10," >>>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/004987.html >>>> >>>> >>>> [2] http://hg.openjdk.java.net/jdk9/consol-proto/ >>>> >>>> [3] JEP 296: Consolidate the JDK Forest into a Single Repository >>>> http://openjdk.java.net/jeps/296 >>>> >>> >> From Alan.Bateman at oracle.com Tue Apr 11 10:19:30 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 11 Apr 2017 11:19:30 +0100 Subject: Question regarding JVM crashes in GC In-Reply-To: References: <0a01fafb-feba-7c57-8fa7-2b768392d5cb@oracle.com> <9f0e4cc9-dcd2-b15c-aba2-b394da0f69fc@redhat.com> Message-ID: On 11/04/2017 09:07, Michael Rasmussen wrote: > : > Yeah, based on information from here we are working on a solution > right now, trying to defer loading as many of our classes as possible > until after VMStart. > I feel confident we'll get it to work, without crashing the JVM :D > I don't know if we have the can_generate_early_vmstart capability enabled. If you do then the VMStart event is posted as early as possible and comes with a warnings in the JVM TI spec: "The VM is capable of executing bytecode but it may not have initialized to the point where it can load classes in modules other than java.base, or even arbitrary classes in java.base. Agents that do load-time instrumentation in this phase must take great care when instrumenting code that potentially executes in this phase. Extreme care should also be taken with JNI FindClass as it may not be possible to load classes and attempts to do so may result in unpredictable behavior, maybe even stability issues on some VM implementations." The concerns in this about stability and "unpredictable behavior" apply to JDK 8 and older releases too of course, early startup has always been delicate. I realize your agent wants to do load time transformation of many core classes but it's always required great care. Many JVM TI agents won't be too concerned with this as they will be content to wait until the VM is full initialized and the VMInit event is posted. -Alan. From aph at redhat.com Tue Apr 11 11:02:26 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 11 Apr 2017 12:02:26 +0100 Subject: Optimizing byte reverse code for int value In-Reply-To: References: <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> Message-ID: <89abbea5-9998-2e4d-62d3-e1f3e9bbd1d5@redhat.com> On 11/04/17 11:36, Michihiro Horie wrote: > Thank you very much for letting us know Unsafe.getIntUnaligned is available in > JDK9. I do agree we should fix Java source code. > We think our byte-reverse optimization would still work on jdk8u as Hiroshi > mentioned. Would you agree on this point? I do, but I do not agree that this patch should necessarily be done in the PowerPC-specific back end. Have you considered it as a generic optimization for all processors? Andrew. From coleen.phillimore at oracle.com Tue Apr 11 11:50:12 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 11 Apr 2017 07:50:12 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <65FA39B2-FA0E-4067-BD7C-2C9671CFC601@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <65FA39B2-FA0E-4067-BD7C-2C9671CFC601@oracle.com> Message-ID: <6978dfd5-927f-011c-5f55-d8810e495fa9@oracle.com> Hi Jiangli, Thank you for looking at this. On 4/10/17 9:44 PM, Jiangli Zhou wrote: > Hi Coleen, > > I noticed there are two comments in Dictionary::do_unloading() refer > to always_strong_oops_do. They should be updated as well. > > 150 // The placeholder array has been handled in always_strong_oops_do. This comment makes no sense in this context so I removed it. > 160 // Non-unloadable classes were handled in always_strong_oops_do This comment doesn't make sense either in this context, and this whole "strongly reachable" idea in this loop is odd. If !ClassUnloading, all of the classes are "strongly reachable" so we shouldn't really bother going through this loop at all. The other "strongly reachable" class is the NULL class loader, where this "if" statement does make sense. I've made changes to this area in following work, so I'll remove this comment for now. Thanks! Coleen > Thanks, > Jiangli > >> On Apr 10, 2017, at 1:18 PM, coleen.phillimore at oracle.com >> wrote: >> >> Summary: remove system dictionary walk and pass strong closure for >> !ClassUnloading >> >> See bug for more details: >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >> >> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >> >> Tested with nightly tier2-5 tests and jprt (runs all GCs) and >> runThese with -XX:-ClassUnloading. >> >> Thanks, >> Coleen > From coleen.phillimore at oracle.com Tue Apr 11 11:52:34 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 11 Apr 2017 07:52:34 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <93fdc779-0c38-facc-9a66-8c4115e86264@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <93fdc779-0c38-facc-9a66-8c4115e86264@oracle.com> Message-ID: On 4/11/17 2:05 AM, David Holmes wrote: > Hi Coleen, > > On 11/04/2017 6:18 AM, coleen.phillimore at oracle.com wrote: >> Summary: remove system dictionary walk and pass strong closure for >> !ClassUnloading > > Can't really comment on functional change but noticed this comment: > > 241 // Then iterate over the protection domain cache to apply the > closure on the > 242 // previously marked ones. > 243 _pd_cache_table->oops_do(blk); Thanks - I tried to follow and update all the comments, but apparently missed some: // Follow oops that are in the protection domain cache table This is what I changed it to. Coleen > > is no longer accurate: there is nothing "before" for the "then" to > follow, and there is no previous marking done. > > Thanks, > David > >> See bug for more details: >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >> >> Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese >> with -XX:-ClassUnloading. >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Tue Apr 11 13:03:26 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 11 Apr 2017 09:03:26 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <58EC7921.3090107@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> Message-ID: <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> On 4/11/17 2:35 AM, Ioi Lam wrote: > Hi Coleen, > > Thanks for doing this clean up. I was guiltily of writing the original > code :-( > > A few questions: > > Why is this block of code moved and the comments dropped? > > 328 void Dictionary::oops_do(OopClosure* f) { > 329 // Only the protection domain oops contain references into the > heap. Iterate > 330 // over all of them. > 331 _pd_cache_table->oops_do(f); > 332 } > 333 > > It would be better to make the changes in-place. I didn't have to change Dictionary::oops_do and moved it to be near always_strong_oops_do(), so I shouldn't have removed the comment. I moved it back but I don't like that it's separated from the other GC functions. With my other change, it'll be closer (I think I'm going to have a merge conflict with myself). > > Also, have you validated that (either with an explicit test, or inside > the debugger) > > [1] live protection domains in _pd_cache_table are properly relocated > during GC? > [2] dead protection domains are removed after class unloading? I ran with runThese (which has lots of class loading and unloading) with logging for both oops_do and unlink functions (removing protection domain entries) but I didn't realize that always_strong_oops_do is never called, so I deleted this function. New webrev (with dictionary::oops_do put back): http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html Thanks! Coleen > > Thanks > - Ioi > > On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >> Summary: remove system dictionary walk and pass strong closure for >> !ClassUnloading >> >> See bug for more details: >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >> >> Tested with nightly tier2-5 tests and jprt (runs all GCs) and >> runThese with -XX:-ClassUnloading. >> >> Thanks, >> Coleen > From martin.doerr at sap.com Tue Apr 11 13:43:04 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 11 Apr 2017 13:43:04 +0000 Subject: Optimizing byte reverse code for int value In-Reply-To: <89abbea5-9998-2e4d-62d3-e1f3e9bbd1d5@redhat.com> References: <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> <89abbea5-9998-2e4d-62d3-e1f3e9bbd1d5@redhat.com> Message-ID: Hi Andrew, thank you for your helpful comments. I fully agree with you. In addition, I noticed that we don't have match rules which exploit byte reverse load/store instructions on PPC64. SPARC already has them: match(Set dst (ReverseBytesI/L/US/S (LoadI src))); match(Set dst (StoreI dst (ReverseBytesI/L/US/S src))); I think we should add them for jdk10. They should be used when the platform endianness doesn't match the bigEndian parameter in Unsafe methods. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Andrew Haley Sent: Dienstag, 11. April 2017 13:02 To: Michihiro Horie Cc: Simonis, Volker ; ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; Hiroshi H Horii Subject: Re: Optimizing byte reverse code for int value On 11/04/17 11:36, Michihiro Horie wrote: > Thank you very much for letting us know Unsafe.getIntUnaligned is available in > JDK9. I do agree we should fix Java source code. > We think our byte-reverse optimization would still work on jdk8u as Hiroshi > mentioned. Would you agree on this point? I do, but I do not agree that this patch should necessarily be done in the PowerPC-specific back end. Have you considered it as a generic optimization for all processors? Andrew. From martin.doerr at sap.com Tue Apr 11 15:12:50 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 11 Apr 2017 15:12:50 +0000 Subject: Optimizing byte reverse code for int value In-Reply-To: References: , <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> <89abbea5-9998-2e4d-62d3-e1f3e9bbd1d5@redhat.com> Message-ID: <622fa3e77da546dfb5155a1e4afacd7c@sap.com> Hi Michihiro, thanks for the quick reply. I think Andrew?s idea is to optimize in the shared code instead of the platform backends. I haven?t thought about where this could be done. Or would it be possible to backport jdk (especially Unsafe) changes? If the required changes are small enough and we don?t have to touch any public interface, this might be an option, too. We?ll appreciate if you take care of the new match rules for PPC64. Thanks a lot. Best regards, Martin From: Michihiro Horie [mailto:HORIE at jp.ibm.com] Sent: Dienstag, 11. April 2017 16:55 To: Doerr, Martin Cc: aph at redhat.com; Hiroshi H Horii ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker Subject: RE: Optimizing byte reverse code for int value Andrew, Martin, Thanks a lot for your helpful feedback. >Have you considered it as a generic optimization for all processors? We would support all processors for our byte-reverse optimization to make it generic. Currently, I just finished adding match rules for little endian and big endian on PPC64, and am testing it in AIX. >In addition, I noticed that we don't have match rules which exploit byte reverse load/store instructions on PPC64. We would like to handle adding match rules for byte reverse load/store instructions on PPC64 for JDK10 if you would not mind. Would it be fine with you? Best regards, -- Michihiro, IBM Research - Tokyo ----- Original message ----- From: "Doerr, Martin" > To: Andrew Haley >, Michihiro Horie/Japan/IBM at IBMJP Cc: "Simonis, Volker" >, "ppc-aix-port-dev at openjdk.java.net" >, "hotspot-dev at openjdk.java.net" >, Hiroshi H Horii/Japan/IBM at IBMJP Subject: RE: Optimizing byte reverse code for int value Date: Tue, Apr 11, 2017 10:44 PM Hi Andrew, thank you for your helpful comments. I fully agree with you. In addition, I noticed that we don't have match rules which exploit byte reverse load/store instructions on PPC64. SPARC already has them: match(Set dst (ReverseBytesI/L/US/S (LoadI src))); match(Set dst (StoreI dst (ReverseBytesI/L/US/S src))); I think we should add them for jdk10. They should be used when the platform endianness doesn't match the bigEndian parameter in Unsafe methods. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Andrew Haley Sent: Dienstag, 11. April 2017 13:02 To: Michihiro Horie > Cc: Simonis, Volker >; ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; Hiroshi H Horii > Subject: Re: Optimizing byte reverse code for int value On 11/04/17 11:36, Michihiro Horie wrote: > Thank you very much for letting us know Unsafe.getIntUnaligned is available in > JDK9. I do agree we should fix Java source code. > We think our byte-reverse optimization would still work on jdk8u as Hiroshi > mentioned. Would you agree on this point? I do, but I do not agree that this patch should necessarily be done in the PowerPC-specific back end. Have you considered it as a generic optimization for all processors? Andrew. From coleen.phillimore at oracle.com Tue Apr 11 15:27:13 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 11 Apr 2017 11:27:13 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> Message-ID: <079da9b2-5652-ae62-5da2-0f2f5b78b736@oracle.com> On 4/11/17 9:03 AM, coleen.phillimore at oracle.com wrote: > > > On 4/11/17 2:35 AM, Ioi Lam wrote: >> Hi Coleen, >> >> Thanks for doing this clean up. I was guiltily of writing the >> original code :-( I should point out that I reviewed it :) Coleen >> >> A few questions: >> >> Why is this block of code moved and the comments dropped? >> >> 328 void Dictionary::oops_do(OopClosure* f) { >> 329 // Only the protection domain oops contain references into the >> heap. Iterate >> 330 // over all of them. >> 331 _pd_cache_table->oops_do(f); >> 332 } >> 333 >> >> It would be better to make the changes in-place. > > I didn't have to change Dictionary::oops_do and moved it to be near > always_strong_oops_do(), so I shouldn't have removed the comment. I > moved it back but I don't like that it's separated from the other GC > functions. With my other change, it'll be closer (I think I'm going > to have a merge conflict with myself). >> >> Also, have you validated that (either with an explicit test, or >> inside the debugger) >> >> [1] live protection domains in _pd_cache_table are properly relocated >> during GC? >> [2] dead protection domains are removed after class unloading? > > I ran with runThese (which has lots of class loading and unloading) > with logging for both oops_do and unlink functions (removing > protection domain entries) but I didn't realize that > always_strong_oops_do is never called, so I deleted this function. > > New webrev (with dictionary::oops_do put back): > > http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html > > > Thanks! > Coleen >> >> Thanks >> - Ioi >> >> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>> Summary: remove system dictionary walk and pass strong closure for >>> !ClassUnloading >>> >>> See bug for more details: >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>> >>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and >>> runThese with -XX:-ClassUnloading. >>> >>> Thanks, >>> Coleen >> > From jan.lahoda at oracle.com Tue Apr 11 17:26:38 2017 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Tue, 11 Apr 2017 19:26:38 +0200 Subject: RFR 9: JDK-8178012: Finish removal of -Xmodule: Message-ID: <58ED11CE.5010608@oracle.com> Hi, javac used to have an option, -Xmodule:, that allowed to compile given sources as if they were part of the specified module. This functionality has been merged into the --patch-module option, but the -Xmodule: option couldn't be fully removed at that time, as some tests and tools (e.g. jtreg) depended on it. The option was however removed from the javac help, etc. Since jtreg can use --patch-module instead of -Xmodule: now, I'd like to propose further demoting the option to an unsupported -XD-Xmodule: setting (the functionality is not removed completely, as there is a small amount of remaining dependencies). The option may be removed completely eventually when the remaining dependencies are resolved. The patch spans many repositories besides langtools, as it adjusts TEST.ROOT files, and the /test/lib/jdk/test/lib/InMemoryJavaCompiler.java. The latter is used by hotspot tests, and this change is a small tweak to translate -Xmodule: to --patch-module. This could be later improved by enhancing the test library API to pass the module name explicitly. Bug: https://bugs.openjdk.java.net/browse/JDK-8178012 Webrev: http://cr.openjdk.java.net/~jlahoda/8178012/webrev.00/ Any feedback is welcome. Thanks, Jan From kim.barrett at oracle.com Tue Apr 11 17:45:18 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 11 Apr 2017 13:45:18 -0400 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: <1213C931-FABD-4BF3-994C-3DA9E9F5D9EA@oracle.com> References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> <1213C931-FABD-4BF3-994C-3DA9E9F5D9EA@oracle.com> Message-ID: > On Apr 10, 2017, at 2:21 PM, Kim Barrett wrote: > >> On Apr 10, 2017, at 12:48 PM, Stefan Karlsson wrote: >> To be consistent with the other code it in that file (res & 1) == 0 should be !(res & 1). > > That would be contrary to hotspot style guide; I think that should take precedence. Stefan and I discussed this privately, and I?ve agreed to go with the locally consistent style here, even though it?s contrary to the hotspot style guide. Part of my decision process is related to the next part of this reply. >> Maybe create a small, trivial cleanup RFE to change the following in the BitMap files: >> * !(res & 1) => (res & 1) == 0 >> * res = res >> 1 => res >>= 1 > > I can do something like that, but would prefer it as a followup. > > There are other similar syntactic / style guide cleanups, such as consistent use of > helpers like bit_in_word, word_index, and bit_index. On further consideration, I don?t think we should do that. Instead, we should deal with the fact that we have 3 functions of significant size and complexity, which differ in small ways, some of which are intentional and some of which are not. We should instead have one parameterized implementation. I?d forgotten that I actually have code that does that. I?d put it aside because I had more pressing matters to deal with and that code needed more work, especially testing. That was actually the driver for 8169039, which is how we got here? From forax at univ-mlv.fr Tue Apr 11 18:35:05 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 11 Apr 2017 20:35:05 +0200 (CEST) Subject: RFR 9: JDK-8178012: Finish removal of -Xmodule: In-Reply-To: <58ED11CE.5010608@oracle.com> References: <58ED11CE.5010608@oracle.com> Message-ID: <882534875.892667.1491935705947.JavaMail.zimbra@u-pem.fr> Yes, please hide -Xmodule, currently people get confused by -Xmodule vs --patch-module, i had several questions related to that at DevoxxFR. R?mi ----- Mail original ----- > De: "Jan Lahoda" > ?: "compiler-dev" , "Java Core Libs" , > hotspot-dev at openjdk.java.net > Envoy?: Mardi 11 Avril 2017 19:26:38 > Objet: RFR 9: JDK-8178012: Finish removal of -Xmodule: > Hi, > > javac used to have an option, -Xmodule:, that allowed to compile given > sources as if they were part of the specified module. > > This functionality has been merged into the --patch-module option, but > the -Xmodule: option couldn't be fully removed at that time, as some > tests and tools (e.g. jtreg) depended on it. The option was however > removed from the javac help, etc. > > Since jtreg can use --patch-module instead of -Xmodule: now, I'd like to > propose further demoting the option to an unsupported -XD-Xmodule: > setting (the functionality is not removed completely, as there is a > small amount of remaining dependencies). The option may be removed > completely eventually when the remaining dependencies are resolved. > > The patch spans many repositories besides langtools, as it adjusts > TEST.ROOT files, and the > /test/lib/jdk/test/lib/InMemoryJavaCompiler.java. The latter > is used by hotspot tests, and this change is a small tweak to translate > -Xmodule: to --patch-module. This could be later improved by enhancing > the test library API to pass the module name explicitly. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8178012 > > Webrev: > http://cr.openjdk.java.net/~jlahoda/8178012/webrev.00/ > > Any feedback is welcome. > > Thanks, > Jan From igor.ignatyev at oracle.com Tue Apr 11 20:33:07 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 11 Apr 2017 13:33:07 -0700 Subject: RFR(XS) : 8178340: remove unneeded "throws" from ProcessTools::createJavaProcessBuilder In-Reply-To: References: <30D206D1-8044-4205-A687-26AF48784FAC@oracle.com> Message-ID: <0F943181-F9CD-471E-8883-837E822CBBD2@oracle.com> Hi David, thanks for review. -- Igor > On Apr 11, 2017, at 12:04 AM, David Holmes wrote: > > Hi Igor, > > This looks fine to me. > > I was a little concerned any tests with a try/catch around these calls would fail to compile due to exception checking, but class Exception is exempt from that checking. > > Thanks, > David > > On 11/04/2017 8:42 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8178340/webrev.00/index.html >>> 8 lines changed: 0 ins; 2 del; 6 mod; >> Hi all, >> >> could you please review this tiny fix which clean up testlibary? >> >> ProcessTools::createJavaProcessBuilder methods can not throw any checked exceptions, but were declared with 'throws Exception', which forced all users of these methods to either have redundant try-catch or throws. the fix cleans up the method declaration in all three copies of testlibary. >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8178340/webrev.00/index.html >> jbs: https://bugs.openjdk.java.net/browse/JDK-8178340 >> testing: hotspot, jdk, langtools and jaxp tests >> >> Thanks, >> -- Igor >> From igor.ignatyev at oracle.com Wed Apr 12 02:29:01 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 11 Apr 2017 19:29:01 -0700 Subject: RFR(S) : 8178415: remove ProcessTools::getPlatformSpecificVMArgs from testlibary Message-ID: <74A681EB-977E-4EA7-B028-36C6C238E20A@oracle.com> http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ > 59 lines changed: 0 ins; 59 del; 0 mod; Hi all, could you please review this small clean up of testlibaries? the only platform specific vm arguments which we had were -d64/32. they were deprecated in jdk9 by 8168010 and will be removed in jdk10 by 8169646. therefore ProcessTools::getPlatformSpecificVMArgs method is not needed anymore and should be removed. JBS: https://bugs.openjdk.java.net/browse/JDK-8178415 webrev: http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ testing: :jdk_tier{1-3}, :hotspot_all 8168010: https://bugs.openjdk.java.net/browse/JDK-8168010 8169646: https://bugs.openjdk.java.net/browse/JDK-8169646 Thanks, -- Igor From david.holmes at oracle.com Wed Apr 12 02:47:31 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Apr 2017 12:47:31 +1000 Subject: RFR(S) : 8178415: remove ProcessTools::getPlatformSpecificVMArgs from testlibary In-Reply-To: <74A681EB-977E-4EA7-B028-36C6C238E20A@oracle.com> References: <74A681EB-977E-4EA7-B028-36C6C238E20A@oracle.com> Message-ID: Hi Igor, On 12/04/2017 12:29 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ >> 59 lines changed: 0 ins; 59 del; 0 mod; > > Hi all, > > could you please review this small clean up of testlibaries? > the only platform specific vm arguments which we had were -d64/32. they were deprecated in jdk9 by 8168010 and will be removed in jdk10 by 8169646. therefore ProcessTools::getPlatformSpecificVMArgs method is not needed anymore and should be removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8178415 > webrev: http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ > testing: :jdk_tier{1-3}, :hotspot_all > 8168010: https://bugs.openjdk.java.net/browse/JDK-8168010 > 8169646: https://bugs.openjdk.java.net/browse/JDK-8169646 These changes look good to me. Thanks, David > Thanks, > -- Igor > From igor.ignatyev at oracle.com Wed Apr 12 02:48:16 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 11 Apr 2017 19:48:16 -0700 Subject: RFR(S) : 8178415: remove ProcessTools::getPlatformSpecificVMArgs from testlibary In-Reply-To: References: <74A681EB-977E-4EA7-B028-36C6C238E20A@oracle.com> Message-ID: <91BF0B5F-7BD2-4928-AFA9-8C0CB8D81D56@oracle.com> Hi David, thank you for such a prompt review! -- Igor > On Apr 11, 2017, at 7:47 PM, David Holmes wrote: > > Hi Igor, > > On 12/04/2017 12:29 PM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ >>> 59 lines changed: 0 ins; 59 del; 0 mod; >> >> Hi all, >> >> could you please review this small clean up of testlibaries? >> the only platform specific vm arguments which we had were -d64/32. they were deprecated in jdk9 by 8168010 and will be removed in jdk10 by 8169646. therefore ProcessTools::getPlatformSpecificVMArgs method is not needed anymore and should be removed. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8178415 >> webrev: http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ >> testing: :jdk_tier{1-3}, :hotspot_all >> 8168010: https://bugs.openjdk.java.net/browse/JDK-8168010 >> 8169646: https://bugs.openjdk.java.net/browse/JDK-8169646 > > These changes look good to me. > > Thanks, > David > >> Thanks, >> -- Igor >> From dmitry.fazunenko at oracle.com Wed Apr 12 07:59:35 2017 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 12 Apr 2017 10:59:35 +0300 Subject: RFR(S) : 8178415: remove ProcessTools::getPlatformSpecificVMArgs from testlibary In-Reply-To: <74A681EB-977E-4EA7-B028-36C6C238E20A@oracle.com> References: <74A681EB-977E-4EA7-B028-36C6C238E20A@oracle.com> Message-ID: <3f0d3013-f86e-893f-f3a0-2773006dd265@oracle.com> Hi Igor, The fix looks, thanks for doing it. Please update the copyrights before integration :) Thanks, Dima On 12.04.2017 5:29, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ >> 59 lines changed: 0 ins; 59 del; 0 mod; > Hi all, > > could you please review this small clean up of testlibaries? > the only platform specific vm arguments which we had were -d64/32. they were deprecated in jdk9 by 8168010 and will be removed in jdk10 by 8169646. therefore ProcessTools::getPlatformSpecificVMArgs method is not needed anymore and should be removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8178415 > webrev: http://cr.openjdk.java.net/~iignatyev/8178415/webrev.00/ > testing: :jdk_tier{1-3}, :hotspot_all > 8168010: https://bugs.openjdk.java.net/browse/JDK-8168010 > 8169646: https://bugs.openjdk.java.net/browse/JDK-8169646 > > Thanks, > -- Igor From volker.simonis at gmail.com Wed Apr 12 09:19:58 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 12 Apr 2017 11:19:58 +0200 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58EC771B.9020202@oracle.com> References: <58EC771B.9020202@oracle.com> Message-ID: Hi Ioi, thanks for considering our ports. At least ppc64 and s390 require some additions tough :) I'm currently working on them and will provide the full ppc64/s390 versions shortly. Regards, Volker On Tue, Apr 11, 2017 at 8:26 AM, Ioi Lam wrote: > Hi,please review the following change > > https://bugs.openjdk.java.net/browse/JDK-8171392 > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constan > tpool_read_only.v01/ > > *Summary:** > * > Before: > + ConstantPool::klass_at(i) finds the Klass from the i-th slot of > ConstantPool. > + When a klass is resolved, the ConstantPool is modified to store the > Klass pointer. > > After: > + ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) > + When a klass is resolved, _resolved_klasses->at(i) is modified. > > > In addition: > + I moved _resolved_references and _reference_map from ConstantPool > to ConstantPoolCache. > + Now _flags is no longer modified for shared ConstantPools. > > As a result, none of the fields in shared ConstantPools are modified at > run time, > so we can move them into the RO region in the CDS archive. > > *Testing:** > * > - Benchmark results show no performance regression, despite the extra > level of > indirection, which has a negligible overhead for the interpreter. > - RBT testing for tier2 and tier3. > > *Ports:** > * > - I have tested only the Oracle-support ports. For the aarch64, ppc and > s390 ports, > I have added some code without testing (it's probably incomplete) > > - Port owners, please check if my patch work for you, and I can > incorporate your > changes in my push. Alternatively, you can wait for my push and > provide fixes > (if necessary) in a new changeset, and I will be happy to sponsor it. > > Thanks > - Ioi > > From ioi.lam at oracle.com Wed Apr 12 14:49:20 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 12 Apr 2017 22:49:20 +0800 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> Message-ID: <58EE3E70.1030901@oracle.com> Looks good. Reviewed. Thanks - Ioi On 4/11/17 9:03 PM, coleen.phillimore at oracle.com wrote: > > > On 4/11/17 2:35 AM, Ioi Lam wrote: >> Hi Coleen, >> >> Thanks for doing this clean up. I was guiltily of writing the >> original code :-( >> >> A few questions: >> >> Why is this block of code moved and the comments dropped? >> >> 328 void Dictionary::oops_do(OopClosure* f) { >> 329 // Only the protection domain oops contain references into the >> heap. Iterate >> 330 // over all of them. >> 331 _pd_cache_table->oops_do(f); >> 332 } >> 333 >> >> It would be better to make the changes in-place. > > I didn't have to change Dictionary::oops_do and moved it to be near > always_strong_oops_do(), so I shouldn't have removed the comment. I > moved it back but I don't like that it's separated from the other GC > functions. With my other change, it'll be closer (I think I'm going > to have a merge conflict with myself). >> >> Also, have you validated that (either with an explicit test, or >> inside the debugger) >> >> [1] live protection domains in _pd_cache_table are properly relocated >> during GC? >> [2] dead protection domains are removed after class unloading? > > I ran with runThese (which has lots of class loading and unloading) > with logging for both oops_do and unlink functions (removing > protection domain entries) but I didn't realize that > always_strong_oops_do is never called, so I deleted this function. > > New webrev (with dictionary::oops_do put back): > > http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html > > > Thanks! > Coleen >> >> Thanks >> - Ioi >> >> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>> Summary: remove system dictionary walk and pass strong closure for >>> !ClassUnloading >>> >>> See bug for more details: >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>> >>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and >>> runThese with -XX:-ClassUnloading. >>> >>> Thanks, >>> Coleen >> > From aph at redhat.com Wed Apr 12 17:22:16 2017 From: aph at redhat.com (Andrew Haley) Date: Wed, 12 Apr 2017 18:22:16 +0100 Subject: SuperWordLoopUnrollAnalysis and loop unrolling Message-ID: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> If I set SuperWordLoopUnrollAnalysis=true, then AArch64 C2 stops after unrolling a simple loop 4 times. If I set SuperWordLoopUnrollAnalysis=false, it stops after unrolling 16 times. Why is is that SuperWordLoopUnrollAnalysis limits unrolling in this way? The info says "Map number of unrolls for main loop via Superword Level Parallelism analysis" but that doesn't help me very much. Only AArch64 and x86 set this option. Thanks, Andrew. From vladimir.kozlov at oracle.com Wed Apr 12 17:56:20 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Apr 2017 10:56:20 -0700 Subject: SuperWordLoopUnrollAnalysis and loop unrolling In-Reply-To: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> References: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> Message-ID: Try to run debug VM with -XX:+TraceSuperWordLoopUnrollAnalysis. It could be https://bugs.openjdk.java.net/browse/JDK-8175096 Vladimir On 4/12/17 10:22 AM, Andrew Haley wrote: > If I set SuperWordLoopUnrollAnalysis=true, then AArch64 C2 stops after > unrolling a simple loop 4 times. If I set > SuperWordLoopUnrollAnalysis=false, it stops after unrolling 16 times. > Why is is that SuperWordLoopUnrollAnalysis limits unrolling in this > way? > > The info says > "Map number of unrolls for main loop via Superword Level Parallelism > analysis" but that doesn't help me very much. Only AArch64 and x86 > set this option. > > Thanks, > > Andrew. > From igor.ignatyev at oracle.com Wed Apr 12 19:03:15 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 12 Apr 2017 12:03:15 -0700 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management Message-ID: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html > 92 lines changed: 36 ins; 36 del; 20 mod; Hi all, could you please review this small fix which removes dependency on java.management module from the testlibrary? jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html testing: :hotspot_all 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 Thanks, -- Igor From igor.ignatyev at oracle.com Wed Apr 12 19:14:20 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 12 Apr 2017 12:14:20 -0700 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management In-Reply-To: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> References: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> Message-ID: <5548AED8-2BEB-4341-88F9-B34C0F827BAD@oracle.com> forgot to update copyrights, http://cr.openjdk.java.net/~iignatyev//8164944/webrev.01/index.html > On Apr 12, 2017, at 12:03 PM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >> 92 lines changed: 36 ins; 36 del; 20 mod; > > Hi all, > > could you please review this small fix which removes dependency on java.management module from the testlibrary? > > jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. > - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. > - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags > > the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. > > after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 > webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html > testing: :hotspot_all > 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 > > Thanks, > -- Igor From vladimir.kozlov at oracle.com Wed Apr 12 19:42:22 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Apr 2017 12:42:22 -0700 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management In-Reply-To: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> References: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> Message-ID: <0c11c59a-3c05-fb5f-36e3-579cd39c0f6d@oracle.com> Looks fine to me. Vladimir On 4/12/17 12:03 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >> 92 lines changed: 36 ins; 36 del; 20 mod; > > Hi all, > > could you please review this small fix which removes dependency on java.management module from the testlibrary? > > jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. > - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. > - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags > > the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. > > after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 > webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html > testing: :hotspot_all > 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 > > Thanks, > -- Igor > From george.triantafillou at oracle.com Wed Apr 12 20:22:13 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 12 Apr 2017 16:22:13 -0400 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management In-Reply-To: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> References: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> Message-ID: <88554a9f-8e8d-9760-4e49-854848548272@oracle.com> Hi Igor, Looks good. -George On 4/12/2017 3:03 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >> 92 lines changed: 36 ins; 36 del; 20 mod; > Hi all, > > could you please review this small fix which removes dependency on java.management module from the testlibrary? > > jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. > - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. > - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags > > the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. > > after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 > webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html > testing: :hotspot_all > 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 > > Thanks, > -- Igor From dmitry.fazunenko at oracle.com Wed Apr 12 20:36:17 2017 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 12 Apr 2017 23:36:17 +0300 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management In-Reply-To: <5548AED8-2BEB-4341-88F9-B34C0F827BAD@oracle.com> References: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> <5548AED8-2BEB-4341-88F9-B34C0F827BAD@oracle.com> Message-ID: <296e94e5-1ee4-3d9b-d7ff-d1103c43a916@oracle.com> Hi Igor, On 12.04.2017 22:14, Igor Ignatyev wrote: > forgot to update copyrights, http://cr.openjdk.java.net/~iignatyev//8164944/webrev.01/index.html It's become much better :) Now the fix looks perfect to me. Thanks for making this change! Thanks, Dima > >> On Apr 12, 2017, at 12:03 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >>> 92 lines changed: 36 ins; 36 del; 20 mod; >> Hi all, >> >> could you please review this small fix which removes dependency on java.management module from the testlibrary? >> >> jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. >> - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. >> - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags >> >> the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. >> >> after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. >> >> jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 >> webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >> testing: :hotspot_all >> 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 >> >> Thanks, >> -- Igor From coleen.phillimore at oracle.com Wed Apr 12 22:12:35 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 12 Apr 2017 18:12:35 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <58EE3E70.1030901@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> <58EE3E70.1030901@oracle.com> Message-ID: <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> Ioi, Thank you for reviewing the code. I've taken this opportunity to move the protectionDomainCache classes into files of their own with this change. I also removed an unused function bucket_size() and removed a comment before ProtectionDomainCacheEntry about it having to go into the dictionary.hpp header file. I also added debug logging for removing protectiondomain entries, and verified entries were deleted, and ran some JDK jtreg protection domain tests. Can you review this new version? Thanks! open webrev at http://cr.openjdk.java.net/~coleenp/8178336.03/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8178336 Thanks, Coleen On 4/12/17 10:49 AM, Ioi Lam wrote: > Looks good. Reviewed. > > Thanks > - Ioi > > On 4/11/17 9:03 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 4/11/17 2:35 AM, Ioi Lam wrote: >>> Hi Coleen, >>> >>> Thanks for doing this clean up. I was guiltily of writing the >>> original code :-( >>> >>> A few questions: >>> >>> Why is this block of code moved and the comments dropped? >>> >>> 328 void Dictionary::oops_do(OopClosure* f) { >>> 329 // Only the protection domain oops contain references into >>> the heap. Iterate >>> 330 // over all of them. >>> 331 _pd_cache_table->oops_do(f); >>> 332 } >>> 333 >>> >>> It would be better to make the changes in-place. >> >> I didn't have to change Dictionary::oops_do and moved it to be near >> always_strong_oops_do(), so I shouldn't have removed the comment. I >> moved it back but I don't like that it's separated from the other GC >> functions. With my other change, it'll be closer (I think I'm going >> to have a merge conflict with myself). >>> >>> Also, have you validated that (either with an explicit test, or >>> inside the debugger) >>> >>> [1] live protection domains in _pd_cache_table are properly >>> relocated during GC? >>> [2] dead protection domains are removed after class unloading? >> >> I ran with runThese (which has lots of class loading and unloading) >> with logging for both oops_do and unlink functions (removing >> protection domain entries) but I didn't realize that >> always_strong_oops_do is never called, so I deleted this function. >> >> New webrev (with dictionary::oops_do put back): >> >> http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html >> >> >> Thanks! >> Coleen >>> >>> Thanks >>> - Ioi >>> >>> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: remove system dictionary walk and pass strong closure for >>>> !ClassUnloading >>>> >>>> See bug for more details: >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>>> >>>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and >>>> runThese with -XX:-ClassUnloading. >>>> >>>> Thanks, >>>> Coleen >>> >> > From coleen.phillimore at oracle.com Wed Apr 12 22:40:58 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 12 Apr 2017 18:40:58 -0400 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58EC771B.9020202@oracle.com> References: <58EC771B.9020202@oracle.com> Message-ID: <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> This looks really good! http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html + // Add one extra element to tags for storing ConstantPool::flags(). + Array* tags = MetadataFactory::new_writeable_array(loader_data, length+1, 0, CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // tags->at(_length) is flags() I think this is left over, since _flags didn't get moved after all. + Klass** adr = this_cp->resolved_klasses()->adr_at(resolved_klass_index); + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); + // The interpreter assumes when the tag is stored, the klass is resolved + // and the Klass* is a klass rather than a Symbol*, so we need + // hardware store ordering here. + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); + return k; The comment still refers to the switch between Symbol* and Klass* in the constant pool. The entry in the Klass array should be NULL. + int name_index = extract_high_short_from_int(*int_at_addr(which)); Can you put klass_name_index_at() in the constantPool.hpp header file (so it's inlined) and have all the places where you get name_index use this function? So we don't have to know in multiple places that extract_high_short_from_int() is where the name index is. And in constantPool.hpp, for unresolved_klass_at_put() add a comment about what the new format of the constant pool entry is {name_index, resolved_klass_index}. I'm happy to see this work nearing completion! The "constant" pool should be constant! thanks, Coleen On 4/11/17 2:26 AM, Ioi Lam wrote: > Hi,please review the following change > https://bugs.openjdk.java.net/browse/JDK-8171392 > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ > *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass > from the i-th slot of ConstantPool. + When a klass is resolved, the > ConstantPool is modified to store the Klass pointer. After: + > ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) > + When a klass is resolved, _resolved_klasses->at(i) is modified. > In addition: + I moved _resolved_references and _reference_map > from ConstantPool to ConstantPoolCache. + Now _flags is no > longer modified for shared ConstantPools. As a result, none of the > fields in shared ConstantPools are modified at run time, so we can > move them into the RO region in the CDS archive. *Testing:** * - > Benchmark results show no performance regression, despite the extra > level of indirection, which has a negligible overhead for the > interpreter. - RBT testing for tier2 and tier3. *Ports:** * - I > have tested only the Oracle-support ports. For the aarch64, ppc and > s390 ports, I have added some code without testing (it's probably > incomplete) - Port owners, please check if my patch work for you, > and I can incorporate your changes in my push. Alternatively, you > can wait for my push and provide fixes (if necessary) in a new > changeset, and I will be happy to sponsor it. Thanks - Ioi From gromero at linux.vnet.ibm.com Wed Apr 12 22:51:39 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 12 Apr 2017 19:51:39 -0300 Subject: [10] RFR (S) 8175813: PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: <58C1AE06.9060609@linux.vnet.ibm.com> References: <58C1AE06.9060609@linux.vnet.ibm.com> Message-ID: <58EEAF7B.6020708@linux.vnet.ibm.com> Hi, Any update on it? Thank you. Regards, Gustavo On 09-03-2017 16:33, Gustavo Romero wrote: > Hi, > > Could the following webrev be reviewed please? > > It improves the numa node detection when non-consecutive or memory-less nodes > exist in the system. > > webrev: http://cr.openjdk.java.net/~gromero/8175813/v2/ > bug : https://bugs.openjdk.java.net/browse/JDK-8175813 > > Currently, although no problem exists when the JVM detects numa nodes that are > consecutive and have memory, for example in a numa topology like: > > available: 2 nodes (0-1) > node 0 cpus: 0 8 16 24 32 > node 0 size: 65258 MB > node 0 free: 34 MB > node 1 cpus: 40 48 56 64 72 > node 1 size: 65320 MB > node 1 free: 150 MB > node distances: > node 0 1 > 0: 10 20 > 1: 20 10, > > it fails on detecting numa nodes to be used in the Parallel GC in a numa > topology like: > > available: 4 nodes (0-1,16-17) > node 0 cpus: 0 8 16 24 32 > node 0 size: 130706 MB > node 0 free: 7729 MB > node 1 cpus: 40 48 56 64 72 > node 1 size: 0 MB > node 1 free: 0 MB > node 16 cpus: 80 88 96 104 112 > node 16 size: 130630 MB > node 16 free: 5282 MB > node 17 cpus: 120 128 136 144 152 > node 17 size: 0 MB > node 17 free: 0 MB > node distances: > node 0 1 16 17 > 0: 10 20 40 40 > 1: 20 10 40 40 > 16: 40 40 10 20 > 17: 40 40 20 10, > > where node 16 is not consecutive in relation to 1 and also nodes 1 and 17 have > no memory. > > If a topology like that exists, os::numa_make_local() will receive a local group > id as a hint that is not available in the system to be bound (it will receive > all nodes from 0 to 17), causing a proliferation of "mbind: Invalid argument" > messages: > > http://cr.openjdk.java.net/~gromero/logs/jdk10_pristine.log > > That change improves the detection by making the JVM numa API aware of the > existence of numa nodes that are non-consecutive from 0 to the highest node > number and also of nodes that might be memory-less nodes, i.e. that might not > be, in libnuma terms, a configured node. Hence just the configured nodes will > be available: > > http://cr.openjdk.java.net/~gromero/logs/jdk10_numa_patched.log > > The change has no effect on numa topologies were the problem does not occur, > i.e. no change in the number of nodes and no change in the cpu to node map. On > numa topologies where memory-less nodes exist (like in the last example above), > cpus from a memory-less node won't be able to bind locally so they are mapped > to the closest node, otherwise they would be not associate to any node and > MutableNUMASpace::cas_allocate() would pick a node randomly, compromising the > performance. > > I found no regressions on x64 for the following numa topology: > > available: 2 nodes (0-1) > node 0 cpus: 0 1 2 3 8 9 10 11 > node 0 size: 24102 MB > node 0 free: 19806 MB > node 1 cpus: 4 5 6 7 12 13 14 15 > node 1 size: 24190 MB > node 1 free: 21951 MB > node distances: > node 0 1 > 0: 10 21 > 1: 21 10 > > I understand that fixing the current numa detection is a prerequisite to enable > UseNUMA by the default [1] and to extend the numa-aware allocation to the G1 GC [2]. > > Thank you. > > > Best regards, > Gustavo > > [1] https://bugs.openjdk.java.net/browse/JDK-8046153 (JEP 163: Enable NUMA Mode by Default When Appropriate) > [2] https://bugs.openjdk.java.net/browse/JDK-8046147 (JEP 157: G1 GC: NUMA-Aware Allocation) > From igor.ignatyev at oracle.com Thu Apr 13 00:15:29 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 12 Apr 2017 17:15:29 -0700 Subject: RFR(S) : 8178695: jdk.test.lib.artifacts.ArtifactContainer has to be public Message-ID: <6D6742A3-7C03-4C66-8D49-DF981B82D46D@oracle.com> http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html > 38 lines changed: 33 ins; 5 del; 0 mod; Hi all, could you please review this small fix for the testlibrary? @ArtifactContainer is created to support repeating @Artifact annotations, but it is package private class, which causes a compilation error if one uses more than one @Artifact. the fix make j.t.l.a.ArtifactContainer publicly accessible. webrev: http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8178695 Thanks, -- Igor From christian.tornqvist at oracle.com Thu Apr 13 00:40:29 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Wed, 12 Apr 2017 20:40:29 -0400 Subject: RFR(S) : 8178695: jdk.test.lib.artifacts.ArtifactContainer has to be public In-Reply-To: <6D6742A3-7C03-4C66-8D49-DF981B82D46D@oracle.com> References: <6D6742A3-7C03-4C66-8D49-DF981B82D46D@oracle.com> Message-ID: Hi Igor, This looks good, thanks for fixing this :) Thanks, Christian > On Apr 12, 2017, at 8:15 PM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html >> 38 lines changed: 33 ins; 5 del; 0 mod; > > Hi all, > > could you please review this small fix for the testlibrary? > > @ArtifactContainer is created to support repeating @Artifact annotations, but it is package private class, which causes a compilation error if one uses more than one @Artifact. the fix make j.t.l.a.ArtifactContainer publicly accessible. > > webrev: http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8178695 > > Thanks, > -- Igor > > From george.triantafillou at oracle.com Thu Apr 13 01:12:06 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 12 Apr 2017 21:12:06 -0400 Subject: RFR(S) : 8178695: jdk.test.lib.artifacts.ArtifactContainer has to be public In-Reply-To: <6D6742A3-7C03-4C66-8D49-DF981B82D46D@oracle.com> References: <6D6742A3-7C03-4C66-8D49-DF981B82D46D@oracle.com> Message-ID: <8024ad57-b5f5-3f95-aed1-505c709b74b6@oracle.com> Hi Igor, Looks good! -George On 4/12/2017 8:15 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html >> 38 lines changed: 33 ins; 5 del; 0 mod; > Hi all, > > could you please review this small fix for the testlibrary? > > @ArtifactContainer is created to support repeating @Artifact annotations, but it is package private class, which causes a compilation error if one uses more than one @Artifact. the fix make j.t.l.a.ArtifactContainer publicly accessible. > > webrev: http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8178695 > > Thanks, > -- Igor > > From david.holmes at oracle.com Thu Apr 13 01:26:11 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Apr 2017 11:26:11 +1000 Subject: linux os processor optimizations for OpenJDK performance enhancement In-Reply-To: References: Message-ID: <86d58d9d-ea45-6732-370f-413ff1b4e8f4@oracle.com> Redirecting to hotspot-dev mailing list, please drop jdk10-dev on future replies. On 13/04/2017 9:39 AM, Ram Krishnan wrote: > We are working on linux os processor specific optimizations centered around > cache partitioning and better processor core thread(s) affinity for OpenJDK > GC performance enhancement with the following benefits > - throughput increase > - minimizing tail latency for latency sensitive applications > - superior isolation in a multi-tenant environment > > ?We have been looking at the G1GC code/documentation and performed some > initial experiments. Based on this, what we understand is that STW events > pause all application threads independent of which regions are being > impacted by garbage collection. For example, if application thread "x" uses > regions 1, 4, 5 and garbage collection is working on regions 2 and 11, > application thread "x" is paused during the STW event. A stop-the-world event does stop all application threads (all JavaThreads in VM terminology) by using a safepoint. I'll let the GC folks comment on the details wrt. G1 region management. Cheers, David > Your expert opinion on this topic is much appreciated. > From igor.ignatyev at oracle.com Thu Apr 13 01:45:54 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 12 Apr 2017 18:45:54 -0700 Subject: RFR(S) : 8178695: jdk.test.lib.artifacts.ArtifactContainer has to be public In-Reply-To: References: <6D6742A3-7C03-4C66-8D49-DF981B82D46D@oracle.com> Message-ID: Hi George, Christian, thank you for your reviews! -- Igor > On Apr 12, 2017, at 5:40 PM, Christian Tornqvist wrote: > > Hi Igor, > > This looks good, thanks for fixing this :) > > Thanks, > Christian > >> On Apr 12, 2017, at 8:15 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html >>> 38 lines changed: 33 ins; 5 del; 0 mod; >> >> Hi all, >> >> could you please review this small fix for the testlibrary? >> >> @ArtifactContainer is created to support repeating @Artifact annotations, but it is package private class, which causes a compilation error if one uses more than one @Artifact. the fix make j.t.l.a.ArtifactContainer publicly accessible. >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8178695/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8178695 >> >> Thanks, >> -- Igor >> >> > From david.holmes at oracle.com Thu Apr 13 01:47:11 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Apr 2017 11:47:11 +1000 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management In-Reply-To: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> References: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> Message-ID: Hi Igor, Overall this looks good. To be clear the old executeTestJvmAllArgs() presumably provides the VM options from @run _and_ @module - right? So if they don't exist in a test we can simply replace the usage of executeTestJvmAllArgs with executeTestJvm - as done in: test/compiler/c2/cr7200264/TestDriver.java And for other tests we either grab the VM input arguments as a separate step, or else reconstruct them as needed (eg for the module flags in DebugOutputTest). Thanks, David On 13/04/2017 5:03 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >> 92 lines changed: 36 ins; 36 del; 20 mod; > > Hi all, > > could you please review this small fix which removes dependency on java.management module from the testlibrary? > > jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. > - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. > - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags > > the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. > > after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 > webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html > testing: :hotspot_all > 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 > > Thanks, > -- Igor > From igor.ignatyev at oracle.com Thu Apr 13 01:56:06 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 12 Apr 2017 18:56:06 -0700 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management In-Reply-To: References: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> Message-ID: Hi David, right. executeTestJvmAllArgs prepends the VM options which the current JVM was started with, which is all options from jtreg harness: @run, @module, and the options specified in the command line by -javaoptions and -vmoptions to the passed arguments. executeTestJvm prepends only the options specified in the command line. so if there is no options from @run or @module, there is no difference b/w executeTestJvmAllArgs and executeTestJvm. in other cases, we need to do some extra steps. -- Igor > On Apr 12, 2017, at 6:47 PM, David Holmes wrote: > > Hi Igor, > > Overall this looks good. > > To be clear the old executeTestJvmAllArgs() presumably provides the VM options from @run _and_ @module - right? So if they don't exist in a test we can simply replace the usage of executeTestJvmAllArgs with executeTestJvm - as done in: > > test/compiler/c2/cr7200264/TestDriver.java > > And for other tests we either grab the VM input arguments as a separate step, or else reconstruct them as needed (eg for the module flags in DebugOutputTest). > > Thanks, > David > > On 13/04/2017 5:03 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >>> 92 lines changed: 36 ins; 36 del; 20 mod; >> >> Hi all, >> >> could you please review this small fix which removes dependency on java.management module from the testlibrary? >> >> jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. >> - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. >> - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags >> >> the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. >> >> after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. >> >> jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 >> webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >> testing: :hotspot_all >> 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 >> >> Thanks, >> -- Igor >> From jiangli.zhou at oracle.com Thu Apr 13 04:48:09 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 12 Apr 2017 21:48:09 -0700 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> <58EE3E70.1030901@oracle.com> <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> Message-ID: <4F8E8FBA-8FC7-4B4E-AB4A-94C917CCB05F@oracle.com> Hi Coleen, Looks good. Thanks, Jiangli > On Apr 12, 2017, at 3:12 PM, coleen.phillimore at oracle.com wrote: > > Ioi, Thank you for reviewing the code. I've taken this opportunity to move the protectionDomainCache classes into files of their own with this change. I also removed an unused function bucket_size() and removed a comment before ProtectionDomainCacheEntry about it having to go into the dictionary.hpp header file. > > I also added debug logging for removing protectiondomain entries, and verified entries were deleted, and ran some JDK jtreg protection domain tests. > > Can you review this new version? Thanks! > > open webrev at http://cr.openjdk.java.net/~coleenp/8178336.03/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8178336 > > Thanks, > Coleen > > On 4/12/17 10:49 AM, Ioi Lam wrote: >> Looks good. Reviewed. >> >> Thanks >> - Ioi >> >> On 4/11/17 9:03 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 4/11/17 2:35 AM, Ioi Lam wrote: >>>> Hi Coleen, >>>> >>>> Thanks for doing this clean up. I was guiltily of writing the original code :-( >>>> >>>> A few questions: >>>> >>>> Why is this block of code moved and the comments dropped? >>>> >>>> 328 void Dictionary::oops_do(OopClosure* f) { >>>> 329 // Only the protection domain oops contain references into the heap. Iterate >>>> 330 // over all of them. >>>> 331 _pd_cache_table->oops_do(f); >>>> 332 } >>>> 333 >>>> >>>> It would be better to make the changes in-place. >>> >>> I didn't have to change Dictionary::oops_do and moved it to be near always_strong_oops_do(), so I shouldn't have removed the comment. I moved it back but I don't like that it's separated from the other GC functions. With my other change, it'll be closer (I think I'm going to have a merge conflict with myself). >>>> >>>> Also, have you validated that (either with an explicit test, or inside the debugger) >>>> >>>> [1] live protection domains in _pd_cache_table are properly relocated during GC? >>>> [2] dead protection domains are removed after class unloading? >>> >>> I ran with runThese (which has lots of class loading and unloading) with logging for both oops_do and unlink functions (removing protection domain entries) but I didn't realize that always_strong_oops_do is never called, so I deleted this function. >>> >>> New webrev (with dictionary::oops_do put back): >>> >>> http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html >>> >>> Thanks! >>> Coleen >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>>>> Summary: remove system dictionary walk and pass strong closure for !ClassUnloading >>>>> >>>>> See bug for more details: >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>>>> >>>>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese with -XX:-ClassUnloading. >>>>> >>>>> Thanks, >>>>> Coleen >>>> >>> >> > From ioi.lam at oracle.com Thu Apr 13 08:56:26 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 13 Apr 2017 16:56:26 +0800 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> Message-ID: <58EF3D3A.6020903@oracle.com> Hi Coleen, Thanks for the comments. Here's a delta from the last patch http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ In addition to your requests, I made these changes: [1] To consolidate the multiple extract_high/low code, I've added CPKlassSlot, so the code is cleaner: CPKlassSlot kslot = this_cp->klass_slot_at(which); int resolved_klass_index = kslot.resolved_klass_index(); int name_index = kslot.name_index(); [2] Renamed ConstantPool::is_shared_quick() to ConstantPool::is_shared(). The C++ compiler should be able to pick this function over MetaspaceObj::is_shared(). [3] Massaged the CDS region size set-up code a little to pass internal tests, because RO/RW ratio has changed. I didn't spend too much time picking the "right" sizes, as this code will be obsoleted soon with JDK-8072061 Thanks - Ioi On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: > > This looks really good! > > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html > > > + // Add one extra element to tags for storing ConstantPool::flags(). > + Array* tags = > MetadataFactory::new_writeable_array(loader_data, length+1, 0, > CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); > // tags->at(_length) is flags() > > > I think this is left over, since _flags didn't get moved after all. > > + Klass** adr = > this_cp->resolved_klasses()->adr_at(resolved_klass_index); > + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); > + // The interpreter assumes when the tag is stored, the klass is > resolved > + // and the Klass* is a klass rather than a Symbol*, so we need > + // hardware store ordering here. > + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); > + return k; > > The comment still refers to the switch between Symbol* and Klass* in > the constant pool. The entry in the Klass array should be NULL. > > + int name_index = extract_high_short_from_int(*int_at_addr(which)); > > Can you put klass_name_index_at() in the constantPool.hpp header file > (so it's inlined) and have all the places where you get name_index use > this function? So we don't have to know in multiple places that > extract_high_short_from_int() is where the name index is. And in > constantPool.hpp, for unresolved_klass_at_put() add a comment about > what the new format of the constant pool entry is {name_index, > resolved_klass_index}. I'm happy to see this work nearing completion! > The "constant" pool should be constant! thanks, Coleen > On 4/11/17 2:26 AM, Ioi Lam wrote: >> Hi,please review the following change >> https://bugs.openjdk.java.net/browse/JDK-8171392 >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass >> from the i-th slot of ConstantPool. + When a klass is resolved, the >> ConstantPool is modified to store the Klass pointer. After: + >> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) >> + When a klass is resolved, _resolved_klasses->at(i) is modified. >> In addition: + I moved _resolved_references and _reference_map >> from ConstantPool to ConstantPoolCache. + Now _flags is no >> longer modified for shared ConstantPools. As a result, none of the >> fields in shared ConstantPools are modified at run time, so we can >> move them into the RO region in the CDS archive. *Testing:** * - >> Benchmark results show no performance regression, despite the extra >> level of indirection, which has a negligible overhead for the >> interpreter. - RBT testing for tier2 and tier3. *Ports:** * - I >> have tested only the Oracle-support ports. For the aarch64, ppc and >> s390 ports, I have added some code without testing (it's probably >> incomplete) - Port owners, please check if my patch work for you, >> and I can incorporate your changes in my push. Alternatively, you >> can wait for my push and provide fixes (if necessary) in a new >> changeset, and I will be happy to sponsor it. Thanks - Ioi From ioi.lam at oracle.com Thu Apr 13 09:22:49 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 13 Apr 2017 17:22:49 +0800 Subject: RFR(S) JDK-8178350 - klassVtable and klassItable should be ValueObj Message-ID: <58EF4369.3050303@oracle.com> Hi, please review this JVM start-up enhancement: https://bugs.openjdk.java.net/browse/JDK-8178350 http://cr.openjdk.java.net/~iklam/jdk10/8178350-klassvtable-as-valueobj.v01/ Klass::vtable() is called frequently inside InstanceKlass::link_class_impl(). We have a start-up benchmark that loads about 4000 classes. Klass::vtable() is called 32458 times and costs about 2~3% of a total elapsed time of 720 ms. Because klassVtable and klassItable are very simple, fixed sized structures, it's much better to allocate them on the stack: OLD: class klassVtable : ResourceObj {...} klassVtable* Klass::vtable() const { return new klassVtable(const_cast(this), start_of_vtable(), vtable_length() / vtableEntry::size()); } NEW: class klassVtable VALUE_OBJ_CLASS_SPEC {...} klassVtable Klass::vtable() const { return klassVtable(const_cast(this), start_of_vtable(), vtable_length() / vtableEntry::size()); } (>> David) I am not sure whether to make klassVtable a StackObj or ValueObj. Since it's returned by a function call, it seems ValueObj is better?? I also removed some ResourceMarks that are obviously not necessary. I left some others unchanged, like here in link_class_impl(): 623 ResourceMark rm(THREAD); 624 vtable().initialize_vtable(true, CHECK_false); 625 itable().initialize_itable(true, CHECK_false); because some code inside initialize_vtable() would do a resource allocation without a ResourceMark. I don't want to touch that for now, as it's unclear to me whether the allocated object(s) should be scoped, or should be returned to a higher scope. I filed JDK-8178712 to track this. Thanks - Ioi From ioi.lam at oracle.com Thu Apr 13 09:30:52 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 13 Apr 2017 17:30:52 +0800 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() Message-ID: <58EF454C.9050307@oracle.com> Hi, please review this small start-up enhancement: https://bugs.openjdk.java.net/browse/JDK-8178543 http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ We have a benchmark that shows Klass::is_shared() is called very frequently during InstanceKlass::link_class_impl, and costs about 2% of the start-up time. The fix is simple -- instead of walking the list of CDS shared regions, use a new bit in Klass::_access_flags Thanks - Ioi From ioi.lam at oracle.com Thu Apr 13 10:19:35 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 13 Apr 2017 18:19:35 +0800 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> <58EE3E70.1030901@oracle.com> <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> Message-ID: <58EF50B7.1050800@oracle.com> Hi Coleen, This is really good. Moving the protectionDomainCache out make it much easier to work on it separately, without getting stuck in what seems to be a million different versions of oops_do(). Everything looks fine. I just have one suggestion: Every time I work on this code, I get confused about what ProtectionDomainEntry is. I need to refer to this comment (which I wrote :-) 137 // Contains the set of approved protection domains that can access 138 // this system dictionary entry. 139 // 140 // This protection domain set is a set of tuples: 141 // 142 // (InstanceKlass C, initiating class loader ICL, Protection Domain PD) 143 // 144 // [Note that C.protection_domain(), which is stored in the java.lang.Class 145 // mirror of C, is NOT the same as PD] 146 // 147 // If such an entry (C, ICL, PD) exists in the table, it means that 148 // it is okay for a class Foo to reference C, where 149 // 150 // Foo.protection_domain() == PD, and 151 // Foo's defining class loader == ICL 152 // 153 // The usage of the PD set can be seen in SystemDictionary::validate_protection_domain() 154 // It is essentially a cache to avoid repeated Java up-calls to 155 // ClassLoader.checkPackageAccess(). 156 // 157 ProtectionDomainEntry* _pd_set; Maybe in the header that declares class ProtectionDomainEntry, add a comment to the effect: "ProtectionDomainEntry doesn't do what you think it does -- it's NOT where InstanceKlass::protection_domain() stores its return value. See comments inside DictionaryEntry for details." Thanks - Ioi On 4/13/17 6:12 AM, coleen.phillimore at oracle.com wrote: > Ioi, Thank you for reviewing the code. I've taken this opportunity > to move the protectionDomainCache classes into files of their own with > this change. I also removed an unused function bucket_size() and > removed a comment before ProtectionDomainCacheEntry about it having to > go into the dictionary.hpp header file. > > I also added debug logging for removing protectiondomain entries, and > verified entries were deleted, and ran some JDK jtreg protection > domain tests. > > Can you review this new version? Thanks! > > open webrev at http://cr.openjdk.java.net/~coleenp/8178336.03/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8178336 > > Thanks, > Coleen > > On 4/12/17 10:49 AM, Ioi Lam wrote: >> Looks good. Reviewed. >> >> Thanks >> - Ioi >> >> On 4/11/17 9:03 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 4/11/17 2:35 AM, Ioi Lam wrote: >>>> Hi Coleen, >>>> >>>> Thanks for doing this clean up. I was guiltily of writing the >>>> original code :-( >>>> >>>> A few questions: >>>> >>>> Why is this block of code moved and the comments dropped? >>>> >>>> 328 void Dictionary::oops_do(OopClosure* f) { >>>> 329 // Only the protection domain oops contain references into >>>> the heap. Iterate >>>> 330 // over all of them. >>>> 331 _pd_cache_table->oops_do(f); >>>> 332 } >>>> 333 >>>> >>>> It would be better to make the changes in-place. >>> >>> I didn't have to change Dictionary::oops_do and moved it to be near >>> always_strong_oops_do(), so I shouldn't have removed the comment. I >>> moved it back but I don't like that it's separated from the other GC >>> functions. With my other change, it'll be closer (I think I'm >>> going to have a merge conflict with myself). >>>> >>>> Also, have you validated that (either with an explicit test, or >>>> inside the debugger) >>>> >>>> [1] live protection domains in _pd_cache_table are properly >>>> relocated during GC? >>>> [2] dead protection domains are removed after class unloading? >>> >>> I ran with runThese (which has lots of class loading and unloading) >>> with logging for both oops_do and unlink functions (removing >>> protection domain entries) but I didn't realize that >>> always_strong_oops_do is never called, so I deleted this function. >>> >>> New webrev (with dictionary::oops_do put back): >>> >>> http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html >>> >>> >>> Thanks! >>> Coleen >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>>>> Summary: remove system dictionary walk and pass strong closure for >>>>> !ClassUnloading >>>>> >>>>> See bug for more details: >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>>>> >>>>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and >>>>> runThese with -XX:-ClassUnloading. >>>>> >>>>> Thanks, >>>>> Coleen >>>> >>> >> > From coleen.phillimore at oracle.com Thu Apr 13 12:10:33 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Apr 2017 08:10:33 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <58EF50B7.1050800@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> <58EE3E70.1030901@oracle.com> <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> <58EF50B7.1050800@oracle.com> Message-ID: <690e79ce-3510-7983-0c9d-3c7c1f66b7ea@oracle.com> On 4/13/17 6:19 AM, Ioi Lam wrote: > Hi Coleen, > > This is really good. Moving the protectionDomainCache out make it much > easier to work on it separately, without getting stuck in what seems > to be a million different versions of oops_do(). > > Everything looks fine. I just have one suggestion: > > Every time I work on this code, I get confused about what > ProtectionDomainEntry is. I need to refer to this comment (which I > wrote :-) > > 137 // Contains the set of approved protection domains that can access > 138 // this system dictionary entry. > 139 // > 140 // This protection domain set is a set of tuples: > 141 // > 142 // (InstanceKlass C, initiating class loader ICL, Protection > Domain PD) > 143 // > 144 // [Note that C.protection_domain(), which is stored in the > java.lang.Class > 145 // mirror of C, is NOT the same as PD] > 146 // > 147 // If such an entry (C, ICL, PD) exists in the table, it means > that > 148 // it is okay for a class Foo to reference C, where > 149 // > 150 // Foo.protection_domain() == PD, and > 151 // Foo's defining class loader == ICL > 152 // > 153 // The usage of the PD set can be seen in > SystemDictionary::validate_protection_domain() > 154 // It is essentially a cache to avoid repeated Java up-calls to > 155 // ClassLoader.checkPackageAccess(). > 156 // > 157 ProtectionDomainEntry* _pd_set; > > > Maybe in the header that declares class ProtectionDomainEntry, add a > comment to the effect: > > "ProtectionDomainEntry doesn't do what you think it does -- it's NOT > where InstanceKlass::protection_domain() stores its return value. See > comments inside DictionaryEntry for details." Yes, I agree, I'll add this comment to the header of protectionDomainCache.hpp: // This class caches the approved protection domains that can access loaded classes. // Dictionary entry pd_set point to entries in this hashtable. Please refer // to dictionary.hpp pd_set for more information about how protection domain entries // are used. // This hashtable is walked for GC, not the entire system dictionary. Thanks for all the help on this change. Coleen > > Thanks > - Ioi > > > On 4/13/17 6:12 AM, coleen.phillimore at oracle.com wrote: >> Ioi, Thank you for reviewing the code. I've taken this opportunity >> to move the protectionDomainCache classes into files of their own >> with this change. I also removed an unused function bucket_size() >> and removed a comment before ProtectionDomainCacheEntry about it >> having to go into the dictionary.hpp header file. >> >> I also added debug logging for removing protectiondomain entries, and >> verified entries were deleted, and ran some JDK jtreg protection >> domain tests. >> >> Can you review this new version? Thanks! >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.03/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >> >> Thanks, >> Coleen >> >> On 4/12/17 10:49 AM, Ioi Lam wrote: >>> Looks good. Reviewed. >>> >>> Thanks >>> - Ioi >>> >>> On 4/11/17 9:03 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 4/11/17 2:35 AM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for doing this clean up. I was guiltily of writing the >>>>> original code :-( >>>>> >>>>> A few questions: >>>>> >>>>> Why is this block of code moved and the comments dropped? >>>>> >>>>> 328 void Dictionary::oops_do(OopClosure* f) { >>>>> 329 // Only the protection domain oops contain references into >>>>> the heap. Iterate >>>>> 330 // over all of them. >>>>> 331 _pd_cache_table->oops_do(f); >>>>> 332 } >>>>> 333 >>>>> >>>>> It would be better to make the changes in-place. >>>> >>>> I didn't have to change Dictionary::oops_do and moved it to be near >>>> always_strong_oops_do(), so I shouldn't have removed the comment. >>>> I moved it back but I don't like that it's separated from the other >>>> GC functions. With my other change, it'll be closer (I think I'm >>>> going to have a merge conflict with myself). >>>>> >>>>> Also, have you validated that (either with an explicit test, or >>>>> inside the debugger) >>>>> >>>>> [1] live protection domains in _pd_cache_table are properly >>>>> relocated during GC? >>>>> [2] dead protection domains are removed after class unloading? >>>> >>>> I ran with runThese (which has lots of class loading and unloading) >>>> with logging for both oops_do and unlink functions (removing >>>> protection domain entries) but I didn't realize that >>>> always_strong_oops_do is never called, so I deleted this function. >>>> >>>> New webrev (with dictionary::oops_do put back): >>>> >>>> http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html >>>> >>>> >>>> Thanks! >>>> Coleen >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: remove system dictionary walk and pass strong closure >>>>>> for !ClassUnloading >>>>>> >>>>>> See bug for more details: >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>>>>> >>>>>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and >>>>>> runThese with -XX:-ClassUnloading. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Thu Apr 13 12:11:14 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Apr 2017 08:11:14 -0400 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <4F8E8FBA-8FC7-4B4E-AB4A-94C917CCB05F@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> <58EE3E70.1030901@oracle.com> <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> <4F8E8FBA-8FC7-4B4E-AB4A-94C917CCB05F@oracle.com> Message-ID: <9bf31953-ce5d-7077-062f-6d5f065fede6@oracle.com> Thank you Jiangli! Coleen On 4/13/17 12:48 AM, Jiangli Zhou wrote: > Hi Coleen, > > Looks good. > > Thanks, > Jiangli > >> On Apr 12, 2017, at 3:12 PM, coleen.phillimore at oracle.com wrote: >> >> Ioi, Thank you for reviewing the code. I've taken this opportunity to move the protectionDomainCache classes into files of their own with this change. I also removed an unused function bucket_size() and removed a comment before ProtectionDomainCacheEntry about it having to go into the dictionary.hpp header file. >> >> I also added debug logging for removing protectiondomain entries, and verified entries were deleted, and ran some JDK jtreg protection domain tests. >> >> Can you review this new version? Thanks! >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.03/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >> >> Thanks, >> Coleen >> >> On 4/12/17 10:49 AM, Ioi Lam wrote: >>> Looks good. Reviewed. >>> >>> Thanks >>> - Ioi >>> >>> On 4/11/17 9:03 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> On 4/11/17 2:35 AM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for doing this clean up. I was guiltily of writing the original code :-( >>>>> >>>>> A few questions: >>>>> >>>>> Why is this block of code moved and the comments dropped? >>>>> >>>>> 328 void Dictionary::oops_do(OopClosure* f) { >>>>> 329 // Only the protection domain oops contain references into the heap. Iterate >>>>> 330 // over all of them. >>>>> 331 _pd_cache_table->oops_do(f); >>>>> 332 } >>>>> 333 >>>>> >>>>> It would be better to make the changes in-place. >>>> I didn't have to change Dictionary::oops_do and moved it to be near always_strong_oops_do(), so I shouldn't have removed the comment. I moved it back but I don't like that it's separated from the other GC functions. With my other change, it'll be closer (I think I'm going to have a merge conflict with myself). >>>>> Also, have you validated that (either with an explicit test, or inside the debugger) >>>>> >>>>> [1] live protection domains in _pd_cache_table are properly relocated during GC? >>>>> [2] dead protection domains are removed after class unloading? >>>> I ran with runThese (which has lots of class loading and unloading) with logging for both oops_do and unlink functions (removing protection domain entries) but I didn't realize that always_strong_oops_do is never called, so I deleted this function. >>>> >>>> New webrev (with dictionary::oops_do put back): >>>> >>>> http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html >>>> >>>> Thanks! >>>> Coleen >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: remove system dictionary walk and pass strong closure for !ClassUnloading >>>>>> >>>>>> See bug for more details: >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>>>>> >>>>>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese with -XX:-ClassUnloading. >>>>>> >>>>>> Thanks, >>>>>> Coleen From coleen.phillimore at oracle.com Thu Apr 13 12:15:11 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Apr 2017 08:15:11 -0400 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: <58EF454C.9050307@oracle.com> References: <58EF454C.9050307@oracle.com> Message-ID: This looks good. Glad there was an access flag bit left. http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/src/share/vm/oops/klassVtable.cpp.udiff.html I'd make it #ifdef ASSERT instead. I don't think anyone's built optimized in a long time tho. Thanks, Coleen On 4/13/17 5:30 AM, Ioi Lam wrote: > Hi, please review this small start-up enhancement: > > https://bugs.openjdk.java.net/browse/JDK-8178543 > http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ > > We have a benchmark that shows Klass::is_shared() is called very > frequently during InstanceKlass::link_class_impl, and costs about 2% > of the start-up time. > > The fix is simple -- instead of walking the list of CDS shared > regions, use a new bit in Klass::_access_flags > > Thanks > - Ioi From ioi.lam at oracle.com Thu Apr 13 13:47:05 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 13 Apr 2017 21:47:05 +0800 Subject: RFR (S) 8178336: Unnecessary SystemDictionary walk for Protection domain liveness In-Reply-To: <690e79ce-3510-7983-0c9d-3c7c1f66b7ea@oracle.com> References: <78e4db4d-d802-ce35-bdd3-f856faf3ac63@oracle.com> <58EC7921.3090107@oracle.com> <012e059d-1466-cb8c-c89b-af08537ff43f@oracle.com> <58EE3E70.1030901@oracle.com> <3e2f8784-2002-6b11-cc9c-e4bfbc1b2244@oracle.com> <58EF50B7.1050800@oracle.com> <690e79ce-3510-7983-0c9d-3c7c1f66b7ea@oracle.com> Message-ID: <72EFA31F-ED72-4665-B75A-38C22250747D@oracle.com> Looks good. Thanks Coleen! Ioi > coleen.phillimore at oracle.com ? 2017?4?13? ??8:10 ??? > > > >> On 4/13/17 6:19 AM, Ioi Lam wrote: >> Hi Coleen, >> >> This is really good. Moving the protectionDomainCache out make it much easier to work on it separately, without getting stuck in what seems to be a million different versions of oops_do(). >> >> Everything looks fine. I just have one suggestion: >> >> Every time I work on this code, I get confused about what ProtectionDomainEntry is. I need to refer to this comment (which I wrote :-) >> >> 137 // Contains the set of approved protection domains that can access >> 138 // this system dictionary entry. >> 139 // >> 140 // This protection domain set is a set of tuples: >> 141 // >> 142 // (InstanceKlass C, initiating class loader ICL, Protection Domain PD) >> 143 // >> 144 // [Note that C.protection_domain(), which is stored in the java.lang.Class >> 145 // mirror of C, is NOT the same as PD] >> 146 // >> 147 // If such an entry (C, ICL, PD) exists in the table, it means that >> 148 // it is okay for a class Foo to reference C, where >> 149 // >> 150 // Foo.protection_domain() == PD, and >> 151 // Foo's defining class loader == ICL >> 152 // >> 153 // The usage of the PD set can be seen in SystemDictionary::validate_protection_domain() >> 154 // It is essentially a cache to avoid repeated Java up-calls to >> 155 // ClassLoader.checkPackageAccess(). >> 156 // >> 157 ProtectionDomainEntry* _pd_set; >> >> >> Maybe in the header that declares class ProtectionDomainEntry, add a comment to the effect: >> >> "ProtectionDomainEntry doesn't do what you think it does -- it's NOT where InstanceKlass::protection_domain() stores its return value. See comments inside DictionaryEntry for details." > > Yes, I agree, I'll add this comment to the header of protectionDomainCache.hpp: > > // This class caches the approved protection domains that can access loaded classes. > // Dictionary entry pd_set point to entries in this hashtable. Please refer > // to dictionary.hpp pd_set for more information about how protection domain entries > // are used. > // This hashtable is walked for GC, not the entire system dictionary. > > Thanks for all the help on this change. > Coleen >> >> Thanks >> - Ioi >> >> >>> On 4/13/17 6:12 AM, coleen.phillimore at oracle.com wrote: >>> Ioi, Thank you for reviewing the code. I've taken this opportunity to move the protectionDomainCache classes into files of their own with this change. I also removed an unused function bucket_size() and removed a comment before ProtectionDomainCacheEntry about it having to go into the dictionary.hpp header file. >>> >>> I also added debug logging for removing protectiondomain entries, and verified entries were deleted, and ran some JDK jtreg protection domain tests. >>> >>> Can you review this new version? Thanks! >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.03/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>> >>> Thanks, >>> Coleen >>> >>>> On 4/12/17 10:49 AM, Ioi Lam wrote: >>>> Looks good. Reviewed. >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> On 4/11/17 9:03 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>>> On 4/11/17 2:35 AM, Ioi Lam wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for doing this clean up. I was guiltily of writing the original code :-( >>>>>> >>>>>> A few questions: >>>>>> >>>>>> Why is this block of code moved and the comments dropped? >>>>>> >>>>>> 328 void Dictionary::oops_do(OopClosure* f) { >>>>>> 329 // Only the protection domain oops contain references into the heap. Iterate >>>>>> 330 // over all of them. >>>>>> 331 _pd_cache_table->oops_do(f); >>>>>> 332 } >>>>>> 333 >>>>>> >>>>>> It would be better to make the changes in-place. >>>>> >>>>> I didn't have to change Dictionary::oops_do and moved it to be near always_strong_oops_do(), so I shouldn't have removed the comment. I moved it back but I don't like that it's separated from the other GC functions. With my other change, it'll be closer (I think I'm going to have a merge conflict with myself). >>>>>> >>>>>> Also, have you validated that (either with an explicit test, or inside the debugger) >>>>>> >>>>>> [1] live protection domains in _pd_cache_table are properly relocated during GC? >>>>>> [2] dead protection domains are removed after class unloading? >>>>> >>>>> I ran with runThese (which has lots of class loading and unloading) with logging for both oops_do and unlink functions (removing protection domain entries) but I didn't realize that always_strong_oops_do is never called, so I deleted this function. >>>>> >>>>> New webrev (with dictionary::oops_do put back): >>>>> >>>>> http://oklahoma.us.oracle.com/~cphillim/webrev/8178336.02/webrev/index.html >>>>> >>>>> Thanks! >>>>> Coleen >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>>> On 4/11/17 4:18 AM, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: remove system dictionary walk and pass strong closure for !ClassUnloading >>>>>>> >>>>>>> See bug for more details: >>>>>>> >>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8178336.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8178336 >>>>>>> >>>>>>> Tested with nightly tier2-5 tests and jprt (runs all GCs) and runThese with -XX:-ClassUnloading. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>> >>>>> >>>> >>> >> > From calvin.cheung at oracle.com Thu Apr 13 14:56:18 2017 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 13 Apr 2017 07:56:18 -0700 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: <58EF454C.9050307@oracle.com> References: <58EF454C.9050307@oracle.com> Message-ID: <58EF9192.1010808@oracle.com> Hi Ioi, The fix looks good to me. thanks, Calvin On 4/13/17, 2:30 AM, Ioi Lam wrote: > Hi, please review this small start-up enhancement: > > https://bugs.openjdk.java.net/browse/JDK-8178543 > http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ > > We have a benchmark that shows Klass::is_shared() is called very > frequently during InstanceKlass::link_class_impl, and costs about 2% > of the start-up time. > > The fix is simple -- instead of walking the list of CDS shared > regions, use a new bit in Klass::_access_flags > > Thanks > - Ioi From ioi.lam at oracle.com Thu Apr 13 17:41:03 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Apr 2017 01:41:03 +0800 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: References: <58EF454C.9050307@oracle.com> Message-ID: <58EFB82F.8060906@oracle.com> Hi Calvin & Coleen, Thanks for the review. I will make the change as Coleen suggested and push. - Ioi On 4/13/17 8:15 PM, coleen.phillimore at oracle.com wrote: > > This looks good. Glad there was an access flag bit left. > > http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/src/share/vm/oops/klassVtable.cpp.udiff.html > > > I'd make it #ifdef ASSERT instead. I don't think anyone's built > optimized in a long time tho. > > Thanks, > Coleen > > On 4/13/17 5:30 AM, Ioi Lam wrote: >> Hi, please review this small start-up enhancement: >> >> https://bugs.openjdk.java.net/browse/JDK-8178543 >> http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ >> >> We have a benchmark that shows Klass::is_shared() is called very >> frequently during InstanceKlass::link_class_impl, and costs about 2% >> of the start-up time. >> >> The fix is simple -- instead of walking the list of CDS shared >> regions, use a new bit in Klass::_access_flags >> >> Thanks >> - Ioi > From jiangli.zhou at oracle.com Thu Apr 13 18:02:15 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Thu, 13 Apr 2017 11:02:15 -0700 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: <58EF454C.9050307@oracle.com> References: <58EF454C.9050307@oracle.com> Message-ID: <95054A6C-EC6D-4A52-8DEE-93EB66CACFD8@oracle.com> Hi Ioi, Could you please explain the change in klassVtable.cpp? Thanks, Jiangli > On Apr 13, 2017, at 2:30 AM, Ioi Lam wrote: > > Hi, please review this small start-up enhancement: > > https://bugs.openjdk.java.net/browse/JDK-8178543 > http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ > > We have a benchmark that shows Klass::is_shared() is called very frequently during InstanceKlass::link_class_impl, and costs about 2% of the start-up time. > > The fix is simple -- instead of walking the list of CDS shared regions, use a new bit in Klass::_access_flags > > Thanks > - Ioi From lois.foltan at oracle.com Thu Apr 13 20:32:00 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 13 Apr 2017 16:32:00 -0400 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58EF3D3A.6020903@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> Message-ID: Hi Ioi, Looks really good. A couple of comments: src/share/vm/classfile/classFileParser.cpp: * line #5676 - I'm not sure I completely understand the logic surrounding anonymous classes. Coleen and I discussed earlier today and I came away from that discussion with the idea that the only classes being patched currently are anonymous classes. So a bit confused as why the check on line #5676 and a check for a java/lang/Class on line #5684. Could the is_anonymous() if statement be combined into the loop? Also why not do this calculation in the rewriter or is that too late? * line #5677, 5692 - a nit but I think the convention is to not have a space after the variable name and between the post increment operator. src/share/vm/classfile/constantPool.hpp: I understand the concept behind _invalid_resolved_klass_index, but it really is not so much invalid as temporary for class redefinition purposes, as you explain in ConstantPool::allocate_resolved_klasses. Please consider renaming to _temp_unresolved_klass_index. And whether you choose to rename the field or not, please add a one line comment ahead of ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses this currently. Great change, thanks! Lois On 4/13/2017 4:56 AM, Ioi Lam wrote: > Hi Coleen, > > Thanks for the comments. Here's a delta from the last patch > > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ > > > In addition to your requests, I made these changes: > > [1] To consolidate the multiple extract_high/low code, I've added > CPKlassSlot, so the code is cleaner: > > CPKlassSlot kslot = this_cp->klass_slot_at(which); > int resolved_klass_index = kslot.resolved_klass_index(); > int name_index = kslot.name_index(); > > [2] Renamed ConstantPool::is_shared_quick() to > ConstantPool::is_shared(). The C++ compiler should be able to pick > this function over MetaspaceObj::is_shared(). > > [3] Massaged the CDS region size set-up code a little to pass internal > tests, because RO/RW ratio has changed. I didn't spend too much time > picking the "right" sizes, as this code will be obsoleted soon with > JDK-8072061 > > Thanks > - Ioi > > On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >> >> This looks really good! >> >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >> >> >> + // Add one extra element to tags for storing ConstantPool::flags(). >> + Array* tags = >> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); >> // tags->at(_length) is flags() >> >> >> I think this is left over, since _flags didn't get moved after all. >> >> + Klass** adr = >> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >> + // The interpreter assumes when the tag is stored, the klass is >> resolved >> + // and the Klass* is a klass rather than a Symbol*, so we need >> + // hardware store ordering here. >> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >> + return k; >> >> The comment still refers to the switch between Symbol* and Klass* in >> the constant pool. The entry in the Klass array should be NULL. >> >> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >> >> Can you put klass_name_index_at() in the constantPool.hpp header file >> (so it's inlined) and have all the places where you get name_index >> use this function? So we don't have to know in multiple places that >> extract_high_short_from_int() is where the name index is. And in >> constantPool.hpp, for unresolved_klass_at_put() add a comment about >> what the new format of the constant pool entry is {name_index, >> resolved_klass_index}. I'm happy to see this work nearing >> completion! The "constant" pool should be constant! thanks, Coleen >> On 4/11/17 2:26 AM, Ioi Lam wrote: >>> Hi,please review the following change >>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>> Klass from the i-th slot of ConstantPool. + When a klass is >>> resolved, the ConstantPool is modified to store the Klass pointer. >>> After: + ConstantPool::klass_at(i) finds the at >>> this->_resolved_klasses->at(i) + When a klass is resolved, >>> _resolved_klasses->at(i) is modified. In addition: + I moved >>> _resolved_references and _reference_map from ConstantPool to >>> ConstantPoolCache. + Now _flags is no longer modified for shared >>> ConstantPools. As a result, none of the fields in shared >>> ConstantPools are modified at run time, so we can move them into >>> the RO region in the CDS archive. *Testing:** * - Benchmark >>> results show no performance regression, despite the extra level >>> of indirection, which has a negligible overhead for the >>> interpreter. - RBT testing for tier2 and tier3. *Ports:** * - I >>> have tested only the Oracle-support ports. For the aarch64, ppc and >>> s390 ports, I have added some code without testing (it's probably >>> incomplete) - Port owners, please check if my patch work for you, >>> and I can incorporate your changes in my push. Alternatively, >>> you can wait for my push and provide fixes (if necessary) in a new >>> changeset, and I will be happy to sponsor it. Thanks - Ioi > From chenyt.cs.sjtu at gmail.com Thu Apr 13 22:02:42 2017 From: chenyt.cs.sjtu at gmail.com (Yuting Chen) Date: Thu, 13 Apr 2017 15:02:42 -0700 Subject: Fwd: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: References: Message-ID: Re-post my previous example. I recently found an interesting class. Hope that I can get answers here. The bytecode is shown as follows. Hotspot can catch some inconsistencies here (Reason: Current frame's flags are not assignable to stack map frame's), while J9 cannot. I thought J9 was wrong until I went into HotSpot's source code and saw some comments in share/vm/classfile/verifier.cpp. It says that the verifier "Return TRUE if all code paths starting with start_bc_offset end in bytecode athrow or loop" (a comment before bool ClassVerifier::ends_in_athrow(u4 start_bc_offset)). Clearly the paths in Vector1(int, int) end in athrow, and the verifier fails. Can anyone explain the reason why the class still cannot pass the verification, and which verifier needs to be fixed. **************** public class Vector1 extends java.util.AbstractList minor version: 0 major version: 52 flags: ACC_PUBLIC, ACC_SUPER Constant pool: #1 = Utf8 Vector1 #2 = Class #1 // Vector1 #3 = Utf8 java/util/AbstractList #4 = Class #3 // java/util/AbstractList #5 = Utf8 #6 = Utf8 (II)V #7 = Utf8 ()V #8 = NameAndType #5:#7 // "":()V #9 = Methodref #4.#8 // java/util/AbstractList."":()V #10 = Utf8 java/lang/Exception #11 = Class #10 // java/lang/Exception #12 = Methodref #11.#8 // java/lang/Exception."":()V #13 = Utf8 main #14 = Utf8 ([Ljava/lang/String;)V #15 = NameAndType #5:#6 // "":(II)V #16 = Methodref #2.#15 // Vector1."":(II)V #17 = Utf8 Code #18 = Utf8 StackMapTable { public Vector1(int, int); descriptor: (II)V flags: ACC_PUBLIC Code: stack=2, locals=3, args_size=3 0: iload_1 1: iflt 11 4: aload_0 5: invokespecial #9 // Method java/util/AbstractList."":()V 8: goto 11 11: new #11 // class java/lang/Exception 14: dup 15: invokespecial #12 // Method java/lang/Exception."":()V 18: athrow StackMapTable: number_of_entries = 1 frame_type = 255 /* full_frame */ offset_delta = 11 locals = [ top, int, int ] stack = [] public static void main(java.lang.String[]); descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC, ACC_STATIC Code: stack=3, locals=1, args_size=1 0: new #2 // class Vector1 3: bipush 10 5: iconst_0 6: invokespecial #16 // Method "":(II)V 9: return } HotSpot reports the next verifyerror, while J9 does not: Exception in thread "main" java.lang.VerifyError: Inconsistent stackmap frames at branch target 11 Exception Details: Location: Vector1.(II)V @11: new Reason: Current frame's flags are not assignable to stack map frame's. Current Frame: bci: @1 flags: { flagThisUninit } locals: { uninitializedThis, integer, integer } stack: { integer } Stackmap Frame: bci: @11 flags: { } locals: { top, integer, integer } stack: { } Bytecode: 0000000: 1b9b 000a 2ab7 0009 a700 03bb 000b 59b7 0000010: 000c bf Stackmap Table: full_frame(@11,{Top,Integer,Integer},{}) From coleen.phillimore at oracle.com Thu Apr 13 22:04:46 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Apr 2017 18:04:46 -0400 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58EF3D3A.6020903@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> Message-ID: Hi Ioi, The changes look good. Thanks for doing klass_slot_at() and I agree is_shared is better than is_shared_quick. thanks, Coleen On 4/13/17 4:56 AM, Ioi Lam wrote: > Hi Coleen, > > Thanks for the comments. Here's a delta from the last patch > > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ > > > In addition to your requests, I made these changes: > > [1] To consolidate the multiple extract_high/low code, I've added > CPKlassSlot, so the code is cleaner: > > CPKlassSlot kslot = this_cp->klass_slot_at(which); > int resolved_klass_index = kslot.resolved_klass_index(); > int name_index = kslot.name_index(); > > [2] Renamed ConstantPool::is_shared_quick() to > ConstantPool::is_shared(). The C++ compiler should be able to pick > this function over MetaspaceObj::is_shared(). > > [3] Massaged the CDS region size set-up code a little to pass internal > tests, because RO/RW ratio has changed. I didn't spend too much time > picking the "right" sizes, as this code will be obsoleted soon with > JDK-8072061 > > Thanks > - Ioi > > On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >> >> This looks really good! >> >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >> >> >> + // Add one extra element to tags for storing ConstantPool::flags(). >> + Array* tags = >> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); >> // tags->at(_length) is flags() >> >> >> I think this is left over, since _flags didn't get moved after all. >> >> + Klass** adr = >> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >> + // The interpreter assumes when the tag is stored, the klass is >> resolved >> + // and the Klass* is a klass rather than a Symbol*, so we need >> + // hardware store ordering here. >> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >> + return k; >> >> The comment still refers to the switch between Symbol* and Klass* in >> the constant pool. The entry in the Klass array should be NULL. >> >> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >> >> Can you put klass_name_index_at() in the constantPool.hpp header file >> (so it's inlined) and have all the places where you get name_index >> use this function? So we don't have to know in multiple places that >> extract_high_short_from_int() is where the name index is. And in >> constantPool.hpp, for unresolved_klass_at_put() add a comment about >> what the new format of the constant pool entry is {name_index, >> resolved_klass_index}. I'm happy to see this work nearing >> completion! The "constant" pool should be constant! thanks, Coleen >> On 4/11/17 2:26 AM, Ioi Lam wrote: >>> Hi,please review the following change >>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>> Klass from the i-th slot of ConstantPool. + When a klass is >>> resolved, the ConstantPool is modified to store the Klass pointer. >>> After: + ConstantPool::klass_at(i) finds the at >>> this->_resolved_klasses->at(i) + When a klass is resolved, >>> _resolved_klasses->at(i) is modified. In addition: + I moved >>> _resolved_references and _reference_map from ConstantPool to >>> ConstantPoolCache. + Now _flags is no longer modified for shared >>> ConstantPools. As a result, none of the fields in shared >>> ConstantPools are modified at run time, so we can move them into >>> the RO region in the CDS archive. *Testing:** * - Benchmark >>> results show no performance regression, despite the extra level >>> of indirection, which has a negligible overhead for the >>> interpreter. - RBT testing for tier2 and tier3. *Ports:** * - I >>> have tested only the Oracle-support ports. For the aarch64, ppc and >>> s390 ports, I have added some code without testing (it's probably >>> incomplete) - Port owners, please check if my patch work for you, >>> and I can incorporate your changes in my push. Alternatively, >>> you can wait for my push and provide fixes (if necessary) in a new >>> changeset, and I will be happy to sponsor it. Thanks - Ioi > From coleen.phillimore at oracle.com Thu Apr 13 22:16:29 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Apr 2017 18:16:29 -0400 Subject: RFR(S) JDK-8178350 - klassVtable and klassItable should be ValueObj In-Reply-To: <58EF4369.3050303@oracle.com> References: <58EF4369.3050303@oracle.com> Message-ID: <2b5f8fc5-7680-c0cc-a1c7-4582230abbf5@oracle.com> I like this change. It's always been a surprise to need a ResourceMark around klass->vtable() calls. On 4/13/17 5:22 AM, Ioi Lam wrote: > Hi, please review this JVM start-up enhancement: > > https://bugs.openjdk.java.net/browse/JDK-8178350 > http://cr.openjdk.java.net/~iklam/jdk10/8178350-klassvtable-as-valueobj.v01/ > > > Klass::vtable() is called frequently inside > InstanceKlass::link_class_impl(). We have a start-up benchmark that > loads about 4000 classes. Klass::vtable() is called 32458 times and > costs about 2~3% of a total elapsed time of 720 ms. > Nice when cleaning something up, you get better performance! > Because klassVtable and klassItable are very simple, fixed sized > structures, it's much better to allocate them on the stack: > > OLD: > > class klassVtable : ResourceObj {...} > klassVtable* Klass::vtable() const { > return new klassVtable(const_cast(this), start_of_vtable(), > vtable_length() / vtableEntry::size()); > } > > NEW: > > class klassVtable VALUE_OBJ_CLASS_SPEC {...} > klassVtable Klass::vtable() const { > return klassVtable(const_cast(this), start_of_vtable(), > vtable_length() / vtableEntry::size()); > } > > (>> David) I am not sure whether to make klassVtable a StackObj or > ValueObj. Since it's returned by a function call, it seems ValueObj is > better?? I don't have a strong opinion about this. ValueObj makes sense because StackObj would be coded as something like, instead of this: + Method * object_resolved_method = object_klass->vtable().method_at(index); Would be this: KlassVtable vt(object_klass); Method* object_resolved_method = vt.method_at(index); So because of how it's called, it makes more sense as a ValueObj (which we want to get rid of). Neither can call CHEAP operator new. > > I also removed some ResourceMarks that are obviously not necessary. I > left some others unchanged, like here in link_class_impl(): > > 623 ResourceMark rm(THREAD); > 624 vtable().initialize_vtable(true, CHECK_false); > 625 itable().initialize_itable(true, CHECK_false); > > because some code inside initialize_vtable() would do a resource > allocation without a ResourceMark. I don't want to touch that for now, > as it's unclear to me whether the allocated object(s) should be > scoped, or should be returned to a higher scope. I filed JDK-8178712 > to track this. > Seems ok to me. thanks, Coleen > Thanks > - Ioi From forax at univ-mlv.fr Thu Apr 13 22:39:37 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 14 Apr 2017 00:39:37 +0200 (CEST) Subject: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: References: Message-ID: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> Disclaimer, i'm not an hotspot guy. J9 seems wrong, to me. If you just take a look to the code, you have a path to go out of the constructor without having 'this' fully initialized, so whatever the VM impelemntations say, this code is not a valid one. Now, Hotspot refuses to merge a path where this is initialized with a path where this is not initialized because it will forget that there is a path where this is not initialized, it seems to be the right thing to do. cheers, R?mi ----- Mail original ----- > De: "Yuting Chen" > ?: hotspot-dev at openjdk.java.net > Envoy?: Vendredi 14 Avril 2017 00:02:42 > Objet: Fwd: HotSpot and IBM's J9 behave a little different when verifying this class > Re-post my previous example. > > I recently found an interesting class. Hope that I can get answers here. > > The bytecode is shown as follows. Hotspot can catch some > inconsistencies here (Reason: Current frame's flags are not assignable > to stack map frame's), while J9 cannot. > > I thought J9 was wrong until I went into HotSpot's source code and saw > some comments in share/vm/classfile/verifier.cpp. It says that the > verifier "Return TRUE if all code paths starting with start_bc_offset > end in bytecode athrow or loop" (a comment before bool > ClassVerifier::ends_in_athrow(u4 start_bc_offset)). Clearly the paths > in Vector1(int, int) end in athrow, and the verifier fails. > > Can anyone explain the reason why the class still cannot pass the > verification, and which verifier needs to be fixed. > > **************** > > public class Vector1 extends java.util.AbstractList > minor version: 0 > major version: 52 > flags: ACC_PUBLIC, ACC_SUPER > Constant pool: > #1 = Utf8 Vector1 > #2 = Class #1 // Vector1 > #3 = Utf8 java/util/AbstractList > #4 = Class #3 // java/util/AbstractList > #5 = Utf8 > #6 = Utf8 (II)V > #7 = Utf8 ()V > #8 = NameAndType #5:#7 // "":()V > #9 = Methodref #4.#8 // java/util/AbstractList."":()V > #10 = Utf8 java/lang/Exception > #11 = Class #10 // java/lang/Exception > #12 = Methodref #11.#8 // java/lang/Exception."":()V > #13 = Utf8 main > #14 = Utf8 ([Ljava/lang/String;)V > #15 = NameAndType #5:#6 // "":(II)V > #16 = Methodref #2.#15 // Vector1."":(II)V > #17 = Utf8 Code > #18 = Utf8 StackMapTable > { > public Vector1(int, int); > descriptor: (II)V > flags: ACC_PUBLIC > Code: > stack=2, locals=3, args_size=3 > 0: iload_1 > 1: iflt 11 > 4: aload_0 > 5: invokespecial #9 // Method > java/util/AbstractList."":()V > 8: goto 11 > 11: new #11 // class java/lang/Exception > 14: dup > 15: invokespecial #12 // Method > java/lang/Exception."":()V > 18: athrow > StackMapTable: number_of_entries = 1 > frame_type = 255 /* full_frame */ > offset_delta = 11 > locals = [ top, int, int ] > stack = [] > > public static void main(java.lang.String[]); > descriptor: ([Ljava/lang/String;)V > flags: ACC_PUBLIC, ACC_STATIC > Code: > stack=3, locals=1, args_size=1 > 0: new #2 // class Vector1 > 3: bipush 10 > 5: iconst_0 > 6: invokespecial #16 // Method "":(II)V > 9: return > } > > HotSpot reports the next verifyerror, while J9 does not: > Exception in thread "main" java.lang.VerifyError: Inconsistent > stackmap frames at branch target 11 > Exception Details: > Location: > Vector1.(II)V @11: new > Reason: > Current frame's flags are not assignable to stack map frame's. > Current Frame: > bci: @1 > flags: { flagThisUninit } > locals: { uninitializedThis, integer, integer } > stack: { integer } > Stackmap Frame: > bci: @11 > flags: { } > locals: { top, integer, integer } > stack: { } > Bytecode: > 0000000: 1b9b 000a 2ab7 0009 a700 03bb 000b 59b7 > 0000010: 000c bf > Stackmap Table: > full_frame(@11,{Top,Integer,Integer},{}) From michael.rasmussen at zeroturnaround.com Thu Apr 13 22:54:50 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Fri, 14 Apr 2017 01:54:50 +0300 Subject: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> References: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> Message-ID: Your problem is, that the stackmap frame in your else has top as local 0, instead of an uninitializedThis, meaning else-block doesn't have flagThisUninit in the flags, so it shouldn't verify. This was changed in 8u121, to conform with the Maintenance Releases of JSRs 336/337 done in 2015. Prior to that, flags was not compared when checking if stackmap frames were compatible, apparently J9 hasn't updated to follow this yet. Also see: http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-January/025781.html /Michael On 14 April 2017 at 01:39, Remi Forax wrote: > Disclaimer, i'm not an hotspot guy. > > J9 seems wrong, to me. > > If you just take a look to the code, > you have a path to go out of the constructor without having 'this' fully initialized, > so whatever the VM impelemntations say, this code is not a valid one. > > Now, Hotspot refuses to merge a path where this is initialized with a path where this is not initialized because it will forget that there is a path where this is not initialized, > it seems to be the right thing to do. > > cheers, > R?mi > > ----- Mail original ----- >> De: "Yuting Chen" >> ?: hotspot-dev at openjdk.java.net >> Envoy?: Vendredi 14 Avril 2017 00:02:42 >> Objet: Fwd: HotSpot and IBM's J9 behave a little different when verifying this class > >> Re-post my previous example. >> >> I recently found an interesting class. Hope that I can get answers here. >> >> The bytecode is shown as follows. Hotspot can catch some >> inconsistencies here (Reason: Current frame's flags are not assignable >> to stack map frame's), while J9 cannot. >> >> I thought J9 was wrong until I went into HotSpot's source code and saw >> some comments in share/vm/classfile/verifier.cpp. It says that the >> verifier "Return TRUE if all code paths starting with start_bc_offset >> end in bytecode athrow or loop" (a comment before bool >> ClassVerifier::ends_in_athrow(u4 start_bc_offset)). Clearly the paths >> in Vector1(int, int) end in athrow, and the verifier fails. >> >> Can anyone explain the reason why the class still cannot pass the >> verification, and which verifier needs to be fixed. >> >> **************** >> >> public class Vector1 extends java.util.AbstractList >> minor version: 0 >> major version: 52 >> flags: ACC_PUBLIC, ACC_SUPER >> Constant pool: >> #1 = Utf8 Vector1 >> #2 = Class #1 // Vector1 >> #3 = Utf8 java/util/AbstractList >> #4 = Class #3 // java/util/AbstractList >> #5 = Utf8 >> #6 = Utf8 (II)V >> #7 = Utf8 ()V >> #8 = NameAndType #5:#7 // "":()V >> #9 = Methodref #4.#8 // java/util/AbstractList."":()V >> #10 = Utf8 java/lang/Exception >> #11 = Class #10 // java/lang/Exception >> #12 = Methodref #11.#8 // java/lang/Exception."":()V >> #13 = Utf8 main >> #14 = Utf8 ([Ljava/lang/String;)V >> #15 = NameAndType #5:#6 // "":(II)V >> #16 = Methodref #2.#15 // Vector1."":(II)V >> #17 = Utf8 Code >> #18 = Utf8 StackMapTable >> { >> public Vector1(int, int); >> descriptor: (II)V >> flags: ACC_PUBLIC >> Code: >> stack=2, locals=3, args_size=3 >> 0: iload_1 >> 1: iflt 11 >> 4: aload_0 >> 5: invokespecial #9 // Method >> java/util/AbstractList."":()V >> 8: goto 11 >> 11: new #11 // class java/lang/Exception >> 14: dup >> 15: invokespecial #12 // Method >> java/lang/Exception."":()V >> 18: athrow >> StackMapTable: number_of_entries = 1 >> frame_type = 255 /* full_frame */ >> offset_delta = 11 >> locals = [ top, int, int ] >> stack = [] >> >> public static void main(java.lang.String[]); >> descriptor: ([Ljava/lang/String;)V >> flags: ACC_PUBLIC, ACC_STATIC >> Code: >> stack=3, locals=1, args_size=1 >> 0: new #2 // class Vector1 >> 3: bipush 10 >> 5: iconst_0 >> 6: invokespecial #16 // Method "":(II)V >> 9: return >> } >> >> HotSpot reports the next verifyerror, while J9 does not: >> Exception in thread "main" java.lang.VerifyError: Inconsistent >> stackmap frames at branch target 11 >> Exception Details: >> Location: >> Vector1.(II)V @11: new >> Reason: >> Current frame's flags are not assignable to stack map frame's. >> Current Frame: >> bci: @1 >> flags: { flagThisUninit } >> locals: { uninitializedThis, integer, integer } >> stack: { integer } >> Stackmap Frame: >> bci: @11 >> flags: { } >> locals: { top, integer, integer } >> stack: { } >> Bytecode: >> 0000000: 1b9b 000a 2ab7 0009 a700 03bb 000b 59b7 >> 0000010: 000c bf >> Stackmap Table: >> full_frame(@11,{Top,Integer,Integer},{}) From chenyt.cs.sjtu at gmail.com Thu Apr 13 23:15:05 2017 From: chenyt.cs.sjtu at gmail.com (Yuting Chen) Date: Thu, 13 Apr 2017 16:15:05 -0700 Subject: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: References: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> Message-ID: Thanks, Remi and Michael. The difference appears only when all of the paths end in athrow. Then what does the comment ("Return TRUE if all code paths starting with start_bc_offset end in bytecode athrow or loop") in verifier.cpp mean? It seems that J9 follows this comment. Both verifiers can decline the class if "athrow" is changed to "return". On Thu, Apr 13, 2017 at 3:54 PM, Michael Rasmussen wrote: > Your problem is, that the stackmap frame in your else has top as local > 0, instead of an uninitializedThis, meaning else-block doesn't have > flagThisUninit in the flags, so it shouldn't verify. > > This was changed in 8u121, to conform with the Maintenance Releases of > JSRs 336/337 done in 2015. > Prior to that, flags was not compared when checking if stackmap frames > were compatible, apparently J9 hasn't updated to follow this yet. > > Also see: http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-January/025781.html > > /Michael > > On 14 April 2017 at 01:39, Remi Forax wrote: >> Disclaimer, i'm not an hotspot guy. >> >> J9 seems wrong, to me. >> >> If you just take a look to the code, >> you have a path to go out of the constructor without having 'this' fully initialized, >> so whatever the VM impelemntations say, this code is not a valid one. >> >> Now, Hotspot refuses to merge a path where this is initialized with a path where this is not initialized because it will forget that there is a path where this is not initialized, >> it seems to be the right thing to do. >> >> cheers, >> R?mi >> >> ----- Mail original ----- >>> De: "Yuting Chen" >>> ?: hotspot-dev at openjdk.java.net >>> Envoy?: Vendredi 14 Avril 2017 00:02:42 >>> Objet: Fwd: HotSpot and IBM's J9 behave a little different when verifying this class >> >>> Re-post my previous example. >>> >>> I recently found an interesting class. Hope that I can get answers here. >>> >>> The bytecode is shown as follows. Hotspot can catch some >>> inconsistencies here (Reason: Current frame's flags are not assignable >>> to stack map frame's), while J9 cannot. >>> >>> I thought J9 was wrong until I went into HotSpot's source code and saw >>> some comments in share/vm/classfile/verifier.cpp. It says that the >>> verifier "Return TRUE if all code paths starting with start_bc_offset >>> end in bytecode athrow or loop" (a comment before bool >>> ClassVerifier::ends_in_athrow(u4 start_bc_offset)). Clearly the paths >>> in Vector1(int, int) end in athrow, and the verifier fails. >>> >>> Can anyone explain the reason why the class still cannot pass the >>> verification, and which verifier needs to be fixed. >>> >>> **************** >>> >>> public class Vector1 extends java.util.AbstractList >>> minor version: 0 >>> major version: 52 >>> flags: ACC_PUBLIC, ACC_SUPER >>> Constant pool: >>> #1 = Utf8 Vector1 >>> #2 = Class #1 // Vector1 >>> #3 = Utf8 java/util/AbstractList >>> #4 = Class #3 // java/util/AbstractList >>> #5 = Utf8 >>> #6 = Utf8 (II)V >>> #7 = Utf8 ()V >>> #8 = NameAndType #5:#7 // "":()V >>> #9 = Methodref #4.#8 // java/util/AbstractList."":()V >>> #10 = Utf8 java/lang/Exception >>> #11 = Class #10 // java/lang/Exception >>> #12 = Methodref #11.#8 // java/lang/Exception."":()V >>> #13 = Utf8 main >>> #14 = Utf8 ([Ljava/lang/String;)V >>> #15 = NameAndType #5:#6 // "":(II)V >>> #16 = Methodref #2.#15 // Vector1."":(II)V >>> #17 = Utf8 Code >>> #18 = Utf8 StackMapTable >>> { >>> public Vector1(int, int); >>> descriptor: (II)V >>> flags: ACC_PUBLIC >>> Code: >>> stack=2, locals=3, args_size=3 >>> 0: iload_1 >>> 1: iflt 11 >>> 4: aload_0 >>> 5: invokespecial #9 // Method >>> java/util/AbstractList."":()V >>> 8: goto 11 >>> 11: new #11 // class java/lang/Exception >>> 14: dup >>> 15: invokespecial #12 // Method >>> java/lang/Exception."":()V >>> 18: athrow >>> StackMapTable: number_of_entries = 1 >>> frame_type = 255 /* full_frame */ >>> offset_delta = 11 >>> locals = [ top, int, int ] >>> stack = [] >>> >>> public static void main(java.lang.String[]); >>> descriptor: ([Ljava/lang/String;)V >>> flags: ACC_PUBLIC, ACC_STATIC >>> Code: >>> stack=3, locals=1, args_size=1 >>> 0: new #2 // class Vector1 >>> 3: bipush 10 >>> 5: iconst_0 >>> 6: invokespecial #16 // Method "":(II)V >>> 9: return >>> } >>> >>> HotSpot reports the next verifyerror, while J9 does not: >>> Exception in thread "main" java.lang.VerifyError: Inconsistent >>> stackmap frames at branch target 11 >>> Exception Details: >>> Location: >>> Vector1.(II)V @11: new >>> Reason: >>> Current frame's flags are not assignable to stack map frame's. >>> Current Frame: >>> bci: @1 >>> flags: { flagThisUninit } >>> locals: { uninitializedThis, integer, integer } >>> stack: { integer } >>> Stackmap Frame: >>> bci: @11 >>> flags: { } >>> locals: { top, integer, integer } >>> stack: { } >>> Bytecode: >>> 0000000: 1b9b 000a 2ab7 0009 a700 03bb 000b 59b7 >>> 0000010: 000c bf >>> Stackmap Table: >>> full_frame(@11,{Top,Integer,Integer},{}) From michael.rasmussen at zeroturnaround.com Thu Apr 13 23:21:58 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Fri, 14 Apr 2017 02:21:58 +0300 Subject: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: References: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> Message-ID: On 14 April 2017 at 02:15, Yuting Chen wrote: > Thanks, Remi and Michael. > > The difference appears only when all of the paths end in athrow. Then > what does the comment ("Return TRUE if all code paths starting with > start_bc_offset end in bytecode athrow or loop") in verifier.cpp mean? > It seems that J9 follows this comment. > > Both verifiers can decline the class if "athrow" is changed to "return". Yes, for constructors, if exit is via return, then there cannot be any uninitializedThis, but allowed if exit is via athrow. But, your problem is, that your two frame aren't assignable to each other. >>> Current Frame: >>> bci: @1 >>> flags: { flagThisUninit } >>> locals: { uninitializedThis, integer, integer } >>> stack: { integer } >>> Stackmap Frame: >>> bci: @11 >>> flags: { } >>> locals: { top, integer, integer } >>> stack: { } While the locals are compatible, the flags are not, you cannot go from { flagThisUninit } to {} http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/share/vm/classfile/stackMapFrame.cpp#l190 /Michael From chenyt.cs.sjtu at gmail.com Thu Apr 13 23:39:22 2017 From: chenyt.cs.sjtu at gmail.com (Yuting Chen) Date: Thu, 13 Apr 2017 16:39:22 -0700 Subject: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: References: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> Message-ID: It seems that J9 forgot to check the flags when athrow follows. On Thu, Apr 13, 2017 at 4:21 PM, Michael Rasmussen wrote: > On 14 April 2017 at 02:15, Yuting Chen wrote: >> Thanks, Remi and Michael. >> >> The difference appears only when all of the paths end in athrow. Then >> what does the comment ("Return TRUE if all code paths starting with >> start_bc_offset end in bytecode athrow or loop") in verifier.cpp mean? >> It seems that J9 follows this comment. >> >> Both verifiers can decline the class if "athrow" is changed to "return". > > Yes, for constructors, if exit is via return, then there cannot be any > uninitializedThis, but allowed if exit is via athrow. > > But, your problem is, that your two frame aren't assignable to each other. >>>> Current Frame: >>>> bci: @1 >>>> flags: { flagThisUninit } >>>> locals: { uninitializedThis, integer, integer } >>>> stack: { integer } >>>> Stackmap Frame: >>>> bci: @11 >>>> flags: { } >>>> locals: { top, integer, integer } >>>> stack: { } > > While the locals are compatible, the flags are not, you cannot go from > { flagThisUninit } to {} > http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/share/vm/classfile/stackMapFrame.cpp#l190 > > /Michael From ioi.lam at oracle.com Fri Apr 14 04:32:16 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Apr 2017 12:32:16 +0800 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: <95054A6C-EC6D-4A52-8DEE-93EB66CACFD8@oracle.com> References: <58EF454C.9050307@oracle.com> <95054A6C-EC6D-4A52-8DEE-93EB66CACFD8@oracle.com> Message-ID: <58F050D0.8060308@oracle.com> Hi Jiangli, Code like this in klassVtable.cpp is necessary: if (is_preinitialized_vtable()) { /// calls _klass->is_shared() // At runtime initialize_vtable is rerun for a shared class // (loaded by the non-boot loader) as part of link_class_impl(). // The dumptime vtable index should be the same as the runtime index. assert(def_vtable_indices->at(i) == initialized, "dump time vtable index is different from runtime index"); } else { def_vtable_indices->at_put(i, initialized); //set vtable index } because the def_vtable_indices for a shared class is in RO space and cannot written into at run time. For this code (before) 1017 if (MetaspaceShared::is_in_shared_space((void*)&_method) && 1018 !MetaspaceShared::remapped_readwrite()) { 1019 // At runtime initialize_itable is rerun as part of link_class_impl() 1020 // for a shared class loaded by the non-boot loader. 1021 // The dumptime itable method entry should be the same as the runtime entry. 1022 assert(_method == m, "sanity"); 1023 } else { 1024 _method = m; 1025 } 1026 } the address (&_method) is either in a vtable or an itable, both of which are in the RO space. Therefore it's OK to write into it. The cost of checking MetaspaceShared::is_in_shared_space((void*)&_method) is high, as at this point we don't readily have a pointer to the corresponding Klass whose i/v table encompasses (&_method). Therefore, I made the assignment unconditional. I kept the assert so we don't lose any functionality. I think the cost of writing into _method is low (compared to the cost of MetaspaceShared::is_in_shared_space) , because it's within an InstanceKlass, which will be written into anyway during InstanceKlass::restore_unshareable_info. AFTER: 1017 #ifdef ASSERT 1018 if (MetaspaceShared::is_in_shared_space((void*)&_method) && 1019 !MetaspaceShared::remapped_readwrite()) { 1020 // At runtime initialize_itable is rerun as part of link_class_impl() 1021 // for a shared class loaded by the non-boot loader. 1022 // The dumptime itable method entry should be the same as the runtime entry. 1023 assert(_method == m, "sanity"); 1024 } 1025 #endif 1026 _method = m; 1027 } Thanks - Ioi On 4/14/17 2:02 AM, Jiangli Zhou wrote: > Hi Ioi, > > Could you please explain the change in klassVtable.cpp? > > Thanks, > Jiangli > >> On Apr 13, 2017, at 2:30 AM, Ioi Lam wrote: >> >> Hi, please review this small start-up enhancement: >> >> https://bugs.openjdk.java.net/browse/JDK-8178543 >> http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ >> >> We have a benchmark that shows Klass::is_shared() is called very frequently during InstanceKlass::link_class_impl, and costs about 2% of the start-up time. >> >> The fix is simple -- instead of walking the list of CDS shared regions, use a new bit in Klass::_access_flags >> >> Thanks >> - Ioi From ioi.lam at oracle.com Fri Apr 14 05:31:33 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Apr 2017 13:31:33 +0800 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> Message-ID: <58F05EB5.10009@oracle.com> HI Lois, Thanks for the review. Please see my comments in-line. On 4/14/17 4:32 AM, Lois Foltan wrote: > Hi Ioi, > > Looks really good. A couple of comments: > > src/share/vm/classfile/classFileParser.cpp: > * line #5676 - I'm not sure I completely understand the logic > surrounding anonymous classes. Coleen and I discussed earlier today > and I came away from that discussion with the idea that the only > classes being patched currently are anonymous classes. Line 5676 ... 5676 if (is_anonymous()) { 5677 _max_num_patched_klasses ++; // for patching the class index 5678 } corresponds to 5361 ik->set_name(_class_name); 5362 5363 if (is_anonymous()) { 5364 // I am well known to myself 5365 patch_class(ik->constants(), _this_class_index, ik, ik->name()); // eagerly resolve 5366 } Even though the class is "anonymous", it actually has a name. ik->name() probably is part of the constant pool, but I am not 100% sure. Also, I would need to search the constant pool to find the index for ik->name(). So I just got lazy here and use the same logic in ConstantPool::patch_class() to append ik->name() to the end of the constant pool. "Anonymous" actually means "the class cannot be looked up by name in the SystemDictionary". I think we need a better terminology :-) > So a bit confused as why the check on line #5676 and a check for a > java/lang/Class on line #5684. 5683 Handle patch = cp_patch_at(i); 5684 if (java_lang_String::is_instance(patch()) || java_lang_Class::is_instance(patch())) { 5685 // We need to append the names of the patched classes to the end of the constant pool, 5686 // because a patched class may have a Utf8 name that's not already included in the 5687 // original constant pool. 5688 // 5689 // Note that a String in cp_patch_at(i) may be used to patch a Utf8, a String, or a Class. 5690 // At this point, we don't know the tag for index i yet, because we haven't parsed the 5691 // constant pool. So we can only assume the worst -- every String is used to patch a Class. 5692 _max_num_patched_klasses ++; Line 5684 checks for all objects in the cp_patch array. Later, when ClassFileParser::patch_constant_pool() is called, any objects that are either Class or String could be treated as a Klass: 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, 725 int index, 726 Handle patch, 727 TRAPS) { ... 732 switch (cp->tag_at(index).value()) { 733 734 case JVM_CONSTANT_UnresolvedClass: { 735 // Patching a class means pre-resolving it. 736 // The name in the constant pool is ignored. 737 if (java_lang_Class::is_instance(patch())) { 738 guarantee_property(!java_lang_Class::is_primitive(patch()), 739 "Illegal class patch at %d in class file %s", 740 index, CHECK); 741 Klass* k = java_lang_Class::as_Klass(patch()); 742 patch_class(cp, index, k, k->name()); 743 } else { 744 guarantee_property(java_lang_String::is_instance(patch()), 745 "Illegal class patch at %d in class file %s", 746 index, CHECK); 747 Symbol* const name = java_lang_String::as_symbol(patch(), CHECK); 748 patch_class(cp, index, NULL, name); 749 } 750 break; 751 } > Could the is_anonymous() if statement be combined into the loop? I think the answer is no. At line 5365, there is no guarantee that ik->name() is in the cp_patch array. 5365 patch_class(ik->constants(), _this_class_index, ik, ik->name()); // eagerly resolve > Also why not do this calculation in the rewriter or is that too late? > Line 5676 and 5684 need to be executed BEFORE the constant pool and the associated tags array is allocated (both of which are fixed size, and cannot be expanded), which is way before the rewriter is run. At this point, we don't know what cp->tag_at(index) is (line #732), so the code needs to make a worst-case estimate on how long the CP/tags should be. > * line #5677, 5692 - a nit but I think the convention is to not have a > space after the variable name and between the post increment operator. > Fixed. > src/share/vm/classfile/constantPool.hpp: > I understand the concept behind _invalid_resolved_klass_index, but it > really is not so much invalid as temporary for class redefinition > purposes, as you explain in ConstantPool::allocate_resolved_klasses. > Please consider renaming to _temp_unresolved_klass_index. And whether > you choose to rename the field or not, please add a one line comment > ahead of ConstantPool::temp_unresolved_klass_at_put that only class > redefinition uses this currently. > Good idea. Will do. Thanks - Ioi > Great change, thanks! > Lois > > On 4/13/2017 4:56 AM, Ioi Lam wrote: >> Hi Coleen, >> >> Thanks for the comments. Here's a delta from the last patch >> >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >> >> >> In addition to your requests, I made these changes: >> >> [1] To consolidate the multiple extract_high/low code, I've added >> CPKlassSlot, so the code is cleaner: >> >> CPKlassSlot kslot = this_cp->klass_slot_at(which); >> int resolved_klass_index = kslot.resolved_klass_index(); >> int name_index = kslot.name_index(); >> >> [2] Renamed ConstantPool::is_shared_quick() to >> ConstantPool::is_shared(). The C++ compiler should be able to pick >> this function over MetaspaceObj::is_shared(). >> >> [3] Massaged the CDS region size set-up code a little to pass >> internal tests, because RO/RW ratio has changed. I didn't spend too >> much time picking the "right" sizes, as this code will be obsoleted >> soon with JDK-8072061 >> >> Thanks >> - Ioi >> >> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>> >>> This looks really good! >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>> >>> >>> + // Add one extra element to tags for storing ConstantPool::flags(). >>> + Array* tags = >>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); >>> // tags->at(_length) is flags() >>> >>> >>> I think this is left over, since _flags didn't get moved after all. >>> >>> + Klass** adr = >>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>> + // The interpreter assumes when the tag is stored, the klass is >>> resolved >>> + // and the Klass* is a klass rather than a Symbol*, so we need >>> + // hardware store ordering here. >>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>> + return k; >>> >>> The comment still refers to the switch between Symbol* and Klass* in >>> the constant pool. The entry in the Klass array should be NULL. >>> >>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>> >>> Can you put klass_name_index_at() in the constantPool.hpp header >>> file (so it's inlined) and have all the places where you get >>> name_index use this function? So we don't have to know in multiple >>> places that extract_high_short_from_int() is where the name index >>> is. And in constantPool.hpp, for unresolved_klass_at_put() add a >>> comment about what the new format of the constant pool entry is >>> {name_index, resolved_klass_index}. I'm happy to see this work >>> nearing completion! The "constant" pool should be constant! thanks, >>> Coleen >>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>> Hi,please review the following change >>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>>> Klass from the i-th slot of ConstantPool. + When a klass is >>>> resolved, the ConstantPool is modified to store the Klass >>>> pointer. After: + ConstantPool::klass_at(i) finds the at >>>> this->_resolved_klasses->at(i) + When a klass is resolved, >>>> _resolved_klasses->at(i) is modified. In addition: + I moved >>>> _resolved_references and _reference_map from ConstantPool to >>>> ConstantPoolCache. + Now _flags is no longer modified for >>>> shared ConstantPools. As a result, none of the fields in shared >>>> ConstantPools are modified at run time, so we can move them into >>>> the RO region in the CDS archive. *Testing:** * - Benchmark >>>> results show no performance regression, despite the extra level >>>> of indirection, which has a negligible overhead for the >>>> interpreter. - RBT testing for tier2 and tier3. *Ports:** * - I >>>> have tested only the Oracle-support ports. For the aarch64, ppc and >>>> s390 ports, I have added some code without testing (it's probably >>>> incomplete) - Port owners, please check if my patch work for you, >>>> and I can incorporate your changes in my push. Alternatively, >>>> you can wait for my push and provide fixes (if necessary) in a new >>>> changeset, and I will be happy to sponsor it. Thanks - Ioi >> > From yang.zhang at linaro.org Fri Apr 14 09:21:30 2017 From: yang.zhang at linaro.org (Yang Zhang) Date: Fri, 14 Apr 2017 17:21:30 +0800 Subject: SuperWordLoopUnrollAnalysis and loop unrolling In-Reply-To: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> References: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> Message-ID: Hi Andrew Now I also investigate the loop unroll mechanism in aarch64/arm64 C2 too. Take the vectAddInt as an example, my debug information is listed as follow: Java example: public static void vectAddInt( int[] a, int[] b, int c) { for (int i = 0; i < LENGTH; i++) { a[i] = b[i] + c; } } For aarch64 C2 with SuperWordLoopUnrollAnalysis=false: loop unroll is controlled by comparing body_size and LoopUnrollLimit ("Unroll loop bodies with node count less than this") in function policy_unroll. When loop unroll is 8 times, body_size is big enough and it stops. For aarch64 C2 with SuperWordLoopUnrollAnalysis=true: First loop unroll is controlled by policy_unroll_slp_analysis. When loop unroll is 4 times, vectorization happens. Then loop unroll is controlled by comparing body_size and LoopUnrollLimit. When loop unroll is 32 times, body_size is big enough and it stops. My test result is just opposite with your description. Could you provide your test case? Regards Yang On 13 April 2017 at 01:22, Andrew Haley wrote: > If I set SuperWordLoopUnrollAnalysis=true, then AArch64 C2 stops after > unrolling a simple loop 4 times. If I set > SuperWordLoopUnrollAnalysis=false, it stops after unrolling 16 times. > Why is is that SuperWordLoopUnrollAnalysis limits unrolling in this > way? > > The info says > "Map number of unrolls for main loop via Superword Level Parallelism > analysis" but that doesn't help me very much. Only AArch64 and x86 > set this option. > > Thanks, > > Andrew. From ioi.lam at oracle.com Fri Apr 14 15:16:18 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Apr 2017 23:16:18 +0800 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: <58F050D0.8060308@oracle.com> References: <58EF454C.9050307@oracle.com> <95054A6C-EC6D-4A52-8DEE-93EB66CACFD8@oracle.com> <58F050D0.8060308@oracle.com> Message-ID: <58F0E7C2.2030809@oracle.com> On 4/14/17 12:32 PM, Ioi Lam wrote: > Hi Jiangli, > > Code like this in klassVtable.cpp is necessary: > > if (is_preinitialized_vtable()) { /// calls > _klass->is_shared() > // At runtime initialize_vtable is rerun for a shared class > // (loaded by the non-boot loader) as part of > link_class_impl(). > // The dumptime vtable index should be the same as the > runtime index. > assert(def_vtable_indices->at(i) == initialized, > "dump time vtable index is different from runtime > index"); > } else { > def_vtable_indices->at_put(i, initialized); //set vtable > index > } > > because the def_vtable_indices for a shared class is in RO space and > cannot written into at run time. > > For this code (before) > > 1017 if (MetaspaceShared::is_in_shared_space((void*)&_method) && > 1018 !MetaspaceShared::remapped_readwrite()) { > 1019 // At runtime initialize_itable is rerun as part of > link_class_impl() > 1020 // for a shared class loaded by the non-boot loader. > 1021 // The dumptime itable method entry should be the same as the > runtime entry. > 1022 assert(_method == m, "sanity"); > 1023 } else { > 1024 _method = m; > 1025 } > 1026 } > > the address (&_method) is either in a vtable or an itable, both of > which are in the RO space. Therefore it's OK to write into it. > Sorry, I meant to say, "both are in the RW space", so it's OK to write. Thanks - Ioi > The cost of checking > MetaspaceShared::is_in_shared_space((void*)&_method) is high, as at > this point we don't readily have a pointer to the corresponding Klass > whose i/v table encompasses (&_method). > > Therefore, I made the assignment unconditional. I kept the assert so > we don't lose any functionality. > > I think the cost of writing into _method is low (compared to the cost > of MetaspaceShared::is_in_shared_space) , because it's within an > InstanceKlass, which will be written into anyway during > InstanceKlass::restore_unshareable_info. > > AFTER: > > 1017 #ifdef ASSERT > 1018 if (MetaspaceShared::is_in_shared_space((void*)&_method) && > 1019 !MetaspaceShared::remapped_readwrite()) { > 1020 // At runtime initialize_itable is rerun as part of > link_class_impl() > 1021 // for a shared class loaded by the non-boot loader. > 1022 // The dumptime itable method entry should be the same as the > runtime entry. > 1023 assert(_method == m, "sanity"); > 1024 } > 1025 #endif > 1026 _method = m; > 1027 } > > Thanks > - Ioi > > On 4/14/17 2:02 AM, Jiangli Zhou wrote: >> Hi Ioi, >> >> Could you please explain the change in klassVtable.cpp? >> >> Thanks, >> Jiangli >> >>> On Apr 13, 2017, at 2:30 AM, Ioi Lam wrote: >>> >>> Hi, please review this small start-up enhancement: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8178543 >>> http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ >>> >>> We have a benchmark that shows Klass::is_shared() is called very >>> frequently during InstanceKlass::link_class_impl, and costs about 2% >>> of the start-up time. >>> >>> The fix is simple -- instead of walking the list of CDS shared >>> regions, use a new bit in Klass::_access_flags >>> >>> Thanks >>> - Ioi > From ioi.lam at oracle.com Fri Apr 14 15:30:22 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Apr 2017 23:30:22 +0800 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58F05EB5.10009@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> Message-ID: <58F0EB0E.60904@oracle.com> On 4/14/17 1:31 PM, Ioi Lam wrote: > HI Lois, > > Thanks for the review. Please see my comments in-line. > > On 4/14/17 4:32 AM, Lois Foltan wrote: >> Hi Ioi, >> >> Looks really good. A couple of comments: >> >> src/share/vm/classfile/classFileParser.cpp: >> * line #5676 - I'm not sure I completely understand the logic >> surrounding anonymous classes. Coleen and I discussed earlier today >> and I came away from that discussion with the idea that the only >> classes being patched currently are anonymous classes. > Line 5676 ... > > 5676 if (is_anonymous()) { > 5677 _max_num_patched_klasses ++; // for patching the class > index > 5678 } > > corresponds to > > 5361 ik->set_name(_class_name); > 5362 > 5363 if (is_anonymous()) { > 5364 // I am well known to myself > 5365 patch_class(ik->constants(), _this_class_index, ik, > ik->name()); // eagerly resolve > 5366 } > > Even though the class is "anonymous", it actually has a name. > ik->name() probably is part of the constant pool, but I am not 100% > sure. Also, I would need to search the constant pool to find the index > for ik->name(). So I just got lazy here and use the same logic in > ConstantPool::patch_class() to append ik->name() to the end of the > constant pool. > > "Anonymous" actually means "the class cannot be looked up by name in > the SystemDictionary". I think we need a better terminology :-) > I finally realized why we need the "eagerly resolve" on line 5365. I'll modify the comments to the following: // _this_class_index is a CONSTANT_Class entry that refers to this // anonymous class itself. If this class needs to refer to its own methods or // fields, it would use a CONSTANT_MethodRef, etc, which would reference // _this_class_index. However, because this class is anonymous (it's // not stored in SystemDictionary), _this_class_index cannot be resolved // with ConstantPool::klass_at_impl, which does a SystemDictionary lookup. // Therefore, we must eagerly resolve _this_class_index now. So, Lois is right. Line 5676 is not necessary. I will revise the code to do the "eager resolution" without using ClassFileParser::patch_class. I'll post the updated code later. Thanks - Ioi >> So a bit confused as why the check on line #5676 and a check for a >> java/lang/Class on line #5684. > 5683 Handle patch = cp_patch_at(i); > 5684 if (java_lang_String::is_instance(patch()) || > java_lang_Class::is_instance(patch())) { > 5685 // We need to append the names of the patched classes > to the end of the constant pool, > 5686 // because a patched class may have a Utf8 name that's > not already included in the > 5687 // original constant pool. > 5688 // > 5689 // Note that a String in cp_patch_at(i) may be used to > patch a Utf8, a String, or a Class. > 5690 // At this point, we don't know the tag for index i > yet, because we haven't parsed the > 5691 // constant pool. So we can only assume the worst -- > every String is used to patch a Class. > 5692 _max_num_patched_klasses ++; > > Line 5684 checks for all objects in the cp_patch array. Later, when > ClassFileParser::patch_constant_pool() is called, any objects that are > either Class or String could be treated as a Klass: > > 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, > 725 int index, > 726 Handle patch, > 727 TRAPS) { > ... > 732 switch (cp->tag_at(index).value()) { > 733 > 734 case JVM_CONSTANT_UnresolvedClass: { > 735 // Patching a class means pre-resolving it. > 736 // The name in the constant pool is ignored. > 737 if (java_lang_Class::is_instance(patch())) { > 738 guarantee_property(!java_lang_Class::is_primitive(patch()), > 739 "Illegal class patch at %d in class > file %s", > 740 index, CHECK); > 741 Klass* k = java_lang_Class::as_Klass(patch()); > 742 patch_class(cp, index, k, k->name()); > 743 } else { > 744 guarantee_property(java_lang_String::is_instance(patch()), > 745 "Illegal class patch at %d in class > file %s", > 746 index, CHECK); > 747 Symbol* const name = java_lang_String::as_symbol(patch(), > CHECK); > 748 patch_class(cp, index, NULL, name); > 749 } > 750 break; > 751 } > >> Could the is_anonymous() if statement be combined into the loop? > > I think the answer is no. At line 5365, there is no guarantee that > ik->name() is in the cp_patch array. > > 5365 patch_class(ik->constants(), _this_class_index, ik, > ik->name()); // eagerly resolve > >> Also why not do this calculation in the rewriter or is that too late? >> > Line 5676 and 5684 need to be executed BEFORE the constant pool and > the associated tags array is allocated (both of which are fixed size, > and cannot be expanded), which is way before the rewriter is run. At > this point, we don't know what cp->tag_at(index) is (line #732), so > the code needs to make a worst-case estimate on how long the CP/tags > should be. > >> * line #5677, 5692 - a nit but I think the convention is to not have >> a space after the variable name and between the post increment operator. >> > Fixed. >> src/share/vm/classfile/constantPool.hpp: >> I understand the concept behind _invalid_resolved_klass_index, but it >> really is not so much invalid as temporary for class redefinition >> purposes, as you explain in ConstantPool::allocate_resolved_klasses. >> Please consider renaming to _temp_unresolved_klass_index. And >> whether you choose to rename the field or not, please add a one line >> comment ahead of ConstantPool::temp_unresolved_klass_at_put that only >> class redefinition uses this currently. >> > Good idea. Will do. > > Thanks > - Ioi > >> Great change, thanks! >> Lois >> >> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>> Hi Coleen, >>> >>> Thanks for the comments. Here's a delta from the last patch >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>> >>> >>> In addition to your requests, I made these changes: >>> >>> [1] To consolidate the multiple extract_high/low code, I've added >>> CPKlassSlot, so the code is cleaner: >>> >>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>> int resolved_klass_index = kslot.resolved_klass_index(); >>> int name_index = kslot.name_index(); >>> >>> [2] Renamed ConstantPool::is_shared_quick() to >>> ConstantPool::is_shared(). The C++ compiler should be able to pick >>> this function over MetaspaceObj::is_shared(). >>> >>> [3] Massaged the CDS region size set-up code a little to pass >>> internal tests, because RO/RW ratio has changed. I didn't spend too >>> much time picking the "right" sizes, as this code will be obsoleted >>> soon with JDK-8072061 >>> >>> Thanks >>> - Ioi >>> >>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> This looks really good! >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>> >>>> >>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>> + Array* tags = >>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, >>>> "invariant"); // tags->at(_length) is flags() >>>> >>>> >>>> I think this is left over, since _flags didn't get moved after all. >>>> >>>> + Klass** adr = >>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>> + // The interpreter assumes when the tag is stored, the klass is >>>> resolved >>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>> + // hardware store ordering here. >>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>> + return k; >>>> >>>> The comment still refers to the switch between Symbol* and Klass* >>>> in the constant pool. The entry in the Klass array should be NULL. >>>> >>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>> >>>> Can you put klass_name_index_at() in the constantPool.hpp header >>>> file (so it's inlined) and have all the places where you get >>>> name_index use this function? So we don't have to know in multiple >>>> places that extract_high_short_from_int() is where the name index >>>> is. And in constantPool.hpp, for unresolved_klass_at_put() add a >>>> comment about what the new format of the constant pool entry is >>>> {name_index, resolved_klass_index}. I'm happy to see this work >>>> nearing completion! The "constant" pool should be constant! >>>> thanks, Coleen >>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>> Hi,please review the following change >>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>>>> Klass from the i-th slot of ConstantPool. + When a klass is >>>>> resolved, the ConstantPool is modified to store the Klass >>>>> pointer. After: + ConstantPool::klass_at(i) finds the at >>>>> this->_resolved_klasses->at(i) + When a klass is resolved, >>>>> _resolved_klasses->at(i) is modified. In addition: + I moved >>>>> _resolved_references and _reference_map from ConstantPool to >>>>> ConstantPoolCache. + Now _flags is no longer modified for >>>>> shared ConstantPools. As a result, none of the fields in shared >>>>> ConstantPools are modified at run time, so we can move them into >>>>> the RO region in the CDS archive. *Testing:** * - Benchmark >>>>> results show no performance regression, despite the extra level >>>>> of indirection, which has a negligible overhead for the >>>>> interpreter. - RBT testing for tier2 and tier3. *Ports:** * - >>>>> I have tested only the Oracle-support ports. For the aarch64, ppc >>>>> and s390 ports, I have added some code without testing (it's >>>>> probably incomplete) - Port owners, please check if my patch >>>>> work for you, and I can incorporate your changes in my push. >>>>> Alternatively, you can wait for my push and provide fixes (if >>>>> necessary) in a new changeset, and I will be happy to sponsor it. >>>>> Thanks - Ioi >>> >> > From aph at redhat.com Fri Apr 14 16:41:13 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 14 Apr 2017 17:41:13 +0100 Subject: SuperWordLoopUnrollAnalysis and loop unrolling In-Reply-To: References: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> Message-ID: <3d2b99e7-a102-a073-23b6-574998027729@redhat.com> On 14/04/17 10:21, Yang Zhang wrote: > My test result is just opposite with your description. Could you > provide your test case? // @Benchmark public int[] sameArrayClass(BenchmarkState state) { for (int i = 0; i < INITSIZE; i++) { state.b[0] = state.b[1]; state.b[1] = state.b[2]; state.b[2] = state.b[3]; state.b[3] = state.b[0]; state.b[0] = state.b[1]; state.b[1] = state.b[2]; state.b[2] = state.b[3]; state.b[3] = state.b[0]; state.b[0] = state.b[1]; state.b[1] = state.b[2]; state.b[2] = state.b[3]; state.b[3] = state.b[0]; state.b[0] = state.b[1]; state.b[1] = state.b[2]; state.b[2] = state.b[3]; state.b[3] = state.b[0]; } return state.b; } This is not vectorizable, but SuperWordLoopUnrollAnalysis=true disables the unrolling which would make it faster. Andrew. From jiangli.zhou at oracle.com Fri Apr 14 18:11:53 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 14 Apr 2017 11:11:53 -0700 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: <58F050D0.8060308@oracle.com> References: <58EF454C.9050307@oracle.com> <95054A6C-EC6D-4A52-8DEE-93EB66CACFD8@oracle.com> <58F050D0.8060308@oracle.com> Message-ID: Hi Ioi, Thanks for the details. Do you know how big is the increase of ?dirty? pages with the specific writes? If the increase of unsharable pages are small then it's okay. It?s a balance between speed and memory. While looking for the callers of itableMethodEntry::initialize(), I found klassItable::initialize_with_method(). It doesn?t seem to be called by anyone. Could you please also remove that as part of your change if you haven?t integrated it. Otherwise, I?ll file a separate bug. Thanks, Jiangli > On Apr 13, 2017, at 9:32 PM, Ioi Lam wrote: > > Hi Jiangli, > > Code like this in klassVtable.cpp is necessary: > > if (is_preinitialized_vtable()) { /// calls _klass->is_shared() > // At runtime initialize_vtable is rerun for a shared class > // (loaded by the non-boot loader) as part of link_class_impl(). > // The dumptime vtable index should be the same as the runtime index. > assert(def_vtable_indices->at(i) == initialized, > "dump time vtable index is different from runtime index"); > } else { > def_vtable_indices->at_put(i, initialized); //set vtable index > } > > because the def_vtable_indices for a shared class is in RO space and cannot written into at run time. > > For this code (before) > > 1017 if (MetaspaceShared::is_in_shared_space((void*)&_method) && > 1018 !MetaspaceShared::remapped_readwrite()) { > 1019 // At runtime initialize_itable is rerun as part of link_class_impl() > 1020 // for a shared class loaded by the non-boot loader. > 1021 // The dumptime itable method entry should be the same as the runtime entry. > 1022 assert(_method == m, "sanity"); > 1023 } else { > 1024 _method = m; > 1025 } > 1026 } > > the address (&_method) is either in a vtable or an itable, both of which are in the RO space. Therefore it's OK to write into it. > > The cost of checking MetaspaceShared::is_in_shared_space((void*)&_method) is high, as at this point we don't readily have a pointer to the corresponding Klass whose i/v table encompasses (&_method). > > Therefore, I made the assignment unconditional. I kept the assert so we don't lose any functionality. > > I think the cost of writing into _method is low (compared to the cost of MetaspaceShared::is_in_shared_space) , because it's within an InstanceKlass, which will be written into anyway during InstanceKlass::restore_unshareable_info. > > AFTER: > > 1017 #ifdef ASSERT > 1018 if (MetaspaceShared::is_in_shared_space((void*)&_method) && > 1019 !MetaspaceShared::remapped_readwrite()) { > 1020 // At runtime initialize_itable is rerun as part of link_class_impl() > 1021 // for a shared class loaded by the non-boot loader. > 1022 // The dumptime itable method entry should be the same as the runtime entry. > 1023 assert(_method == m, "sanity"); > 1024 } > 1025 #endif > 1026 _method = m; > 1027 } > > Thanks > - Ioi > > On 4/14/17 2:02 AM, Jiangli Zhou wrote: >> Hi Ioi, >> >> Could you please explain the change in klassVtable.cpp? >> >> Thanks, >> Jiangli >> >>> On Apr 13, 2017, at 2:30 AM, Ioi Lam wrote: >>> >>> Hi, please review this small start-up enhancement: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8178543 >>> http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ >>> >>> We have a benchmark that shows Klass::is_shared() is called very frequently during InstanceKlass::link_class_impl, and costs about 2% of the start-up time. >>> >>> The fix is simple -- instead of walking the list of CDS shared regions, use a new bit in Klass::_access_flags >>> >>> Thanks >>> - Ioi > From lois.foltan at oracle.com Fri Apr 14 18:31:07 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 14 Apr 2017 14:31:07 -0400 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58F0EB0E.60904@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> Message-ID: On 4/14/2017 11:30 AM, Ioi Lam wrote: > > > On 4/14/17 1:31 PM, Ioi Lam wrote: >> HI Lois, >> >> Thanks for the review. Please see my comments in-line. >> >> On 4/14/17 4:32 AM, Lois Foltan wrote: >>> Hi Ioi, >>> >>> Looks really good. A couple of comments: >>> >>> src/share/vm/classfile/classFileParser.cpp: >>> * line #5676 - I'm not sure I completely understand the logic >>> surrounding anonymous classes. Coleen and I discussed earlier today >>> and I came away from that discussion with the idea that the only >>> classes being patched currently are anonymous classes. >> Line 5676 ... >> >> 5676 if (is_anonymous()) { >> 5677 _max_num_patched_klasses ++; // for patching the >> class index >> 5678 } >> >> corresponds to >> >> 5361 ik->set_name(_class_name); >> 5362 >> 5363 if (is_anonymous()) { >> 5364 // I am well known to myself >> 5365 patch_class(ik->constants(), _this_class_index, ik, >> ik->name()); // eagerly resolve >> 5366 } >> >> Even though the class is "anonymous", it actually has a name. >> ik->name() probably is part of the constant pool, but I am not 100% >> sure. Also, I would need to search the constant pool to find the >> index for ik->name(). So I just got lazy here and use the same logic >> in ConstantPool::patch_class() to append ik->name() to the end of the >> constant pool. >> >> "Anonymous" actually means "the class cannot be looked up by name in >> the SystemDictionary". I think we need a better terminology :-) >> > > I finally realized why we need the "eagerly resolve" on line 5365. > I'll modify the comments to the following: > > // _this_class_index is a CONSTANT_Class entry that refers to this > // anonymous class itself. If this class needs to refer to its own > methods or > // fields, it would use a CONSTANT_MethodRef, etc, which would > reference > // _this_class_index. However, because this class is anonymous (it's > // not stored in SystemDictionary), _this_class_index cannot be > resolved > // with ConstantPool::klass_at_impl, which does a SystemDictionary > lookup. > // Therefore, we must eagerly resolve _this_class_index now. > > So, Lois is right. Line 5676 is not necessary. I will revise the code > to do the "eager resolution" without using > ClassFileParser::patch_class. I'll post the updated code later. Thanks Ioi for studying this and explaining! Look forward to seeing the updated webrev. Lois > > Thanks > - Ioi > >>> So a bit confused as why the check on line #5676 and a check for a >>> java/lang/Class on line #5684. >> 5683 Handle patch = cp_patch_at(i); >> 5684 if (java_lang_String::is_instance(patch()) || >> java_lang_Class::is_instance(patch())) { >> 5685 // We need to append the names of the patched classes >> to the end of the constant pool, >> 5686 // because a patched class may have a Utf8 name that's >> not already included in the >> 5687 // original constant pool. >> 5688 // >> 5689 // Note that a String in cp_patch_at(i) may be used to >> patch a Utf8, a String, or a Class. >> 5690 // At this point, we don't know the tag for index i >> yet, because we haven't parsed the >> 5691 // constant pool. So we can only assume the worst -- >> every String is used to patch a Class. >> 5692 _max_num_patched_klasses ++; >> >> Line 5684 checks for all objects in the cp_patch array. Later, when >> ClassFileParser::patch_constant_pool() is called, any objects that >> are either Class or String could be treated as a Klass: >> >> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >> 725 int index, >> 726 Handle patch, >> 727 TRAPS) { >> ... >> 732 switch (cp->tag_at(index).value()) { >> 733 >> 734 case JVM_CONSTANT_UnresolvedClass: { >> 735 // Patching a class means pre-resolving it. >> 736 // The name in the constant pool is ignored. >> 737 if (java_lang_Class::is_instance(patch())) { >> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >> 739 "Illegal class patch at %d in class >> file %s", >> 740 index, CHECK); >> 741 Klass* k = java_lang_Class::as_Klass(patch()); >> 742 patch_class(cp, index, k, k->name()); >> 743 } else { >> 744 guarantee_property(java_lang_String::is_instance(patch()), >> 745 "Illegal class patch at %d in class >> file %s", >> 746 index, CHECK); >> 747 Symbol* const name = >> java_lang_String::as_symbol(patch(), CHECK); >> 748 patch_class(cp, index, NULL, name); >> 749 } >> 750 break; >> 751 } >> >>> Could the is_anonymous() if statement be combined into the loop? >> >> I think the answer is no. At line 5365, there is no guarantee that >> ik->name() is in the cp_patch array. >> >> 5365 patch_class(ik->constants(), _this_class_index, ik, >> ik->name()); // eagerly resolve >> >>> Also why not do this calculation in the rewriter or is that too late? >>> >> Line 5676 and 5684 need to be executed BEFORE the constant pool and >> the associated tags array is allocated (both of which are fixed size, >> and cannot be expanded), which is way before the rewriter is run. At >> this point, we don't know what cp->tag_at(index) is (line #732), so >> the code needs to make a worst-case estimate on how long the CP/tags >> should be. >> >>> * line #5677, 5692 - a nit but I think the convention is to not have >>> a space after the variable name and between the post increment >>> operator. >>> >> Fixed. >>> src/share/vm/classfile/constantPool.hpp: >>> I understand the concept behind _invalid_resolved_klass_index, but >>> it really is not so much invalid as temporary for class redefinition >>> purposes, as you explain in >>> ConstantPool::allocate_resolved_klasses. Please consider renaming >>> to _temp_unresolved_klass_index. And whether you choose to rename >>> the field or not, please add a one line comment ahead of >>> ConstantPool::temp_unresolved_klass_at_put that only class >>> redefinition uses this currently. >>> >> Good idea. Will do. >> >> Thanks >> - Ioi >> >>> Great change, thanks! >>> Lois >>> >>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>> Hi Coleen, >>>> >>>> Thanks for the comments. Here's a delta from the last patch >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>> >>>> >>>> In addition to your requests, I made these changes: >>>> >>>> [1] To consolidate the multiple extract_high/low code, I've added >>>> CPKlassSlot, so the code is cleaner: >>>> >>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>> int name_index = kslot.name_index(); >>>> >>>> [2] Renamed ConstantPool::is_shared_quick() to >>>> ConstantPool::is_shared(). The C++ compiler should be able to pick >>>> this function over MetaspaceObj::is_shared(). >>>> >>>> [3] Massaged the CDS region size set-up code a little to pass >>>> internal tests, because RO/RW ratio has changed. I didn't spend too >>>> much time picking the "right" sizes, as this code will be obsoleted >>>> soon with JDK-8072061 >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> This looks really good! >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>> >>>>> >>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>> + Array* tags = >>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, >>>>> "invariant"); // tags->at(_length) is flags() >>>>> >>>>> >>>>> I think this is left over, since _flags didn't get moved after all. >>>>> >>>>> + Klass** adr = >>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>> resolved >>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>> + // hardware store ordering here. >>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>> + return k; >>>>> >>>>> The comment still refers to the switch between Symbol* and Klass* >>>>> in the constant pool. The entry in the Klass array should be NULL. >>>>> >>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>> >>>>> Can you put klass_name_index_at() in the constantPool.hpp header >>>>> file (so it's inlined) and have all the places where you get >>>>> name_index use this function? So we don't have to know in >>>>> multiple places that extract_high_short_from_int() is where the >>>>> name index is. And in constantPool.hpp, for >>>>> unresolved_klass_at_put() add a comment about what the new format >>>>> of the constant pool entry is {name_index, resolved_klass_index}. >>>>> I'm happy to see this work nearing completion! The "constant" >>>>> pool should be constant! thanks, Coleen >>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>> Hi,please review the following change >>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>>>>> Klass from the i-th slot of ConstantPool. + When a klass is >>>>>> resolved, the ConstantPool is modified to store the Klass >>>>>> pointer. After: + ConstantPool::klass_at(i) finds the at >>>>>> this->_resolved_klasses->at(i) + When a klass is resolved, >>>>>> _resolved_klasses->at(i) is modified. In addition: + I >>>>>> moved _resolved_references and _reference_map from >>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no >>>>>> longer modified for shared ConstantPools. As a result, none of >>>>>> the fields in shared ConstantPools are modified at run time, so >>>>>> we can move them into the RO region in the CDS archive. >>>>>> *Testing:** * - Benchmark results show no performance >>>>>> regression, despite the extra level of indirection, which has a >>>>>> negligible overhead for the interpreter. - RBT testing for >>>>>> tier2 and tier3. *Ports:** * - I have tested only the >>>>>> Oracle-support ports. For the aarch64, ppc and s390 ports, I have >>>>>> added some code without testing (it's probably incomplete) - >>>>>> Port owners, please check if my patch work for you, and I can >>>>>> incorporate your changes in my push. Alternatively, you can wait >>>>>> for my push and provide fixes (if necessary) in a new changeset, >>>>>> and I will be happy to sponsor it. Thanks - Ioi >>>> >>> >> > From serguei.spitsyn at oracle.com Fri Apr 14 20:45:22 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 14 Apr 2017 13:45:22 -0700 Subject: RFR(S): 8164944: Refactor ProcessTools to get rid of dependency on java.management In-Reply-To: <5548AED8-2BEB-4341-88F9-B34C0F827BAD@oracle.com> References: <9054D440-C6C9-4D97-AEB9-25CD3057564F@oracle.com> <5548AED8-2BEB-4341-88F9-B34C0F827BAD@oracle.com> Message-ID: Igor, Looks good. Thanks, Serguei On 4/12/17 12:14, Igor Ignatyev wrote: > forgot to update copyrights, http://cr.openjdk.java.net/~iignatyev//8164944/webrev.01/index.html > >> On Apr 12, 2017, at 12:03 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >>> 92 lines changed: 36 ins; 36 del; 20 mod; >> Hi all, >> >> could you please review this small fix which removes dependency on java.management module from the testlibrary? >> >> jdk.test.lib.process.ProcessTools class has getVmInputArgs method which depend on java.management, this method is used only by j.d.t.process.ProcessTools::executeTestJvmAll and j.t.l.cli.CommandLineOptionTest class. >> - ProcessTools::executeTestJvmAll is a variation of executeTestJvm which prepends vm flags from jtreg @run. >> - CommandLineOptionTest is a library class for cli testing, it uses flags from ProcessTools::getVmInputArgs to create a new jvm with all flags >> >> the fix removes ProcessTools::executeTestJvmAll method, moves getVmInputArgs to a separate class jdk.test.lib.management.InputArguments. c.c.s.scenario.Executor and j.t.l.cli.CommandLineOptionTest are updated to use getVmInputArgs from that class. c.j.compilerToVM.DebugOutputTest is refactored to pass add-exports and patch module directly. all the affected classes and obviously updated to use executeTestJvm instead of executeTestJvmAll. >> >> after this fix many hotspot tests won't depend on java.management, hence that module might be removed from their @modules, this will be done separately by 8178416. >> >> jbs: https://bugs.openjdk.java.net/browse/JDK-8164944 >> webrev: http://cr.openjdk.java.net/~iignatyev//8164944/webrev.00/index.html >> testing: :hotspot_all >> 8178416: https://bugs.openjdk.java.net/browse/JDK-8178416 >> >> Thanks, >> -- Igor From ioi.lam at oracle.com Sat Apr 15 00:39:00 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 15 Apr 2017 08:39:00 +0800 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: References: <58EF454C.9050307@oracle.com> <95054A6C-EC6D-4A52-8DEE-93EB66CACFD8@oracle.com> <58F050D0.8060308@oracle.com> Message-ID: <58F16BA4.6090407@oracle.com> On 4/15/17 2:11 AM, Jiangli Zhou wrote: > Hi Ioi, > > Thanks for the details. Do you know how big is the increase of ?dirty? pages with the specific writes? If the increase of unsharable pages are small then it's okay. It?s a balance between speed and memory. Hi Jiangli, I ran the start-up benchmark in gdb and set a break point at exit, and check the smaps file: BEFORE: 801400000-802280000 rw-p 008e4000 fc:01 17360993 /tmp/iklam/bench2/cds.old.jsa Size: 14848 kB Rss: 14772 kB Pss: 14772 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 228 kB Private_Dirty: 14544 kB Referenced: 14772 kB Anonymous: 14544 kB AFTER: 801400000-802280000 rw-p 008e4000 fc:01 17360995 /tmp/iklam/bench2/cds.new.jsa Size: 14848 kB Rss: 14772 kB Pss: 14772 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 228 kB Private_Dirty: 14544 kB Referenced: 14772 kB Anonymous: 14544 kB As you can see, almost the entire RW section is dirty, and there's no difference with or without the change. So I think the change is pretty safe. > While looking for the callers of itableMethodEntry::initialize(), I found klassItable::initialize_with_method(). It doesn?t seem to be called by anyone. Could you please also remove that as part of your change if you haven?t integrated it. Otherwise, I?ll file a separate bug. Thanks for finding the dead code, I will take that out. Thanks - Ioi > Thanks, > Jiangli > >> On Apr 13, 2017, at 9:32 PM, Ioi Lam wrote: >> >> Hi Jiangli, >> >> Code like this in klassVtable.cpp is necessary: >> >> if (is_preinitialized_vtable()) { /// calls _klass->is_shared() >> // At runtime initialize_vtable is rerun for a shared class >> // (loaded by the non-boot loader) as part of link_class_impl(). >> // The dumptime vtable index should be the same as the runtime index. >> assert(def_vtable_indices->at(i) == initialized, >> "dump time vtable index is different from runtime index"); >> } else { >> def_vtable_indices->at_put(i, initialized); //set vtable index >> } >> >> because the def_vtable_indices for a shared class is in RO space and cannot written into at run time. >> >> For this code (before) >> >> 1017 if (MetaspaceShared::is_in_shared_space((void*)&_method) && >> 1018 !MetaspaceShared::remapped_readwrite()) { >> 1019 // At runtime initialize_itable is rerun as part of link_class_impl() >> 1020 // for a shared class loaded by the non-boot loader. >> 1021 // The dumptime itable method entry should be the same as the runtime entry. >> 1022 assert(_method == m, "sanity"); >> 1023 } else { >> 1024 _method = m; >> 1025 } >> 1026 } >> >> the address (&_method) is either in a vtable or an itable, both of which are in the RO space. Therefore it's OK to write into it. >> >> The cost of checking MetaspaceShared::is_in_shared_space((void*)&_method) is high, as at this point we don't readily have a pointer to the corresponding Klass whose i/v table encompasses (&_method). >> >> Therefore, I made the assignment unconditional. I kept the assert so we don't lose any functionality. >> >> I think the cost of writing into _method is low (compared to the cost of MetaspaceShared::is_in_shared_space) , because it's within an InstanceKlass, which will be written into anyway during InstanceKlass::restore_unshareable_info. >> >> AFTER: >> >> 1017 #ifdef ASSERT >> 1018 if (MetaspaceShared::is_in_shared_space((void*)&_method) && >> 1019 !MetaspaceShared::remapped_readwrite()) { >> 1020 // At runtime initialize_itable is rerun as part of link_class_impl() >> 1021 // for a shared class loaded by the non-boot loader. >> 1022 // The dumptime itable method entry should be the same as the runtime entry. >> 1023 assert(_method == m, "sanity"); >> 1024 } >> 1025 #endif >> 1026 _method = m; >> 1027 } >> >> Thanks >> - Ioi >> >> On 4/14/17 2:02 AM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> Could you please explain the change in klassVtable.cpp? >>> >>> Thanks, >>> Jiangli >>> >>>> On Apr 13, 2017, at 2:30 AM, Ioi Lam wrote: >>>> >>>> Hi, please review this small start-up enhancement: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8178543 >>>> http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ >>>> >>>> We have a benchmark that shows Klass::is_shared() is called very frequently during InstanceKlass::link_class_impl, and costs about 2% of the start-up time. >>>> >>>> The fix is simple -- instead of walking the list of CDS shared regions, use a new bit in Klass::_access_flags >>>> >>>> Thanks >>>> - Ioi From jiangli.zhou at oracle.com Sat Apr 15 01:53:01 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 14 Apr 2017 18:53:01 -0700 Subject: RFR[XS] JDK-8178543 - Optimize Klass::is_shared() In-Reply-To: <58F16BA4.6090407@oracle.com> References: <58EF454C.9050307@oracle.com> <95054A6C-EC6D-4A52-8DEE-93EB66CACFD8@oracle.com> <58F050D0.8060308@oracle.com> <58F16BA4.6090407@oracle.com> Message-ID: Hi Ioi, > On Apr 14, 2017, at 5:39 PM, Ioi Lam wrote: > > On 4/15/17 2:11 AM, Jiangli Zhou wrote: >> Hi Ioi, >> >> Thanks for the details. Do you know how big is the increase of ?dirty? pages with the specific writes? If the increase of unsharable pages are small then it's okay. It?s a balance between speed and memory. > > Hi Jiangli, > > I ran the start-up benchmark in gdb and set a break point at exit, and check the smaps file: > > BEFORE: > > 801400000-802280000 rw-p 008e4000 fc:01 17360993 /tmp/iklam/bench2/cds.old.jsa > Size: 14848 kB > Rss: 14772 kB > Pss: 14772 kB > Shared_Clean: 0 kB > Shared_Dirty: 0 kB > Private_Clean: 228 kB > Private_Dirty: 14544 kB > Referenced: 14772 kB > Anonymous: 14544 kB > > AFTER: > > 801400000-802280000 rw-p 008e4000 fc:01 17360995 /tmp/iklam/bench2/cds.new.jsa > Size: 14848 kB > Rss: 14772 kB > Pss: 14772 kB > Shared_Clean: 0 kB > Shared_Dirty: 0 kB > Private_Clean: 228 kB > Private_Dirty: 14544 kB > Referenced: 14772 kB > Anonymous: 14544 kB > > As you can see, almost the entire RW section is dirty, and there's no difference with or without the change. So I think the change is pretty safe. Thanks for looking into that. > >> While looking for the callers of itableMethodEntry::initialize(), I found klassItable::initialize_with_method(). It doesn?t seem to be called by anyone. Could you please also remove that as part of your change if you haven?t integrated it. Otherwise, I?ll file a separate bug. > > Thanks for finding the dead code, I will take that out. Thanks! Jiangli > > Thanks > - Ioi >> Thanks, >> Jiangli >> >>> On Apr 13, 2017, at 9:32 PM, Ioi Lam wrote: >>> >>> Hi Jiangli, >>> >>> Code like this in klassVtable.cpp is necessary: >>> >>> if (is_preinitialized_vtable()) { /// calls _klass->is_shared() >>> // At runtime initialize_vtable is rerun for a shared class >>> // (loaded by the non-boot loader) as part of link_class_impl(). >>> // The dumptime vtable index should be the same as the runtime index. >>> assert(def_vtable_indices->at(i) == initialized, >>> "dump time vtable index is different from runtime index"); >>> } else { >>> def_vtable_indices->at_put(i, initialized); //set vtable index >>> } >>> >>> because the def_vtable_indices for a shared class is in RO space and cannot written into at run time. >>> >>> For this code (before) >>> >>> 1017 if (MetaspaceShared::is_in_shared_space((void*)&_method) && >>> 1018 !MetaspaceShared::remapped_readwrite()) { >>> 1019 // At runtime initialize_itable is rerun as part of link_class_impl() >>> 1020 // for a shared class loaded by the non-boot loader. >>> 1021 // The dumptime itable method entry should be the same as the runtime entry. >>> 1022 assert(_method == m, "sanity"); >>> 1023 } else { >>> 1024 _method = m; >>> 1025 } >>> 1026 } >>> >>> the address (&_method) is either in a vtable or an itable, both of which are in the RO space. Therefore it's OK to write into it. >>> >>> The cost of checking MetaspaceShared::is_in_shared_space((void*)&_method) is high, as at this point we don't readily have a pointer to the corresponding Klass whose i/v table encompasses (&_method). >>> >>> Therefore, I made the assignment unconditional. I kept the assert so we don't lose any functionality. >>> >>> I think the cost of writing into _method is low (compared to the cost of MetaspaceShared::is_in_shared_space) , because it's within an InstanceKlass, which will be written into anyway during InstanceKlass::restore_unshareable_info. >>> >>> AFTER: >>> >>> 1017 #ifdef ASSERT >>> 1018 if (MetaspaceShared::is_in_shared_space((void*)&_method) && >>> 1019 !MetaspaceShared::remapped_readwrite()) { >>> 1020 // At runtime initialize_itable is rerun as part of link_class_impl() >>> 1021 // for a shared class loaded by the non-boot loader. >>> 1022 // The dumptime itable method entry should be the same as the runtime entry. >>> 1023 assert(_method == m, "sanity"); >>> 1024 } >>> 1025 #endif >>> 1026 _method = m; >>> 1027 } >>> >>> Thanks >>> - Ioi >>> >>> On 4/14/17 2:02 AM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> Could you please explain the change in klassVtable.cpp? >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On Apr 13, 2017, at 2:30 AM, Ioi Lam wrote: >>>>> >>>>> Hi, please review this small start-up enhancement: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8178543 >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8178543-klass-is-shared.v01/ >>>>> >>>>> We have a benchmark that shows Klass::is_shared() is called very frequently during InstanceKlass::link_class_impl, and costs about 2% of the start-up time. >>>>> >>>>> The fix is simple -- instead of walking the list of CDS shared regions, use a new bit in Klass::_access_flags >>>>> >>>>> Thanks >>>>> - Ioi > From ioi.lam at oracle.com Sun Apr 16 09:33:03 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 16 Apr 2017 17:33:03 +0800 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> Message-ID: <58F33A4F.70104@oracle.com> Hi Lois, I have updated the patch to include your comments, and fixes the handling of anonymous classes. I also added some more comments regarding the _temp_resolved_klass_index: (delta from last webrev) http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ (full webrev) http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ Thanks - Ioi On 4/15/17 2:31 AM, Lois Foltan wrote: > On 4/14/2017 11:30 AM, Ioi Lam wrote: >> >> >> On 4/14/17 1:31 PM, Ioi Lam wrote: >>> HI Lois, >>> >>> Thanks for the review. Please see my comments in-line. >>> >>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>> Hi Ioi, >>>> >>>> Looks really good. A couple of comments: >>>> >>>> src/share/vm/classfile/classFileParser.cpp: >>>> * line #5676 - I'm not sure I completely understand the logic >>>> surrounding anonymous classes. Coleen and I discussed earlier >>>> today and I came away from that discussion with the idea that the >>>> only classes being patched currently are anonymous classes. >>> Line 5676 ... >>> >>> 5676 if (is_anonymous()) { >>> 5677 _max_num_patched_klasses ++; // for patching the >>> class index >>> 5678 } >>> >>> corresponds to >>> >>> 5361 ik->set_name(_class_name); >>> 5362 >>> 5363 if (is_anonymous()) { >>> 5364 // I am well known to myself >>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>> ik->name()); // eagerly resolve >>> 5366 } >>> >>> Even though the class is "anonymous", it actually has a name. >>> ik->name() probably is part of the constant pool, but I am not 100% >>> sure. Also, I would need to search the constant pool to find the >>> index for ik->name(). So I just got lazy here and use the same logic >>> in ConstantPool::patch_class() to append ik->name() to the end of >>> the constant pool. >>> >>> "Anonymous" actually means "the class cannot be looked up by name in >>> the SystemDictionary". I think we need a better terminology :-) >>> >> >> I finally realized why we need the "eagerly resolve" on line 5365. >> I'll modify the comments to the following: >> >> // _this_class_index is a CONSTANT_Class entry that refers to this >> // anonymous class itself. If this class needs to refer to its >> own methods or >> // fields, it would use a CONSTANT_MethodRef, etc, which would >> reference >> // _this_class_index. However, because this class is anonymous (it's >> // not stored in SystemDictionary), _this_class_index cannot be >> resolved >> // with ConstantPool::klass_at_impl, which does a >> SystemDictionary lookup. >> // Therefore, we must eagerly resolve _this_class_index now. >> >> So, Lois is right. Line 5676 is not necessary. I will revise the code >> to do the "eager resolution" without using >> ClassFileParser::patch_class. I'll post the updated code later. > > Thanks Ioi for studying this and explaining! Look forward to seeing > the updated webrev. > Lois > >> >> Thanks >> - Ioi >> >>>> So a bit confused as why the check on line #5676 and a check for a >>>> java/lang/Class on line #5684. >>> 5683 Handle patch = cp_patch_at(i); >>> 5684 if (java_lang_String::is_instance(patch()) || >>> java_lang_Class::is_instance(patch())) { >>> 5685 // We need to append the names of the patched classes >>> to the end of the constant pool, >>> 5686 // because a patched class may have a Utf8 name >>> that's not already included in the >>> 5687 // original constant pool. >>> 5688 // >>> 5689 // Note that a String in cp_patch_at(i) may be used >>> to patch a Utf8, a String, or a Class. >>> 5690 // At this point, we don't know the tag for index i >>> yet, because we haven't parsed the >>> 5691 // constant pool. So we can only assume the worst -- >>> every String is used to patch a Class. >>> 5692 _max_num_patched_klasses ++; >>> >>> Line 5684 checks for all objects in the cp_patch array. Later, when >>> ClassFileParser::patch_constant_pool() is called, any objects that >>> are either Class or String could be treated as a Klass: >>> >>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>> 725 int index, >>> 726 Handle patch, >>> 727 TRAPS) { >>> ... >>> 732 switch (cp->tag_at(index).value()) { >>> 733 >>> 734 case JVM_CONSTANT_UnresolvedClass: { >>> 735 // Patching a class means pre-resolving it. >>> 736 // The name in the constant pool is ignored. >>> 737 if (java_lang_Class::is_instance(patch())) { >>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>> 739 "Illegal class patch at %d in class >>> file %s", >>> 740 index, CHECK); >>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>> 742 patch_class(cp, index, k, k->name()); >>> 743 } else { >>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>> 745 "Illegal class patch at %d in class >>> file %s", >>> 746 index, CHECK); >>> 747 Symbol* const name = >>> java_lang_String::as_symbol(patch(), CHECK); >>> 748 patch_class(cp, index, NULL, name); >>> 749 } >>> 750 break; >>> 751 } >>> >>>> Could the is_anonymous() if statement be combined into the loop? >>> >>> I think the answer is no. At line 5365, there is no guarantee that >>> ik->name() is in the cp_patch array. >>> >>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>> ik->name()); // eagerly resolve >>> >>>> Also why not do this calculation in the rewriter or is that too >>>> late? >>>> >>> Line 5676 and 5684 need to be executed BEFORE the constant pool and >>> the associated tags array is allocated (both of which are fixed >>> size, and cannot be expanded), which is way before the rewriter is >>> run. At this point, we don't know what cp->tag_at(index) is (line >>> #732), so the code needs to make a worst-case estimate on how long >>> the CP/tags should be. >>> >>>> * line #5677, 5692 - a nit but I think the convention is to not >>>> have a space after the variable name and between the post increment >>>> operator. >>>> >>> Fixed. >>>> src/share/vm/classfile/constantPool.hpp: >>>> I understand the concept behind _invalid_resolved_klass_index, but >>>> it really is not so much invalid as temporary for class >>>> redefinition purposes, as you explain in >>>> ConstantPool::allocate_resolved_klasses. Please consider renaming >>>> to _temp_unresolved_klass_index. And whether you choose to rename >>>> the field or not, please add a one line comment ahead of >>>> ConstantPool::temp_unresolved_klass_at_put that only class >>>> redefinition uses this currently. >>>> >>> Good idea. Will do. >>> >>> Thanks >>> - Ioi >>> >>>> Great change, thanks! >>>> Lois >>>> >>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for the comments. Here's a delta from the last patch >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>> >>>>> >>>>> In addition to your requests, I made these changes: >>>>> >>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>> CPKlassSlot, so the code is cleaner: >>>>> >>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>> int name_index = kslot.name_index(); >>>>> >>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick >>>>> this function over MetaspaceObj::is_shared(). >>>>> >>>>> [3] Massaged the CDS region size set-up code a little to pass >>>>> internal tests, because RO/RW ratio has changed. I didn't spend >>>>> too much time picking the "right" sizes, as this code will be >>>>> obsoleted soon with JDK-8072061 >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> This looks really good! >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>> >>>>>> >>>>>> + // Add one extra element to tags for storing >>>>>> ConstantPool::flags(). >>>>>> + Array* tags = >>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, >>>>>> 0, CHECK_NULL); ... + assert(tags->length()-1 == _length, >>>>>> "invariant"); // tags->at(_length) is flags() >>>>>> >>>>>> >>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>> >>>>>> + Klass** adr = >>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>> resolved >>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>> + // hardware store ordering here. >>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>> + return k; >>>>>> >>>>>> The comment still refers to the switch between Symbol* and Klass* >>>>>> in the constant pool. The entry in the Klass array should be NULL. >>>>>> >>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>> >>>>>> Can you put klass_name_index_at() in the constantPool.hpp header >>>>>> file (so it's inlined) and have all the places where you get >>>>>> name_index use this function? So we don't have to know in >>>>>> multiple places that extract_high_short_from_int() is where the >>>>>> name index is. And in constantPool.hpp, for >>>>>> unresolved_klass_at_put() add a comment about what the new format >>>>>> of the constant pool entry is {name_index, resolved_klass_index}. >>>>>> I'm happy to see this work nearing completion! The "constant" >>>>>> pool should be constant! thanks, Coleen >>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>> Hi,please review the following change >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>>>>>> Klass from the i-th slot of ConstantPool. + When a klass is >>>>>>> resolved, the ConstantPool is modified to store the Klass >>>>>>> pointer. After: + ConstantPool::klass_at(i) finds the at >>>>>>> this->_resolved_klasses->at(i) + When a klass is resolved, >>>>>>> _resolved_klasses->at(i) is modified. In addition: + I >>>>>>> moved _resolved_references and _reference_map from >>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no >>>>>>> longer modified for shared ConstantPools. As a result, none of >>>>>>> the fields in shared ConstantPools are modified at run time, so >>>>>>> we can move them into the RO region in the CDS archive. >>>>>>> *Testing:** * - Benchmark results show no performance >>>>>>> regression, despite the extra level of indirection, which has a >>>>>>> negligible overhead for the interpreter. - RBT testing for >>>>>>> tier2 and tier3. *Ports:** * - I have tested only the >>>>>>> Oracle-support ports. For the aarch64, ppc and s390 ports, I >>>>>>> have added some code without testing (it's probably >>>>>>> incomplete) - Port owners, please check if my patch work for >>>>>>> you, and I can incorporate your changes in my push. >>>>>>> Alternatively, you can wait for my push and provide fixes (if >>>>>>> necessary) in a new changeset, and I will be happy to sponsor >>>>>>> it. Thanks - Ioi >>>>> >>>> >>> >> > From lois.foltan at oracle.com Mon Apr 17 12:41:23 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 17 Apr 2017 08:41:23 -0400 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58F33A4F.70104@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> Message-ID: <151da513-5459-36ef-0417-fd0aa210c7b6@oracle.com> On 4/16/2017 5:33 AM, Ioi Lam wrote: > Hi Lois, > > I have updated the patch to include your comments, and fixes the > handling of anonymous classes. I also added some more comments > regarding the _temp_resolved_klass_index: Reviewed, looks good! Thank you for making those changes. Lois > > (delta from last webrev) > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ > > > (full webrev) > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ > > > Thanks > - Ioi > > On 4/15/17 2:31 AM, Lois Foltan wrote: >> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>> >>> >>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>> HI Lois, >>>> >>>> Thanks for the review. Please see my comments in-line. >>>> >>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>> Hi Ioi, >>>>> >>>>> Looks really good. A couple of comments: >>>>> >>>>> src/share/vm/classfile/classFileParser.cpp: >>>>> * line #5676 - I'm not sure I completely understand the logic >>>>> surrounding anonymous classes. Coleen and I discussed earlier >>>>> today and I came away from that discussion with the idea that the >>>>> only classes being patched currently are anonymous classes. >>>> Line 5676 ... >>>> >>>> 5676 if (is_anonymous()) { >>>> 5677 _max_num_patched_klasses ++; // for patching the >>>> class index >>>> 5678 } >>>> >>>> corresponds to >>>> >>>> 5361 ik->set_name(_class_name); >>>> 5362 >>>> 5363 if (is_anonymous()) { >>>> 5364 // I am well known to myself >>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>> ik->name()); // eagerly resolve >>>> 5366 } >>>> >>>> Even though the class is "anonymous", it actually has a name. >>>> ik->name() probably is part of the constant pool, but I am not 100% >>>> sure. Also, I would need to search the constant pool to find the >>>> index for ik->name(). So I just got lazy here and use the same >>>> logic in ConstantPool::patch_class() to append ik->name() to the >>>> end of the constant pool. >>>> >>>> "Anonymous" actually means "the class cannot be looked up by name >>>> in the SystemDictionary". I think we need a better terminology :-) >>>> >>> >>> I finally realized why we need the "eagerly resolve" on line 5365. >>> I'll modify the comments to the following: >>> >>> // _this_class_index is a CONSTANT_Class entry that refers to this >>> // anonymous class itself. If this class needs to refer to its >>> own methods or >>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>> reference >>> // _this_class_index. However, because this class is anonymous >>> (it's >>> // not stored in SystemDictionary), _this_class_index cannot be >>> resolved >>> // with ConstantPool::klass_at_impl, which does a >>> SystemDictionary lookup. >>> // Therefore, we must eagerly resolve _this_class_index now. >>> >>> So, Lois is right. Line 5676 is not necessary. I will revise the >>> code to do the "eager resolution" without using >>> ClassFileParser::patch_class. I'll post the updated code later. >> >> Thanks Ioi for studying this and explaining! Look forward to seeing >> the updated webrev. >> Lois >> >>> >>> Thanks >>> - Ioi >>> >>>>> So a bit confused as why the check on line #5676 and a check for a >>>>> java/lang/Class on line #5684. >>>> 5683 Handle patch = cp_patch_at(i); >>>> 5684 if (java_lang_String::is_instance(patch()) || >>>> java_lang_Class::is_instance(patch())) { >>>> 5685 // We need to append the names of the patched >>>> classes to the end of the constant pool, >>>> 5686 // because a patched class may have a Utf8 name >>>> that's not already included in the >>>> 5687 // original constant pool. >>>> 5688 // >>>> 5689 // Note that a String in cp_patch_at(i) may be used >>>> to patch a Utf8, a String, or a Class. >>>> 5690 // At this point, we don't know the tag for index i >>>> yet, because we haven't parsed the >>>> 5691 // constant pool. So we can only assume the worst -- >>>> every String is used to patch a Class. >>>> 5692 _max_num_patched_klasses ++; >>>> >>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>> ClassFileParser::patch_constant_pool() is called, any objects that >>>> are either Class or String could be treated as a Klass: >>>> >>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>> 725 int index, >>>> 726 Handle patch, >>>> 727 TRAPS) { >>>> ... >>>> 732 switch (cp->tag_at(index).value()) { >>>> 733 >>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>> 735 // Patching a class means pre-resolving it. >>>> 736 // The name in the constant pool is ignored. >>>> 737 if (java_lang_Class::is_instance(patch())) { >>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>> 739 "Illegal class patch at %d in class >>>> file %s", >>>> 740 index, CHECK); >>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>> 742 patch_class(cp, index, k, k->name()); >>>> 743 } else { >>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>> 745 "Illegal class patch at %d in class >>>> file %s", >>>> 746 index, CHECK); >>>> 747 Symbol* const name = >>>> java_lang_String::as_symbol(patch(), CHECK); >>>> 748 patch_class(cp, index, NULL, name); >>>> 749 } >>>> 750 break; >>>> 751 } >>>> >>>>> Could the is_anonymous() if statement be combined into the loop? >>>> >>>> I think the answer is no. At line 5365, there is no guarantee that >>>> ik->name() is in the cp_patch array. >>>> >>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>> ik->name()); // eagerly resolve >>>> >>>>> Also why not do this calculation in the rewriter or is that too >>>>> late? >>>>> >>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and >>>> the associated tags array is allocated (both of which are fixed >>>> size, and cannot be expanded), which is way before the rewriter is >>>> run. At this point, we don't know what cp->tag_at(index) is (line >>>> #732), so the code needs to make a worst-case estimate on how long >>>> the CP/tags should be. >>>> >>>>> * line #5677, 5692 - a nit but I think the convention is to not >>>>> have a space after the variable name and between the post >>>>> increment operator. >>>>> >>>> Fixed. >>>>> src/share/vm/classfile/constantPool.hpp: >>>>> I understand the concept behind _invalid_resolved_klass_index, but >>>>> it really is not so much invalid as temporary for class >>>>> redefinition purposes, as you explain in >>>>> ConstantPool::allocate_resolved_klasses. Please consider renaming >>>>> to _temp_unresolved_klass_index. And whether you choose to rename >>>>> the field or not, please add a one line comment ahead of >>>>> ConstantPool::temp_unresolved_klass_at_put that only class >>>>> redefinition uses this currently. >>>>> >>>> Good idea. Will do. >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Great change, thanks! >>>>> Lois >>>>> >>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>> >>>>>> >>>>>> In addition to your requests, I made these changes: >>>>>> >>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>> CPKlassSlot, so the code is cleaner: >>>>>> >>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>> int name_index = kslot.name_index(); >>>>>> >>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>> ConstantPool::is_shared(). The C++ compiler should be able to >>>>>> pick this function over MetaspaceObj::is_shared(). >>>>>> >>>>>> [3] Massaged the CDS region size set-up code a little to pass >>>>>> internal tests, because RO/RW ratio has changed. I didn't spend >>>>>> too much time picking the "right" sizes, as this code will be >>>>>> obsoleted soon with JDK-8072061 >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> This looks really good! >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>> >>>>>>> >>>>>>> + // Add one extra element to tags for storing >>>>>>> ConstantPool::flags(). >>>>>>> + Array* tags = >>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, >>>>>>> 0, CHECK_NULL); ... + assert(tags->length()-1 == _length, >>>>>>> "invariant"); // tags->at(_length) is flags() >>>>>>> >>>>>>> >>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>> >>>>>>> + Klass** adr = >>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>> + // The interpreter assumes when the tag is stored, the klass >>>>>>> is resolved >>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>> + // hardware store ordering here. >>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>> + return k; >>>>>>> >>>>>>> The comment still refers to the switch between Symbol* and >>>>>>> Klass* in the constant pool. The entry in the Klass array >>>>>>> should be NULL. >>>>>>> >>>>>>> + int name_index = >>>>>>> extract_high_short_from_int(*int_at_addr(which)); >>>>>>> >>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header >>>>>>> file (so it's inlined) and have all the places where you get >>>>>>> name_index use this function? So we don't have to know in >>>>>>> multiple places that extract_high_short_from_int() is where the >>>>>>> name index is. And in constantPool.hpp, for >>>>>>> unresolved_klass_at_put() add a comment about what the new >>>>>>> format of the constant pool entry is {name_index, >>>>>>> resolved_klass_index}. I'm happy to see this work nearing >>>>>>> completion! The "constant" pool should be constant! thanks, Coleen >>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>> Hi,please review the following change >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>>>>>>> Klass from the i-th slot of ConstantPool. + When a klass is >>>>>>>> resolved, the ConstantPool is modified to store the Klass >>>>>>>> pointer. After: + ConstantPool::klass_at(i) finds the at >>>>>>>> this->_resolved_klasses->at(i) + When a klass is resolved, >>>>>>>> _resolved_klasses->at(i) is modified. In addition: + I >>>>>>>> moved _resolved_references and _reference_map from >>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no >>>>>>>> longer modified for shared ConstantPools. As a result, none >>>>>>>> of the fields in shared ConstantPools are modified at run time, >>>>>>>> so we can move them into the RO region in the CDS archive. >>>>>>>> *Testing:** * - Benchmark results show no performance >>>>>>>> regression, despite the extra level of indirection, which has a >>>>>>>> negligible overhead for the interpreter. - RBT testing for >>>>>>>> tier2 and tier3. *Ports:** * - I have tested only the >>>>>>>> Oracle-support ports. For the aarch64, ppc and s390 ports, I >>>>>>>> have added some code without testing (it's probably >>>>>>>> incomplete) - Port owners, please check if my patch work for >>>>>>>> you, and I can incorporate your changes in my push. >>>>>>>> Alternatively, you can wait for my push and provide fixes (if >>>>>>>> necessary) in a new changeset, and I will be happy to sponsor >>>>>>>> it. Thanks - Ioi >>>>>> >>>>> >>>> >>> >> > From ramkri123 at gmail.com Thu Apr 13 01:30:55 2017 From: ramkri123 at gmail.com (Ram Krishnan) Date: Wed, 12 Apr 2017 18:30:55 -0700 Subject: linux os processor optimizations for OpenJDK performance enhancement In-Reply-To: <86d58d9d-ea45-6732-370f-413ff1b4e8f4@oracle.com> References: <86d58d9d-ea45-6732-370f-413ff1b4e8f4@oracle.com> Message-ID: Many thanks David for the immediate reply. Thanks, Ramki On Wed, Apr 12, 2017 at 6:26 PM, David Holmes wrote: > Redirecting to hotspot-dev mailing list, please drop jdk10-dev on future > replies. > > On 13/04/2017 9:39 AM, Ram Krishnan wrote: > >> We are working on linux os processor specific optimizations centered >> around >> cache partitioning and better processor core thread(s) affinity for >> OpenJDK >> GC performance enhancement with the following benefits >> - throughput increase >> - minimizing tail latency for latency sensitive applications >> - superior isolation in a multi-tenant environment >> >> ?We have been looking at the G1GC code/documentation and performed some >> initial experiments. Based on this, what we understand is that STW events >> pause all application threads independent of which regions are being >> impacted by garbage collection. For example, if application thread "x" >> uses >> regions 1, 4, 5 and garbage collection is working on regions 2 and 11, >> application thread "x" is paused during the STW event. >> > > A stop-the-world event does stop all application threads (all JavaThreads > in VM terminology) by using a safepoint. > > I'll let the GC folks comment on the details wrt. G1 region management. > > Cheers, > David > > Your expert opinion on this topic is much appreciated. >> >> -- Thanks, Ramki From chenyt.cs.sjtu at gmail.com Thu Apr 13 21:48:59 2017 From: chenyt.cs.sjtu at gmail.com (Yuting Chen) Date: Thu, 13 Apr 2017 14:48:59 -0700 Subject: HotSpot and IBM's J9 behave a little different when verifying this class Message-ID: I recently found an interesting class. Hope that I can get answers here. The bytecode is shown as follows. Hotspot can catch some inconsistencies here (Reason: Current frame's flags are not assignable to stack map frame's), while J9 cannot. I thought J9 was wrong until I went into HotSpot's source code and saw some comments in share/vm/classfile/verifier.cpp. It says that the verifier "Return TRUE if all code paths starting with start_bc_offset end in bytecode athrow or loop" (a comment before bool ClassVerifier::ends_in_athrow(u4 start_bc_offset)). Clearly the paths in Vector1(int, int) end in athrow, and the verifier fails. Can anyone explain the reason why the class still cannot pass the verification, and which verifier needs to be fixed. **************** public class Vector1 extends java.util.AbstractList minor version: 0 major version: 52 flags: ACC_PUBLIC, ACC_SUPER Constant pool: #1 = Utf8 Vector1 #2 = Class #1 // Vector1 #3 = Utf8 java/util/AbstractList #4 = Class #3 // java/util/AbstractList #5 = Utf8 #6 = Utf8 (II)V #7 = Utf8 ()V #8 = NameAndType #5:#7 // "":()V #9 = Methodref #4.#8 // java/util/AbstractList."":()V #10 = Utf8 java/lang/Exception #11 = Class #10 // java/lang/Exception #12 = Methodref #11.#8 // java/lang/Exception."":()V #13 = Utf8 main #14 = Utf8 ([Ljava/lang/String;)V #15 = NameAndType #5:#6 // "":(II)V #16 = Methodref #2.#15 // Vector1."":(II)V #17 = Utf8 Code #18 = Utf8 StackMapTable { public Vector1(int, int); descriptor: (II)V flags: ACC_PUBLIC Code: stack=2, locals=3, args_size=3 0: iload_1 1: iflt 11 4: aload_0 5: invokespecial #9 // Method java/util/AbstractList."":()V 8: goto 11 11: new #11 // class java/lang/Exception 14: dup 15: invokespecial #12 // Method java/lang/Exception."":()V 18: athrow StackMapTable: number_of_entries = 1 frame_type = 255 /* full_frame */ offset_delta = 11 locals = [ top, int, int ] stack = [] public static void main(java.lang.String[]); descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC, ACC_STATIC Code: stack=3, locals=1, args_size=1 0: new #2 // class Vector1 3: bipush 10 5: iconst_0 6: invokespecial #16 // Method "":(II)V 9: return } HotSpot reports the next verifyerror, while J9 does not: Exception in thread "main" java.lang.VerifyError: Inconsistent stackmap frames at branch target 11 Exception Details: Location: Vector1.(II)V @11: new Reason: Current frame's flags are not assignable to stack map frame's. Current Frame: bci: @1 flags: { flagThisUninit } locals: { uninitializedThis, integer, integer } stack: { integer } Stackmap Frame: bci: @11 flags: { } locals: { top, integer, integer } stack: { } Bytecode: 0000000: 1b9b 000a 2ab7 0009 a700 03bb 000b 59b7 0000010: 000c bf Stackmap Table: full_frame(@11,{Top,Integer,Integer},{}) From t.p.ellison at gmail.com Mon Apr 17 19:57:55 2017 From: t.p.ellison at gmail.com (Tim Ellison) Date: Mon, 17 Apr 2017 21:57:55 +0200 Subject: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: References: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> Message-ID: I will bring this to the attention of the J9 verifier team and ask them to comment on the behavior. Regards, Tim On 14 Apr 2017 00:39, "Yuting Chen" wrote: > It seems that J9 forgot to check the flags when athrow follows. > > On Thu, Apr 13, 2017 at 4:21 PM, Michael Rasmussen > wrote: > > On 14 April 2017 at 02:15, Yuting Chen wrote: > >> Thanks, Remi and Michael. > >> > >> The difference appears only when all of the paths end in athrow. Then > >> what does the comment ("Return TRUE if all code paths starting with > >> start_bc_offset end in bytecode athrow or loop") in verifier.cpp mean? > >> It seems that J9 follows this comment. > >> > >> Both verifiers can decline the class if "athrow" is changed to "return". > > > > Yes, for constructors, if exit is via return, then there cannot be any > > uninitializedThis, but allowed if exit is via athrow. > > > > But, your problem is, that your two frame aren't assignable to each > other. > >>>> Current Frame: > >>>> bci: @1 > >>>> flags: { flagThisUninit } > >>>> locals: { uninitializedThis, integer, integer } > >>>> stack: { integer } > >>>> Stackmap Frame: > >>>> bci: @11 > >>>> flags: { } > >>>> locals: { top, integer, integer } > >>>> stack: { } > > > > While the locals are compatible, the flags are not, you cannot go from > > { flagThisUninit } to {} > > http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/ > src/share/vm/classfile/stackMapFrame.cpp#l190 > > > > /Michael > From chenyt.cs.sjtu at gmail.com Tue Apr 18 00:50:58 2017 From: chenyt.cs.sjtu at gmail.com (Yuting Chen) Date: Mon, 17 Apr 2017 17:50:58 -0700 Subject: HotSpot and IBM's J9 behave a little different when verifying this class In-Reply-To: References: <1117183455.1833927.1492123177051.JavaMail.zimbra@u-pem.fr> Message-ID: The bug has been fixed. They suggested me to change a new version. After all, it's interesting. On Mon, Apr 17, 2017 at 12:57 PM, Tim Ellison wrote: > I will bring this to the attention of the J9 verifier team and ask them to > comment on the behavior. > > Regards, > Tim > > On 14 Apr 2017 00:39, "Yuting Chen" wrote: >> >> It seems that J9 forgot to check the flags when athrow follows. >> >> On Thu, Apr 13, 2017 at 4:21 PM, Michael Rasmussen >> wrote: >> > On 14 April 2017 at 02:15, Yuting Chen wrote: >> >> Thanks, Remi and Michael. >> >> >> >> The difference appears only when all of the paths end in athrow. Then >> >> what does the comment ("Return TRUE if all code paths starting with >> >> start_bc_offset end in bytecode athrow or loop") in verifier.cpp mean? >> >> It seems that J9 follows this comment. >> >> >> >> Both verifiers can decline the class if "athrow" is changed to >> >> "return". >> > >> > Yes, for constructors, if exit is via return, then there cannot be any >> > uninitializedThis, but allowed if exit is via athrow. >> > >> > But, your problem is, that your two frame aren't assignable to each >> > other. >> >>>> Current Frame: >> >>>> bci: @1 >> >>>> flags: { flagThisUninit } >> >>>> locals: { uninitializedThis, integer, integer } >> >>>> stack: { integer } >> >>>> Stackmap Frame: >> >>>> bci: @11 >> >>>> flags: { } >> >>>> locals: { top, integer, integer } >> >>>> stack: { } >> > >> > While the locals are compatible, the flags are not, you cannot go from >> > { flagThisUninit } to {} >> > >> > http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/share/vm/classfile/stackMapFrame.cpp#l190 >> > >> > /Michael From yang.zhang at linaro.org Tue Apr 18 06:42:13 2017 From: yang.zhang at linaro.org (Yang Zhang) Date: Tue, 18 Apr 2017 14:42:13 +0800 Subject: SuperWordLoopUnrollAnalysis and loop unrolling In-Reply-To: <3d2b99e7-a102-a073-23b6-574998027729@redhat.com> References: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> <3d2b99e7-a102-a073-23b6-574998027729@redhat.com> Message-ID: Hi Andrew I have run this test case. For aarch64 C2 with SuperWordLoopUnrollAnalysis=false: Loop unroll is still controlled by comparing body_size. When loop unroll is 16 times, body_size is big enough and it stops. For aarch64 C2 with SuperWordLoopUnrollAnalysis=true: First loop unroll is controlled by policy_unroll_slp_analysis. When loop unroll is 4 times, vectorization happens. If vectorization succeeds, loop unroll would be instigated more by set_major_progress until body_size is big enough. if vectorization fails, loop unroll stops. I haven't run the performance test. But I think in modern CPUs instructions are run out of order. Loop unroll doesn't always bring performance improvement. Regards Yang On 15 April 2017 at 00:41, Andrew Haley wrote: > On 14/04/17 10:21, Yang Zhang wrote: >> My test result is just opposite with your description. Could you >> provide your test case? > > // @Benchmark > public int[] sameArrayClass(BenchmarkState state) { > for (int i = 0; i < INITSIZE; i++) { > state.b[0] = state.b[1]; > state.b[1] = state.b[2]; > state.b[2] = state.b[3]; > state.b[3] = state.b[0]; > > state.b[0] = state.b[1]; > state.b[1] = state.b[2]; > state.b[2] = state.b[3]; > state.b[3] = state.b[0]; > > state.b[0] = state.b[1]; > state.b[1] = state.b[2]; > state.b[2] = state.b[3]; > state.b[3] = state.b[0]; > > state.b[0] = state.b[1]; > state.b[1] = state.b[2]; > state.b[2] = state.b[3]; > state.b[3] = state.b[0]; > } > return state.b; > } > > This is not vectorizable, but SuperWordLoopUnrollAnalysis=true disables > the unrolling which would make it faster. > > Andrew. > From aph at redhat.com Tue Apr 18 07:40:54 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 18 Apr 2017 08:40:54 +0100 Subject: SuperWordLoopUnrollAnalysis and loop unrolling In-Reply-To: References: <6296b3df-cb61-8484-96bd-5b95c6ff0f85@redhat.com> <3d2b99e7-a102-a073-23b6-574998027729@redhat.com> Message-ID: <0d98837e-cf1a-af06-75c0-3dd55865e161@redhat.com> On 18/04/17 07:42, Yang Zhang wrote: > I have run this test case. > > For aarch64 C2 with SuperWordLoopUnrollAnalysis=false: > Loop unroll is still controlled by comparing body_size. When loop > unroll is 16 times, body_size is big enough and it stops. > > For aarch64 C2 with SuperWordLoopUnrollAnalysis=true: > First loop unroll is controlled by policy_unroll_slp_analysis. When > loop unroll is 4 times, vectorization happens. > If vectorization succeeds, loop unroll would be instigated more by > set_major_progress until body_size is big enough. > if vectorization fails, loop unroll stops. > > I haven't run the performance test. I don't really understand what your point is. How can you have run the test case but not the performance test? I think that you should look at the performance in order to understand the issue. Andrew. From martin.doerr at sap.com Tue Apr 18 10:36:44 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 18 Apr 2017 10:36:44 +0000 Subject: [8u] Request for enhancement backport approval for 8144019 Message-ID: <6ec6869db4444edf87fb5f10c98d69ed@sap.com> Hi Michihiro, there are still shared code parts which should not be backported with this change. I have a couple of requests and suggestions. c1_Compilation.hpp I think it would be better to change #ifndef PPC to #ifndef PPC32 as it restricts the size unnecessarily for PPC64. In addition, the change will be PPC64 only if you remove the type cast. c1_GraphBuilder.cpp, globals.hpp I think AlwaysSafeConstructors should not get introduced in jdk8u. c1_LIRGenerator.hpp, c1_RangeCheckElimination.hpp Delete operators need to be changed to fix slow debug build on AIX (was done by a later change): void operator delete(void* p) { ShouldNotReachHere(); } void operator delete[](void* p) { ShouldNotReachHere(); } c1_Runtime1.cpp I think the PPC32 code should not get removed. Please use #ifdef PPC32 instead? deoptimization.cpp, deoptimization.hpp I think the exec_mode change should not be backported as it is a part of a jdk9 feature. As this is a major feature backport, it will motivate us to accept this change if we can get a commitment from your side that you maintain the PPC64 part of the change until end of life of jdk8u. That includes testing on all PPC64 platforms and making sure future critical bug fixes get applied. Best regards, Martin From: Michihiro Horie [mailto:HORIE at jp.ibm.com] Sent: Dienstag, 18. April 2017 05:25 To: hotspot-dev at openjdk.java.net Cc: Hiroshi H Horii ; Tim Ellison ; Gustavo Bueno Romero ; Doerr, Martin ; Simonis, Volker ; rob.mckenna at oracle.com Subject: [8u] Request for enhancement backport approval for 8144019 Hello, (I resent this message because my previous one was not listed in the hotspot-dev archive.) Would you please approve the backport of the following ppc64 change to jdk8u-dev? Since this backport includes changes in shared code, we would like hotspot team to verify that the changes will have no impact on x86. We have already confirmed these changes have no impact on x86 with jtreg. 8144019: PPC64 C1: Introduce Client Compiler Bug: https://bugs.openjdk.java.net/browse/JDK-8144019 Webrev: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/4a24de859a87 Review: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2015-December/020229.html URL: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/4a24de859a87 [Webrev for backport to 8u] http://cr.openjdk.java.net/~horii/jdk8u_support_tiered_ppc64le/hotspot/webrev02/ http://cr.openjdk.java.net/~horii/jdk8u_support_tiered_ppc64le/jdk/webrev02/ [A rationale for why the enhancement should be backported] C1 and Tiered Compilation provide faster startup for JVMs. In recent popular workloads, such as Hadoop and Spark (via Yarn), multiple JVMs concurrently run with their short life cycles. C1 and Tiered Compilation significantly shorten execution time of these workloads. For example, the above changes provide 15% improvement of execution time of Spark (2.1.0) q16 on POWER8 3.325MHz with 1.0 scale factor. [Tests] We observed the same test results in jtreg with/without above changes on x86, ppc64, and ppc64le. Also, we have run SPECjbb2015, Spark TPC-DS queries, and DaCapo benchmark suite. We confirmed all benchmarks worked well. [Risks] Most of changes in shared codes have effects on only ppc64 and ppc64le. src/share/vm/c1/c1_Compilation.hpp: 1 line changed: 0 ins; 0 del; 1 mod; 340 unchg src/share/vm/c1/c1_GraphBuilder.cpp: 10 lines changed: 9 ins; 0 del; 1 mod; 4460 unchg src/share/vm/c1/c1_IR.cpp: 2 lines changed: 1 ins; 0 del; 1 mod; 1404 unchg src/share/vm/c1/c1_IR.hpp: 7 lines changed: 6 ins; 0 del; 1 mod; 357 unchg src/share/vm/c1/c1_LIR.hpp: 2 lines changed: 0 ins; 0 del; 2 mod; 2506 unchg src/share/vm/c1/c1_LIRGenerator.cpp: 10 lines changed: 8 ins; 0 del; 2 mod; 3673 unchg src/share/vm/c1/c1_LIRGenerator.hpp: 6 lines changed: 5 ins; 0 del; 1 mod; 644 unchg src/share/vm/c1/c1_RangeCheckElimination.hpp: 7 lines changed: 6 ins; 0 del; 1 mod; 243 unchg src/share/vm/c1/c1_Runtime1.cpp: 8 lines changed: 0 ins; 7 del; 1 mod; 1483 unchg src/share/vm/runtime/deoptimization.cpp: 10 lines changed: 2 ins; 0 del; 8 mod; 2074 unchg src/share/vm/runtime/deoptimization.hpp: 5 lines changed: 1 ins; 0 del; 4 mod; 385 unchg src/share/vm/runtime/globals.hpp: 3 lines changed: 3 ins; 0 del; 0 mod; 4027 unchg Changes in src/cpu/ppc will add C1 for ppc64 and ppc64le. We believe that impacts on Interpreter and C2 are limited on ppc64 and ppc64le. Best regards, -- Michihiro, IBM Research - Tokyo From volker.simonis at gmail.com Tue Apr 18 13:09:12 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 18 Apr 2017 15:09:12 +0200 Subject: Result: New hotspot Group Member: Andrew Haley Message-ID: Voting for Andrew Haley [1] is now closed. Yes: 13 Veto: 0 Abstain: 0 According to the Bylaws definition of Three-Vote Consensus, this is sufficient to approve the nomination. Regards, Volker [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-March/thread.html#26291 From ramkri123 at gmail.com Tue Apr 18 14:34:09 2017 From: ramkri123 at gmail.com (Ram Krishnan) Date: Tue, 18 Apr 2017 07:34:09 -0700 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> Message-ID: Please find detailed proposal below, looking forward to your comments. "Minimize application tail latency using cache-partitioning-aware G1GC" -- https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbBZTclOWyg0arhuycXN94/edit Thanks, Ramki On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels wrote: > Maybe it would be better to concentrate the processor optimizations on > accessors and barrriers without introducing a completely new GC > architecture. I can imagine that especially in the area of NUMA, TLAB, huge > pages, cache consistency and possibly MMX extensions there is some > potential. > > Abandoning the global STW - while it seems like a pretty powerful change - > is I guess not a good starter exercise. Especially since it is not only a > question of mutator threads. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ------------------------------ > *From:* hotspot-gc-dev on > behalf of Ram Krishnan > *Sent:* Friday, April 14, 2017 6:36:27 AM > *To:* Asif Qamar; Andrew Haley; hotspot-gc-dev at openjdk.java.net > *Subject:* Re: linux os processor optimizations for OpenJDK GC > performance enhancement > > Thanks Andrew. > > ?>>Surely there is: a thread could have its TLAB allocated from a region > >>local to that socket (or core), and the GC thread for that region > >>could run on the same socket. It only works for young gen, but that's > >>a lot of the problem. > > A clarification -- does the TLAB allocation apply to tenured space also? > If not, the above would work only for young gen cases where there is no > promotion to tenured right? > > Thanks, > Ramki > > On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan > wrote: > >> >> ---------- Forwarded message ---------- >> From: >> ?? >> Andrew Haley >> Date: Thu, Apr 13, 2017 at 9:52 AM >> Subject: Re: linux os processor optimizations for OpenJDK GC performance >> enhancement >> To: >> ?? >> hotspot-gc-dev at openjdk.java.net >> >> >> On 13/04/17 16:33, Kim Barrett wrote: >> > An application thread may touch memory in any region; there is no >> > notion of a thread being "scoped" to a specific set of regions. While >> > it might happen that a thread would only touch regions not being >> > worked on by the collector, there is no a priori way to know that. >> >> ?? >> Surely there is: a thread could have its TLAB allocated from a region >> local to that socket (or core), and the GC thread for that region >> could run on the same socket. It only works for young gen, but that's >> a lot of the problem. >> >> Andrew. >> >> >> >> >> -- >> Thanks, >> Ramki >> > > > > -- > Thanks, > Ramki > -- Thanks, Ramki From frederic.parain at oracle.com Tue Apr 18 14:47:02 2017 From: frederic.parain at oracle.com (Frederic Parain) Date: Tue, 18 Apr 2017 10:47:02 -0400 Subject: RFR: JDK-8172791 fixing issues with Reserved Stack Area Message-ID: <3cd1c352-33fa-96fe-d2f6-a78a25c0543e@oracle.com> Greetings, Please review this fix which addresses several issues with the ReservedStackArea implementation: 1 - the method look_for_reserved_stack_annotated_method() fails to detect in-lined annotated methods, making the annotation ineffective for in-lined methods 2 - sometime an assertion failure related to reserved area status occurs after incorrect restoring of guards pages during a return from runtime Bug: https://bugs.openjdk.java.net/browse/JDK-8172791 webrev: http://cr.openjdk.java.net/~fparain/8172791/webrev.00/index.html This fix has been contributed by Andrew Haley. Thank you, Fred From david.holmes at oracle.com Tue Apr 18 20:55:24 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Apr 2017 06:55:24 +1000 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> Message-ID: <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> Hi Ramki, On 19/04/2017 12:34 AM, Ram Krishnan wrote: > Please find detailed proposal below, looking forward to your comments. > > "Minimize application tail latency using cache-partitioning-aware G1GC" -- > https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbBZTclOWyg0arhuycXN94/edit All contributions to OpenJDK need to be hosted on OpenJDK infrastructure not on external systems like the above. Also I can not see you listed as an OCA signatory. Are you an OpenJDK contributor? Thanks, David ----- > Thanks, > Ramki > > On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels > wrote: > >> Maybe it would be better to concentrate the processor optimizations on >> accessors and barrriers without introducing a completely new GC >> architecture. I can imagine that especially in the area of NUMA, TLAB, huge >> pages, cache consistency and possibly MMX extensions there is some >> potential. >> >> Abandoning the global STW - while it seems like a pretty powerful change - >> is I guess not a good starter exercise. Especially since it is not only a >> question of mutator threads. >> >> Gruss >> Bernd >> -- >> http://bernd.eckenfels.net >> ------------------------------ >> *From:* hotspot-gc-dev on >> behalf of Ram Krishnan >> *Sent:* Friday, April 14, 2017 6:36:27 AM >> *To:* Asif Qamar; Andrew Haley; hotspot-gc-dev at openjdk.java.net >> *Subject:* Re: linux os processor optimizations for OpenJDK GC >> performance enhancement >> >> Thanks Andrew. >> >> ?>>Surely there is: a thread could have its TLAB allocated from a region >>>> local to that socket (or core), and the GC thread for that region >>>> could run on the same socket. It only works for young gen, but that's >>>> a lot of the problem. >> >> A clarification -- does the TLAB allocation apply to tenured space also? >> If not, the above would work only for young gen cases where there is no >> promotion to tenured right? >> >> Thanks, >> Ramki >> >> On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan >> wrote: >> >>> >>> ---------- Forwarded message ---------- >>> From: >>> ?? >>> Andrew Haley >>> Date: Thu, Apr 13, 2017 at 9:52 AM >>> Subject: Re: linux os processor optimizations for OpenJDK GC performance >>> enhancement >>> To: >>> ?? >>> hotspot-gc-dev at openjdk.java.net >>> >>> >>> On 13/04/17 16:33, Kim Barrett wrote: >>>> An application thread may touch memory in any region; there is no >>>> notion of a thread being "scoped" to a specific set of regions. While >>>> it might happen that a thread would only touch regions not being >>>> worked on by the collector, there is no a priori way to know that. >>> >>> ?? >>> Surely there is: a thread could have its TLAB allocated from a region >>> local to that socket (or core), and the GC thread for that region >>> could run on the same socket. It only works for young gen, but that's >>> a lot of the problem. >>> >>> Andrew. >>> >>> >>> >>> >>> -- >>> Thanks, >>> Ramki >>> >> >> >> >> -- >> Thanks, >> Ramki >> > > > From igor.ignatyev at oracle.com Tue Apr 18 22:12:48 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 18 Apr 2017 15:12:48 -0700 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests Message-ID: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > 69903 lines changed: 69903 ins; 0 del; 0 mod; (69524 lines are generated) Hi all, could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 testing: applications/jcstress Thanks, -- Igor From doug.simon at oracle.com Tue Apr 18 22:13:50 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 00:13:50 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal Message-ID: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. The primary requirement for this is to remove all usage of JDK internals from Graal. While this most involves changes in Graal, there remain a few capabilities that must be exposed via JVMCI. Namely: 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize can use system properties that are guaranteed not to have been modified by application code (Graal initialization is lazy and may occur after application code has been run). 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal optimized implementation of the Truffle runtime. These capabilities have been added to jdk.vm.ci.services.Services. A comment has also been added to this class to denote the current methods to be removed in a subsequent bug to update the JDK copy of Graal with the upstream version which no longer uses the methods. The methods destined for removal are: exportJVMCITo(Class requestor) load(Class service) loadSingle(Class service, boolean required) The first one is no longer needed as JVMCI exports itself to Graal service providers it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API is only useful for JVMCI-8 where a hidden JVMCI class loader is used. These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra minor changes were required: 1. In JDK-8177663, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException on all error paths except one which throws AssertionError. The latter was a mistake and has been fixed to also throw IllegalArgumentException. 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and remove use of JDK internal API from the jdk.aot module. The qualified exports to jdk.vm.internal.compiler and jdk.aot can then be removed. -Doug https://bugs.openjdk.java.net/browse/JDK-8177845 http://cr.openjdk.java.net/~dnsimon/8177845/ [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess From ramkri123 at gmail.com Tue Apr 18 22:27:31 2017 From: ramkri123 at gmail.com (Ram Krishnan) Date: Tue, 18 Apr 2017 15:27:31 -0700 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> Message-ID: Hi David, Thanks for the clarification. I have signed the OCA and mailed it to oracle-ca_us(at)oracle.com. Any help to expedite processing would be much appreciated. We are seeing promising POC results (details in the google doc) for this proposal -- would really appreciate your help in moving this forward. Thanks, Ramki On Tue, Apr 18, 2017 at 1:55 PM, David Holmes wrote: > Hi Ramki, > > On 19/04/2017 12:34 AM, Ram Krishnan wrote: > >> Please find detailed proposal below, looking forward to your comments. >> >> "Minimize application tail latency using cache-partitioning-aware G1GC" -- >> https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbB >> ZTclOWyg0arhuycXN94/edit >> > > All contributions to OpenJDK need to be hosted on OpenJDK infrastructure > not on external systems like the above. > > Also I can not see you listed as an OCA signatory. Are you an OpenJDK > contributor? > > Thanks, > David > ----- > > Thanks, >> Ramki >> >> On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels > > >> wrote: >> >> Maybe it would be better to concentrate the processor optimizations on >>> accessors and barrriers without introducing a completely new GC >>> architecture. I can imagine that especially in the area of NUMA, TLAB, >>> huge >>> pages, cache consistency and possibly MMX extensions there is some >>> potential. >>> >>> Abandoning the global STW - while it seems like a pretty powerful change >>> - >>> is I guess not a good starter exercise. Especially since it is not only a >>> question of mutator threads. >>> >>> Gruss >>> Bernd >>> -- >>> http://bernd.eckenfels.net >>> ------------------------------ >>> *From:* hotspot-gc-dev on >>> behalf of Ram Krishnan >>> *Sent:* Friday, April 14, 2017 6:36:27 AM >>> *To:* Asif Qamar; Andrew Haley; hotspot-gc-dev at openjdk.java.net >>> *Subject:* Re: linux os processor optimizations for OpenJDK GC >>> performance enhancement >>> >>> Thanks Andrew. >>> >>> ?>>Surely there is: a thread could have its TLAB allocated from a region >>> >>>> local to that socket (or core), and the GC thread for that region >>>>> could run on the same socket. It only works for young gen, but that's >>>>> a lot of the problem. >>>>> >>>> >>> A clarification -- does the TLAB allocation apply to tenured space also? >>> If not, the above would work only for young gen cases where there is no >>> promotion to tenured right? >>> >>> Thanks, >>> Ramki >>> >>> On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan >>> wrote: >>> >>> >>>> ---------- Forwarded message ---------- >>>> From: >>>> ?? >>>> Andrew Haley >>>> Date: Thu, Apr 13, 2017 at 9:52 AM >>>> Subject: Re: linux os processor optimizations for OpenJDK GC performance >>>> enhancement >>>> To: >>>> ?? >>>> hotspot-gc-dev at openjdk.java.net >>>> >>>> >>>> On 13/04/17 16:33, Kim Barrett wrote: >>>> >>>>> An application thread may touch memory in any region; there is no >>>>> notion of a thread being "scoped" to a specific set of regions. While >>>>> it might happen that a thread would only touch regions not being >>>>> worked on by the collector, there is no a priori way to know that. >>>>> >>>> >>>> ?? >>>> Surely there is: a thread could have its TLAB allocated from a region >>>> local to that socket (or core), and the GC thread for that region >>>> could run on the same socket. It only works for young gen, but that's >>>> a lot of the problem. >>>> >>>> Andrew. >>>> >>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> Ramki >>>> >>>> >>> >>> >>> -- >>> Thanks, >>> Ramki >>> >>> >> >> >> -- Thanks, Ramki From vladimir.kozlov at oracle.com Tue Apr 18 23:34:19 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 18 Apr 2017 16:34:19 -0700 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> Message-ID: <565e6b33-844a-d08e-2e79-39f69c25e36c@oracle.com> Looks fine to me. Vladimir On 4/18/17 3:12 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >> 69903 lines changed: 69903 ins; 0 del; 0 mod; > (69524 lines are generated) > > Hi all, > > could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? > > webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 > testing: applications/jcstress > > Thanks, > -- Igor > From mandy.chung at oracle.com Wed Apr 19 00:02:43 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 18 Apr 2017 17:02:43 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> Message-ID: <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> > On Apr 18, 2017, at 3:13 PM, Doug Simon wrote: > > Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. > : > http://cr.openjdk.java.net/~dnsimon/8177845/ Thanks for making this change. This would simplify the way to replace JDK?s graal with the lab graal. A couple of comments: jdk/internal/misc/VM.java 161 * Note that the saved system properties do not include 162 * the ones set by sun.misc.Version.init(). sun.misc.Version is no longer present in JDK 9. Renamed to java.lang.VersionProps jdk/vm/ci/services/Services.java 67 Class.forName("jdk.vm.ci.runtime.JVMCI?); JVMCI class is local in jdk.internal.vm.ci module. An alternative may be to provide a static initialize method rather than Class::forName. jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java 211 for (HotSpotJVMCIBackendFactory factory : ServiceLoader.load(HotSpotJVMCIBackendFactory.class)) { This uses the thread context class loader to load providers. Services::load uses the system class loader to load providers. I suspect you want this to use the system class loader here too. jdk/vm/ci/services/JVMCIServiceLocator.java 78 for (JVMCIServiceLocator access : ServiceLoader.load(JVMCIServiceLocator.class)) { Same comment as above. I think you want to use system class loader to load providers. Mandy From david.holmes at oracle.com Wed Apr 19 00:42:02 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Apr 2017 10:42:02 +1000 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> Message-ID: Hi Ramki, On 19/04/2017 8:27 AM, Ram Krishnan wrote: > Hi David, > > Thanks for the clarification. > > I have signed the OCA and mailed it to oracle-ca_us(at)oracle.com > . Any help to expedite processing would be much > appreciated. Can't help with that I'm afraid. :) > We are seeing promising POC results (details in the google doc) for this > proposal -- would really appreciate your help in moving this forward. If you email me a text/html version of the document I can host it on cr.openjdk.java.net temporarily. For this to become a JEP you will need a sponsor with the necessary OpenJDK credentials. http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html Cheers, David > Thanks, > Ramki > > On Tue, Apr 18, 2017 at 1:55 PM, David Holmes > wrote: > > Hi Ramki, > > On 19/04/2017 12:34 AM, Ram Krishnan wrote: > > Please find detailed proposal below, looking forward to your > comments. > > "Minimize application tail latency using > cache-partitioning-aware G1GC" -- > https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbBZTclOWyg0arhuycXN94/edit > > > > All contributions to OpenJDK need to be hosted on OpenJDK > infrastructure not on external systems like the above. > > Also I can not see you listed as an OCA signatory. Are you an > OpenJDK contributor? > > Thanks, > David > ----- > > Thanks, > Ramki > > On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels > > > wrote: > > Maybe it would be better to concentrate the processor > optimizations on > accessors and barrriers without introducing a completely new GC > architecture. I can imagine that especially in the area of > NUMA, TLAB, huge > pages, cache consistency and possibly MMX extensions there > is some > potential. > > Abandoning the global STW - while it seems like a pretty > powerful change - > is I guess not a good starter exercise. Especially since it > is not only a > question of mutator threads. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ------------------------------ > *From:* hotspot-gc-dev > > on > behalf of Ram Krishnan > > *Sent:* Friday, April 14, 2017 6:36:27 AM > *To:* Asif Qamar; Andrew Haley; > hotspot-gc-dev at openjdk.java.net > > *Subject:* Re: linux os processor optimizations for OpenJDK GC > performance enhancement > > Thanks Andrew. > > ?>>Surely there is: a thread could have its TLAB allocated > from a region > > local to that socket (or core), and the GC thread > for that region > could run on the same socket. It only works for > young gen, but that's > a lot of the problem. > > > A clarification -- does the TLAB allocation apply to tenured > space also? > If not, the above would work only for young gen cases where > there is no > promotion to tenured right? > > Thanks, > Ramki > > On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan > > > wrote: > > > ---------- Forwarded message ---------- > From: > ?? > Andrew Haley > > Date: Thu, Apr 13, 2017 at 9:52 AM > Subject: Re: linux os processor optimizations for > OpenJDK GC performance > enhancement > To: > ?? > hotspot-gc-dev at openjdk.java.net > > > > On 13/04/17 16:33, Kim Barrett wrote: > > An application thread may touch memory in any > region; there is no > notion of a thread being "scoped" to a specific set > of regions. While > it might happen that a thread would only touch > regions not being > worked on by the collector, there is no a priori way > to know that. > > > ?? > Surely there is: a thread could have its TLAB allocated > from a region > local to that socket (or core), and the GC thread for > that region > could run on the same socket. It only works for young > gen, but that's > a lot of the problem. > > Andrew. > > > > > -- > Thanks, > Ramki > > > > > -- > Thanks, > Ramki > > > > > > > > -- > Thanks, > Ramki From david.holmes at oracle.com Wed Apr 19 01:28:11 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Apr 2017 11:28:11 +1000 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> Message-ID: <5a1da3e4-622b-66a4-1228-4a7e3ebbee91@oracle.com> Hi Igor, On 19/04/2017 8:12 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >> 69903 lines changed: 69903 ins; 0 del; 0 mod; > (69524 lines are generated) > > Hi all, > > could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? > > webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 > testing: applications/jcstress This looks really good! Only concern is that according to the docs jtreg doesn't seem to support multiple test declarations in a single file ?? Maybe the docs are out of date? http://openjdk.java.net/jtreg/tag-spec.html A few grammatical nits in the README: 23 The tests located under this directory run tests from Java Concurrency Stress from Java -> from the Java 24 [1] aka jcstress test suite. This suite aims to verify the correctness of Suggest: test suite [1] (a.k.a. jcstress). This suite ... 25 concurrency support in JDK. in JDK -> in the JDK 27 All the tests are run thought test driver class -- JcstressRunner, which s/thought/through the/ 29 spawns a new JVM to run one jsctress test and checks that it is finished is finished -> finishes 33 changed, one should make corresponding changes in the artifact description in in the -> to the 36 */Test.java files should never be modified directly, because they are Insert "The" at start. --- General Java style nits re long ling wrapping. You seem to be using a consistent "indent by 8" style when wrapping long lines. The "usual" style guidelines suggest a more structured indentation style, for example: - when wrapping the arguments to a call align after the opening ( of the call ie: throw new Error("TESTBUG: Can not resolve artifacts for " + JcstressRunner.class.getName(), e); - when you have chained method calls, try to align at the dots ie: return artifacts.get("org.openjdk.jcstress.jcstress-tests-all-0.3") .toAbsolutePath(); and ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(getCmd(args)) .redirectErrorStream(true) .redirectOutput(out.toFile()); --- test/applications/jcstress/TestGenerator.java 44 * This is a test generator, should be run only when jsctress is changed Typo: jsctress -> jcstress 51 * Use jcstress test suite to generates generates -> generate Strictly speaking the copyright years should only be updated when there is a substantive change to the text of the file, so updating whenever the files are regenerated is not really correct. This may need to be manually adjusted when commiting updated files ... though if there is no change other than to copyright year then the changes can simply be ignored (hg revert) and no update pushed. Thanks, David ------ > Thanks, > -- Igor > From ramkri123 at gmail.com Wed Apr 19 01:38:17 2017 From: ramkri123 at gmail.com (Ram Krishnan) Date: Tue, 18 Apr 2017 18:38:17 -0700 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> Message-ID: Hi David, Many thanks, please find attached text version of document for temporary hosting. Thanks, Ramki On Tue, Apr 18, 2017 at 5:42 PM, David Holmes wrote: > Hi Ramki, > > On 19/04/2017 8:27 AM, Ram Krishnan wrote: > >> Hi David, >> >> Thanks for the clarification. >> >> I have signed the OCA and mailed it to oracle-ca_us(at)oracle.com >> . Any help to expedite processing would be much >> appreciated. >> > > Can't help with that I'm afraid. :) > > We are seeing promising POC results (details in the google doc) for this >> proposal -- would really appreciate your help in moving this forward. >> > > If you email me a text/html version of the document I can host it on > cr.openjdk.java.net temporarily. For this to become a JEP you will need a > sponsor with the necessary OpenJDK credentials. > > http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > > Cheers, > David > > > Thanks, >> Ramki >> >> On Tue, Apr 18, 2017 at 1:55 PM, David Holmes > > wrote: >> >> Hi Ramki, >> >> On 19/04/2017 12:34 AM, Ram Krishnan wrote: >> >> Please find detailed proposal below, looking forward to your >> comments. >> >> "Minimize application tail latency using >> cache-partitioning-aware G1GC" -- >> https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbB >> ZTclOWyg0arhuycXN94/edit >> > BZTclOWyg0arhuycXN94/edit> >> >> >> All contributions to OpenJDK need to be hosted on OpenJDK >> infrastructure not on external systems like the above. >> >> Also I can not see you listed as an OCA signatory. Are you an >> OpenJDK contributor? >> >> Thanks, >> David >> ----- >> >> Thanks, >> Ramki >> >> On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels >> > >> wrote: >> >> Maybe it would be better to concentrate the processor >> optimizations on >> accessors and barrriers without introducing a completely new >> GC >> architecture. I can imagine that especially in the area of >> NUMA, TLAB, huge >> pages, cache consistency and possibly MMX extensions there >> is some >> potential. >> >> Abandoning the global STW - while it seems like a pretty >> powerful change - >> is I guess not a good starter exercise. Especially since it >> is not only a >> question of mutator threads. >> >> Gruss >> Bernd >> -- >> http://bernd.eckenfels.net >> ------------------------------ >> *From:* hotspot-gc-dev >> > > on >> behalf of Ram Krishnan > > >> *Sent:* Friday, April 14, 2017 6:36:27 AM >> *To:* Asif Qamar; Andrew Haley; >> hotspot-gc-dev at openjdk.java.net >> >> *Subject:* Re: linux os processor optimizations for OpenJDK GC >> performance enhancement >> >> Thanks Andrew. >> >> ?>>Surely there is: a thread could have its TLAB allocated >> from a region >> >> local to that socket (or core), and the GC thread >> for that region >> could run on the same socket. It only works for >> young gen, but that's >> a lot of the problem. >> >> >> A clarification -- does the TLAB allocation apply to tenured >> space also? >> If not, the above would work only for young gen cases where >> there is no >> promotion to tenured right? >> >> Thanks, >> Ramki >> >> On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan >> > >> wrote: >> >> >> ---------- Forwarded message ---------- >> From: >> ?? >> Andrew Haley > >> Date: Thu, Apr 13, 2017 at 9:52 AM >> Subject: Re: linux os processor optimizations for >> OpenJDK GC performance >> enhancement >> To: >> ?? >> hotspot-gc-dev at openjdk.java.net >> >> >> >> On 13/04/17 16:33, Kim Barrett wrote: >> >> An application thread may touch memory in any >> region; there is no >> notion of a thread being "scoped" to a specific set >> of regions. While >> it might happen that a thread would only touch >> regions not being >> worked on by the collector, there is no a priori way >> to know that. >> >> >> ?? >> Surely there is: a thread could have its TLAB allocated >> from a region >> local to that socket (or core), and the GC thread for >> that region >> could run on the same socket. It only works for young >> gen, but that's >> a lot of the problem. >> >> Andrew. >> >> >> >> >> -- >> Thanks, >> Ramki >> >> >> >> >> -- >> Thanks, >> Ramki >> >> >> >> >> >> >> >> -- >> Thanks, >> Ramki >> > -- Thanks, Ramki -------------- next part -------------- ?Minimize application tail latency using cache-partitioning-aware G1GC Author: Ram (Ramki) Krishnan (ramkri123 at gmail.com) Organization: Supportvectors on behalf of Red Hat Created: 2017/4/16 Type: Feature State: Draft Component: vm/gc Collaborators: Asif Qamar (asif at asifqamar.com), Red Hat Summary ------- Enhance G1 to improve application throughput and isolation and reduce application tail latency on systems with cache-partitioning-aware processors. The goal is to address both intra and inter JVM scenarios. Non-Goals --------- Extend cache-partitioning-awareness to work on any OS other than Linux. Extend cache-partitioning-awareness to work on any other systems other than those with Intel processors with Resource Director Technology (RDT) Cache Allocation Technology (CAT) feature [1] [4] support. Cache interaction between JVM and other applications on the same system. Motivation ---------- Caching is an important part of multi-core, multi-socket system memory access hierarchy with each level of cache improving the memory access latency by an order of magnitude. The sizes of caches are different by orders of magnitude across the hierarchy as compared to the system memory [2] [3]. Keeping the working set consistent and avoiding thrashing is critical to maximizing the cache performance and thus, overall system performance. The Last-level Cache (LLC) is a shared resource across all the cores in a socket and cause for contention among applications with differing Quality of Service (QoS) requirements running in different cores. In this context, JVM garbage collection operation performs read and/or write operations on several memory locations thus substantially thrashing the LLC working set for currently executing applications, especially the latency-sensitive ones. By isolating the JVM garbage collection operation to a certain region of the LLC, the noisy-neighbor impact on currently executing applications is minimized which improves application throughput and reduces application tail latency. Description ----------- There is a LLC cache-partitioning-manager which interfaces with the HW through the Linux kernel and maintains partitions of different sizes. The cache-partitioning-manager is an infrastructure orchestration component (this can be implemented as a system component or an external component) and will be leveraged by this proposal. With this background, there are two distinct use cases in a system which benefit from this approach. Use Case 1: Interaction between multiple JVMs Any garbage collection activity in one JVM has a substantial noisy-neighbor impact on the applications executing in other JVMs because of LLC cache working set thrashing. To address this, each JVM has an option to specify a GC LLC cache partition name as follows ?-XX: G1GcCpuLLCCachePartitionName=Part-jvm-gc-interaction? during startup (the choice of the name is strictly exemplary). The cache partition name is unique per system. All the GC threads running in the context of the JVM are mapped to use the aforementioned L3 cache partition during the entire lifetime of the JVM. Other threads running in the context of the JVM use the entire L3 cache and *are not* restricted to the aforementioned cache partition. This is achieved by calling the appropriate Intel RDT CAT APIs in the context of the JVM. There can be multiple cache partitions for JVMs with differing QoS requirements, for example one for JVMs hosting low-latency applications and another for JVMs hosting normal applications. The low-latency JVM LLC partition would typically be larger than the normal JVM LLC partition so that the low-latency GC activities are prioritized and complete faster. In a multi-socket system, the LLC cache partition could apply to more than one socket; this is a LLC cache-partitioning manager configuration option. Use Case 2: Interaction between young GC threads and application threads within a JVM Within a JVM, young GC garbage collection within a Thread Local Allocation Buffer (TLAB) *will not* prevent other application threads from running. In this scenario, the young GC garbage collection thread has a substantial noisy-neighbor impact on other executing application threads because of LLC cache working set thrashing. To address this, each JVM has an option to specify a GC LLC cache partition name as follows ?-XX: G1GcCpuLLCCachePartitionName=Part-jvm-gc-local? during startup (the choice of the name is strictly exemplary). The cache partition name is unique per system. All the young GC threads operating within a TLAB use this LLC cache partition during garbage collection. Other threads running in the context of the JVM use the entire L3 cache and *are not* restricted to the aforementioned cache partition. This is achieved by calling the appropriate Intel RDT CAT APIs in the context of the JVM. In a multi-socket system, the LLC cache partition could apply to more than one socket; this is a LLC cache-partitioning manager configuration option. If there are multiple JVMs in a system, the aforementioned cache partition is shared across them; the size is set appropriately based on the number of JVMs. Summarizing cases 1 and 2 for single and multiple JVMs in a system: Single JVM in a system ?-XX: G1GcCpuLLCCachePartitionName=Part-jvm-gc-local? for young GC threads operating within a TLAB within a JVM. Multiple JVMs in a system ?-XX: G1GcCpuLLCCachePartitionName=Part-jvm-gc-local? for young GC threads operating within a TLAB within a JVM. This cache partition is shared across multiple JVMs in a system. The size is set appropriately based on the number of JVMs and the probability of concurrent garbage collection events. ?-XX: G1GcCpuLLCCachePartitionName=Part-jvm-gc-interaction? for all GC threads within a JVM. This cache partition is shared across multiple JVMs in a system. The size is set appropriately based on the number of JVMs and the probability of concurrent garbage collection events. G1 Analytics Enhancements The following G1 Analytics enhancements will be integrated into G1 for troubleshooting and performance tuning - Intel RDT Cache Monitoring Technology (CMT) [5] for monitoring cache partition usage - Intel RDT Memory Bandwidth Monitoring (MBM) [6] for monitoring memory bandwidth usage Testing ------- Normal testing with the flags (?-XX: G1GcCpuLLCCachePartitionName=AA?, ?-XX: G1GcCpuLLCCachePartitionName=BB??) should flush out any correctness issues. Testing needs specific hardware which is described in [4]. Performance testing of a system would involve fine tuning of the number of cache partitions and their sizes based on the applications in a JVM and the number of JVMs leveraging G1 analytics enhancements and existing Linux perf tools for monitoring cache hits and misses. LLC cache partitioning will likely increase the memory accesses and thus memory bandwidth. This has to be carefully measured and fine tuned using G1 analytics enhancements and existing Linux perf tools. Risks and Assumptions --------------------- The cache partitioning function depends on the CPU microarchitecture with the assumption that the memory addresses are uniformly distributed over the partition with a high probability. This behavior might vary with different generations of microarchitectures from the same processor vendor and across different processor vendors. Performance testing leveraging G1 Analytics enhancements as described above will address this. Proof of Concept ---------------- To demonstrate the value of cache partitioning for G1GC, a proof of concept (POC) was put together, details are below. The POC results clearly demonstrate that LLC cache partitioning is clearly beneficial for G1GC. In this case LLC maps to L3 cache. Happy to share the code patch if there is interest. POC code changes - LLC Cache partition of size specified by -LLCCachePartitionPercent= of L3 cache of total size 25600 Kbytes was created - All G1GC threads were mapped to this cache partition for their entire lifetime POC results A Jtreg test hotspot/test/gc/g1/TestStringDeduplicationFullGC.java was was repeatedly executed with different LLC cache partition sizes. As the partition size reduces, G1GC takes substantially more time to complete. * Partition percent - 5% - approx. time - 10.7 seconds * Partition percent - 10% - approx. time - 6.7 seconds * Partition percent - 25% - approx. time - 5.8 seconds * Partition percent - 50% - approx. time - 5.6 seconds * Partition percent - 100% (entire cache) - approx. time - 5.5 seconds POC system configuration unknown485d60a3a27d.attlocal.net:/root->lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 20 On-line CPU(s) list: 0-19 Thread(s) per core: 2 Core(s) per socket: 10 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 79 Model name: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz Stepping: 1 CPU MHz: 1200.170 BogoMIPS: 4394.42 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 25600K NUMA node0 CPU(s): 0-19 unknown485d60a3a27d.attlocal.net:/root->uname -a Linux unknown485d60a3a27d.attlocal.net 4.11.0-rc5+ #5 SMP Tue Apr 4 12:59:58 PDT 2017 x86_64 x86_64 x86_64 GNU/Linux Follow-up work -------------- G1 self-tuning to dynamically refine cache partition: Fine tuning of the number of cache partitions and their sizes based on the applications in a JVM and the number of JVMs involves substantial manual effort. By using predictive analytics techniques such as time series analysis on the G1 analytics data, the appropriate number of cache partitions and their sizes can be automatically determined as a function time. L1/L2 cache partitioning: Extend the LLC partitioning to L1 and/or cache partitioning to further improve GC performance especially for young GC garbage collection within a TLAB. LLC code segment partitioning: Leverage Intel RDT Code Data Partitioning (CDP) [7] to protect latency-sensitive application code segments in the LLC to further improve GC performance. Orchestration Ecosystem Integration ----------------------------------- Popular orchestration ecosystems such as Kubernetes support Intel RDT as part of node capability discovery [8]. LLC cache partitioning aware JVMs naturally integrate into this ecosystem. References ---------- [1] ?Intel RDT CAT,? https://software.intel.com/en-us/articles/introduction-to-cache-allocation-technology [2] ?Intel Broadwell Microarchitecture,? https://en.wikichip.org/wiki/intel/microarchitectures/broadwell [3] ?Cavium ThunderX,? http://www.cavium.com/pdfFiles/ThunderX_PB_p12_Rev1.pdf [4] ?Intel(R) RDT hardware support,? https://github.com/01org/intel-cmt-cat/blob/master/README [5] ?Intel RDT CMT,? https://software.intel.com/en-us/blogs/2014/06/18/benefit-of-cache-monitoring [6] ?Intel RDT MBM,? https://software.intel.com/en-us/articles/introduction-to-memory-bandwidth-monitoring [7] ?Intel RDT CDP,? https://software.intel.com/en-us/articles/introduction-to-code-and-data-prioritization-with-usage-models [8] ?Kubernetes Node Feature Discovery,? https://github.com/kubernetes-incubator/node-feature-discovery From igor.ignatyev at oracle.com Wed Apr 19 02:51:42 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 18 Apr 2017 19:51:42 -0700 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <5a1da3e4-622b-66a4-1228-4a7e3ebbee91@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <5a1da3e4-622b-66a4-1228-4a7e3ebbee91@oracle.com> Message-ID: <33C90E84-6872-4DC4-B6F9-74055D943876@oracle.com> Hi David, thank you for your comments, I've tried to address them all, please look at the incremental webrev: http://cr.openjdk.java.net/~iignatyev/8178788/webrev.00-01/index.html regarding multiple test declarations, jtreg supports that, but in the current version it's not fully/officially supported and hence it's not documented. Jon has fixed a few jtreg bugs[1-3] related to this and AFAIK is working towards full support, so I hope it will be supported and documented in the next promoted build of jtreg. [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7900144 : Implement multiple tests in one file in jtreg [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7901914 : Failure to find multiple @test in a single file [3] https://bugs.openjdk.java.net/browse/CODETOOLS-7901940 : support multiple @test in one test file Thanks, -- Igor > On Apr 18, 2017, at 6:28 PM, David Holmes wrote: > > Hi Igor, > > On 19/04/2017 8:12 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>> 69903 lines changed: 69903 ins; 0 del; 0 mod; >> (69524 lines are generated) >> >> Hi all, >> >> could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 >> testing: applications/jcstress > > This looks really good! Only concern is that according to the docs jtreg doesn't seem to support multiple test declarations in a single file ?? Maybe the docs are out of date? > > http://openjdk.java.net/jtreg/tag-spec.html > > A few grammatical nits in the README: > > 23 The tests located under this directory run tests from Java Concurrency Stress > > from Java -> from the Java > > 24 [1] aka jcstress test suite. This suite aims to verify the correctness of > > Suggest: > > test suite [1] (a.k.a. jcstress). This suite ... > > 25 concurrency support in JDK. > > in JDK -> in the JDK > > 27 All the tests are run thought test driver class -- JcstressRunner, which > > s/thought/through the/ > > 29 spawns a new JVM to run one jsctress test and checks that it is finished > > is finished -> finishes > > 33 changed, one should make corresponding changes in the artifact description in > > in the -> to the > > 36 */Test.java files should never be modified directly, because they are > > Insert "The" at start. > > --- > > General Java style nits re long ling wrapping. You seem to be using a consistent "indent by 8" style when wrapping long lines. The "usual" style guidelines suggest a more structured indentation style, for example: > > - when wrapping the arguments to a call align after the opening ( of the call ie: > > throw new Error("TESTBUG: Can not resolve artifacts for " > + JcstressRunner.class.getName(), e); > > - when you have chained method calls, try to align at the dots ie: > > return artifacts.get("org.openjdk.jcstress.jcstress-tests-all-0.3") > .toAbsolutePath(); > > and > > ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(getCmd(args)) > .redirectErrorStream(true) > .redirectOutput(out.toFile()); > > --- > > test/applications/jcstress/TestGenerator.java > > 44 * This is a test generator, should be run only when jsctress is changed > > Typo: jsctress -> jcstress > > 51 * Use jcstress test suite to generates > > generates -> generate > > Strictly speaking the copyright years should only be updated when there is a substantive change to the text of the file, so updating whenever the files are regenerated is not really correct. This may need to be manually adjusted when commiting updated files ... though if there is no change other than to copyright year then the changes can simply be ignored (hg revert) and no update pushed. > > Thanks, > David > ------ > >> Thanks, >> -- Igor >> From frank.yuan at oracle.com Wed Apr 19 03:32:49 2017 From: frank.yuan at oracle.com (Frank Yuan) Date: Wed, 19 Apr 2017 11:32:49 +0800 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> Message-ID: <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> Hi Igor This is really a valuable work! Would you like to pull your artifacts up to the common repo hg.openjdk.java.net/jdk10/hs/test? That more people(e.g. corelibs :)) can benefit from this. Thanks Frank Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests Date: Tue, 18 Apr 2017 15:12:48 -0700 From: Igor Ignatyev To: hotspot-compiler-dev at openjdk.java.net compiler , HotSpot Open Source Developers http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > 69903 lines changed: 69903 ins; 0 del; 0 mod; (69524 lines are generated) Hi all, could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 testing: applications/jcstress Thanks, -- Igor From igor.ignatyev at oracle.com Wed Apr 19 05:47:28 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 18 Apr 2017 22:47:28 -0700 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> Message-ID: <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> Hi Felix, currently, /test is not a jtreg testbase and it is not supported by make and the rest of infra, therefore it's not possible to have tests there. I don't want to mix up this fix w/ infra/make related work, and would prefer to think about this movement after (or as a part of) JEP 296 Consolidate the JDK Forest into a Single Repository. actually, corelibs people are more than welcome to run hotspot/test, we (hotspot team) run jdk/test quite often and would really love to see other people running our tests before pushing changes which can affect hotspot and/or its tests. -- Igor > On Apr 18, 2017, at 8:32 PM, Frank Yuan wrote: > > Hi Igor > > This is really a valuable work! > > Would you like to pull your artifacts up to the common repo hg.openjdk.java.net/jdk10/hs/test? That more people(e.g. corelibs J) can benefit from this. > > Thanks > Frank > > > Subject: > RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests > Date: > Tue, 18 Apr 2017 15:12:48 -0700 > From: > Igor Ignatyev > To: > hotspot-compiler-dev at openjdk.java.net compiler , HotSpot Open Source Developers > > > http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > > 69903 lines changed: 69903 ins; 0 del; 0 mod; > (69524 lines are generated) > > Hi all, > > could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? > > webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 > testing: applications/jcstress > > Thanks, > -- Igor > From david.holmes at oracle.com Wed Apr 19 06:08:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Apr 2017 16:08:14 +1000 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> Message-ID: <15bf57e1-a53b-07c1-1216-deacdb0d8231@oracle.com> On 19/04/2017 11:38 AM, Ram Krishnan wrote: > Hi David, > > Many thanks, please find attached text version of document for temporary > hosting. Hosted at: http://cr.openjdk.java.net/~dholmes/JEP-cache-partitioning-v1.txt David > > Thanks, > Ramki > > On Tue, Apr 18, 2017 at 5:42 PM, David Holmes > wrote: > > Hi Ramki, > > On 19/04/2017 8:27 AM, Ram Krishnan wrote: > > Hi David, > > Thanks for the clarification. > > I have signed the OCA and mailed it to > oracle-ca_us(at)oracle.com > . Any help to expedite processing would be much > appreciated. > > > Can't help with that I'm afraid. :) > > We are seeing promising POC results (details in the google doc) > for this > proposal -- would really appreciate your help in moving this > forward. > > > If you email me a text/html version of the document I can host it on > cr.openjdk.java.net temporarily. For > this to become a JEP you will need a sponsor with the necessary > OpenJDK credentials. > > http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > > > Cheers, > David > > > Thanks, > Ramki > > On Tue, Apr 18, 2017 at 1:55 PM, David Holmes > > >> wrote: > > Hi Ramki, > > On 19/04/2017 12:34 AM, Ram Krishnan wrote: > > Please find detailed proposal below, looking forward to your > comments. > > "Minimize application tail latency using > cache-partitioning-aware G1GC" -- > > https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbBZTclOWyg0arhuycXN94/edit > > > > > > > All contributions to OpenJDK need to be hosted on OpenJDK > infrastructure not on external systems like the above. > > Also I can not see you listed as an OCA signatory. Are you an > OpenJDK contributor? > > Thanks, > David > ----- > > Thanks, > Ramki > > On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels > > >> > wrote: > > Maybe it would be better to concentrate the processor > optimizations on > accessors and barrriers without introducing a > completely new GC > architecture. I can imagine that especially in the > area of > NUMA, TLAB, huge > pages, cache consistency and possibly MMX extensions > there > is some > potential. > > Abandoning the global STW - while it seems like a pretty > powerful change - > is I guess not a good starter exercise. Especially > since it > is not only a > question of mutator threads. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ------------------------------ > *From:* hotspot-gc-dev > > >> on > behalf of Ram Krishnan > >> > *Sent:* Friday, April 14, 2017 6:36:27 AM > *To:* Asif Qamar; Andrew Haley; > hotspot-gc-dev at openjdk.java.net > > > > *Subject:* Re: linux os processor optimizations for > OpenJDK GC > performance enhancement > > Thanks Andrew. > > ?>>Surely there is: a thread could have its TLAB > allocated > from a region > > local to that socket (or core), and the GC > thread > for that region > could run on the same socket. It only works for > young gen, but that's > a lot of the problem. > > > A clarification -- does the TLAB allocation apply to > tenured > space also? > If not, the above would work only for young gen > cases where > there is no > promotion to tenured right? > > Thanks, > Ramki > > On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan > > >> > wrote: > > > ---------- Forwarded message ---------- > From: > ?? > Andrew Haley >> > Date: Thu, Apr 13, 2017 at 9:52 AM > Subject: Re: linux os processor optimizations for > OpenJDK GC performance > enhancement > To: > ?? > hotspot-gc-dev at openjdk.java.net > > > > > > On 13/04/17 16:33, Kim Barrett wrote: > > An application thread may touch memory in any > region; there is no > notion of a thread being "scoped" to a > specific set > of regions. While > it might happen that a thread would only touch > regions not being > worked on by the collector, there is no a > priori way > to know that. > > > ?? > Surely there is: a thread could have its TLAB > allocated > from a region > local to that socket (or core), and the GC > thread for > that region > could run on the same socket. It only works for > young > gen, but that's > a lot of the problem. > > Andrew. > > > > > -- > Thanks, > Ramki > > > > > -- > Thanks, > Ramki > > > > > > > > -- > Thanks, > Ramki > > > > > -- > Thanks, > Ramki From david.holmes at oracle.com Wed Apr 19 06:17:28 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Apr 2017 16:17:28 +1000 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <33C90E84-6872-4DC4-B6F9-74055D943876@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <5a1da3e4-622b-66a4-1228-4a7e3ebbee91@oracle.com> <33C90E84-6872-4DC4-B6F9-74055D943876@oracle.com> Message-ID: <28eafe0f-c951-7a24-2fa0-7e994db7beec@oracle.com> Hi Igor, Thanks for the jtreg info! You missed a couple of things in the README: 27 All the tests are run thought test driver class -- JcstressRunner, which s/thought/through the/ And I just noticed an additional typo: 29 spawns a new JVM to run one jsctress test jsctress -> jcstress Thanks, David ----- On 19/04/2017 12:51 PM, Igor Ignatyev wrote: > Hi David, > > thank you for your comments, I've tried to address them all, please look at the incremental webrev: http://cr.openjdk.java.net/~iignatyev/8178788/webrev.00-01/index.html > > regarding multiple test declarations, jtreg supports that, but in the current version it's not fully/officially supported and hence it's not documented. Jon has fixed a few jtreg bugs[1-3] related to this and AFAIK is working towards full support, so I hope it will be supported and documented in the next promoted build of jtreg. > > [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7900144 : Implement multiple tests in one file in jtreg > [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7901914 : Failure to find multiple @test in a single file > [3] https://bugs.openjdk.java.net/browse/CODETOOLS-7901940 : support multiple @test in one test file > > Thanks, > -- Igor > >> On Apr 18, 2017, at 6:28 PM, David Holmes wrote: >> >> Hi Igor, >> >> On 19/04/2017 8:12 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>>> 69903 lines changed: 69903 ins; 0 del; 0 mod; >>> (69524 lines are generated) >>> >>> Hi all, >>> >>> could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 >>> testing: applications/jcstress >> >> This looks really good! Only concern is that according to the docs jtreg doesn't seem to support multiple test declarations in a single file ?? Maybe the docs are out of date? > >> >> http://openjdk.java.net/jtreg/tag-spec.html >> >> A few grammatical nits in the README: >> >> 23 The tests located under this directory run tests from Java Concurrency Stress >> >> from Java -> from the Java >> >> 24 [1] aka jcstress test suite. This suite aims to verify the correctness of >> >> Suggest: >> >> test suite [1] (a.k.a. jcstress). This suite ... >> >> 25 concurrency support in JDK. >> >> in JDK -> in the JDK >> >> 27 All the tests are run thought test driver class -- JcstressRunner, which >> >> s/thought/through the/ >> >> 29 spawns a new JVM to run one jsctress test and checks that it is finished >> >> is finished -> finishes >> >> 33 changed, one should make corresponding changes in the artifact description in >> >> in the -> to the >> >> 36 */Test.java files should never be modified directly, because they are >> >> Insert "The" at start. >> >> --- >> >> General Java style nits re long ling wrapping. You seem to be using a consistent "indent by 8" style when wrapping long lines. The "usual" style guidelines suggest a more structured indentation style, for example: >> >> - when wrapping the arguments to a call align after the opening ( of the call ie: >> >> throw new Error("TESTBUG: Can not resolve artifacts for " >> + JcstressRunner.class.getName(), e); >> >> - when you have chained method calls, try to align at the dots ie: >> >> return artifacts.get("org.openjdk.jcstress.jcstress-tests-all-0.3") >> .toAbsolutePath(); >> >> and >> >> ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(getCmd(args)) >> .redirectErrorStream(true) >> .redirectOutput(out.toFile()); >> >> --- >> >> test/applications/jcstress/TestGenerator.java >> >> 44 * This is a test generator, should be run only when jsctress is changed >> >> Typo: jsctress -> jcstress >> >> 51 * Use jcstress test suite to generates >> >> generates -> generate >> >> Strictly speaking the copyright years should only be updated when there is a substantive change to the text of the file, so updating whenever the files are regenerated is not really correct. This may need to be manually adjusted when commiting updated files ... though if there is no change other than to copyright year then the changes can simply be ignored (hg revert) and no update pushed. >> >> Thanks, >> David >> ------ >> >>> Thanks, >>> -- Igor >>> > From frank.yuan at oracle.com Wed Apr 19 06:39:01 2017 From: frank.yuan at oracle.com (Frank Yuan) Date: Wed, 19 Apr 2017 14:39:01 +0800 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> Message-ID: <029e01d2b8d7$a3211760$e9634620$@oracle.com> Hi Igor The artifact which I talked about is the wrapper(JcstressRunner.java), I supposed there would be more people to create their tests with jcstress facility if the wrapper is put to a common place, there have been e.g. failure_handler, lib in hg.openjdk.java.net/jdk10/hs/test. If we put the wrapper into hotspot repo, even after JEP 296, it would be under test/hotspot/. Certainly, it's up to you:) Thanks Frank From: Igor Ignatyev [mailto:igor.ignatyev at oracle.com] Sent: Wednesday, April 19, 2017 1:47 PM To: Frank Yuan Cc: HotSpot Open Source Developers Subject: Re: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests Hi Felix, currently, /test is not a jtreg testbase and it is not supported by make and the rest of infra, therefore it's not possible to have tests there. I don't want to mix up this fix w/ infra/make related work, and would prefer to think about this movement after (or as a part of) JEP 296 Consolidate the JDK Forest into a Single Repository. actually, corelibs people are more than welcome to run hotspot/test, we (hotspot team) run jdk/test quite often and would really love to see other people running our tests before pushing changes which can affect hotspot and/or its tests. -- Igor On Apr 18, 2017, at 8:32 PM, Frank Yuan > wrote: Hi Igor This is really a valuable work! Would you like to pull your artifacts up to the common repo hg.openjdk.java.net/jdk10/hs/test? That more people(e.g. corelibs :)) can benefit from this. Thanks Frank Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests Date: Tue, 18 Apr 2017 15:12:48 -0700 From: Igor Ignatyev To: hotspot-compiler-dev at openjdk.java.net compiler , HotSpot Open Source Developers http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > 69903 lines changed: 69903 ins; 0 del; 0 mod; (69524 lines are generated) Hi all, could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 testing: applications/jcstress Thanks, -- Igor From kim.barrett at oracle.com Wed Apr 19 06:40:35 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 19 Apr 2017 02:40:35 -0400 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> <1213C931-FABD-4BF3-994C-3DA9E9F5D9EA@oracle.com> Message-ID: > On Apr 11, 2017, at 1:45 PM, Kim Barrett wrote: > >> On Apr 10, 2017, at 2:21 PM, Kim Barrett wrote: >> >>> On Apr 10, 2017, at 12:48 PM, Stefan Karlsson wrote: >>> To be consistent with the other code it in that file (res & 1) == 0 should be !(res & 1). >> >> That would be contrary to hotspot style guide; I think that should take precedence. > > Stefan and I discussed this privately, and I?ve agreed to go with the locally consistent style here, > even though it?s contrary to the hotspot style guide. Part of my decision process is related to > the next part of this reply. Here?s the updated webrev: http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.02/ Need a second reviewer, in addition to waiting for Stefan. From doug.simon at oracle.com Wed Apr 19 07:06:38 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 09:06:38 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> Message-ID: <385D6A4A-0C1F-4CCD-8806-5665052A81AA@oracle.com> > On 19 Apr 2017, at 02:02, Mandy Chung wrote: > > >> On Apr 18, 2017, at 3:13 PM, Doug Simon wrote: >> >> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >> : >> http://cr.openjdk.java.net/~dnsimon/8177845/ > > Thanks for making this change. This would simplify the way to replace JDK?s graal with the lab graal. A couple of comments: > > jdk/internal/misc/VM.java > > 161 * Note that the saved system properties do not include > 162 * the ones set by sun.misc.Version.init(). > > sun.misc.Version is no longer present in JDK 9. Renamed to java.lang.VersionProps Sorry, the webrev was out of date. I've updated it and also fixed the comment for VM::getSavedProperty. > jdk/vm/ci/services/Services.java > 67 Class.forName("jdk.vm.ci.runtime.JVMCI?); > > JVMCI class is local in jdk.internal.vm.ci module. An alternative may be > to provide a static initialize method rather than Class::forName. JVMCI is broken into fine grained "projects" and the jdk.vm.ci.runtime[1] project depends on the jdk.vm.ci.services project[2] so I cannot make a direct reference without introducing a circular dependency. I don't expect the reflection cost to be significant in practice. > jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java > 211 for (HotSpotJVMCIBackendFactory factory : ServiceLoader.load(HotSpotJVMCIBackendFactory.class)) { > > This uses the thread context class loader to load providers. Services::load uses > the system class loader to load providers. I suspect you want this to use the > system class loader here too. > > jdk/vm/ci/services/JVMCIServiceLocator.java > 78 for (JVMCIServiceLocator access : ServiceLoader.load(JVMCIServiceLocator.class)) { > > Same comment as above. I think you want to use system class loader to load providers. Yes, I think you're right. I've updated the webrev with that change. Thanks for the review. -Doug [1] http://hg.openjdk.java.net/jdk9/dev/hotspot/file/560d7aa083a2/.mx.jvmci/suite.py#l94 [2] http://hg.openjdk.java.net/jdk9/dev/hotspot/file/560d7aa083a2/.mx.jvmci/suite.py#l45 From doug.simon at oracle.com Wed Apr 19 07:16:08 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 09:16:08 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal Message-ID: (adding hotspot-compiler-dev) Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. The primary requirement for this is to remove all usage of JDK internals from Graal. While this most involves changes in Graal, there remain a few capabilities that must be exposed via JVMCI. Namely: 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize can use system properties that are guaranteed not to have been modified by application code (Graal initialization is lazy and may occur after application code has been run). 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal optimized implementation of the Truffle runtime. These capabilities have been added to jdk.vm.ci.services.Services. A comment has also been added to this class to denote the current methods to be removed in a subsequent bug to update the JDK copy of Graal with the upstream version which no longer uses the methods. The methods destined for removal are: exportJVMCITo(Class requestor) load(Class service) loadSingle(Class service, boolean required) The first one is no longer needed as JVMCI exports itself to Graal service providers it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API is only useful for JVMCI-8 where a hidden JVMCI class loader is used. These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra minor changes were required: 1. In JDK-8177673, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException on all error paths except one which throws AssertionError. The latter was a mistake and has been fixed to also throw IllegalArgumentException. 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and remove use of JDK internal API from the jdk.aot module. The qualified exports to jdk.vm.internal.compiler and jdk.aot can then be removed. -Doug https://bugs.openjdk.java.net/browse/JDK-8177845 http://cr.openjdk.java.net/~dnsimon/8177845/ [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess From doug.simon at oracle.com Wed Apr 19 07:18:03 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 09:18:03 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> Message-ID: (adding hotspot-compiler-dev) > On 19 Apr 2017, at 02:02, Mandy Chung wrote: > > >> On Apr 18, 2017, at 3:13 PM, Doug Simon wrote: >> >> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >> : >> http://cr.openjdk.java.net/~dnsimon/8177845/ > > Thanks for making this change. This would simplify the way to replace JDK?s graal with the lab graal. A couple of comments: > > jdk/internal/misc/VM.java > > 161 * Note that the saved system properties do not include > 162 * the ones set by sun.misc.Version.init(). > > sun.misc.Version is no longer present in JDK 9. Renamed to java.lang.VersionProps Sorry, the webrev was out of date. I've updated it and also fixed the comment for VM::getSavedProperty. > jdk/vm/ci/services/Services.java > 67 Class.forName("jdk.vm.ci.runtime.JVMCI?); > > JVMCI class is local in jdk.internal.vm.ci module. An alternative may be > to provide a static initialize method rather than Class::forName. JVMCI is broken into fine grained "projects" and the jdk.vm.ci.runtime[1] project depends on the jdk.vm.ci.services project[2] so I cannot make a direct reference without introducing a circular dependency. I don't expect the reflection cost to be significant in practice. > jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java > 211 for (HotSpotJVMCIBackendFactory factory : ServiceLoader.load(HotSpotJVMCIBackendFactory.class)) { > > This uses the thread context class loader to load providers. Services::load uses > the system class loader to load providers. I suspect you want this to use the > system class loader here too. > > jdk/vm/ci/services/JVMCIServiceLocator.java > 78 for (JVMCIServiceLocator access : ServiceLoader.load(JVMCIServiceLocator.class)) { > > Same comment as above. I think you want to use system class loader to load providers. Yes, I think you're right. I've updated the webrev with that change. Thanks for the review. -Doug [1] http://hg.openjdk.java.net/jdk9/dev/hotspot/file/560d7aa083a2/.mx.jvmci/suite.py#l94 [2] http://hg.openjdk.java.net/jdk9/dev/hotspot/file/560d7aa083a2/.mx.jvmci/suite.py#l45 From peter.levart at gmail.com Wed Apr 19 07:37:38 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 19 Apr 2017 09:37:38 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> Message-ID: Hi Doug, Mandy, Just a minor comment on the VM class changes... Note that Properties class is thread-safe, while HashMap isn't. But I think this should not be a problem here, as during initPhase1(), as far as I know, no threads apart from main thread are started yet (is this true?), so anyone accessing getSavedProperty(ies) will either be the main thread itself or a thread started afterwards, so there is a happens-before relationship. If this is true, then we could wrap the copy of saved properties with an unmodifiable map view before setting the savedProps field so that getSavedProperties() could always return the same instance. Like for example in the following alternative change: http://cr.openjdk.java.net/~plevart/jdk9-dev/8177845_VM.getSavedProperties/webrev.01/ What do you think? Regards, Peter On 04/19/2017 02:02 AM, Mandy Chung wrote: >> On Apr 18, 2017, at 3:13 PM, Doug Simon wrote: >> >> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >> : >> http://cr.openjdk.java.net/~dnsimon/8177845/ > Thanks for making this change. This would simplify the way to replace JDK?s graal with the lab graal. A couple of comments: > > jdk/internal/misc/VM.java > > 161 * Note that the saved system properties do not include > 162 * the ones set by sun.misc.Version.init(). > > sun.misc.Version is no longer present in JDK 9. Renamed to java.lang.VersionProps > > jdk/vm/ci/services/Services.java > 67 Class.forName("jdk.vm.ci.runtime.JVMCI?); > > JVMCI class is local in jdk.internal.vm.ci module. An alternative may be > to provide a static initialize method rather than Class::forName. > > jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java > 211 for (HotSpotJVMCIBackendFactory factory : ServiceLoader.load(HotSpotJVMCIBackendFactory.class)) { > > This uses the thread context class loader to load providers. Services::load uses > the system class loader to load providers. I suspect you want this to use the > system class loader here too. > > jdk/vm/ci/services/JVMCIServiceLocator.java > 78 for (JVMCIServiceLocator access : ServiceLoader.load(JVMCIServiceLocator.class)) { > > Same comment as above. I think you want to use system class loader to load providers. > > Mandy From peter.levart at gmail.com Wed Apr 19 07:42:27 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 19 Apr 2017 09:42:27 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> Message-ID: <49b6a3ac-da15-7635-ab26-07d21de19b40@gmail.com> On 04/19/2017 09:37 AM, Peter Levart wrote: > > > http://cr.openjdk.java.net/~plevart/jdk9-dev/8177845_VM.getSavedProperties/webrev.01/ > Also, while we are at it, the following javadocs in the getSavedProperty() do not apply any more: 138 * It accesses a private copy of the system properties so 139 * that user's locking of the system properties object will not 140 * cause the library to deadlock. In JDK 9, Properties class does not use locking any more on the Properties instance for get()/getProperty() methods... Regards, Peter From peter.levart at gmail.com Wed Apr 19 08:12:59 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 19 Apr 2017 10:12:59 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <49b6a3ac-da15-7635-ab26-07d21de19b40@gmail.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> <49b6a3ac-da15-7635-ab26-07d21de19b40@gmail.com> Message-ID: On 04/19/2017 09:42 AM, Peter Levart wrote: > > > On 04/19/2017 09:37 AM, Peter Levart wrote: >> >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/8177845_VM.getSavedProperties/webrev.01/ >> > > Also, while we are at it, the following javadocs in the > getSavedProperty() do not apply any more: > > 138 * It accesses a private copy of the system properties so > 139 * that user's locking of the system properties object will not > 140 * cause the library to deadlock. > > In JDK 9, Properties class does not use locking any more on the > Properties instance for get()/getProperty() methods... > > Regards, Peter > I also noticed the following comment: // TODO: the Property Management needs to be refactored and // the appropriate prop keys need to be accessible to the // calling classes to avoid duplication of keys. private static Map savedProps; ...which is not entirely true. Neither keys nor values are duplicated (they are just referenced in the new copy of the Properties/Map object). What is duplicated is an excessive amount of internal objects, such as array slots and Map.Entry objects. If this is a concern, then we could use the new immutable Map implementation that is available in JDK 9, so the following lines in my webrev: 181 @SuppressWarnings("unchecked") 182 Map sp = new HashMap<>((Map)props); 183 // only main thread is running at this time, so savedProps and 184 // its content will be correctly published to threads started later 185 savedProps = Collections.unmodifiableMap(sp); Could be changed into: @SuppressWarnings("unchecked") Map sp = Map.ofEntries(props.entrySet().toArray(new Map.Entry[0])); // only main thread is running at this time, so savedProps // will be correctly published to threads started later savedProps = sp; ...to save some excessive space (the implementation is a linear-probe hashtable which keeps keys and values directly in an array without wrapping them with Map.Entry objects). Regards, Peter From doug.simon at oracle.com Wed Apr 19 09:25:54 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 11:25:54 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> <49b6a3ac-da15-7635-ab26-07d21de19b40@gmail.com> Message-ID: Hi Peter, All of your suggestions look good. I've updated http://cr.openjdk.java.net/~dnsimon/8177845/jdk/src/java.base/share/classes/jdk/internal/misc/VM.java.udiff.html to include them (please check I didn't make any copy errors in the process). I was not aware of the new Map.ofEntries method. Nice to see more space efficient map implementations being added to the JDK. Thanks! -Doug > On 19 Apr 2017, at 10:12, Peter Levart wrote: > > > > On 04/19/2017 09:42 AM, Peter Levart wrote: >> >> >> On 04/19/2017 09:37 AM, Peter Levart wrote: >>> >>> >>> http://cr.openjdk.java.net/~plevart/jdk9-dev/8177845_VM.getSavedProperties/webrev.01/ >> >> Also, while we are at it, the following javadocs in the getSavedProperty() do not apply any more: >> >> 138 * It accesses a private copy of the system properties so >> 139 * that user's locking of the system properties object will not >> 140 * cause the library to deadlock. >> >> In JDK 9, Properties class does not use locking any more on the Properties instance for get()/getProperty() methods... >> >> Regards, Peter >> > > I also noticed the following comment: > > // TODO: the Property Management needs to be refactored and > // the appropriate prop keys need to be accessible to the > // calling classes to avoid duplication of keys. > private static Map savedProps; > > ...which is not entirely true. Neither keys nor values are duplicated (they are just referenced in the new copy of the Properties/Map object). What is duplicated is an excessive amount of internal objects, such as array slots and Map.Entry objects. If this is a concern, then we could use the new immutable Map implementation that is available in JDK 9, so the following lines in my webrev: > > 181 @SuppressWarnings("unchecked") > 182 Map sp = new HashMap<>((Map)props); > 183 // only main thread is running at this time, so savedProps and > 184 // its content will be correctly published to threads started later > 185 savedProps = Collections.unmodifiableMap(sp); > > Could be changed into: > > @SuppressWarnings("unchecked") > Map sp = > Map.ofEntries(props.entrySet().toArray(new Map.Entry[0])); > // only main thread is running at this time, so savedProps > // will be correctly published to threads started later > savedProps = sp; > > ...to save some excessive space (the implementation is a linear-probe hashtable which keeps keys and values directly in an array without wrapping them with Map.Entry objects). > > Regards, Peter > From peter.levart at gmail.com Wed Apr 19 09:49:08 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 19 Apr 2017 11:49:08 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> <49b6a3ac-da15-7635-ab26-07d21de19b40@gmail.com> Message-ID: Hi Simon, On 04/19/2017 11:25 AM, Doug Simon wrote: > Hi Peter, > > All of your suggestions look good. I've updated http://cr.openjdk.java.net/~dnsimon/8177845/jdk/src/java.base/share/classes/jdk/internal/misc/VM.java.udiff.html to include them (please check I didn't make any copy errors in the process). Looks good. > > I was not aware of the new Map.ofEntries method. Nice to see more space efficient map implementations being added to the JDK. Admittedly, I used this method in a way not envisioned by the author. Maybe there's a reason there is no Map.copyOf(Map) method there, which would make this even simpler. If there was one, it would be too easy to (mis)use it instead of Collections.unmodifiableMap(Map), albeit with a slightly different semantics, and force re-hashing-copying of big maps where there is no need to do that. But it would be a pretty nice replacement for the following idiom: Collections.unmodifiableMap(new HashMap<>(someMap)) Regards, Peter > > Thanks! > > -Doug > >> On 19 Apr 2017, at 10:12, Peter Levart wrote: >> >> >> >> On 04/19/2017 09:42 AM, Peter Levart wrote: >>> >>> On 04/19/2017 09:37 AM, Peter Levart wrote: >>>> >>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/8177845_VM.getSavedProperties/webrev.01/ >>> Also, while we are at it, the following javadocs in the getSavedProperty() do not apply any more: >>> >>> 138 * It accesses a private copy of the system properties so >>> 139 * that user's locking of the system properties object will not >>> 140 * cause the library to deadlock. >>> >>> In JDK 9, Properties class does not use locking any more on the Properties instance for get()/getProperty() methods... >>> >>> Regards, Peter >>> >> I also noticed the following comment: >> >> // TODO: the Property Management needs to be refactored and >> // the appropriate prop keys need to be accessible to the >> // calling classes to avoid duplication of keys. >> private static Map savedProps; >> >> ...which is not entirely true. Neither keys nor values are duplicated (they are just referenced in the new copy of the Properties/Map object). What is duplicated is an excessive amount of internal objects, such as array slots and Map.Entry objects. If this is a concern, then we could use the new immutable Map implementation that is available in JDK 9, so the following lines in my webrev: >> >> 181 @SuppressWarnings("unchecked") >> 182 Map sp = new HashMap<>((Map)props); >> 183 // only main thread is running at this time, so savedProps and >> 184 // its content will be correctly published to threads started later >> 185 savedProps = Collections.unmodifiableMap(sp); >> >> Could be changed into: >> >> @SuppressWarnings("unchecked") >> Map sp = >> Map.ofEntries(props.entrySet().toArray(new Map.Entry[0])); >> // only main thread is running at this time, so savedProps >> // will be correctly published to threads started later >> savedProps = sp; >> >> ...to save some excessive space (the implementation is a linear-probe hashtable which keeps keys and values directly in an array without wrapping them with Map.Entry objects). >> >> Regards, Peter >> From stefan.karlsson at oracle.com Wed Apr 19 10:01:52 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 19 Apr 2017 12:01:52 +0200 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> <1213C931-FABD-4BF3-994C-3DA9E9F5D9EA@oracle.com> Message-ID: On 2017-04-19 08:40, Kim Barrett wrote: >> On Apr 11, 2017, at 1:45 PM, Kim Barrett wrote: >> >>> On Apr 10, 2017, at 2:21 PM, Kim Barrett wrote: >>> >>>> On Apr 10, 2017, at 12:48 PM, Stefan Karlsson wrote: >>>> To be consistent with the other code it in that file (res & 1) == 0 should be !(res & 1). >>> >>> That would be contrary to hotspot style guide; I think that should take precedence. >> >> Stefan and I discussed this privately, and I?ve agreed to go with the locally consistent style here, >> even though it?s contrary to the hotspot style guide. Part of my decision process is related to >> the next part of this reply. > > Here?s the updated webrev: > http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.02/ > > Need a second reviewer, in addition to waiting for Stefan. > Looks good. Thanks, StefanK From peter.levart at gmail.com Wed Apr 19 10:01:51 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 19 Apr 2017 12:01:51 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> <49b6a3ac-da15-7635-ab26-07d21de19b40@gmail.com> Message-ID: <8a7d3c56-0190-710a-8f21-0f957ebb95a2@gmail.com> On 04/19/2017 11:49 AM, Peter Levart wrote: >> I was not aware of the new Map.ofEntries method. Nice to see more >> space efficient map implementations being added to the JDK. > > Admittedly, I used this method in a way not envisioned by the author. > Maybe there's a reason there is no Map.copyOf(Map) method there, which > would make this even simpler. If there was one, it would be too easy > to (mis)use it instead of Collections.unmodifiableMap(Map), albeit > with a slightly different semantics, and force re-hashing-copying of > big maps where there is no need to do that. But it would be a pretty > nice replacement for the following idiom: > > Collections.unmodifiableMap(new HashMap<>(someMap)) Sometimes I wish that besides public, protected, "package-private" and private, Java also had an "expert protected" access modifier ;-) Regards, Peter From Alan.Bateman at oracle.com Wed Apr 19 13:41:24 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Apr 2017 14:41:24 +0100 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> Message-ID: On 19/04/2017 08:37, Peter Levart wrote: > : > > Note that Properties class is thread-safe, while HashMap isn't. But I > think this should not be a problem here, as during initPhase1(), as > far as I know, no threads apart from main thread are started yet (is > this true?), so anyone accessing getSavedProperty(ies) will either be > the main thread itself or a thread started afterwards, so there is a > happens-before relationship. The ReferenceHandler and Finalizer threads are started early, before initPhase1. However, both should immediately block and in the case of the Finalizer, await the completion of initPhase1 before polling. -Alan From Alan.Bateman at oracle.com Wed Apr 19 14:04:10 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 19 Apr 2017 15:04:10 +0100 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> Message-ID: <56bb9632-e8d7-60be-0832-fa86c20a36ef@oracle.com> On 18/04/2017 23:13, Doug Simon wrote: > : > > https://bugs.openjdk.java.net/browse/JDK-8177845 > http://cr.openjdk.java.net/~dnsimon/8177845/ > I'm happy to see jdk.internal.vm.compiler changing to be an upgradable module and the patching foo going away. If I read the changes correctly then I can extend JVMCIServiceLocator and the construction of my sub-class will open up all packages in jdk.internal.vm.ci to me. It is there any to tie this to -XX:+EnableJVMCI? -Alan From ramkri123 at gmail.com Wed Apr 19 14:04:40 2017 From: ramkri123 at gmail.com (Ram Krishnan) Date: Wed, 19 Apr 2017 07:04:40 -0700 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: <15bf57e1-a53b-07c1-1216-deacdb0d8231@oracle.com> References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> <15bf57e1-a53b-07c1-1216-deacdb0d8231@oracle.com> Message-ID: Many thanks David. Thanks, Ramki On Tue, Apr 18, 2017 at 11:08 PM, David Holmes wrote: > On 19/04/2017 11:38 AM, Ram Krishnan wrote: > >> Hi David, >> >> Many thanks, please find attached text version of document for temporary >> hosting. >> > > Hosted at: http://cr.openjdk.java.net/~dholmes/JEP-cache-partitioning- > v1.txt > > David > > >> Thanks, >> Ramki >> >> On Tue, Apr 18, 2017 at 5:42 PM, David Holmes > > wrote: >> >> Hi Ramki, >> >> On 19/04/2017 8:27 AM, Ram Krishnan wrote: >> >> Hi David, >> >> Thanks for the clarification. >> >> I have signed the OCA and mailed it to >> oracle-ca_us(at)oracle.com >> . Any help to expedite processing would be >> much >> appreciated. >> >> >> Can't help with that I'm afraid. :) >> >> We are seeing promising POC results (details in the google doc) >> for this >> proposal -- would really appreciate your help in moving this >> forward. >> >> >> If you email me a text/html version of the document I can host it on >> cr.openjdk.java.net temporarily. For >> this to become a JEP you will need a sponsor with the necessary >> OpenJDK credentials. >> >> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html >> >> >> Cheers, >> David >> >> >> Thanks, >> Ramki >> >> On Tue, Apr 18, 2017 at 1:55 PM, David Holmes >> >> > >> wrote: >> >> Hi Ramki, >> >> On 19/04/2017 12:34 AM, Ram Krishnan wrote: >> >> Please find detailed proposal below, looking forward to >> your >> comments. >> >> "Minimize application tail latency using >> cache-partitioning-aware G1GC" -- >> >> https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbB >> ZTclOWyg0arhuycXN94/edit >> > BZTclOWyg0arhuycXN94/edit> >> >> > BZTclOWyg0arhuycXN94/edit >> > BZTclOWyg0arhuycXN94/edit>> >> >> >> All contributions to OpenJDK need to be hosted on OpenJDK >> infrastructure not on external systems like the above. >> >> Also I can not see you listed as an OCA signatory. Are you an >> OpenJDK contributor? >> >> Thanks, >> David >> ----- >> >> Thanks, >> Ramki >> >> On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels >> >> >> >> wrote: >> >> Maybe it would be better to concentrate the processor >> optimizations on >> accessors and barrriers without introducing a >> completely new GC >> architecture. I can imagine that especially in the >> area of >> NUMA, TLAB, huge >> pages, cache consistency and possibly MMX extensions >> there >> is some >> potential. >> >> Abandoning the global STW - while it seems like a >> pretty >> powerful change - >> is I guess not a good starter exercise. Especially >> since it >> is not only a >> question of mutator threads. >> >> Gruss >> Bernd >> -- >> http://bernd.eckenfels.net >> ------------------------------ >> *From:* hotspot-gc-dev >> > >> > >> on >> behalf of Ram Krishnan > >> > >> >> *Sent:* Friday, April 14, 2017 6:36:27 AM >> *To:* Asif Qamar; Andrew Haley; >> hotspot-gc-dev at openjdk.java.net >> >> > > >> *Subject:* Re: linux os processor optimizations for >> OpenJDK GC >> performance enhancement >> >> Thanks Andrew. >> >> ?>>Surely there is: a thread could have its TLAB >> allocated >> from a region >> >> local to that socket (or core), and the GC >> thread >> for that region >> could run on the same socket. It only works >> for >> young gen, but that's >> a lot of the problem. >> >> >> A clarification -- does the TLAB allocation apply to >> tenured >> space also? >> If not, the above would work only for young gen >> cases where >> there is no >> promotion to tenured right? >> >> Thanks, >> Ramki >> >> On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan >> >> >> >> wrote: >> >> >> ---------- Forwarded message ---------- >> From: >> ?? >> Andrew Haley > > >> >> Date: Thu, Apr 13, 2017 at 9:52 AM >> Subject: Re: linux os processor optimizations for >> OpenJDK GC performance >> enhancement >> To: >> ?? >> hotspot-gc-dev at openjdk.java.net >> >> > > >> >> >> On 13/04/17 16:33, Kim Barrett wrote: >> >> An application thread may touch memory in any >> region; there is no >> notion of a thread being "scoped" to a >> specific set >> of regions. While >> it might happen that a thread would only touch >> regions not being >> worked on by the collector, there is no a >> priori way >> to know that. >> >> >> ?? >> Surely there is: a thread could have its TLAB >> allocated >> from a region >> local to that socket (or core), and the GC >> thread for >> that region >> could run on the same socket. It only works for >> young >> gen, but that's >> a lot of the problem. >> >> Andrew. >> >> >> >> >> -- >> Thanks, >> Ramki >> >> >> >> >> -- >> Thanks, >> Ramki >> >> >> >> >> >> >> >> -- >> Thanks, >> Ramki >> >> >> >> >> -- >> Thanks, >> Ramki >> > -- Thanks, Ramki From doug.simon at oracle.com Wed Apr 19 14:16:44 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 16:16:44 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <56bb9632-e8d7-60be-0832-fa86c20a36ef@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <56bb9632-e8d7-60be-0832-fa86c20a36ef@oracle.com> Message-ID: <28A2A0B1-3898-43DB-BFA1-011B64D2CE69@oracle.com> > On 19 Apr 2017, at 16:04, Alan Bateman wrote: > > On 18/04/2017 23:13, Doug Simon wrote: > >> : >> >> https://bugs.openjdk.java.net/browse/JDK-8177845 >> http://cr.openjdk.java.net/~dnsimon/8177845/ >> > I'm happy to see jdk.internal.vm.compiler changing to be an upgradable module and the patching foo going away. Yes, I'm delighted to see the command line required for using upstream Graal shrinking! > If I read the changes correctly then I can extend JVMCIServiceLocator and the construction of my sub-class will open up all packages in jdk.internal.vm.ci to me. It is there any to tie this to -XX:+EnableJVMCI? We could but I'm not sure what it would buy you. The service lookup only originates from the JVMCI runtime and the initialization of JVMCI already checks EnableJVMCI[1] -Doug [1] http://hg.openjdk.java.net/jdk9/dev/hotspot/file/4368832d1991/src/share/vm/jvmci/jvmciRuntime.cpp#l634 From peter.levart at gmail.com Wed Apr 19 15:03:04 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 19 Apr 2017 17:03:04 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> Message-ID: Hi Alan, On 04/19/2017 03:41 PM, Alan Bateman wrote: > On 19/04/2017 08:37, Peter Levart wrote: > >> : >> >> Note that Properties class is thread-safe, while HashMap isn't. But I >> think this should not be a problem here, as during initPhase1(), as >> far as I know, no threads apart from main thread are started yet (is >> this true?), so anyone accessing getSavedProperty(ies) will either be >> the main thread itself or a thread started afterwards, so there is a >> happens-before relationship. > The ReferenceHandler and Finalizer threads are started early, before > initPhase1. However, both should immediately block and in the case of > the Finalizer, await the completion of initPhase1 before polling. > > -Alan Just out of curiosity, what guarantees are there that no code before VM.saveAndRemoveProperties() ever executes Integer.valueOf(int) method for example? Note that this call is implicitly hidden in autoboxing of int(s)... If this happens, then class initialization of Integer.IntegerCache may fail, because it is using VM.getSavedProperty(): public static String getSavedProperty(String key) { if (savedProps.isEmpty()) throw new IllegalStateException("Should be non-empty if initialized"); return savedProps.getProperty(key); } Hm.... Peter From kim.barrett at oracle.com Wed Apr 19 16:49:59 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 19 Apr 2017 12:49:59 -0400 Subject: RFR: 8178352: BitMap::get_next_zero_offset may give wrong result on Mac In-Reply-To: References: <4a3f5ef6-0ce0-19da-1288-01271ddf9041@oracle.com> <10D07D0F-0050-49E4-9C74-9803727EFF73@oracle.com> <1213C931-FABD-4BF3-994C-3DA9E9F5D9EA@oracle.com> Message-ID: <0B37DCC8-88C4-413A-880D-63AB6BD99D59@oracle.com> > On Apr 19, 2017, at 6:01 AM, Stefan Karlsson wrote: > > On 2017-04-19 08:40, Kim Barrett wrote: >> Here?s the updated webrev: >> http://cr.openjdk.java.net/~kbarrett/8178352/hotspot.02/ >> >> Need a second reviewer, in addition to waiting for Stefan. >> > > Looks good. Thanks. From harold.seigel at oracle.com Wed Apr 19 16:57:57 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 19 Apr 2017 12:57:57 -0400 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> Message-ID: <7b8acb29-2ef4-0229-b791-78988bf30432@oracle.com> Hi Igor, How long will these tests typically take to run? Thanks, Harold On 4/19/2017 1:47 AM, Igor Ignatyev wrote: > Hi Felix, > > currently, /test is not a jtreg testbase and it is not supported by make and the rest of infra, therefore it's not possible to have tests there. I don't want to mix up this fix w/ infra/make related work, and would prefer to think about this movement after (or as a part of) JEP 296 Consolidate the JDK Forest into a Single Repository. > > actually, corelibs people are more than welcome to run hotspot/test, we (hotspot team) run jdk/test quite often and would really love to see other people running our tests before pushing changes which can affect hotspot and/or its tests. > > -- Igor > > >> On Apr 18, 2017, at 8:32 PM, Frank Yuan wrote: >> >> Hi Igor >> >> This is really a valuable work! >> >> Would you like to pull your artifacts up to the common repo hg.openjdk.java.net/jdk10/hs/test? That more people(e.g. corelibs J) can benefit from this. >> >> Thanks >> Frank >> >> >> Subject: >> RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests >> Date: >> Tue, 18 Apr 2017 15:12:48 -0700 >> From: >> Igor Ignatyev >> To: >> hotspot-compiler-dev at openjdk.java.net compiler , HotSpot Open Source Developers >> >> >> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>> 69903 lines changed: 69903 ins; 0 del; 0 mod; >> (69524 lines are generated) >> >> Hi all, >> >> could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 >> testing: applications/jcstress >> >> Thanks, >> -- Igor >> From george.triantafillou at oracle.com Wed Apr 19 17:43:09 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 19 Apr 2017 13:43:09 -0400 Subject: RFR(xs) JDK-8163861 JNI NewString() and GetStringLength() documentation incorrect Message-ID: Please review this very small fix to the spec for JDK-8163861: JBS: https://bugs.openjdk.java.net/browse/JDK-8163861 webrev: http://cr.openjdk.java.net/~gtriantafill/8163861-webrev/webrev The change provides a more precise definition for NewString() and GetStringLength(). Thanks. -George From igor.ignatyev at oracle.com Wed Apr 19 17:47:42 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 19 Apr 2017 10:47:42 -0700 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <7b8acb29-2ef4-0229-b791-78988bf30432@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> <7b8acb29-2ef4-0229-b791-78988bf30432@oracle.com> Message-ID: <4B673007-C1CF-4F06-934D-05AC5DFA6286@oracle.com> Hi Harold, on my laptop, one test takes about 50 seconds to finish, so all 11554 tests will take around 577 700 seconds, which is about 1 week of machine time. I should have mentioned that at the very begging, we are not proposing to run them very frequently. -- Igor > On Apr 19, 2017, at 9:57 AM, harold seigel wrote: > > Hi Igor, > > How long will these tests typically take to run? > > Thanks, Harold > > > On 4/19/2017 1:47 AM, Igor Ignatyev wrote: >> Hi Felix, >> >> currently, /test is not a jtreg testbase and it is not supported by make and the rest of infra, therefore it's not possible to have tests there. I don't want to mix up this fix w/ infra/make related work, and would prefer to think about this movement after (or as a part of) JEP 296 Consolidate the JDK Forest into a Single Repository. >> >> actually, corelibs people are more than welcome to run hotspot/test, we (hotspot team) run jdk/test quite often and would really love to see other people running our tests before pushing changes which can affect hotspot and/or its tests. >> >> -- Igor >> >> >>> On Apr 18, 2017, at 8:32 PM, Frank Yuan wrote: >>> >>> Hi Igor >>> This is really a valuable work! >>> Would you like to pull your artifacts up to the common repo hg.openjdk.java.net/jdk10/hs/test? > That more people(e.g. corelibs J) can benefit from this. >>> Thanks >>> Frank >>> Subject: >>> RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests >>> Date: >>> Tue, 18 Apr 2017 15:12:48 -0700 >>> From: >>> Igor Ignatyev > > >>> To: >>> hotspot-compiler-dev at openjdk.java.net > compiler > >, HotSpot Open Source Developers > > >>> >>> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > >>>> 69903 lines changed: 69903 ins; 0 del; 0 mod; >>> (69524 lines are generated) >>> Hi all, >>> could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? >>> webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 > >>> testing: applications/jcstress >>> Thanks, >>> -- Igor From mandy.chung at oracle.com Wed Apr 19 18:01:33 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Apr 2017 11:01:33 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> Message-ID: <078DD9E5-1A87-4F82-8698-3E19757DEDE4@oracle.com> > On Apr 19, 2017, at 8:03 AM, Peter Levart wrote: > > Hi Alan, > > On 04/19/2017 03:41 PM, Alan Bateman wrote: >> On 19/04/2017 08:37, Peter Levart wrote: >> >>> : >>> >>> Note that Properties class is thread-safe, while HashMap isn't. But I think this should not be a problem here, as during initPhase1(), as far as I know, no threads apart from main thread are started yet (is this true?), so anyone accessing getSavedProperty(ies) will either be the main thread itself or a thread started afterwards, so there is a happens-before relationship. >> The ReferenceHandler and Finalizer threads are started early, before initPhase1. However, both should immediately block and in the case of the Finalizer, await the completion of initPhase1 before polling. >> >> -Alan > > Just out of curiosity, what guarantees are there that no code before VM.saveAndRemoveProperties() ever executes Integer.valueOf(int) method for example? Note that this call is implicitly hidden in autoboxing of int(s)... > > If this happens, then class initialization of Integer.IntegerCache may fail, because it is using VM.getSavedProperty(): > > public static String getSavedProperty(String key) { > if (savedProps.isEmpty()) > throw new IllegalStateException("Should be non-empty if initialized"); > > return savedProps.getProperty(key); > } > > Hm.... We should catch this issue and detect if VM.getSavedProperty is called when init level < 1 during early startup. Mandy From igor.ignatyev at oracle.com Wed Apr 19 18:06:07 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 19 Apr 2017 11:06:07 -0700 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <029e01d2b8d7$a3211760$e9634620$@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> <029e01d2b8d7$a3211760$e9634620$@oracle.com> Message-ID: Hi Felix, I understand your point and use-case and believe it is important and useful, however I don't think we should do that now. 1st of all, current jcstress wrapper is able to run precompiled tests from jsctress-tests-all, so if people write their own jcstress based tests, they will still need to use jcstress to build and run them. 2nd, hotspot/test/applications/jcstress tests depend on the version of jcstress, and currently the version is defined in wrapper, so I'd prefer to have the wrapper as much closer to these tests as possible. that is to say, if we have high need for running custom jcstress tests, then we should invest more in jcstress driver/wrapper (and maybe infra) so it will be possible to have tests in jcstress format as part of our test bases, but this is out of scope of this RFE. Thanks, -- Igor > On Apr 18, 2017, at 11:39 PM, Frank Yuan wrote: > > Hi Igor > > The artifact which I talked about is the wrapper(JcstressRunner.java), I supposed there would be more people to create their tests with jcstress facility if the wrapper is put to a common place, there have been e.g. failure_handler, lib in hg.openjdk.java.net/jdk10/hs/test . > > If we put the wrapper into hotspot repo, even after JEP 296, it would be under test/hotspot/? > > Certainly, it?s up to youJ > > > Thanks > Frank > > From: Igor Ignatyev [mailto:igor.ignatyev at oracle.com] > Sent: Wednesday, April 19, 2017 1:47 PM > To: Frank Yuan > Cc: HotSpot Open Source Developers > Subject: Re: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests > > Hi Felix, > > currently, /test is not a jtreg testbase and it is not supported by make and the rest of infra, therefore it's not possible to have tests there. I don't want to mix up this fix w/ infra/make related work, and would prefer to think about this movement after (or as a part of) JEP 296 Consolidate the JDK Forest into a Single Repository. > > actually, corelibs people are more than welcome to run hotspot/test, we (hotspot team) run jdk/test quite often and would really love to see other people running our tests before pushing changes which can affect hotspot and/or its tests. > > -- Igor > > >> On Apr 18, 2017, at 8:32 PM, Frank Yuan > wrote: >> >> Hi Igor >> >> This is really a valuable work! >> >> Would you like to pull your artifacts up to the common repo hg.openjdk.java.net/jdk10/hs/test? That more people(e.g. corelibs J) can benefit from this. >> >> Thanks >> Frank >> >> >> Subject: >> RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests >> Date: >> Tue, 18 Apr 2017 15:12:48 -0700 >> From: >> Igor Ignatyev >> To: >> hotspot-compiler-dev at openjdk.java.net compiler , HotSpot Open Source Developers >> >> >> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >> > 69903 lines changed: 69903 ins; 0 del; 0 mod; >> (69524 lines are generated) >> >> Hi all, >> >> could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 >> testing: applications/jcstress >> >> Thanks, >> -- Igor From cthalinger at twitter.com Wed Apr 19 18:24:48 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Wed, 19 Apr 2017 08:24:48 -1000 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: Message-ID: I lost track a bit but why are we exporting jdk.vm.ci.services to the world now? module jdk.internal.vm.ci { - exports jdk.vm.ci.services to jdk.internal.vm.compiler; + exports jdk.vm.ci.services; > On Apr 18, 2017, at 9:16 PM, Doug Simon wrote: > > (adding hotspot-compiler-dev) > > Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. > The primary requirement for this is to remove all usage of JDK internals from Graal. > While this most involves changes in Graal, there remain a few capabilities that must > be exposed via JVMCI. Namely: > > 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize > can use system properties that are guaranteed not to have been modified by application > code (Graal initialization is lazy and may occur after application code has been run). > > 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal > optimized implementation of the Truffle runtime. > > These capabilities have been added to jdk.vm.ci.services.Services. A comment has > also been added to this class to denote the current methods to be removed > in a subsequent bug to update the JDK copy of Graal with the upstream version which > no longer uses the methods. The methods destined for removal are: > > exportJVMCITo(Class requestor) > load(Class service) > loadSingle(Class service, boolean required) > > The first one is no longer needed as JVMCI exports itself to Graal service providers > it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. > The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API > is only useful for JVMCI-8 where a hidden JVMCI class loader is used. > > These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra > minor changes were required: > > 1. In JDK-8177673, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException > on all error paths except one which throws AssertionError. The latter was a mistake and has been > fixed to also throw IllegalArgumentException. > 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. > Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. > This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. > > As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and > remove use of JDK internal API from the jdk.aot module. The qualified exports to > jdk.vm.internal.compiler and jdk.aot can then be removed. > > -Doug > > https://bugs.openjdk.java.net/browse/JDK-8177845 > http://cr.openjdk.java.net/~dnsimon/8177845/ > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess > > > From mandy.chung at oracle.com Wed Apr 19 18:38:04 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Apr 2017 11:38:04 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: Message-ID: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. Mandy > On Apr 19, 2017, at 11:24 AM, Christian Thalinger wrote: > > I lost track a bit but why are we exporting jdk.vm.ci.services to the world now? > > module jdk.internal.vm.ci { > - exports jdk.vm.ci.services to jdk.internal.vm.compiler; > + exports jdk.vm.ci.services; > >> On Apr 18, 2017, at 9:16 PM, Doug Simon wrote: >> >> (adding hotspot-compiler-dev) >> >> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >> The primary requirement for this is to remove all usage of JDK internals from Graal. >> While this most involves changes in Graal, there remain a few capabilities that must >> be exposed via JVMCI. Namely: >> >> 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize >> can use system properties that are guaranteed not to have been modified by application >> code (Graal initialization is lazy and may occur after application code has been run). >> >> 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal >> optimized implementation of the Truffle runtime. >> >> These capabilities have been added to jdk.vm.ci.services.Services. A comment has >> also been added to this class to denote the current methods to be removed >> in a subsequent bug to update the JDK copy of Graal with the upstream version which >> no longer uses the methods. The methods destined for removal are: >> >> exportJVMCITo(Class requestor) >> load(Class service) >> loadSingle(Class service, boolean required) >> >> The first one is no longer needed as JVMCI exports itself to Graal service providers >> it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. >> The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API >> is only useful for JVMCI-8 where a hidden JVMCI class loader is used. >> >> These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra >> minor changes were required: >> >> 1. In JDK-8177673, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException >> on all error paths except one which throws AssertionError. The latter was a mistake and has been >> fixed to also throw IllegalArgumentException. >> 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. >> Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. >> This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. >> >> As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and >> remove use of JDK internal API from the jdk.aot module. The qualified exports to >> jdk.vm.internal.compiler and jdk.aot can then be removed. >> >> -Doug >> >> https://bugs.openjdk.java.net/browse/JDK-8177845 >> http://cr.openjdk.java.net/~dnsimon/8177845/ >> >> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess >> >> >> > From cthalinger at twitter.com Wed Apr 19 18:55:29 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Wed, 19 Apr 2017 08:55:29 -1000 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> Message-ID: <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> > On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: > > Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. > > The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? After all, this is a security issue. > > Mandy > >> On Apr 19, 2017, at 11:24 AM, Christian Thalinger wrote: >> >> I lost track a bit but why are we exporting jdk.vm.ci.services to the world now? >> >> module jdk.internal.vm.ci { >> - exports jdk.vm.ci.services to jdk.internal.vm.compiler; >> + exports jdk.vm.ci.services; >> >>> On Apr 18, 2017, at 9:16 PM, Doug Simon wrote: >>> >>> (adding hotspot-compiler-dev) >>> >>> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >>> The primary requirement for this is to remove all usage of JDK internals from Graal. >>> While this most involves changes in Graal, there remain a few capabilities that must >>> be exposed via JVMCI. Namely: >>> >>> 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize >>> can use system properties that are guaranteed not to have been modified by application >>> code (Graal initialization is lazy and may occur after application code has been run). >>> >>> 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal >>> optimized implementation of the Truffle runtime. >>> >>> These capabilities have been added to jdk.vm.ci.services.Services. A comment has >>> also been added to this class to denote the current methods to be removed >>> in a subsequent bug to update the JDK copy of Graal with the upstream version which >>> no longer uses the methods. The methods destined for removal are: >>> >>> exportJVMCITo(Class requestor) >>> load(Class service) >>> loadSingle(Class service, boolean required) >>> >>> The first one is no longer needed as JVMCI exports itself to Graal service providers >>> it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. >>> The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API >>> is only useful for JVMCI-8 where a hidden JVMCI class loader is used. >>> >>> These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra >>> minor changes were required: >>> >>> 1. In JDK-8177673, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException >>> on all error paths except one which throws AssertionError. The latter was a mistake and has been >>> fixed to also throw IllegalArgumentException. >>> 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. >>> Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. >>> This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. >>> >>> As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and >>> remove use of JDK internal API from the jdk.aot module. The qualified exports to >>> jdk.vm.internal.compiler and jdk.aot can then be removed. >>> >>> -Doug >>> >>> https://bugs.openjdk.java.net/browse/JDK-8177845 >>> http://cr.openjdk.java.net/~dnsimon/8177845/ >>> >>> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess >>> >>> >>> >> > From kim.barrett at oracle.com Wed Apr 19 18:59:09 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 19 Apr 2017 14:59:09 -0400 Subject: RFR: 8178836: Improve PtrQueue index abstraction Message-ID: <596AF348-8332-4EE4-9A79-B4016D18AF7E@oracle.com> Please review this API cleanup of PtrQueue and related classes. PtrQueue internally represents buffer indices as byte offsets, so that generated code for queue insertion doesn't need to deal with scaling. Most clients (other than the queue insertion code generators) don't need to know about that representation, and are simplified by dealing with indices and sizes in the natural units of an array of pointers. This change eliminates the leakage of the byte offset representation to those other clients. CR: https://bugs.openjdk.java.net/browse/JDK-8178836 Webrev: http://cr.openjdk.java.net/~kbarrett/8178836/hotspot.00/ Testing: JPRT rbt hs-tier2, hs-tier{3,4}-gc From mandy.chung at oracle.com Wed Apr 19 19:04:49 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Apr 2017 12:04:49 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> Message-ID: > On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: > > >> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >> >> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >> >> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. > > This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? It's unqualified and no restriction in this change. Mandy From kim.barrett at oracle.com Wed Apr 19 19:04:56 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 19 Apr 2017 15:04:56 -0400 Subject: RFR: 8178836: Improve PtrQueue index abstraction In-Reply-To: <596AF348-8332-4EE4-9A79-B4016D18AF7E@oracle.com> References: <596AF348-8332-4EE4-9A79-B4016D18AF7E@oracle.com> Message-ID: [oops, wrong list. this was supposed to go to hotspot-gc-dev] > On Apr 19, 2017, at 2:59 PM, Kim Barrett wrote: > > Please review this API cleanup of PtrQueue and related classes. > > PtrQueue internally represents buffer indices as byte offsets, so that > generated code for queue insertion doesn't need to deal with scaling. > > Most clients (other than the queue insertion code generators) don't > need to know about that representation, and are simplified by dealing > with indices and sizes in the natural units of an array of pointers. > > This change eliminates the leakage of the byte offset representation > to those other clients. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8178836 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8178836/hotspot.00/ > > Testing: > JPRT > rbt hs-tier2, hs-tier{3,4}-gc From doug.simon at oracle.com Wed Apr 19 19:07:06 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 21:07:06 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <078DD9E5-1A87-4F82-8698-3E19757DEDE4@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> <078DD9E5-1A87-4F82-8698-3E19757DEDE4@oracle.com> Message-ID: <2FDA9652-F376-439B-AFA6-DF929DB3D403@oracle.com> > On 19 Apr 2017, at 20:01, Mandy Chung wrote: > >> >> On Apr 19, 2017, at 8:03 AM, Peter Levart wrote: >> >> Hi Alan, >> >> On 04/19/2017 03:41 PM, Alan Bateman wrote: >>> On 19/04/2017 08:37, Peter Levart wrote: >>> >>>> : >>>> >>>> Note that Properties class is thread-safe, while HashMap isn't. But I think this should not be a problem here, as during initPhase1(), as far as I know, no threads apart from main thread are started yet (is this true?), so anyone accessing getSavedProperty(ies) will either be the main thread itself or a thread started afterwards, so there is a happens-before relationship. >>> The ReferenceHandler and Finalizer threads are started early, before initPhase1. However, both should immediately block and in the case of the Finalizer, await the completion of initPhase1 before polling. >>> >>> -Alan >> >> Just out of curiosity, what guarantees are there that no code before VM.saveAndRemoveProperties() ever executes Integer.valueOf(int) method for example? Note that this call is implicitly hidden in autoboxing of int(s)... >> >> If this happens, then class initialization of Integer.IntegerCache may fail, because it is using VM.getSavedProperty(): >> >> public static String getSavedProperty(String key) { >> if (savedProps.isEmpty()) >> throw new IllegalStateException("Should be non-empty if initialized"); >> >> return savedProps.getProperty(key); >> } >> >> Hm.... > > We should catch this issue and detect if VM.getSavedProperty is called when init level < 1 during early startup. What are you proposing? Some extra testing of VM startup to see if this is an issue? If so, can you please suggest what testing is to be performed. -Doug From mandy.chung at oracle.com Wed Apr 19 19:20:10 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Apr 2017 12:20:10 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <2FDA9652-F376-439B-AFA6-DF929DB3D403@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <808007C7-8B14-46BB-A932-9EA9DE2285A9@oracle.com> <078DD9E5-1A87-4F82-8698-3E19757DEDE4@oracle.com> <2FDA9652-F376-439B-AFA6-DF929DB3D403@oracle.com> Message-ID: <08B17CC8-CAA5-44AD-BD85-ED8B72326C73@oracle.com> > On Apr 19, 2017, at 12:07 PM, Doug Simon wrote: > > >> On 19 Apr 2017, at 20:01, Mandy Chung wrote: >> >> We should catch this issue and detect if VM.getSavedProperty is called when init level < 1 during early startup. > > What are you proposing? Some extra testing of VM startup to see if this is an issue? If so, can you please suggest what testing is to be performed. This is an existing issue, not related to your change. I will create JBS issue to track it. What you have is fine. Mandy From doug.simon at oracle.com Wed Apr 19 19:27:38 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 21:27:38 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> Message-ID: > On 19 Apr 2017, at 21:04, Mandy Chung wrote: > > >> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >> >> >>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>> >>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>> >>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >> >> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? > > It's unqualified and no restriction in this change. The public methods currently in jdk.vm.ci.services are: 1. JVMCIServiceLocator.getProvider(Class) 2. JVMCIServiceLocator.getProviders(Class) 3. Services.initializeJVMCI() 4. Services.getSavedProperties() 5. Services.exportJVMCITo(Class) 6. Services.load(Class) 7. Services.loadSingle(Class, boolean) 1 should be made protected. I'll update the webrev with this change. 2 should check for JVMCIPermission. I'll update the webrev with this change. 3 is harmless from a security perspective in my opinion. 4 checks for JVMCIPermission. 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). Does this address the security concerns? -Doug From cthalinger at twitter.com Wed Apr 19 19:40:19 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Wed, 19 Apr 2017 09:40:19 -1000 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> Message-ID: > On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: > >> >> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >> >> >>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>> >>> >>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>> >>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>> >>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>> >>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >> >> It's unqualified and no restriction in this change. > > The public methods currently in jdk.vm.ci.services are: > > 1. JVMCIServiceLocator.getProvider(Class) > 2. JVMCIServiceLocator.getProviders(Class) > 3. Services.initializeJVMCI() > 4. Services.getSavedProperties() > 5. Services.exportJVMCITo(Class) > 6. Services.load(Class) > 7. Services.loadSingle(Class, boolean) > > 1 should be made protected. I'll update the webrev with this change. Good. > > 2 should check for JVMCIPermission. I'll update the webrev with this change. Good. > > 3 is harmless from a security perspective in my opinion. Would be good if one of Oracle?s security engineers could take a quick look just to be sure. > > 4 checks for JVMCIPermission. Ok. > > 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). About this, will this Graal update happen for JDK 9? It?s awfully late in the cycle... > > Does this address the security concerns? Mostly, yes. Thanks. > > -Doug From vladimir.kozlov at oracle.com Wed Apr 19 19:46:05 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Apr 2017 12:46:05 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> Message-ID: Hi Doug, Can you consider using other name and not JDK9 for new JVMCI class? It will be used in JDK 10 too: jdk.vm.ci.services.internal.JDK9; Thanks, Vladimir On 4/18/17 3:13 PM, Doug Simon wrote: > Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. > The primary requirement for this is to remove all usage of JDK internals from Graal. > While this most involves changes in Graal, there remain a few capabilities that must > be exposed via JVMCI. Namely: > > 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize > can use system properties that are guaranteed not to have been modified by application > code (Graal initialization is lazy and may occur after application code has been run). > > 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal > optimized implementation of the Truffle runtime. > > These capabilities have been added to jdk.vm.ci.services.Services. A comment has > also been added to this class to denote the current methods to be removed > in a subsequent bug to update the JDK copy of Graal with the upstream version which > no longer uses the methods. The methods destined for removal are: > > exportJVMCITo(Class requestor) > load(Class service) > loadSingle(Class service, boolean required) > > The first one is no longer needed as JVMCI exports itself to Graal service providers > it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. > The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API > is only useful for JVMCI-8 where a hidden JVMCI class loader is used. > > These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra > minor changes were required: > > 1. In JDK-8177663, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException > on all error paths except one which throws AssertionError. The latter was a mistake and has been > fixed to also throw IllegalArgumentException. > 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. > Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. > This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. > > As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and > remove use of JDK internal API from the jdk.aot module. The qualified exports to > jdk.vm.internal.compiler and jdk.aot can then be removed. > > -Doug > > https://bugs.openjdk.java.net/browse/JDK-8177845 > http://cr.openjdk.java.net/~dnsimon/8177845/ > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess > > > From doug.simon at oracle.com Wed Apr 19 20:02:46 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 22:02:46 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> Message-ID: <69BA05B6-C33A-4D97-A323-3E3216515A38@oracle.com> Sure - how about good old Util? ;-) I'm open to other suggestions. Sent from my iPhone > On Apr 19, 2017, at 9:46 PM, Vladimir Kozlov wrote: > > Hi Doug, > > Can you consider using other name and not JDK9 for new JVMCI class? It will be used in JDK 10 too: > > jdk.vm.ci.services.internal.JDK9; > > Thanks, > Vladimir > >> On 4/18/17 3:13 PM, Doug Simon wrote: >> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >> The primary requirement for this is to remove all usage of JDK internals from Graal. >> While this most involves changes in Graal, there remain a few capabilities that must >> be exposed via JVMCI. Namely: >> >> 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize >> can use system properties that are guaranteed not to have been modified by application >> code (Graal initialization is lazy and may occur after application code has been run). >> >> 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal >> optimized implementation of the Truffle runtime. >> >> These capabilities have been added to jdk.vm.ci.services.Services. A comment has >> also been added to this class to denote the current methods to be removed >> in a subsequent bug to update the JDK copy of Graal with the upstream version which >> no longer uses the methods. The methods destined for removal are: >> >> exportJVMCITo(Class requestor) >> load(Class service) >> loadSingle(Class service, boolean required) >> >> The first one is no longer needed as JVMCI exports itself to Graal service providers >> it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. >> The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API >> is only useful for JVMCI-8 where a hidden JVMCI class loader is used. >> >> These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra >> minor changes were required: >> >> 1. In JDK-8177663, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException >> on all error paths except one which throws AssertionError. The latter was a mistake and has been >> fixed to also throw IllegalArgumentException. >> 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. >> Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. >> This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. >> >> As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and >> remove use of JDK internal API from the jdk.aot module. The qualified exports to >> jdk.vm.internal.compiler and jdk.aot can then be removed. >> >> -Doug >> >> https://bugs.openjdk.java.net/browse/JDK-8177845 >> http://cr.openjdk.java.net/~dnsimon/8177845/ >> >> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess >> >> >> From vladimir.kozlov at oracle.com Wed Apr 19 20:29:24 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Apr 2017 13:29:24 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <69BA05B6-C33A-4D97-A323-3E3216515A38@oracle.com> References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <69BA05B6-C33A-4D97-A323-3E3216515A38@oracle.com> Message-ID: ReflectionAccessJDK ? Based on your comment in the file. Vladimir On 4/19/17 1:02 PM, Doug Simon wrote: > Sure - how about good old Util? ;-) I'm open to other suggestions. > > Sent from my iPhone > >> On Apr 19, 2017, at 9:46 PM, Vladimir Kozlov wrote: >> >> Hi Doug, >> >> Can you consider using other name and not JDK9 for new JVMCI class? It will be used in JDK 10 too: >> >> jdk.vm.ci.services.internal.JDK9; >> >> Thanks, >> Vladimir >> >>> On 4/18/17 3:13 PM, Doug Simon wrote: >>> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >>> The primary requirement for this is to remove all usage of JDK internals from Graal. >>> While this most involves changes in Graal, there remain a few capabilities that must >>> be exposed via JVMCI. Namely: >>> >>> 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize >>> can use system properties that are guaranteed not to have been modified by application >>> code (Graal initialization is lazy and may occur after application code has been run). >>> >>> 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal >>> optimized implementation of the Truffle runtime. >>> >>> These capabilities have been added to jdk.vm.ci.services.Services. A comment has >>> also been added to this class to denote the current methods to be removed >>> in a subsequent bug to update the JDK copy of Graal with the upstream version which >>> no longer uses the methods. The methods destined for removal are: >>> >>> exportJVMCITo(Class requestor) >>> load(Class service) >>> loadSingle(Class service, boolean required) >>> >>> The first one is no longer needed as JVMCI exports itself to Graal service providers >>> it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. >>> The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API >>> is only useful for JVMCI-8 where a hidden JVMCI class loader is used. >>> >>> These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra >>> minor changes were required: >>> >>> 1. In JDK-8177663, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException >>> on all error paths except one which throws AssertionError. The latter was a mistake and has been >>> fixed to also throw IllegalArgumentException. >>> 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. >>> Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. >>> This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. >>> >>> As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and >>> remove use of JDK internal API from the jdk.aot module. The qualified exports to >>> jdk.vm.internal.compiler and jdk.aot can then be removed. >>> >>> -Doug >>> >>> https://bugs.openjdk.java.net/browse/JDK-8177845 >>> http://cr.openjdk.java.net/~dnsimon/8177845/ >>> >>> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess >>> >>> >>> > From doug.simon at oracle.com Wed Apr 19 21:12:22 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 23:12:22 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> Message-ID: <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> > On 19 Apr 2017, at 21:40, Christian Thalinger wrote: > >> >> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >> >>> >>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>> >>> >>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>> >>>> >>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>> >>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>> >>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>> >>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>> >>> It's unqualified and no restriction in this change. >> >> The public methods currently in jdk.vm.ci.services are: >> >> 1. JVMCIServiceLocator.getProvider(Class) >> 2. JVMCIServiceLocator.getProviders(Class) >> 3. Services.initializeJVMCI() >> 4. Services.getSavedProperties() >> 5. Services.exportJVMCITo(Class) >> 6. Services.load(Class) >> 7. Services.loadSingle(Class, boolean) >> >> 1 should be made protected. I'll update the webrev with this change. > > Good. > >> >> 2 should check for JVMCIPermission. I'll update the webrev with this change. > > Good. > >> >> 3 is harmless from a security perspective in my opinion. > > Would be good if one of Oracle?s security engineers could take a quick look just to be sure. Vladimir, can you please bring this to the attention of the relevant engineer. >> >> 4 checks for JVMCIPermission. > > Ok. > >> >> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). > > About this, will this Graal update happen for JDK 9? Yes. > It?s awfully late in the cycle... These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). -Doug From doug.simon at oracle.com Wed Apr 19 21:26:28 2017 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 19 Apr 2017 23:26:28 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> Message-ID: <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: 1. JVMCIServiceLocator.getProvider(Class) is now protected 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK -Doug > On 19 Apr 2017, at 23:12, Doug Simon wrote: > >> >> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >> >>> >>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>> >>>> >>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>> >>>> >>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>> >>>>> >>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>> >>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>> >>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>> >>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>> >>>> It's unqualified and no restriction in this change. >>> >>> The public methods currently in jdk.vm.ci.services are: >>> >>> 1. JVMCIServiceLocator.getProvider(Class) >>> 2. JVMCIServiceLocator.getProviders(Class) >>> 3. Services.initializeJVMCI() >>> 4. Services.getSavedProperties() >>> 5. Services.exportJVMCITo(Class) >>> 6. Services.load(Class) >>> 7. Services.loadSingle(Class, boolean) >>> >>> 1 should be made protected. I'll update the webrev with this change. >> >> Good. >> >>> >>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >> >> Good. >> >>> >>> 3 is harmless from a security perspective in my opinion. >> >> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. > > Vladimir, can you please bring this to the attention of the relevant engineer. > >>> >>> 4 checks for JVMCIPermission. >> >> Ok. >> >>> >>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >> >> About this, will this Graal update happen for JDK 9? > > Yes. > >> It?s awfully late in the cycle... > > These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). > > -Doug From igor.ignatyev at oracle.com Wed Apr 19 21:45:16 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 19 Apr 2017 14:45:16 -0700 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <28eafe0f-c951-7a24-2fa0-7e994db7beec@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <5a1da3e4-622b-66a4-1228-4a7e3ebbee91@oracle.com> <33C90E84-6872-4DC4-B6F9-74055D943876@oracle.com> <28eafe0f-c951-7a24-2fa0-7e994db7beec@oracle.com> Message-ID: Hi David, thank for catching those, the incremental webrev : http://cr.openjdk.java.net/~iignatyev/8178788/webrev.01-02/ Cheers, -- Igor > On Apr 18, 2017, at 11:17 PM, David Holmes wrote: > > Hi Igor, > > Thanks for the jtreg info! > > You missed a couple of things in the README: > > 27 All the tests are run thought test driver class -- JcstressRunner, which > > s/thought/through the/ > > And I just noticed an additional typo: > > 29 spawns a new JVM to run one jsctress test > > jsctress -> jcstress > > Thanks, > David > ----- > > On 19/04/2017 12:51 PM, Igor Ignatyev wrote: >> Hi David, >> >> thank you for your comments, I've tried to address them all, please look at the incremental webrev: http://cr.openjdk.java.net/~iignatyev/8178788/webrev.00-01/index.html >> >> regarding multiple test declarations, jtreg supports that, but in the current version it's not fully/officially supported and hence it's not documented. Jon has fixed a few jtreg bugs[1-3] related to this and AFAIK is working towards full support, so I hope it will be supported and documented in the next promoted build of jtreg. >> >> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7900144 : Implement multiple tests in one file in jtreg >> [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7901914 : Failure to find multiple @test in a single file >> [3] https://bugs.openjdk.java.net/browse/CODETOOLS-7901940 : support multiple @test in one test file >> >> Thanks, >> -- Igor >> >>> On Apr 18, 2017, at 6:28 PM, David Holmes wrote: >>> >>> Hi Igor, >>> >>> On 19/04/2017 8:12 AM, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>>>> 69903 lines changed: 69903 ins; 0 del; 0 mod; >>>> (69524 lines are generated) >>>> >>>> Hi all, >>>> >>>> could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? >>>> >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 >>>> testing: applications/jcstress >>> >>> This looks really good! Only concern is that according to the docs jtreg doesn't seem to support multiple test declarations in a single file ?? Maybe the docs are out of date? >> >>> >>> http://openjdk.java.net/jtreg/tag-spec.html >>> >>> A few grammatical nits in the README: >>> >>> 23 The tests located under this directory run tests from Java Concurrency Stress >>> >>> from Java -> from the Java >>> >>> 24 [1] aka jcstress test suite. This suite aims to verify the correctness of >>> >>> Suggest: >>> >>> test suite [1] (a.k.a. jcstress). This suite ... >>> >>> 25 concurrency support in JDK. >>> >>> in JDK -> in the JDK >>> >>> 27 All the tests are run thought test driver class -- JcstressRunner, which >>> >>> s/thought/through the/ >>> >>> 29 spawns a new JVM to run one jsctress test and checks that it is finished >>> >>> is finished -> finishes >>> >>> 33 changed, one should make corresponding changes in the artifact description in >>> >>> in the -> to the >>> >>> 36 */Test.java files should never be modified directly, because they are >>> >>> Insert "The" at start. >>> >>> --- >>> >>> General Java style nits re long ling wrapping. You seem to be using a consistent "indent by 8" style when wrapping long lines. The "usual" style guidelines suggest a more structured indentation style, for example: >>> >>> - when wrapping the arguments to a call align after the opening ( of the call ie: >>> >>> throw new Error("TESTBUG: Can not resolve artifacts for " >>> + JcstressRunner.class.getName(), e); >>> >>> - when you have chained method calls, try to align at the dots ie: >>> >>> return artifacts.get("org.openjdk.jcstress.jcstress-tests-all-0.3") >>> .toAbsolutePath(); >>> >>> and >>> >>> ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(getCmd(args)) >>> .redirectErrorStream(true) >>> .redirectOutput(out.toFile()); >>> >>> --- >>> >>> test/applications/jcstress/TestGenerator.java >>> >>> 44 * This is a test generator, should be run only when jsctress is changed >>> >>> Typo: jsctress -> jcstress >>> >>> 51 * Use jcstress test suite to generates >>> >>> generates -> generate >>> >>> Strictly speaking the copyright years should only be updated when there is a substantive change to the text of the file, so updating whenever the files are regenerated is not really correct. This may need to be manually adjusted when commiting updated files ... though if there is no change other than to copyright year then the changes can simply be ignored (hg revert) and no update pushed. >>> >>> Thanks, >>> David >>> ------ >>> >>>> Thanks, >>>> -- Igor >>>> >> From vladimir.kozlov at oracle.com Wed Apr 19 22:28:09 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Apr 2017 15:28:09 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> Message-ID: <21999ed5-23aa-939b-f1e5-1eb08b3c03e4@oracle.com> Hotspot changes looks good to me. Thanks, Vladimir On 4/19/17 2:26 PM, Doug Simon wrote: > I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: > > 1. JVMCIServiceLocator.getProvider(Class) is now protected > 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission > 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK > > -Doug > >> On 19 Apr 2017, at 23:12, Doug Simon wrote: >> >>> >>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>> >>>> >>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>> >>>>> >>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>> >>>>> >>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>> >>>>>> >>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>> >>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>> >>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>> >>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>> >>>>> It's unqualified and no restriction in this change. >>>> >>>> The public methods currently in jdk.vm.ci.services are: >>>> >>>> 1. JVMCIServiceLocator.getProvider(Class) >>>> 2. JVMCIServiceLocator.getProviders(Class) >>>> 3. Services.initializeJVMCI() >>>> 4. Services.getSavedProperties() >>>> 5. Services.exportJVMCITo(Class) >>>> 6. Services.load(Class) >>>> 7. Services.loadSingle(Class, boolean) >>>> >>>> 1 should be made protected. I'll update the webrev with this change. >>> >>> Good. >>> >>>> >>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>> >>> Good. >>> >>>> >>>> 3 is harmless from a security perspective in my opinion. >>> >>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >> >> Vladimir, can you please bring this to the attention of the relevant engineer. >> >>>> >>>> 4 checks for JVMCIPermission. >>> >>> Ok. >>> >>>> >>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>> >>> About this, will this Graal update happen for JDK 9? >> >> Yes. >> >>> It?s awfully late in the cycle... >> >> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >> >> -Doug > From vladimir.kozlov at oracle.com Wed Apr 19 22:31:25 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Apr 2017 15:31:25 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> Message-ID: Doug, Can you point (link) particular code which needs to be reviewed? And what security issues could be? Thanks, Vladimir On 4/19/17 2:12 PM, Doug Simon wrote: >> 3. Services.initializeJVMCI() >>> 3 is harmless from a security perspective in my opinion. >> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. > Vladimir, can you please bring this to the attention of the relevant engineer. > From david.holmes at oracle.com Wed Apr 19 23:16:39 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Apr 2017 09:16:39 +1000 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <5a1da3e4-622b-66a4-1228-4a7e3ebbee91@oracle.com> <33C90E84-6872-4DC4-B6F9-74055D943876@oracle.com> <28eafe0f-c951-7a24-2fa0-7e994db7beec@oracle.com> Message-ID: <25befcfc-9f25-70de-5a90-0b76e34fc737@oracle.com> Looks good! Sorry I should have said no new webrev needed :) Thanks, David On 20/04/2017 7:45 AM, Igor Ignatyev wrote: > Hi David, > > thank for catching those, the incremental webrev : http://cr.openjdk.java.net/~iignatyev/8178788/webrev.01-02/ > > Cheers, > -- Igor > >> On Apr 18, 2017, at 11:17 PM, David Holmes wrote: >> >> Hi Igor, >> >> Thanks for the jtreg info! >> >> You missed a couple of things in the README: >> >> 27 All the tests are run thought test driver class -- JcstressRunner, which >> >> s/thought/through the/ >> >> And I just noticed an additional typo: >> >> 29 spawns a new JVM to run one jsctress test >> >> jsctress -> jcstress >> >> Thanks, >> David >> ----- >> >> On 19/04/2017 12:51 PM, Igor Ignatyev wrote: >>> Hi David, >>> >>> thank you for your comments, I've tried to address them all, please look at the incremental webrev: http://cr.openjdk.java.net/~iignatyev/8178788/webrev.00-01/index.html >>> >>> regarding multiple test declarations, jtreg supports that, but in the current version it's not fully/officially supported and hence it's not documented. Jon has fixed a few jtreg bugs[1-3] related to this and AFAIK is working towards full support, so I hope it will be supported and documented in the next promoted build of jtreg. >>> >>> [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7900144 : Implement multiple tests in one file in jtreg >>> [2] https://bugs.openjdk.java.net/browse/CODETOOLS-7901914 : Failure to find multiple @test in a single file >>> [3] https://bugs.openjdk.java.net/browse/CODETOOLS-7901940 : support multiple @test in one test file >>> >>> Thanks, >>> -- Igor >>> >>>> On Apr 18, 2017, at 6:28 PM, David Holmes wrote: >>>> >>>> Hi Igor, >>>> >>>> On 19/04/2017 8:12 AM, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>>>>> 69903 lines changed: 69903 ins; 0 del; 0 mod; >>>>> (69524 lines are generated) >>>>> >>>>> Hi all, >>>>> >>>>> could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? >>>>> >>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 >>>>> testing: applications/jcstress >>>> >>>> This looks really good! Only concern is that according to the docs jtreg doesn't seem to support multiple test declarations in a single file ?? Maybe the docs are out of date? >>> >>>> >>>> http://openjdk.java.net/jtreg/tag-spec.html >>>> >>>> A few grammatical nits in the README: >>>> >>>> 23 The tests located under this directory run tests from Java Concurrency Stress >>>> >>>> from Java -> from the Java >>>> >>>> 24 [1] aka jcstress test suite. This suite aims to verify the correctness of >>>> >>>> Suggest: >>>> >>>> test suite [1] (a.k.a. jcstress). This suite ... >>>> >>>> 25 concurrency support in JDK. >>>> >>>> in JDK -> in the JDK >>>> >>>> 27 All the tests are run thought test driver class -- JcstressRunner, which >>>> >>>> s/thought/through the/ >>>> >>>> 29 spawns a new JVM to run one jsctress test and checks that it is finished >>>> >>>> is finished -> finishes >>>> >>>> 33 changed, one should make corresponding changes in the artifact description in >>>> >>>> in the -> to the >>>> >>>> 36 */Test.java files should never be modified directly, because they are >>>> >>>> Insert "The" at start. >>>> >>>> --- >>>> >>>> General Java style nits re long ling wrapping. You seem to be using a consistent "indent by 8" style when wrapping long lines. The "usual" style guidelines suggest a more structured indentation style, for example: >>>> >>>> - when wrapping the arguments to a call align after the opening ( of the call ie: >>>> >>>> throw new Error("TESTBUG: Can not resolve artifacts for " >>>> + JcstressRunner.class.getName(), e); >>>> >>>> - when you have chained method calls, try to align at the dots ie: >>>> >>>> return artifacts.get("org.openjdk.jcstress.jcstress-tests-all-0.3") >>>> .toAbsolutePath(); >>>> >>>> and >>>> >>>> ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(getCmd(args)) >>>> .redirectErrorStream(true) >>>> .redirectOutput(out.toFile()); >>>> >>>> --- >>>> >>>> test/applications/jcstress/TestGenerator.java >>>> >>>> 44 * This is a test generator, should be run only when jsctress is changed >>>> >>>> Typo: jsctress -> jcstress >>>> >>>> 51 * Use jcstress test suite to generates >>>> >>>> generates -> generate >>>> >>>> Strictly speaking the copyright years should only be updated when there is a substantive change to the text of the file, so updating whenever the files are regenerated is not really correct. This may need to be manually adjusted when commiting updated files ... though if there is no change other than to copyright year then the changes can simply be ignored (hg revert) and no update pushed. >>>> >>>> Thanks, >>>> David >>>> ------ >>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>> > From david.holmes at oracle.com Thu Apr 20 00:48:26 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Apr 2017 10:48:26 +1000 Subject: RFR(xs) JDK-8163861 JNI NewString() and GetStringLength() documentation incorrect In-Reply-To: References: Message-ID: <26815c67-1e2c-30cb-6435-2be6f713f42a@oracle.com> Hi George, Apologies in advance - this is not "xs" in the discussion. But this needs to go through CSR process (which is not yet available for 10) so it can't be pushed yet anyway (sorry). But consider this my CSR review. :) On 20/04/2017 3:43 AM, George Triantafillou wrote: > Please review this very small fix to the spec for JDK-8163861: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8163861 > > webrev: http://cr.openjdk.java.net/~gtriantafill/8163861-webrev/webrev > > > The change provides a more precise definition for NewString() and > GetStringLength(). Thanks. As the bug report states 'most of the Java documention was updated to deal with this', so I think it important to ensure any changes are consistent with that updated documentation and terminology. Specifically we need to refer to the String class API specification and the Character class specification which defines Unicode Character Representations for the Java platform. Looking at the JDK 8 JNI spec as referenced in the bug report we first see: jsize GetStringLength(JNIEnv *env, jstring string); Returns the length (the count of Unicode characters) of a Java string. This should be functionally equivalent to java.lang.String.length(). If anything the "count of Unicode characters" should just be deleted as this method is intended to return the length of the Java string, just as String.length does. The phrase "count of Unicode characters" is at best "loose" as the reporter notes (and this terminology is mis-used in a few of the JNI string functions!). The String.length() method states: "The length is equal to the number of Unicode code units in the string." where "Unicode code units" is defined in the Character class. So the correct fix would be to change "characters" to "code units". However that there are two descriptions of how the function behaves - in javadoc terminology we have one description in the main body: 4168

Returns the length (the count of Unicode 4169 characters) of a Java string.

and one equivalent to @returns: 4180

RETURNS:

4181 4182

Returns the length of the Java string.

The latter is perfectly and accurately correct! If you want to know how the length of a Java String is defined then you need to see the String class. So I would say that the correct fix here is to either delete "(the count of Unicode characters)", or else change "characters" to "code units" as previously discussed. --- Turning to NewString .... the JNI spec states: "Constructs a new java.lang.String object from an array of Unicode characters." and the @param equivalent for 'len' states: "len: length of the Unicode string. May be 0." So initially we have the problematic use of "Unicode characters" again; then we have the problem as to what the definition of the length of a "Unicode string" is. I find no help elsewhere in the JNI spec to clarify this (despite a very lengthy discussion on its use of UTF-8 encodings). I can only assume, based on the implementation, that the expectation is that the array of "Unicode characters" is in UTF-16 format. In which case a more technically accurate fix would be to say: "len: length of the Unicode string in UTF-16 code units. May be 0." Thanks, David PS. I will also add the above to the bug report. > -George From kim.barrett at oracle.com Thu Apr 20 06:17:55 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 20 Apr 2017 02:17:55 -0400 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation Message-ID: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Please review this addition of the count_trailing_zeros function. Unfortunately, there isn't an obvious direct and portable way to write such a function. But supported hardware architectures generally provide an efficient implementation, e.g. a single instruction or a short sequence. Compilers often provide "built-in" or "intrinsic" access to that hardware implementation, or one can use inline assembler. If a platform doesn't have such a built-in efficient implementation, the function can be implemented using de Bruijn sequences as discussed in the literature. But all of the OpenJDK-supported platforms provide an efficient implementation, so we aren't providing such a fallback. As part of reviewing this change, please feel free to suggest alternative ways to structure the code. I'm not completely happy with the way I've done it, but alternatives I've tried seemed worse. CR: https://bugs.openjdk.java.net/browse/JDK-8179004 Webrev: http://cr.openjdk.java.net/~kbarrett/8179004/hotspot.00/ Testing: Unit test for new function. Experimented with modifying BitMap::get_next_one_offset and the like to replace the existing for-loop with a call to this new function, and measured substantial speedups on all tested platforms for a test program that counts the bits in a bitmap by iterating calls to that search function. The speedup varies with the density of set bits, with very sparse or nearly full being similar to the existing code, since the bit counting is not the dominant factor in those situations. But in between give speedups of factors of x1.5 to x5 or more, depending on the density and the platform. From joe.darcy at oracle.com Thu Apr 20 06:53:28 2017 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 19 Apr 2017 23:53:28 -0700 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Message-ID: Hi Kim, What relation does this work have to the java.lang.Integer.numberOfTrailingZeros?() method which is marked as @HotSpotIntrinsicCandidate? Thanks, -Joe On 4/19/2017 11:17 PM, Kim Barrett wrote: > Please review this addition of the count_trailing_zeros function. > > Unfortunately, there isn't an obvious direct and portable way to write > such a function. But supported hardware architectures generally > provide an efficient implementation, e.g. a single instruction or a > short sequence. Compilers often provide "built-in" or "intrinsic" > access to that hardware implementation, or one can use inline > assembler. > > If a platform doesn't have such a built-in efficient implementation, > the function can be implemented using de Bruijn sequences as discussed > in the literature. But all of the OpenJDK-supported platforms provide > an efficient implementation, so we aren't providing such a fallback. > > As part of reviewing this change, please feel free to suggest > alternative ways to structure the code. I'm not completely happy with > the way I've done it, but alternatives I've tried seemed worse. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8179004 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8179004/hotspot.00/ > > Testing: > Unit test for new function. > > Experimented with modifying BitMap::get_next_one_offset and the like > to replace the existing for-loop with a call to this new function, and > measured substantial speedups on all tested platforms for a test > program that counts the bits in a bitmap by iterating calls to that > search function. The speedup varies with the density of set bits, with > very sparse or nearly full being similar to the existing code, since > the bit counting is not the dominant factor in those situations. But > in between give speedups of factors of x1.5 to x5 or more, depending > on the density and the platform. > From frank.yuan at oracle.com Thu Apr 20 07:16:30 2017 From: frank.yuan at oracle.com (Frank Yuan) Date: Thu, 20 Apr 2017 15:16:30 +0800 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> <024601d2b8bd$9fe0b840$dfa228c0$@oracle.com> <76E3B690-77FA-4BBA-92A2-77DF03271FE3@oracle.com> <029e01d2b8d7$a3211760$e9634620$@oracle.com> Message-ID: <048e01d2b9a6$0a711ae0$1f5350a0$@oracle.com> Understand. Alright, let?s talk off this thread for what out of scope of this RFE. Thanks Frank From: Igor Ignatyev [mailto:igor.ignatyev at oracle.com] Sent: Thursday, April 20, 2017 2:06 AM To: Frank Yuan Cc: HotSpot Open Source Developers Subject: Re: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests Hi Felix, I understand your point and use-case and believe it is important and useful, however I don't think we should do that now. 1st of all, current jcstress wrapper is able to run precompiled tests from jsctress-tests-all, so if people write their own jcstress based tests, they will still need to use jcstress to build and run them. 2nd, hotspot/test/applications/jcstress tests depend on the version of jcstress, and currently the version is defined in wrapper, so I'd prefer to have the wrapper as much closer to these tests as possible. that is to say, if we have high need for running custom jcstress tests, then we should invest more in jcstress driver/wrapper (and maybe infra) so it will be possible to have tests in jcstress format as part of our test bases, but this is out of scope of this RFE. Thanks, -- Igor On Apr 18, 2017, at 11:39 PM, Frank Yuan > wrote: Hi Igor The artifact which I talked about is the wrapper(JcstressRunner.java), I supposed there would be more people to create their tests with jcstress facility if the wrapper is put to a common place, there have been e.g. failure_handler, lib in hg.openjdk.java.net/jdk10/hs/test. If we put the wrapper into hotspot repo, even after JEP 296, it would be under test/hotspot/? Certainly, it?s up to you:) Thanks Frank From: Igor Ignatyev [mailto:igor.ignatyev at oracle.com] Sent: Wednesday, April 19, 2017 1:47 PM To: Frank Yuan Cc: HotSpot Open Source Developers Subject: Re: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests Hi Felix, currently, /test is not a jtreg testbase and it is not supported by make and the rest of infra, therefore it's not possible to have tests there. I don't want to mix up this fix w/ infra/make related work, and would prefer to think about this movement after (or as a part of) JEP 296 Consolidate the JDK Forest into a Single Repository. actually, corelibs people are more than welcome to run hotspot/test, we (hotspot team) run jdk/test quite often and would really love to see other people running our tests before pushing changes which can affect hotspot and/or its tests. -- Igor On Apr 18, 2017, at 8:32 PM, Frank Yuan < frank.yuan at oracle.com> wrote: Hi Igor This is really a valuable work! Would you like to pull your artifacts up to the common repo hg.openjdk.java.net/jdk10/hs/test? That more people(e.g. corelibs :)) can benefit from this. Thanks Frank Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests Date: Tue, 18 Apr 2017 15:12:48 -0700 From: Igor Ignatyev To: hotspot-compiler-dev at openjdk.java.net compiler , HotSpot Open Source Developers http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > 69903 lines changed: 69903 ins; 0 del; 0 mod; (69524 lines are generated) Hi all, could you please review this patch which adds a jtreg test wrapper for jcstress test suite and jtreg tests which run jsctress tests thru this wrapper? webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 testing: applications/jcstress Thanks, -- Igor From kim.barrett at oracle.com Thu Apr 20 07:29:41 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 20 Apr 2017 03:29:41 -0400 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Message-ID: > On Apr 20, 2017, at 2:53 AM, joe darcy wrote: > > Hi Kim, > > What relation does this work have to the java.lang.Integer.numberOfTrailingZeros?() method which is marked as @HotSpotIntrinsicCandidate? I?m going to guess little or none. This change is about providing a C++ callable routine for use in within the VM. From doug.simon at oracle.com Thu Apr 20 07:49:10 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 20 Apr 2017 09:49:10 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> Message-ID: > On 20 Apr 2017, at 00:31, Vladimir Kozlov wrote: > > Doug, > > Can you point (link) particular code which needs to be reviewed? And what security issues could be? I believe Chris had possible concerns with Services.initializeJVMCI() (at http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.services/src/jdk/vm/ci/services/Services.java.udiff.html). As previously stated, I can't see any security issues with this method which is why it doesn't check JVMCIPermission. -Doug > > Thanks, > Vladimir > > On 4/19/17 2:12 PM, Doug Simon wrote: >>> 3. Services.initializeJVMCI() > >>>> 3 is harmless from a security perspective in my opinion. >>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >> Vladimir, can you please bring this to the attention of the relevant engineer. >> From george.triantafillou at oracle.com Thu Apr 20 14:25:18 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Thu, 20 Apr 2017 10:25:18 -0400 Subject: RFR(xs) JDK-8163861 JNI NewString() and GetStringLength() documentation incorrect In-Reply-To: <26815c67-1e2c-30cb-6435-2be6f713f42a@oracle.com> References: <26815c67-1e2c-30cb-6435-2be6f713f42a@oracle.com> Message-ID: Hi David, Thanks for the review. I wasn't aware that the CSR isn't yet available for 10. Do you know when it will be available? On 4/19/2017 8:48 PM, David Holmes wrote: > Hi George, > > Apologies in advance - this is not "xs" in the discussion. But this > needs to go through CSR process (which is not yet available for 10) so > it can't be pushed yet anyway (sorry). But consider this my CSR > review. :) > > On 20/04/2017 3:43 AM, George Triantafillou wrote: >> Please review this very small fix to the spec for JDK-8163861: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8163861 >> >> webrev: http://cr.openjdk.java.net/~gtriantafill/8163861-webrev/webrev >> >> >> The change provides a more precise definition for NewString() and >> GetStringLength(). Thanks. > > As the bug report states 'most of the Java documention was updated to > deal with this', so I think it important to ensure any changes are > consistent with that updated documentation and terminology. > Specifically we need to refer to the String class API specification > and the Character class specification which defines Unicode Character > Representations for the Java platform. > > Looking at the JDK 8 JNI spec as referenced in the bug report we first > see: > > jsize GetStringLength(JNIEnv *env, jstring string); > Returns the length (the count of Unicode characters) of a Java string. > > This should be functionally equivalent to java.lang.String.length(). > If anything the "count of Unicode characters" should just be deleted > as this method is intended to return the length of the Java string, > just as String.length does. The phrase "count of Unicode characters" > is at best "loose" as the reporter notes (and this terminology is > mis-used in a few of the JNI string functions!). The String.length() > method states: > > "The length is equal to the number of Unicode code units in the string." > > where "Unicode code units" is defined in the Character class. > > So the correct fix would be to change "characters" to "code units". > > However that there are two descriptions of how the function behaves - > in javadoc terminology we have one description in the main body: > > 4168

Returns the length (the count of Unicode > 4169 characters) of a Java string.

> > and one equivalent to @returns: > > 4180

RETURNS:

> 4181 > 4182

Returns the length of the Java string.

> > The latter is perfectly and accurately correct! If you want to know > how the length of a Java String is defined then you need to see the > String class. > > So I would say that the correct fix here is to either delete "(the > count of Unicode characters)", or else change "characters" to "code > units" as previously discussed. That sounds reasonable! > > --- > > Turning to NewString .... the JNI spec states: > > "Constructs a new java.lang.String object from an array of Unicode > characters." > > and the @param equivalent for 'len' states: > > "len: length of the Unicode string. May be 0." > > So initially we have the problematic use of "Unicode characters" > again; then we have the problem as to what the definition of the > length of a "Unicode string" is. > > I find no help elsewhere in the JNI spec to clarify this (despite a > very lengthy discussion on its use of UTF-8 encodings). I can only > assume, based on the implementation, that the expectation is that the > array of "Unicode characters" is in UTF-16 format. In which case a > more technically accurate fix would be to say: > > "len: length of the Unicode string in UTF-16 code units. May be 0." > > Thanks, > David > > PS. I will also add the above to the bug report. Thanks, I appreciate it. -George > >> -George From volker.simonis at gmail.com Thu Apr 20 16:02:04 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 20 Apr 2017 18:02:04 +0200 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58F33A4F.70104@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> Message-ID: Hi Ioi, thanks once again for considering our ports! Please find the required additions for ppc64/s390x in the following webrew (which is based upon your latest v03 patch): http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly code. I did some tests and I think it should be correct, but maybe you still find some improvements :) Besides that, I have some general questions/comments regarding your change: 1. In constantPool.hpp, why don't you declare the '_name_index' and '_resolved_klass_index' fields with type 'jushort'? As far as I can see, they can only hold 16-bit values anyway. It would also save you some space and several asserts (e.g. in unresolved_klass_at_put(): 274 assert((name_index & 0xffff0000) == 0, "must be"); 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); 2. What do you mean by: 106 // ... will be changed to support compressed pointers 107 Array* _resolved_klasses; 3. Why don't we need the call to "release_tag_at_put()" in "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int class_index, Klass* k)" is used from "ClassFileParser::fill_instance_klass() and before your change that function used the previous version of "klass_at_put(int class_index, Klass* k)" which did call "release_tag_at_put()". 4. In ConstantPool::copy_entry_to() you've changed the behavior for tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was copied to the new constant pool if one existed but now you always only copy a class_index to the new constant pool (even if a resolved klass existed). Is that OK? E.g. won't this lead to a new resolving for the new constant pool and will this have performance impacts or other side effects? Thanks again for doing this nice change and best regards, Volker On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: > Hi Lois, > > I have updated the patch to include your comments, and fixes the handling of > anonymous classes. I also added some more comments regarding the > _temp_resolved_klass_index: > > (delta from last webrev) > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ > > (full webrev) > http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ > > Thanks > - Ioi > > > On 4/15/17 2:31 AM, Lois Foltan wrote: >> >> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>> >>> >>> >>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>> >>>> HI Lois, >>>> >>>> Thanks for the review. Please see my comments in-line. >>>> >>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>> >>>>> Hi Ioi, >>>>> >>>>> Looks really good. A couple of comments: >>>>> >>>>> src/share/vm/classfile/classFileParser.cpp: >>>>> * line #5676 - I'm not sure I completely understand the logic >>>>> surrounding anonymous classes. Coleen and I discussed earlier today and I >>>>> came away from that discussion with the idea that the only classes being >>>>> patched currently are anonymous classes. >>>> >>>> Line 5676 ... >>>> >>>> 5676 if (is_anonymous()) { >>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>> index >>>> 5678 } >>>> >>>> corresponds to >>>> >>>> 5361 ik->set_name(_class_name); >>>> 5362 >>>> 5363 if (is_anonymous()) { >>>> 5364 // I am well known to myself >>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>> ik->name()); // eagerly resolve >>>> 5366 } >>>> >>>> Even though the class is "anonymous", it actually has a name. ik->name() >>>> probably is part of the constant pool, but I am not 100% sure. Also, I would >>>> need to search the constant pool to find the index for ik->name(). So I just >>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>> append ik->name() to the end of the constant pool. >>>> >>>> "Anonymous" actually means "the class cannot be looked up by name in the >>>> SystemDictionary". I think we need a better terminology :-) >>>> >>> >>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>> modify the comments to the following: >>> >>> // _this_class_index is a CONSTANT_Class entry that refers to this >>> // anonymous class itself. If this class needs to refer to its own >>> methods or >>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>> reference >>> // _this_class_index. However, because this class is anonymous (it's >>> // not stored in SystemDictionary), _this_class_index cannot be >>> resolved >>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>> lookup. >>> // Therefore, we must eagerly resolve _this_class_index now. >>> >>> So, Lois is right. Line 5676 is not necessary. I will revise the code to >>> do the "eager resolution" without using ClassFileParser::patch_class. I'll >>> post the updated code later. >> >> >> Thanks Ioi for studying this and explaining! Look forward to seeing the >> updated webrev. >> Lois >> >>> >>> Thanks >>> - Ioi >>> >>>>> So a bit confused as why the check on line #5676 and a check for a >>>>> java/lang/Class on line #5684. >>>> >>>> 5683 Handle patch = cp_patch_at(i); >>>> 5684 if (java_lang_String::is_instance(patch()) || >>>> java_lang_Class::is_instance(patch())) { >>>> 5685 // We need to append the names of the patched classes to >>>> the end of the constant pool, >>>> 5686 // because a patched class may have a Utf8 name that's >>>> not already included in the >>>> 5687 // original constant pool. >>>> 5688 // >>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>> patch a Utf8, a String, or a Class. >>>> 5690 // At this point, we don't know the tag for index i yet, >>>> because we haven't parsed the >>>> 5691 // constant pool. So we can only assume the worst -- >>>> every String is used to patch a Class. >>>> 5692 _max_num_patched_klasses ++; >>>> >>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>> either Class or String could be treated as a Klass: >>>> >>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>> 725 int index, >>>> 726 Handle patch, >>>> 727 TRAPS) { >>>> ... >>>> 732 switch (cp->tag_at(index).value()) { >>>> 733 >>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>> 735 // Patching a class means pre-resolving it. >>>> 736 // The name in the constant pool is ignored. >>>> 737 if (java_lang_Class::is_instance(patch())) { >>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>> 739 "Illegal class patch at %d in class file >>>> %s", >>>> 740 index, CHECK); >>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>> 742 patch_class(cp, index, k, k->name()); >>>> 743 } else { >>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>> 745 "Illegal class patch at %d in class file >>>> %s", >>>> 746 index, CHECK); >>>> 747 Symbol* const name = java_lang_String::as_symbol(patch(), >>>> CHECK); >>>> 748 patch_class(cp, index, NULL, name); >>>> 749 } >>>> 750 break; >>>> 751 } >>>> >>>>> Could the is_anonymous() if statement be combined into the loop? >>>> >>>> >>>> I think the answer is no. At line 5365, there is no guarantee that >>>> ik->name() is in the cp_patch array. >>>> >>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>> ik->name()); // eagerly resolve >>>> >>>>> Also why not do this calculation in the rewriter or is that too late? >>>>> >>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and the >>>> associated tags array is allocated (both of which are fixed size, and cannot >>>> be expanded), which is way before the rewriter is run. At this point, we >>>> don't know what cp->tag_at(index) is (line #732), so the code needs to make >>>> a worst-case estimate on how long the CP/tags should be. >>>> >>>>> * line #5677, 5692 - a nit but I think the convention is to not have a >>>>> space after the variable name and between the post increment operator. >>>>> >>>> Fixed. >>>>> >>>>> src/share/vm/classfile/constantPool.hpp: >>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>> really is not so much invalid as temporary for class redefinition purposes, >>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please consider >>>>> renaming to _temp_unresolved_klass_index. And whether you choose to rename >>>>> the field or not, please add a one line comment ahead of >>>>> ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses >>>>> this currently. >>>>> >>>> Good idea. Will do. >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Great change, thanks! >>>>> Lois >>>>> >>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>> >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>> >>>>>> In addition to your requests, I made these changes: >>>>>> >>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>> CPKlassSlot, so the code is cleaner: >>>>>> >>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>> int name_index = kslot.name_index(); >>>>>> >>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick this >>>>>> function over MetaspaceObj::is_shared(). >>>>>> >>>>>> [3] Massaged the CDS region size set-up code a little to pass internal >>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time picking >>>>>> the "right" sizes, as this code will be obsoleted soon with JDK-8072061 >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> >>>>>>> This looks really good! >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>> >>>>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>>>> + Array* tags = >>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // >>>>>>> tags->at(_length) is flags() >>>>>>> >>>>>>> >>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>> >>>>>>> + Klass** adr = >>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>> resolved >>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>> + // hardware store ordering here. >>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>> + return k; >>>>>>> >>>>>>> The comment still refers to the switch between Symbol* and Klass* in >>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>> >>>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>>> >>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header file >>>>>>> (so it's inlined) and have all the places where you get name_index use this >>>>>>> function? So we don't have to know in multiple places that >>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about what the >>>>>>> new format of the constant pool entry is {name_index, resolved_klass_index}. >>>>>>> I'm happy to see this work nearing completion! The "constant" pool should >>>>>>> be constant! thanks, Coleen >>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>> >>>>>>>> Hi,please review the following change >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from >>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool >>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + >>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. In >>>>>>>> addition: + I moved _resolved_references and _reference_map from >>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no longer >>>>>>>> modified for shared ConstantPools. As a result, none of the fields in >>>>>>>> shared ConstantPools are modified at run time, so we can move them into the >>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results show no >>>>>>>> performance regression, despite the extra level of indirection, which has a >>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 and >>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support ports. For the >>>>>>>> aarch64, ppc and s390 ports, I have added some code without testing (it's >>>>>>>> probably incomplete) - Port owners, please check if my patch work for you, >>>>>>>> and I can incorporate your changes in my push. Alternatively, you can wait >>>>>>>> for my push and provide fixes (if necessary) in a new changeset, and I will >>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>> >>>>>> >>>>> >>>> >>> >> > From doug.simon at oracle.com Thu Apr 20 18:50:16 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 20 Apr 2017 20:50:16 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <816F9D53-B11D-48F0-B7BE-4A5E32AD92ED@oracle.com> <69BA05B6-C33A-4D97-A323-3E3216515A38@oracle.com> Message-ID: > On 19 Apr 2017, at 22:29, Vladimir Kozlov wrote: > > ReflectionAccessJDK ? Based on your comment in the file. > > Vladimir > > On 4/19/17 1:02 PM, Doug Simon wrote: >> Sure - how about good old Util? ;-) I'm open to other suggestions. >> >> Sent from my iPhone >> >>> On Apr 19, 2017, at 9:46 PM, Vladimir Kozlov wrote: >>> >>> Hi Doug, >>> >>> Can you consider using other name and not JDK9 for new JVMCI class? It will be used in JDK 10 too: >>> >>> jdk.vm.ci.services.internal.JDK9; >>> >>> Thanks, >>> Vladimir >>> >>>> On 4/18/17 3:13 PM, Doug Simon wrote: >>>> Please review these changes that make jdk.internal.vm.compiler an upgradable compiler. >>>> The primary requirement for this is to remove all usage of JDK internals from Graal. >>>> While this most involves changes in Graal, there remain a few capabilities that must >>>> be exposed via JVMCI. Namely: >>>> >>>> 1. Graal needs a copy of jdk.internal.misc.VM.savedProps so that it can Graal initialize >>>> can use system properties that are guaranteed not to have been modified by application >>>> code (Graal initialization is lazy and may occur after application code has been run). >>>> >>>> 2. Truffle needs to be able to trigger JVMCI initialization so that it can select the Graal >>>> optimized implementation of the Truffle runtime. >>>> >>>> These capabilities have been added to jdk.vm.ci.services.Services. A comment has >>>> also been added to this class to denote the current methods to be removed >>>> in a subsequent bug to update the JDK copy of Graal with the upstream version which >>>> no longer uses the methods. The methods destined for removal are: >>>> >>>> exportJVMCITo(Class requestor) >>>> load(Class service) >>>> loadSingle(Class service, boolean required) >>>> >>>> The first one is no longer needed as JVMCI exports itself to Graal service providers >>>> it loads via private API and Graal re-exports[1] JVMCI to any module the extends Graal. >>>> The latter 2 are no longer required as Graal uses the standard ServiceLoader. This API >>>> is only useful for JVMCI-8 where a hidden JVMCI class loader is used. >>>> >>>> These changes have been tested against upstream Graal. To make the Graal tests pass, 2 extra >>>> minor changes were required: >>>> >>>> 1. In JDK-8177663, HotSpotMemoryAccessProviderImpl::checkRead was added and throws IllegalArgumentException >>>> on all error paths except one which throws AssertionError. The latter was a mistake and has been >>>> fixed to also throw IllegalArgumentException. >>>> 2. An invalid assertion has been removed from HotSpotResolvedObjectTypeImpl::isDefinitelyResolvedWithRespectTo. >>>> Up to JDK 8, it was true that all classes in java.* packages were loaded by the boot class loader. >>>> This is no longer true in JDK 9 with classes like java.sql being loaded by platform class loader. >>>> >>>> As hinted above, a subsequent bug will be required to pull in the latest upstream Graal and >>>> remove use of JDK internal API from the jdk.aot module. The qualified exports to >>>> jdk.vm.internal.compiler and jdk.aot can then be removed. >>>> >>>> -Doug >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8177845 >>>> http://cr.openjdk.java.net/~dnsimon/8177845/ >>>> >>>> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#IndirectQualifiedReflectiveAccess >>>> >>>> >>>> >> From doug.simon at oracle.com Thu Apr 20 18:50:56 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 20 Apr 2017 20:50:56 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> Message-ID: <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to the version compatible with the JVMCI API changes. In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these tests better isolates them from parts of the runtime they are not aiming to test. org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:563) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) at java.base/java.lang.Thread.run(Thread.java:844) Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) ... 26 more -Doug > On 19 Apr 2017, at 23:26, Doug Simon wrote: > > I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: > > 1. JVMCIServiceLocator.getProvider(Class) is now protected > 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission > 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK > > -Doug > >> On 19 Apr 2017, at 23:12, Doug Simon wrote: >> >>> >>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>> >>>> >>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>> >>>>> >>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>> >>>>> >>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>> >>>>>> >>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>> >>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>> >>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>> >>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>> >>>>> It's unqualified and no restriction in this change. >>>> >>>> The public methods currently in jdk.vm.ci.services are: >>>> >>>> 1. JVMCIServiceLocator.getProvider(Class) >>>> 2. JVMCIServiceLocator.getProviders(Class) >>>> 3. Services.initializeJVMCI() >>>> 4. Services.getSavedProperties() >>>> 5. Services.exportJVMCITo(Class) >>>> 6. Services.load(Class) >>>> 7. Services.loadSingle(Class, boolean) >>>> >>>> 1 should be made protected. I'll update the webrev with this change. >>> >>> Good. >>> >>>> >>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>> >>> Good. >>> >>>> >>>> 3 is harmless from a security perspective in my opinion. >>> >>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >> >> Vladimir, can you please bring this to the attention of the relevant engineer. >> >>>> >>>> 4 checks for JVMCIPermission. >>> >>> Ok. >>> >>>> >>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>> >>> About this, will this Graal update happen for JDK 9? >> >> Yes. >> >>> It?s awfully late in the cycle... >> >> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >> >> -Doug > From lutz.schmidt at sap.com Thu Apr 20 21:24:23 2017 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Thu, 20 Apr 2017 21:24:23 +0000 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> Message-ID: <31A1E52F-E10F-4193-AE22-DC9FBF173253@sap.com> Hi Volker, you additions are ok for me - they are as good as it gets when you add pointer indirections. My ok statement holds for s390x in particular, but ppc looks good as well. Let?s see if Martin has extended comments on ppc. Regards, Lutz > On 20 Apr 2017, at 18:02, Volker Simonis wrote: > > Hi Ioi, > > thanks once again for considering our ports! Please find the required > additions for ppc64/s390x in the following webrew (which is based upon > your latest v03 patch): > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ > > @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly > code. I did some tests and I think it should be correct, but maybe you > still find some improvements :) > > Besides that, I have some general questions/comments regarding your change: > > 1. In constantPool.hpp, why don't you declare the '_name_index' and > '_resolved_klass_index' fields with type 'jushort'? As far as I can > see, they can only hold 16-bit values anyway. It would also save you > some space and several asserts (e.g. in unresolved_klass_at_put(): > > > 274 assert((name_index & 0xffff0000) == 0, "must be"); > 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); > > 2. What do you mean by: > > 106 // ... will be changed to support compressed pointers > 107 Array* _resolved_klasses; > > 3. Why don't we need the call to "release_tag_at_put()" in > "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int > class_index, Klass* k)" is used from > "ClassFileParser::fill_instance_klass() and before your change that > function used the previous version of "klass_at_put(int class_index, > Klass* k)" which did call "release_tag_at_put()". > > 4. In ConstantPool::copy_entry_to() you've changed the behavior for > tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, > JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was > copied to the new constant pool if one existed but now you always only > copy a class_index to the new constant pool (even if a resolved klass > existed). Is that OK? E.g. won't this lead to a new resolving for the > new constant pool and will this have performance impacts or other side > effects? > > Thanks again for doing this nice change and best regards, > Volker > > > On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >> Hi Lois, >> >> I have updated the patch to include your comments, and fixes the handling of >> anonymous classes. I also added some more comments regarding the >> _temp_resolved_klass_index: >> >> (delta from last webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >> >> (full webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >> >> Thanks >> - Ioi >> >> >> On 4/15/17 2:31 AM, Lois Foltan wrote: >>> >>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>> >>>> >>>> >>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>> >>>>> HI Lois, >>>>> >>>>> Thanks for the review. Please see my comments in-line. >>>>> >>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>> >>>>>> Hi Ioi, >>>>>> >>>>>> Looks really good. A couple of comments: >>>>>> >>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>> surrounding anonymous classes. Coleen and I discussed earlier today and I >>>>>> came away from that discussion with the idea that the only classes being >>>>>> patched currently are anonymous classes. >>>>> >>>>> Line 5676 ... >>>>> >>>>> 5676 if (is_anonymous()) { >>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>> index >>>>> 5678 } >>>>> >>>>> corresponds to >>>>> >>>>> 5361 ik->set_name(_class_name); >>>>> 5362 >>>>> 5363 if (is_anonymous()) { >>>>> 5364 // I am well known to myself >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> 5366 } >>>>> >>>>> Even though the class is "anonymous", it actually has a name. ik->name() >>>>> probably is part of the constant pool, but I am not 100% sure. Also, I would >>>>> need to search the constant pool to find the index for ik->name(). So I just >>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>> append ik->name() to the end of the constant pool. >>>>> >>>>> "Anonymous" actually means "the class cannot be looked up by name in the >>>>> SystemDictionary". I think we need a better terminology :-) >>>>> >>>> >>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>> modify the comments to the following: >>>> >>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>> // anonymous class itself. If this class needs to refer to its own >>>> methods or >>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>> reference >>>> // _this_class_index. However, because this class is anonymous (it's >>>> // not stored in SystemDictionary), _this_class_index cannot be >>>> resolved >>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>> lookup. >>>> // Therefore, we must eagerly resolve _this_class_index now. >>>> >>>> So, Lois is right. Line 5676 is not necessary. I will revise the code to >>>> do the "eager resolution" without using ClassFileParser::patch_class. I'll >>>> post the updated code later. >>> >>> >>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>> updated webrev. >>> Lois >>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>> java/lang/Class on line #5684. >>>>> >>>>> 5683 Handle patch = cp_patch_at(i); >>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>> java_lang_Class::is_instance(patch())) { >>>>> 5685 // We need to append the names of the patched classes to >>>>> the end of the constant pool, >>>>> 5686 // because a patched class may have a Utf8 name that's >>>>> not already included in the >>>>> 5687 // original constant pool. >>>>> 5688 // >>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>> patch a Utf8, a String, or a Class. >>>>> 5690 // At this point, we don't know the tag for index i yet, >>>>> because we haven't parsed the >>>>> 5691 // constant pool. So we can only assume the worst -- >>>>> every String is used to patch a Class. >>>>> 5692 _max_num_patched_klasses ++; >>>>> >>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>> either Class or String could be treated as a Klass: >>>>> >>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>> 725 int index, >>>>> 726 Handle patch, >>>>> 727 TRAPS) { >>>>> ... >>>>> 732 switch (cp->tag_at(index).value()) { >>>>> 733 >>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>> 735 // Patching a class means pre-resolving it. >>>>> 736 // The name in the constant pool is ignored. >>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>> 739 "Illegal class patch at %d in class file >>>>> %s", >>>>> 740 index, CHECK); >>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>> 742 patch_class(cp, index, k, k->name()); >>>>> 743 } else { >>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>> 745 "Illegal class patch at %d in class file >>>>> %s", >>>>> 746 index, CHECK); >>>>> 747 Symbol* const name = java_lang_String::as_symbol(patch(), >>>>> CHECK); >>>>> 748 patch_class(cp, index, NULL, name); >>>>> 749 } >>>>> 750 break; >>>>> 751 } >>>>> >>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>> >>>>> >>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>> ik->name() is in the cp_patch array. >>>>> >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> >>>>>> Also why not do this calculation in the rewriter or is that too late? >>>>>> >>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and the >>>>> associated tags array is allocated (both of which are fixed size, and cannot >>>>> be expanded), which is way before the rewriter is run. At this point, we >>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to make >>>>> a worst-case estimate on how long the CP/tags should be. >>>>> >>>>>> * line #5677, 5692 - a nit but I think the convention is to not have a >>>>>> space after the variable name and between the post increment operator. >>>>>> >>>>> Fixed. >>>>>> >>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>> really is not so much invalid as temporary for class redefinition purposes, >>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please consider >>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to rename >>>>>> the field or not, please add a one line comment ahead of >>>>>> ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses >>>>>> this currently. >>>>>> >>>>> Good idea. Will do. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Great change, thanks! >>>>>> Lois >>>>>> >>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>> >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>> >>>>>>> In addition to your requests, I made these changes: >>>>>>> >>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>> >>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>> int name_index = kslot.name_index(); >>>>>>> >>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick this >>>>>>> function over MetaspaceObj::is_shared(). >>>>>>> >>>>>>> [3] Massaged the CDS region size set-up code a little to pass internal >>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time picking >>>>>>> the "right" sizes, as this code will be obsoleted soon with JDK-8072061 >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>> >>>>>>>> >>>>>>>> This looks really good! >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>> >>>>>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>>>>> + Array* tags = >>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // >>>>>>>> tags->at(_length) is flags() >>>>>>>> >>>>>>>> >>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>> >>>>>>>> + Klass** adr = >>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>> resolved >>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>> + // hardware store ordering here. >>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>> + return k; >>>>>>>> >>>>>>>> The comment still refers to the switch between Symbol* and Klass* in >>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>> >>>>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>>>> >>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header file >>>>>>>> (so it's inlined) and have all the places where you get name_index use this >>>>>>>> function? So we don't have to know in multiple places that >>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about what the >>>>>>>> new format of the constant pool entry is {name_index, resolved_klass_index}. >>>>>>>> I'm happy to see this work nearing completion! The "constant" pool should >>>>>>>> be constant! thanks, Coleen >>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> Hi,please review the following change >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from >>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool >>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + >>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. In >>>>>>>>> addition: + I moved _resolved_references and _reference_map from >>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no longer >>>>>>>>> modified for shared ConstantPools. As a result, none of the fields in >>>>>>>>> shared ConstantPools are modified at run time, so we can move them into the >>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results show no >>>>>>>>> performance regression, despite the extra level of indirection, which has a >>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 and >>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support ports. For the >>>>>>>>> aarch64, ppc and s390 ports, I have added some code without testing (it's >>>>>>>>> probably incomplete) - Port owners, please check if my patch work for you, >>>>>>>>> and I can incorporate your changes in my push. Alternatively, you can wait >>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, and I will >>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> From chris.plummer at oracle.com Fri Apr 21 05:54:40 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 20 Apr 2017 22:54:40 -0700 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Message-ID: On 4/19/17 11:17 PM, Kim Barrett wrote: > Please review this addition of the count_trailing_zeros function. > > Unfortunately, there isn't an obvious direct and portable way to write > such a function. But supported hardware architectures generally > provide an efficient implementation, e.g. a single instruction or a > short sequence. Compilers often provide "built-in" or "intrinsic" > access to that hardware implementation, or one can use inline > assembler. > > If a platform doesn't have such a built-in efficient implementation, > the function can be implemented using de Bruijn sequences as discussed > in the literature. But all of the OpenJDK-supported platforms provide > an efficient implementation, so we aren't providing such a fallback. > > As part of reviewing this change, please feel free to suggest > alternative ways to structure the code. I'm not completely happy with > the way I've done it, but alternatives I've tried seemed worse. Are you talking about the #if/#elseif structure and the ////// delimiters. I'd suggest replacing the ////// with a block comment that stands out. Something like: /************** * GCC Support **************/ Otherwise it looks fine to me, but I'm no expert in this area. The testing looks satisfactory and the bitmap results you are getting are reason enough to add this functionality. Chris > > CR: > https://bugs.openjdk.java.net/browse/JDK-8179004 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8179004/hotspot.00/ > > Testing: > Unit test for new function. > > Experimented with modifying BitMap::get_next_one_offset and the like > to replace the existing for-loop with a call to this new function, and > measured substantial speedups on all tested platforms for a test > program that counts the bits in a bitmap by iterating calls to that > search function. The speedup varies with the density of set bits, with > very sparse or nearly full being similar to the existing code, since > the bit counting is not the dominant factor in those situations. But > in between give speedups of factors of x1.5 to x5 or more, depending > on the density and the platform. > From david.holmes at oracle.com Fri Apr 21 07:07:53 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Apr 2017 17:07:53 +1000 Subject: RFR(xs) JDK-8163861 JNI NewString() and GetStringLength() documentation incorrect In-Reply-To: References: <26815c67-1e2c-30cb-6435-2be6f713f42a@oracle.com> Message-ID: On 21/04/2017 12:25 AM, George Triantafillou wrote: > Hi David, > > Thanks for the review. I wasn't aware that the CSR isn't yet available > for 10. Do you know when it will be available? "real soon now" David > On 4/19/2017 8:48 PM, David Holmes wrote: >> Hi George, >> >> Apologies in advance - this is not "xs" in the discussion. But this >> needs to go through CSR process (which is not yet available for 10) so >> it can't be pushed yet anyway (sorry). But consider this my CSR >> review. :) >> >> On 20/04/2017 3:43 AM, George Triantafillou wrote: >>> Please review this very small fix to the spec for JDK-8163861: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8163861 >>> >>> webrev: http://cr.openjdk.java.net/~gtriantafill/8163861-webrev/webrev >>> >>> >>> The change provides a more precise definition for NewString() and >>> GetStringLength(). Thanks. >> >> As the bug report states 'most of the Java documention was updated to >> deal with this', so I think it important to ensure any changes are >> consistent with that updated documentation and terminology. >> Specifically we need to refer to the String class API specification >> and the Character class specification which defines Unicode Character >> Representations for the Java platform. >> >> Looking at the JDK 8 JNI spec as referenced in the bug report we first >> see: >> >> jsize GetStringLength(JNIEnv *env, jstring string); >> Returns the length (the count of Unicode characters) of a Java string. >> >> This should be functionally equivalent to java.lang.String.length(). >> If anything the "count of Unicode characters" should just be deleted >> as this method is intended to return the length of the Java string, >> just as String.length does. The phrase "count of Unicode characters" >> is at best "loose" as the reporter notes (and this terminology is >> mis-used in a few of the JNI string functions!). The String.length() >> method states: >> >> "The length is equal to the number of Unicode code units in the string." >> >> where "Unicode code units" is defined in the Character class. >> >> So the correct fix would be to change "characters" to "code units". >> >> However that there are two descriptions of how the function behaves - >> in javadoc terminology we have one description in the main body: >> >> 4168

Returns the length (the count of Unicode >> 4169 characters) of a Java string.

>> >> and one equivalent to @returns: >> >> 4180

RETURNS:

>> 4181 >> 4182

Returns the length of the Java string.

>> >> The latter is perfectly and accurately correct! If you want to know >> how the length of a Java String is defined then you need to see the >> String class. >> >> So I would say that the correct fix here is to either delete "(the >> count of Unicode characters)", or else change "characters" to "code >> units" as previously discussed. > That sounds reasonable! > >> >> --- >> >> Turning to NewString .... the JNI spec states: >> >> "Constructs a new java.lang.String object from an array of Unicode >> characters." >> >> and the @param equivalent for 'len' states: >> >> "len: length of the Unicode string. May be 0." >> >> So initially we have the problematic use of "Unicode characters" >> again; then we have the problem as to what the definition of the >> length of a "Unicode string" is. >> >> I find no help elsewhere in the JNI spec to clarify this (despite a >> very lengthy discussion on its use of UTF-8 encodings). I can only >> assume, based on the implementation, that the expectation is that the >> array of "Unicode characters" is in UTF-16 format. In which case a >> more technically accurate fix would be to say: >> >> "len: length of the Unicode string in UTF-16 code units. May be 0." >> >> Thanks, >> David >> >> PS. I will also add the above to the bug report. > Thanks, I appreciate it. > > -George >> >>> -George > From doug.simon at oracle.com Fri Apr 21 11:46:11 2017 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 21 Apr 2017 13:46:11 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> Message-ID: <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> There has been some discussion about whether we want to update Graal in the JDK at this late stage. The main (only?) risk is a regression in the AOT tool. If we don't update Graal from upstream, then the qualified exports from JVMCI to jdk.internal.vm.compiler cannot be removed in JDK 9. Note that in addition to updating Graal to remove the qualified exports, there would also need to be changes in the relevant make files to add --add-exports options when compiling Graal and jaotc as they use the dynamically exported JVMCI packages. I have an updated hotspot patch that adapts Graal to the JVMCI API changes: http://cr.openjdk.java.net/~dnsimon/8177845/hotspot.02/ Note that this patch does not include the changes removing use of JDK internal API from Graal. Cherry picking those upstream Graal changes would be more work than simply doing a complete update from upstream Graal. As I see it, there are 2 options here: 1. Go with the current webrev (including hotspot.02 patch) and live with the qualified exports. 2. Go with the current webrev (including hotspot.02 patch) and create a follow up bug to update Graal from upstream, perform the relevant make file changes and remove the qualified exports. I've tested the current webrev (including hotspot.02 patch) against both upstream Graal and with jprt. -Doug > On 20 Apr 2017, at 20:50, Doug Simon wrote: > > I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an > exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg > tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. > Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to > the version compatible with the JVMCI API changes. > > In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these > tests better isolates them from parts of the runtime they are not aiming to test. > > org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties > at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) > at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) > at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) > at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) > at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) > at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) > at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) > at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) > at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) > at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) > at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:563) > at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) > at java.base/java.lang.Thread.run(Thread.java:844) > Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties > at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) > ... 26 more > > -Doug > >> On 19 Apr 2017, at 23:26, Doug Simon wrote: >> >> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: >> >> 1. JVMCIServiceLocator.getProvider(Class) is now protected >> 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission >> 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK >> >> -Doug >> >>> On 19 Apr 2017, at 23:12, Doug Simon wrote: >>> >>>> >>>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>>> >>>>> >>>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>>> >>>>>> >>>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>>> >>>>>> >>>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>>> >>>>>>> >>>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>>> >>>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>>> >>>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>>> >>>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>>> >>>>>> It's unqualified and no restriction in this change. >>>>> >>>>> The public methods currently in jdk.vm.ci.services are: >>>>> >>>>> 1. JVMCIServiceLocator.getProvider(Class) >>>>> 2. JVMCIServiceLocator.getProviders(Class) >>>>> 3. Services.initializeJVMCI() >>>>> 4. Services.getSavedProperties() >>>>> 5. Services.exportJVMCITo(Class) >>>>> 6. Services.load(Class) >>>>> 7. Services.loadSingle(Class, boolean) >>>>> >>>>> 1 should be made protected. I'll update the webrev with this change. >>>> >>>> Good. >>>> >>>>> >>>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>>> >>>> Good. >>>> >>>>> >>>>> 3 is harmless from a security perspective in my opinion. >>>> >>>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >>> >>> Vladimir, can you please bring this to the attention of the relevant engineer. >>> >>>>> >>>>> 4 checks for JVMCIPermission. >>>> >>>> Ok. >>>> >>>>> >>>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>>> >>>> About this, will this Graal update happen for JDK 9? >>> >>> Yes. >>> >>>> It?s awfully late in the cycle... >>> >>> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >>> >>> -Doug >> > From volker.simonis at gmail.com Fri Apr 21 13:27:03 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 21 Apr 2017 15:27:03 +0200 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <31A1E52F-E10F-4193-AE22-DC9FBF173253@sap.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> <31A1E52F-E10F-4193-AE22-DC9FBF173253@sap.com> Message-ID: Thanks, Lutz! On Thu, Apr 20, 2017 at 11:24 PM, Schmidt, Lutz wrote: > Hi Volker, > > you additions are ok for me - they are as good as it gets when you add pointer indirections. My ok statement holds for s390x in particular, but ppc looks good as well. Let?s see if Martin has extended comments on ppc. > > Regards, Lutz > > >> On 20 Apr 2017, at 18:02, Volker Simonis wrote: >> >> Hi Ioi, >> >> thanks once again for considering our ports! Please find the required >> additions for ppc64/s390x in the following webrew (which is based upon >> your latest v03 patch): >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ >> >> @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly >> code. I did some tests and I think it should be correct, but maybe you >> still find some improvements :) >> >> Besides that, I have some general questions/comments regarding your change: >> >> 1. In constantPool.hpp, why don't you declare the '_name_index' and >> '_resolved_klass_index' fields with type 'jushort'? As far as I can >> see, they can only hold 16-bit values anyway. It would also save you >> some space and several asserts (e.g. in unresolved_klass_at_put(): >> >> >> 274 assert((name_index & 0xffff0000) == 0, "must be"); >> 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); >> >> 2. What do you mean by: >> >> 106 // ... will be changed to support compressed pointers >> 107 Array* _resolved_klasses; >> >> 3. Why don't we need the call to "release_tag_at_put()" in >> "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int >> class_index, Klass* k)" is used from >> "ClassFileParser::fill_instance_klass() and before your change that >> function used the previous version of "klass_at_put(int class_index, >> Klass* k)" which did call "release_tag_at_put()". >> >> 4. In ConstantPool::copy_entry_to() you've changed the behavior for >> tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, >> JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was >> copied to the new constant pool if one existed but now you always only >> copy a class_index to the new constant pool (even if a resolved klass >> existed). Is that OK? E.g. won't this lead to a new resolving for the >> new constant pool and will this have performance impacts or other side >> effects? >> >> Thanks again for doing this nice change and best regards, >> Volker >> >> >> On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >>> Hi Lois, >>> >>> I have updated the patch to include your comments, and fixes the handling of >>> anonymous classes. I also added some more comments regarding the >>> _temp_resolved_klass_index: >>> >>> (delta from last webrev) >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >>> >>> (full webrev) >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >>> >>> Thanks >>> - Ioi >>> >>> >>> On 4/15/17 2:31 AM, Lois Foltan wrote: >>>> >>>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>>> >>>>> >>>>> >>>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>>> >>>>>> HI Lois, >>>>>> >>>>>> Thanks for the review. Please see my comments in-line. >>>>>> >>>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>>> >>>>>>> Hi Ioi, >>>>>>> >>>>>>> Looks really good. A couple of comments: >>>>>>> >>>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>>> surrounding anonymous classes. Coleen and I discussed earlier today and I >>>>>>> came away from that discussion with the idea that the only classes being >>>>>>> patched currently are anonymous classes. >>>>>> >>>>>> Line 5676 ... >>>>>> >>>>>> 5676 if (is_anonymous()) { >>>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>>> index >>>>>> 5678 } >>>>>> >>>>>> corresponds to >>>>>> >>>>>> 5361 ik->set_name(_class_name); >>>>>> 5362 >>>>>> 5363 if (is_anonymous()) { >>>>>> 5364 // I am well known to myself >>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>> ik->name()); // eagerly resolve >>>>>> 5366 } >>>>>> >>>>>> Even though the class is "anonymous", it actually has a name. ik->name() >>>>>> probably is part of the constant pool, but I am not 100% sure. Also, I would >>>>>> need to search the constant pool to find the index for ik->name(). So I just >>>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>>> append ik->name() to the end of the constant pool. >>>>>> >>>>>> "Anonymous" actually means "the class cannot be looked up by name in the >>>>>> SystemDictionary". I think we need a better terminology :-) >>>>>> >>>>> >>>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>>> modify the comments to the following: >>>>> >>>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>>> // anonymous class itself. If this class needs to refer to its own >>>>> methods or >>>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>>> reference >>>>> // _this_class_index. However, because this class is anonymous (it's >>>>> // not stored in SystemDictionary), _this_class_index cannot be >>>>> resolved >>>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>>> lookup. >>>>> // Therefore, we must eagerly resolve _this_class_index now. >>>>> >>>>> So, Lois is right. Line 5676 is not necessary. I will revise the code to >>>>> do the "eager resolution" without using ClassFileParser::patch_class. I'll >>>>> post the updated code later. >>>> >>>> >>>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>>> updated webrev. >>>> Lois >>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>>> java/lang/Class on line #5684. >>>>>> >>>>>> 5683 Handle patch = cp_patch_at(i); >>>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>>> java_lang_Class::is_instance(patch())) { >>>>>> 5685 // We need to append the names of the patched classes to >>>>>> the end of the constant pool, >>>>>> 5686 // because a patched class may have a Utf8 name that's >>>>>> not already included in the >>>>>> 5687 // original constant pool. >>>>>> 5688 // >>>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>>> patch a Utf8, a String, or a Class. >>>>>> 5690 // At this point, we don't know the tag for index i yet, >>>>>> because we haven't parsed the >>>>>> 5691 // constant pool. So we can only assume the worst -- >>>>>> every String is used to patch a Class. >>>>>> 5692 _max_num_patched_klasses ++; >>>>>> >>>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>>> either Class or String could be treated as a Klass: >>>>>> >>>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>>> 725 int index, >>>>>> 726 Handle patch, >>>>>> 727 TRAPS) { >>>>>> ... >>>>>> 732 switch (cp->tag_at(index).value()) { >>>>>> 733 >>>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>>> 735 // Patching a class means pre-resolving it. >>>>>> 736 // The name in the constant pool is ignored. >>>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>>> 739 "Illegal class patch at %d in class file >>>>>> %s", >>>>>> 740 index, CHECK); >>>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>>> 742 patch_class(cp, index, k, k->name()); >>>>>> 743 } else { >>>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>>> 745 "Illegal class patch at %d in class file >>>>>> %s", >>>>>> 746 index, CHECK); >>>>>> 747 Symbol* const name = java_lang_String::as_symbol(patch(), >>>>>> CHECK); >>>>>> 748 patch_class(cp, index, NULL, name); >>>>>> 749 } >>>>>> 750 break; >>>>>> 751 } >>>>>> >>>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>>> >>>>>> >>>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>>> ik->name() is in the cp_patch array. >>>>>> >>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>> ik->name()); // eagerly resolve >>>>>> >>>>>>> Also why not do this calculation in the rewriter or is that too late? >>>>>>> >>>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and the >>>>>> associated tags array is allocated (both of which are fixed size, and cannot >>>>>> be expanded), which is way before the rewriter is run. At this point, we >>>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to make >>>>>> a worst-case estimate on how long the CP/tags should be. >>>>>> >>>>>>> * line #5677, 5692 - a nit but I think the convention is to not have a >>>>>>> space after the variable name and between the post increment operator. >>>>>>> >>>>>> Fixed. >>>>>>> >>>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>>> really is not so much invalid as temporary for class redefinition purposes, >>>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please consider >>>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to rename >>>>>>> the field or not, please add a one line comment ahead of >>>>>>> ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses >>>>>>> this currently. >>>>>>> >>>>>> Good idea. Will do. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>>> Great change, thanks! >>>>>>> Lois >>>>>>> >>>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>>> >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>>> >>>>>>>> In addition to your requests, I made these changes: >>>>>>>> >>>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>>> >>>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>>> int name_index = kslot.name_index(); >>>>>>>> >>>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick this >>>>>>>> function over MetaspaceObj::is_shared(). >>>>>>>> >>>>>>>> [3] Massaged the CDS region size set-up code a little to pass internal >>>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time picking >>>>>>>> the "right" sizes, as this code will be obsoleted soon with JDK-8072061 >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> This looks really good! >>>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>>> >>>>>>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>>>>>> + Array* tags = >>>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // >>>>>>>>> tags->at(_length) is flags() >>>>>>>>> >>>>>>>>> >>>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>>> >>>>>>>>> + Klass** adr = >>>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>>> resolved >>>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>>> + // hardware store ordering here. >>>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>>> + return k; >>>>>>>>> >>>>>>>>> The comment still refers to the switch between Symbol* and Klass* in >>>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>>> >>>>>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>>>>> >>>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header file >>>>>>>>> (so it's inlined) and have all the places where you get name_index use this >>>>>>>>> function? So we don't have to know in multiple places that >>>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about what the >>>>>>>>> new format of the constant pool entry is {name_index, resolved_klass_index}. >>>>>>>>> I'm happy to see this work nearing completion! The "constant" pool should >>>>>>>>> be constant! thanks, Coleen >>>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>>> >>>>>>>>>> Hi,please review the following change >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from >>>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool >>>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>>> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + >>>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. In >>>>>>>>>> addition: + I moved _resolved_references and _reference_map from >>>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no longer >>>>>>>>>> modified for shared ConstantPools. As a result, none of the fields in >>>>>>>>>> shared ConstantPools are modified at run time, so we can move them into the >>>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results show no >>>>>>>>>> performance regression, despite the extra level of indirection, which has a >>>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 and >>>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support ports. For the >>>>>>>>>> aarch64, ppc and s390 ports, I have added some code without testing (it's >>>>>>>>>> probably incomplete) - Port owners, please check if my patch work for you, >>>>>>>>>> and I can incorporate your changes in my push. Alternatively, you can wait >>>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, and I will >>>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> > From martin.doerr at sap.com Fri Apr 21 13:29:53 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 21 Apr 2017 13:29:53 +0000 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <31A1E52F-E10F-4193-AE22-DC9FBF173253@sap.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> <31A1E52F-E10F-4193-AE22-DC9FBF173253@sap.com> Message-ID: Hi, the PPC64 (big and little Endian) and s390x parts look correct. Thanks for porting them, Volker. On PPC64, the memory barrier is very tricky, but I have convinced myself that it's correct. The _resolved_klasses array gets allocated during initialization and just the tag gets patched after class resolution. So it is sufficient to order only the load of the tag wrt. the load of the Klass* which is exactly what you have implemented. @Ioi: I think it would be nice to have a little more comments at all places which use release_tag... or release_store_ptr and the respective acquire functions. It takes quite some time to find out what exactly needs to be ordered with what, where are possible races etc. But it's just on my wish list, not a must have for your change. Thanks also from my side for taking care of our platforms. Best regards, Martin -----Original Message----- From: Schmidt, Lutz Sent: Donnerstag, 20. April 2017 23:24 To: Volker Simonis Cc: Ioi Lam ; Doerr, Martin ; Lois Foltan ; HotSpot Open Source Developers Subject: Re: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only Hi Volker, you additions are ok for me - they are as good as it gets when you add pointer indirections. My ok statement holds for s390x in particular, but ppc looks good as well. Let?s see if Martin has extended comments on ppc. Regards, Lutz > On 20 Apr 2017, at 18:02, Volker Simonis wrote: > > Hi Ioi, > > thanks once again for considering our ports! Please find the required > additions for ppc64/s390x in the following webrew (which is based upon > your latest v03 patch): > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ > > @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly > code. I did some tests and I think it should be correct, but maybe you > still find some improvements :) > > Besides that, I have some general questions/comments regarding your change: > > 1. In constantPool.hpp, why don't you declare the '_name_index' and > '_resolved_klass_index' fields with type 'jushort'? As far as I can > see, they can only hold 16-bit values anyway. It would also save you > some space and several asserts (e.g. in unresolved_klass_at_put(): > > > 274 assert((name_index & 0xffff0000) == 0, "must be"); > 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); > > 2. What do you mean by: > > 106 // ... will be changed to support compressed pointers > 107 Array* _resolved_klasses; > > 3. Why don't we need the call to "release_tag_at_put()" in > "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int > class_index, Klass* k)" is used from > "ClassFileParser::fill_instance_klass() and before your change that > function used the previous version of "klass_at_put(int class_index, > Klass* k)" which did call "release_tag_at_put()". > > 4. In ConstantPool::copy_entry_to() you've changed the behavior for > tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, > JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was > copied to the new constant pool if one existed but now you always only > copy a class_index to the new constant pool (even if a resolved klass > existed). Is that OK? E.g. won't this lead to a new resolving for the > new constant pool and will this have performance impacts or other side > effects? > > Thanks again for doing this nice change and best regards, > Volker > > > On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >> Hi Lois, >> >> I have updated the patch to include your comments, and fixes the handling of >> anonymous classes. I also added some more comments regarding the >> _temp_resolved_klass_index: >> >> (delta from last webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >> >> (full webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >> >> Thanks >> - Ioi >> >> >> On 4/15/17 2:31 AM, Lois Foltan wrote: >>> >>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>> >>>> >>>> >>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>> >>>>> HI Lois, >>>>> >>>>> Thanks for the review. Please see my comments in-line. >>>>> >>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>> >>>>>> Hi Ioi, >>>>>> >>>>>> Looks really good. A couple of comments: >>>>>> >>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>> surrounding anonymous classes. Coleen and I discussed earlier today and I >>>>>> came away from that discussion with the idea that the only classes being >>>>>> patched currently are anonymous classes. >>>>> >>>>> Line 5676 ... >>>>> >>>>> 5676 if (is_anonymous()) { >>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>> index >>>>> 5678 } >>>>> >>>>> corresponds to >>>>> >>>>> 5361 ik->set_name(_class_name); >>>>> 5362 >>>>> 5363 if (is_anonymous()) { >>>>> 5364 // I am well known to myself >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> 5366 } >>>>> >>>>> Even though the class is "anonymous", it actually has a name. ik->name() >>>>> probably is part of the constant pool, but I am not 100% sure. Also, I would >>>>> need to search the constant pool to find the index for ik->name(). So I just >>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>> append ik->name() to the end of the constant pool. >>>>> >>>>> "Anonymous" actually means "the class cannot be looked up by name in the >>>>> SystemDictionary". I think we need a better terminology :-) >>>>> >>>> >>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>> modify the comments to the following: >>>> >>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>> // anonymous class itself. If this class needs to refer to its own >>>> methods or >>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>> reference >>>> // _this_class_index. However, because this class is anonymous (it's >>>> // not stored in SystemDictionary), _this_class_index cannot be >>>> resolved >>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>> lookup. >>>> // Therefore, we must eagerly resolve _this_class_index now. >>>> >>>> So, Lois is right. Line 5676 is not necessary. I will revise the code to >>>> do the "eager resolution" without using ClassFileParser::patch_class. I'll >>>> post the updated code later. >>> >>> >>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>> updated webrev. >>> Lois >>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>> java/lang/Class on line #5684. >>>>> >>>>> 5683 Handle patch = cp_patch_at(i); >>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>> java_lang_Class::is_instance(patch())) { >>>>> 5685 // We need to append the names of the patched classes to >>>>> the end of the constant pool, >>>>> 5686 // because a patched class may have a Utf8 name that's >>>>> not already included in the >>>>> 5687 // original constant pool. >>>>> 5688 // >>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>> patch a Utf8, a String, or a Class. >>>>> 5690 // At this point, we don't know the tag for index i yet, >>>>> because we haven't parsed the >>>>> 5691 // constant pool. So we can only assume the worst -- >>>>> every String is used to patch a Class. >>>>> 5692 _max_num_patched_klasses ++; >>>>> >>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>> either Class or String could be treated as a Klass: >>>>> >>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>> 725 int index, >>>>> 726 Handle patch, >>>>> 727 TRAPS) { >>>>> ... >>>>> 732 switch (cp->tag_at(index).value()) { >>>>> 733 >>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>> 735 // Patching a class means pre-resolving it. >>>>> 736 // The name in the constant pool is ignored. >>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>> 739 "Illegal class patch at %d in class file >>>>> %s", >>>>> 740 index, CHECK); >>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>> 742 patch_class(cp, index, k, k->name()); >>>>> 743 } else { >>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>> 745 "Illegal class patch at %d in class file >>>>> %s", >>>>> 746 index, CHECK); >>>>> 747 Symbol* const name = java_lang_String::as_symbol(patch(), >>>>> CHECK); >>>>> 748 patch_class(cp, index, NULL, name); >>>>> 749 } >>>>> 750 break; >>>>> 751 } >>>>> >>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>> >>>>> >>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>> ik->name() is in the cp_patch array. >>>>> >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> >>>>>> Also why not do this calculation in the rewriter or is that too late? >>>>>> >>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and the >>>>> associated tags array is allocated (both of which are fixed size, and cannot >>>>> be expanded), which is way before the rewriter is run. At this point, we >>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to make >>>>> a worst-case estimate on how long the CP/tags should be. >>>>> >>>>>> * line #5677, 5692 - a nit but I think the convention is to not have a >>>>>> space after the variable name and between the post increment operator. >>>>>> >>>>> Fixed. >>>>>> >>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>> really is not so much invalid as temporary for class redefinition purposes, >>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please consider >>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to rename >>>>>> the field or not, please add a one line comment ahead of >>>>>> ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses >>>>>> this currently. >>>>>> >>>>> Good idea. Will do. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Great change, thanks! >>>>>> Lois >>>>>> >>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>> >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>> >>>>>>> In addition to your requests, I made these changes: >>>>>>> >>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>> >>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>> int name_index = kslot.name_index(); >>>>>>> >>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick this >>>>>>> function over MetaspaceObj::is_shared(). >>>>>>> >>>>>>> [3] Massaged the CDS region size set-up code a little to pass internal >>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time picking >>>>>>> the "right" sizes, as this code will be obsoleted soon with JDK-8072061 >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>> >>>>>>>> >>>>>>>> This looks really good! >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>> >>>>>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>>>>> + Array* tags = >>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // >>>>>>>> tags->at(_length) is flags() >>>>>>>> >>>>>>>> >>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>> >>>>>>>> + Klass** adr = >>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>> resolved >>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>> + // hardware store ordering here. >>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>> + return k; >>>>>>>> >>>>>>>> The comment still refers to the switch between Symbol* and Klass* in >>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>> >>>>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>>>> >>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header file >>>>>>>> (so it's inlined) and have all the places where you get name_index use this >>>>>>>> function? So we don't have to know in multiple places that >>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about what the >>>>>>>> new format of the constant pool entry is {name_index, resolved_klass_index}. >>>>>>>> I'm happy to see this work nearing completion! The "constant" pool should >>>>>>>> be constant! thanks, Coleen >>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> Hi,please review the following change >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from >>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool >>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + >>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. In >>>>>>>>> addition: + I moved _resolved_references and _reference_map from >>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no longer >>>>>>>>> modified for shared ConstantPools. As a result, none of the fields in >>>>>>>>> shared ConstantPools are modified at run time, so we can move them into the >>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results show no >>>>>>>>> performance regression, despite the extra level of indirection, which has a >>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 and >>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support ports. For the >>>>>>>>> aarch64, ppc and s390 ports, I have added some code without testing (it's >>>>>>>>> probably incomplete) - Port owners, please check if my patch work for you, >>>>>>>>> and I can incorporate your changes in my push. Alternatively, you can wait >>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, and I will >>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> From volker.simonis at gmail.com Fri Apr 21 13:32:23 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 21 Apr 2017 15:32:23 +0200 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> <31A1E52F-E10F-4193-AE22-DC9FBF173253@sap.com> Message-ID: Thanks for looking at the change Martin! On Fri, Apr 21, 2017 at 3:29 PM, Doerr, Martin wrote: > Hi, > > the PPC64 (big and little Endian) and s390x parts look correct. Thanks for porting them, Volker. > > On PPC64, the memory barrier is very tricky, but I have convinced myself that it's correct. > The _resolved_klasses array gets allocated during initialization and just the tag gets patched after class resolution. > So it is sufficient to order only the load of the tag wrt. the load of the Klass* which is exactly what you have implemented. > > @Ioi: I think it would be nice to have a little more comments at all places which use release_tag... or release_store_ptr and the respective acquire functions. > It takes quite some time to find out what exactly needs to be ordered with what, where are possible races etc. > But it's just on my wish list, not a must have for your change. Thanks also from my side for taking care of our platforms. > > Best regards, > Martin > > > -----Original Message----- > From: Schmidt, Lutz > Sent: Donnerstag, 20. April 2017 23:24 > To: Volker Simonis > Cc: Ioi Lam ; Doerr, Martin ; Lois Foltan ; HotSpot Open Source Developers > Subject: Re: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only > > Hi Volker, > > you additions are ok for me - they are as good as it gets when you add pointer indirections. My ok statement holds for s390x in particular, but ppc looks good as well. Let?s see if Martin has extended comments on ppc. > > Regards, Lutz > > >> On 20 Apr 2017, at 18:02, Volker Simonis wrote: >> >> Hi Ioi, >> >> thanks once again for considering our ports! Please find the required >> additions for ppc64/s390x in the following webrew (which is based upon >> your latest v03 patch): >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ >> >> @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly >> code. I did some tests and I think it should be correct, but maybe you >> still find some improvements :) >> >> Besides that, I have some general questions/comments regarding your change: >> >> 1. In constantPool.hpp, why don't you declare the '_name_index' and >> '_resolved_klass_index' fields with type 'jushort'? As far as I can >> see, they can only hold 16-bit values anyway. It would also save you >> some space and several asserts (e.g. in unresolved_klass_at_put(): >> >> >> 274 assert((name_index & 0xffff0000) == 0, "must be"); >> 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); >> >> 2. What do you mean by: >> >> 106 // ... will be changed to support compressed pointers >> 107 Array* _resolved_klasses; >> >> 3. Why don't we need the call to "release_tag_at_put()" in >> "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int >> class_index, Klass* k)" is used from >> "ClassFileParser::fill_instance_klass() and before your change that >> function used the previous version of "klass_at_put(int class_index, >> Klass* k)" which did call "release_tag_at_put()". >> >> 4. In ConstantPool::copy_entry_to() you've changed the behavior for >> tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, >> JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was >> copied to the new constant pool if one existed but now you always only >> copy a class_index to the new constant pool (even if a resolved klass >> existed). Is that OK? E.g. won't this lead to a new resolving for the >> new constant pool and will this have performance impacts or other side >> effects? >> >> Thanks again for doing this nice change and best regards, >> Volker >> >> >> On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >>> Hi Lois, >>> >>> I have updated the patch to include your comments, and fixes the handling of >>> anonymous classes. I also added some more comments regarding the >>> _temp_resolved_klass_index: >>> >>> (delta from last webrev) >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >>> >>> (full webrev) >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >>> >>> Thanks >>> - Ioi >>> >>> >>> On 4/15/17 2:31 AM, Lois Foltan wrote: >>>> >>>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>>> >>>>> >>>>> >>>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>>> >>>>>> HI Lois, >>>>>> >>>>>> Thanks for the review. Please see my comments in-line. >>>>>> >>>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>>> >>>>>>> Hi Ioi, >>>>>>> >>>>>>> Looks really good. A couple of comments: >>>>>>> >>>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>>> surrounding anonymous classes. Coleen and I discussed earlier today and I >>>>>>> came away from that discussion with the idea that the only classes being >>>>>>> patched currently are anonymous classes. >>>>>> >>>>>> Line 5676 ... >>>>>> >>>>>> 5676 if (is_anonymous()) { >>>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>>> index >>>>>> 5678 } >>>>>> >>>>>> corresponds to >>>>>> >>>>>> 5361 ik->set_name(_class_name); >>>>>> 5362 >>>>>> 5363 if (is_anonymous()) { >>>>>> 5364 // I am well known to myself >>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>> ik->name()); // eagerly resolve >>>>>> 5366 } >>>>>> >>>>>> Even though the class is "anonymous", it actually has a name. ik->name() >>>>>> probably is part of the constant pool, but I am not 100% sure. Also, I would >>>>>> need to search the constant pool to find the index for ik->name(). So I just >>>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>>> append ik->name() to the end of the constant pool. >>>>>> >>>>>> "Anonymous" actually means "the class cannot be looked up by name in the >>>>>> SystemDictionary". I think we need a better terminology :-) >>>>>> >>>>> >>>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>>> modify the comments to the following: >>>>> >>>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>>> // anonymous class itself. If this class needs to refer to its own >>>>> methods or >>>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>>> reference >>>>> // _this_class_index. However, because this class is anonymous (it's >>>>> // not stored in SystemDictionary), _this_class_index cannot be >>>>> resolved >>>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>>> lookup. >>>>> // Therefore, we must eagerly resolve _this_class_index now. >>>>> >>>>> So, Lois is right. Line 5676 is not necessary. I will revise the code to >>>>> do the "eager resolution" without using ClassFileParser::patch_class. I'll >>>>> post the updated code later. >>>> >>>> >>>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>>> updated webrev. >>>> Lois >>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>>> java/lang/Class on line #5684. >>>>>> >>>>>> 5683 Handle patch = cp_patch_at(i); >>>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>>> java_lang_Class::is_instance(patch())) { >>>>>> 5685 // We need to append the names of the patched classes to >>>>>> the end of the constant pool, >>>>>> 5686 // because a patched class may have a Utf8 name that's >>>>>> not already included in the >>>>>> 5687 // original constant pool. >>>>>> 5688 // >>>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>>> patch a Utf8, a String, or a Class. >>>>>> 5690 // At this point, we don't know the tag for index i yet, >>>>>> because we haven't parsed the >>>>>> 5691 // constant pool. So we can only assume the worst -- >>>>>> every String is used to patch a Class. >>>>>> 5692 _max_num_patched_klasses ++; >>>>>> >>>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>>> either Class or String could be treated as a Klass: >>>>>> >>>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>>> 725 int index, >>>>>> 726 Handle patch, >>>>>> 727 TRAPS) { >>>>>> ... >>>>>> 732 switch (cp->tag_at(index).value()) { >>>>>> 733 >>>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>>> 735 // Patching a class means pre-resolving it. >>>>>> 736 // The name in the constant pool is ignored. >>>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>>> 739 "Illegal class patch at %d in class file >>>>>> %s", >>>>>> 740 index, CHECK); >>>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>>> 742 patch_class(cp, index, k, k->name()); >>>>>> 743 } else { >>>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>>> 745 "Illegal class patch at %d in class file >>>>>> %s", >>>>>> 746 index, CHECK); >>>>>> 747 Symbol* const name = java_lang_String::as_symbol(patch(), >>>>>> CHECK); >>>>>> 748 patch_class(cp, index, NULL, name); >>>>>> 749 } >>>>>> 750 break; >>>>>> 751 } >>>>>> >>>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>>> >>>>>> >>>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>>> ik->name() is in the cp_patch array. >>>>>> >>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>> ik->name()); // eagerly resolve >>>>>> >>>>>>> Also why not do this calculation in the rewriter or is that too late? >>>>>>> >>>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and the >>>>>> associated tags array is allocated (both of which are fixed size, and cannot >>>>>> be expanded), which is way before the rewriter is run. At this point, we >>>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to make >>>>>> a worst-case estimate on how long the CP/tags should be. >>>>>> >>>>>>> * line #5677, 5692 - a nit but I think the convention is to not have a >>>>>>> space after the variable name and between the post increment operator. >>>>>>> >>>>>> Fixed. >>>>>>> >>>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>>> really is not so much invalid as temporary for class redefinition purposes, >>>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please consider >>>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to rename >>>>>>> the field or not, please add a one line comment ahead of >>>>>>> ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses >>>>>>> this currently. >>>>>>> >>>>>> Good idea. Will do. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>>> Great change, thanks! >>>>>>> Lois >>>>>>> >>>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>>> >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>>> >>>>>>>> In addition to your requests, I made these changes: >>>>>>>> >>>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>>> >>>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>>> int name_index = kslot.name_index(); >>>>>>>> >>>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick this >>>>>>>> function over MetaspaceObj::is_shared(). >>>>>>>> >>>>>>>> [3] Massaged the CDS region size set-up code a little to pass internal >>>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time picking >>>>>>>> the "right" sizes, as this code will be obsoleted soon with JDK-8072061 >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> This looks really good! >>>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>>> >>>>>>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>>>>>> + Array* tags = >>>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // >>>>>>>>> tags->at(_length) is flags() >>>>>>>>> >>>>>>>>> >>>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>>> >>>>>>>>> + Klass** adr = >>>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>>> resolved >>>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>>> + // hardware store ordering here. >>>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>>> + return k; >>>>>>>>> >>>>>>>>> The comment still refers to the switch between Symbol* and Klass* in >>>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>>> >>>>>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>>>>> >>>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header file >>>>>>>>> (so it's inlined) and have all the places where you get name_index use this >>>>>>>>> function? So we don't have to know in multiple places that >>>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about what the >>>>>>>>> new format of the constant pool entry is {name_index, resolved_klass_index}. >>>>>>>>> I'm happy to see this work nearing completion! The "constant" pool should >>>>>>>>> be constant! thanks, Coleen >>>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>>> >>>>>>>>>> Hi,please review the following change >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from >>>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool >>>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>>> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + >>>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. In >>>>>>>>>> addition: + I moved _resolved_references and _reference_map from >>>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no longer >>>>>>>>>> modified for shared ConstantPools. As a result, none of the fields in >>>>>>>>>> shared ConstantPools are modified at run time, so we can move them into the >>>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results show no >>>>>>>>>> performance regression, despite the extra level of indirection, which has a >>>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 and >>>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support ports. For the >>>>>>>>>> aarch64, ppc and s390 ports, I have added some code without testing (it's >>>>>>>>>> probably incomplete) - Port owners, please check if my patch work for you, >>>>>>>>>> and I can incorporate your changes in my push. Alternatively, you can wait >>>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, and I will >>>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> > From kim.barrett at oracle.com Fri Apr 21 17:41:14 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Apr 2017 13:41:14 -0400 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Message-ID: > On Apr 21, 2017, at 1:54 AM, Chris Plummer wrote: > > On 4/19/17 11:17 PM, Kim Barrett wrote: >> Please review this addition of the count_trailing_zeros function. >> >> [?] >> As part of reviewing this change, please feel free to suggest >> alternative ways to structure the code. I'm not completely happy with >> the way I've done it, but alternatives I've tried seemed worse. > Are you talking about the #if/#elseif structure and the ////// delimiters. I'd suggest replacing the ////// with a block comment that stands out. Something like: > > /************** > * GCC Support > **************/ Sure, that makes things stand out a bit more. Updated webrev (only comment changes in count_trailing_zeros.hpp) http://cr.openjdk.java.net/~kbarrett/8179004/hotspot.01/ What I'm looking for is whether this TARGET_COMPILER dispatch with the corresponding code is okay, or whether some other structure would be preferred. Some alternatives I considered were: * Always #include OS_CPU_INCLUDE(count_trailing_zeros). But that proliforates files which will be duplicates (possibly duplicates by #include of some shared toolchain-related include). * Dispatch to #include "utilities/count_trailing_zeros_.hpp" similarly to what globalDefinitions.hpp does. This adds 4 small files, which seems a little excessive. * Put these toolchain-specific definitions in the corresponding globalDefintions_.hpp. But the #include of those files by globalDefinitions.hpp happens very early in that file, before things like the uintx type are defined. Also, presently debug.hpp can't be #include'd by globalDefinitions.hpp, so the latter does similar things "by hand". (I think that's something that ought to be fixed.) > Otherwise it looks fine to me, but I'm no expert in this area. The testing looks satisfactory and the bitmap results you are getting are reason enough to add this functionality. Thanks. Do you want to be counted as a reviewer? From chris.plummer at oracle.com Fri Apr 21 18:21:05 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 21 Apr 2017 11:21:05 -0700 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Message-ID: <43f813fb-cb78-eff9-0fb5-bef99d047f25@oracle.com> On 4/21/17 10:41 AM, Kim Barrett wrote: >> On Apr 21, 2017, at 1:54 AM, Chris Plummer wrote: >> >> On 4/19/17 11:17 PM, Kim Barrett wrote: >>> Please review this addition of the count_trailing_zeros function. >>> >>> [?] >>> As part of reviewing this change, please feel free to suggest >>> alternative ways to structure the code. I'm not completely happy with >>> the way I've done it, but alternatives I've tried seemed worse. >> Are you talking about the #if/#elseif structure and the ////// delimiters. I'd suggest replacing the ////// with a block comment that stands out. Something like: >> >> /************** >> * GCC Support >> **************/ > Sure, that makes things stand out a bit more. > > Updated webrev (only comment changes in count_trailing_zeros.hpp) > http://cr.openjdk.java.net/~kbarrett/8179004/hotspot.01/ > > What I'm looking for is whether this TARGET_COMPILER dispatch with the > corresponding code is okay, or whether some other structure would be > preferred. > > Some alternatives I considered were: > > * Always #include OS_CPU_INCLUDE(count_trailing_zeros). But that > proliforates files which will be duplicates (possibly duplicates > by #include of some shared toolchain-related include). > > * Dispatch to #include "utilities/count_trailing_zeros_.hpp" > similarly to what globalDefinitions.hpp does. This adds 4 small > files, which seems a little excessive. globalDefinitions.hpp seems to be the only other case of using in the source. For globalDefinitions.hpp, there is a fair amount of code in each globalDefinitions_.hpp header file, so I think in this case it does help organize the source and adds to readability. In your case we aren't talking about much code, and it may do more harm than good to separate the various impls out this way. > > * Put these toolchain-specific definitions in the corresponding > globalDefintions_.hpp. But the #include of those files by > globalDefinitions.hpp happens very early in that file, before things > like the uintx type are defined. Also, presently debug.hpp can't > be #include'd by globalDefinitions.hpp, so the latter does similar > things "by hand". (I think that's something that ought to be fixed.) I also had this thought before reading your comment above. If your code was all toolchain independent, would you still consider putting it in globalDefintions.hpp? If not, I don't necessarily think that it makes sense to put it there just to piggyback on globalDefinitions.hpp the logic. I skimmed through globalDefintions.hpp and couldn't get enough a good feel for whether or not your API belongs there. > >> Otherwise it looks fine to me, but I'm no expert in this area. The testing looks satisfactory and the bitmap results you are getting are reason enough to add this functionality. > Thanks. Do you want to be counted as a reviewer? Sure. FYI I'm not a (R)eviewer. Chris > From HORIE at jp.ibm.com Fri Apr 21 18:52:49 2017 From: HORIE at jp.ibm.com (Michihiro Horie) Date: Sat, 22 Apr 2017 03:52:49 +0900 Subject: Optimizing byte reverse code for int value In-Reply-To: References: , <622fa3e77da546dfb5155a1e4afacd7c@sap.com>, , <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> <89abbea5-9998-2e4d-62d3-e1f3e9bbd1d5@redhat.com> Message-ID: I noticed my silly mistake that previous change does not work in big endian, although I ran jtreg.. This time I checked also with my micro benchmark ReadFloatTest.java. Would you review the following newest change? Webrev: http://cr.openjdk.java.net/~horii/8178294/webrev.02/ (See attached file: ReadFloatTest.java) Best regards, -- Michihiro, IBM Research - Tokyo ----- Original message ----- From: Michihiro Horie/Japan/IBM To: martin.doerr at sap.com Cc: aph at redhat.com, Hiroshi H Horii/Japan/IBM at IBMJP, hotspot-dev at openjdk.java.net, ppc-aix-port-dev at openjdk.java.net, volker.simonis at sap.com, Gustavo Bueno Romero/Brazil/IBM at IBMBR Subject: RE: Optimizing byte reverse code for int value Date: Sat, Apr 22, 2017 1:18 AM Would you review following change for jdk8? Webrev: http://cr.openjdk.java.net/~horii/8178294/webrev.01/ Our byte-reverse optimization now works in shared code. I tested it with jtreg on x86, ppc64, and ppc64le. Best regards, -- Michihiro, IBM Research - Tokyo ----- Original message ----- From: "Doerr, Martin" To: Michihiro Horie/Japan/IBM at IBMJP Cc: "aph at redhat.com" , Hiroshi H Horii/Japan/IBM at IBMJP, "hotspot-dev at openjdk.java.net" , "ppc-aix-port-dev at openjdk.java.net" , "Simonis, Volker" Subject: RE: Optimizing byte reverse code for int value Date: Wed, Apr 12, 2017 12:13 AM Hi Michihiro, thanks for the quick reply. I think Andrew?s idea is to optimize in the shared code instead of the platform backends. I haven?t thought about where this could be done. Or would it be possible to backport jdk (especially Unsafe) changes? If the required changes are small enough and we don?t have to touch any public interface, this might be an option, too. We?ll appreciate if you take care of the new match rules for PPC64. Thanks a lot. Best regards, Martin From: Michihiro Horie [mailto:HORIE at jp.ibm.com] Sent: Dienstag, 11. April 2017 16:55 To: Doerr, Martin Cc: aph at redhat.com; Hiroshi H Horii ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker Subject: RE: Optimizing byte reverse code for int value Andrew, Martin, Thanks a lot for your helpful feedback. >Have you considered it as a generic optimization for all processors? We would support all processors for our byte-reverse optimization to make it generic. Currently, I just finished adding match rules for little endian and big endian on PPC64, and am testing it in AIX. >In addition, I noticed that we don't have match rules which exploit byte reverse load/store instructions on PPC64. We would like to handle adding match rules for byte reverse load/store instructions on PPC64 for JDK10 if you would not mind. Would it be fine with you? Best regards, -- Michihiro, IBM Research - Tokyo ----- Original message ----- From: "Doerr, Martin" To: Andrew Haley , Michihiro Horie/Japan/IBM at IBMJP Cc: "Simonis, Volker" , " ppc-aix-port-dev at openjdk.java.net" , " hotspot-dev at openjdk.java.net" , Hiroshi H Horii/Japan/IBM at IBMJP Subject: RE: Optimizing byte reverse code for int value Date: Tue, Apr 11, 2017 10:44 PM Hi Andrew, thank you for your helpful comments. I fully agree with you. In addition, I noticed that we don't have match rules which exploit byte reverse load/store instructions on PPC64. SPARC already has them: match(Set dst (ReverseBytesI/L/US/S (LoadI src))); match(Set dst (StoreI dst (ReverseBytesI/L/US/S src))); I think we should add them for jdk10. They should be used when the platform endianness doesn't match the bigEndian parameter in Unsafe methods. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Andrew Haley Sent: Dienstag, 11. April 2017 13:02 To: Michihiro Horie Cc: Simonis, Volker ; ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; Hiroshi H Horii Subject: Re: Optimizing byte reverse code for int value On 11/04/17 11:36, Michihiro Horie wrote: > Thank you very much for letting us know Unsafe.getIntUnaligned is available in > JDK9. I do agree we should fix Java source code. > We think our byte-reverse optimization would still work on jdk8u as Hiroshi > mentioned. Would you agree on this point? I do, but I do not agree that this patch should necessarily be done in the PowerPC-specific back end. Have you considered it as a generic optimization for all processors? Andrew. From joe.darcy at oracle.com Fri Apr 21 21:53:12 2017 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Fri, 21 Apr 2017 14:53:12 -0700 Subject: RFR(xs) JDK-8163861 JNI NewString() and GetStringLength() documentation incorrect In-Reply-To: References: <26815c67-1e2c-30cb-6435-2be6f713f42a@oracle.com> Message-ID: <58FA7F48.7070206@oracle.com> On 4/21/2017 12:07 AM, David Holmes wrote: > On 21/04/2017 12:25 AM, George Triantafillou wrote: >> Hi David, >> >> Thanks for the review. I wasn't aware that the CSR isn't yet available >> for 10. Do you know when it will be available? > > "real soon now" FYI, related discussions about CSR occurring over at csr-discuss at openjdk.java.net. -Joe From kim.barrett at oracle.com Fri Apr 21 22:41:35 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Apr 2017 18:41:35 -0400 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: <43f813fb-cb78-eff9-0fb5-bef99d047f25@oracle.com> References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> <43f813fb-cb78-eff9-0fb5-bef99d047f25@oracle.com> Message-ID: <5AE9AD5E-B3D2-42BA-A5E7-7A71AD3BFEE0@oracle.com> > On Apr 21, 2017, at 2:21 PM, Chris Plummer wrote: > > On 4/21/17 10:41 AM, Kim Barrett wrote: >> * Put these toolchain-specific definitions in the corresponding >> globalDefintions_.hpp. But the #include of those files by >> globalDefinitions.hpp happens very early in that file, before things >> like the uintx type are defined. Also, presently debug.hpp can't >> be #include'd by globalDefinitions.hpp, so the latter does similar >> things "by hand". (I think that's something that ought to be fixed.) > I also had this thought before reading your comment above. If your code was all toolchain independent, would you still consider putting it in globalDefintions.hpp? If not, I don't necessarily think that it makes sense to put it there just to piggyback on globalDefinitions.hpp the logic. I skimmed through globalDefintions.hpp and couldn't get enough a good feel for whether or not your API belongs there. globalDefinitions.hpp contains a lot of little utility functions, and this one (and some related ones that I'm not (yet) proposing) fall into that category. So I think it fits. OTOH, I pretty strongly dislike files like globalDefinitions.hpp. If it were up to me, that file would get some significant refactoring, with several new files being pulled out of it. But the noted structural issues make this function not fit in there very nicely at present. >>> Otherwise it looks fine to me, but I'm no expert in this area. The testing looks satisfactory and the bitmap results you are getting are reason enough to add this functionality. >> Thanks. Do you want to be counted as a reviewer? > Sure. FYI I'm not a (R)eviewer. OK. Thanks for the review. From kim.barrett at oracle.com Sat Apr 22 17:03:49 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 22 Apr 2017 13:03:49 -0400 Subject: RFR: 8179181: Cleanup BitMap search API Message-ID: <09D5CEBD-41FA-4570-9CD2-9591FD501CBD@oracle.com> Please review this small simplification of the BitMap search API. The change consists of dropping the _inline suffix from some function names, and removing the now redundant (and also declared inline) forwarding wrappers. CR: https://bugs.openjdk.java.net/browse/JDK-8179181 Webrev: http://cr.openjdk.java.net/~kbarrett/8179181/hotspot.00/ Testing: JPRT From david.holmes at oracle.com Sun Apr 23 02:47:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Sun, 23 Apr 2017 12:47:17 +1000 Subject: RFR: 8179181: Cleanup BitMap search API In-Reply-To: <09D5CEBD-41FA-4570-9CD2-9591FD501CBD@oracle.com> References: <09D5CEBD-41FA-4570-9CD2-9591FD501CBD@oracle.com> Message-ID: <1bbe2dc3-a848-b76d-9308-2d5e10eff411@oracle.com> Hi Kim, On 23/04/2017 3:03 AM, Kim Barrett wrote: > Please review this small simplification of the BitMap search API. The > change consists of dropping the _inline suffix from some function > names, and removing the now redundant (and also declared inline) > forwarding wrappers. That was one seriously confused API! Presumably the intent was to have inlined and non-lined variants, but as you note everything was inline anyway. :( Cleanup looks good. Thanks, David > CR: > https://bugs.openjdk.java.net/browse/JDK-8179181 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8179181/hotspot.00/ > > Testing: > JPRT > > From kim.barrett at oracle.com Sun Apr 23 04:49:08 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 23 Apr 2017 00:49:08 -0400 Subject: RFR: 8179181: Cleanup BitMap search API In-Reply-To: <1bbe2dc3-a848-b76d-9308-2d5e10eff411@oracle.com> References: <09D5CEBD-41FA-4570-9CD2-9591FD501CBD@oracle.com> <1bbe2dc3-a848-b76d-9308-2d5e10eff411@oracle.com> Message-ID: <21181218-7956-411D-8AE3-46C6184B26AE@oracle.com> > On Apr 22, 2017, at 10:47 PM, David Holmes wrote: > > Hi Kim, > > On 23/04/2017 3:03 AM, Kim Barrett wrote: >> Please review this small simplification of the BitMap search API. The >> change consists of dropping the _inline suffix from some function >> names, and removing the now redundant (and also declared inline) >> forwarding wrappers. > > That was one seriously confused API! Presumably the intent was to have inlined and non-lined variants, but as you note everything was inline anyway. :( I particularly liked the "Non-inline versionsof the above.? comment, and yet, there they are in the .inline.hpp? > Cleanup looks good. Thanks. > > Thanks, > David > >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8179181 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8179181/hotspot.00/ >> >> Testing: >> JPRT From coleen.phillimore at oracle.com Sun Apr 23 08:13:22 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sun, 23 Apr 2017 09:13:22 +0100 Subject: RFR: 8179181: Cleanup BitMap search API In-Reply-To: <21181218-7956-411D-8AE3-46C6184B26AE@oracle.com> References: <09D5CEBD-41FA-4570-9CD2-9591FD501CBD@oracle.com> <1bbe2dc3-a848-b76d-9308-2d5e10eff411@oracle.com> <21181218-7956-411D-8AE3-46C6184B26AE@oracle.com> Message-ID: <698dc93e-7f14-b758-84de-a07c759439df@oracle.com> Looks good. Second review. On 4/23/17 5:49 AM, Kim Barrett wrote: >> On Apr 22, 2017, at 10:47 PM, David Holmes wrote: >> >> Hi Kim, >> >> On 23/04/2017 3:03 AM, Kim Barrett wrote: >>> Please review this small simplification of the BitMap search API. The >>> change consists of dropping the _inline suffix from some function >>> names, and removing the now redundant (and also declared inline) >>> forwarding wrappers. >> That was one seriously confused API! Presumably the intent was to have inlined and non-lined variants, but as you note everything was inline anyway. :( > I particularly liked the "Non-inline versionsof the above.? comment, and yet, there they are in the .inline.hpp? Seriously, thanks for cleaning this up. Coleen > >> Cleanup looks good. > Thanks. > >> Thanks, >> David >> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8179181 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8179181/hotspot.00/ >>> >>> Testing: >>> JPRT > From coleen.phillimore at oracle.com Sun Apr 23 08:41:11 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sun, 23 Apr 2017 09:41:11 +0100 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> Message-ID: On 4/20/17 5:02 PM, Volker Simonis wrote: > Hi Ioi, > > thanks once again for considering our ports! Please find the required > additions for ppc64/s390x in the following webrew (which is based upon > your latest v03 patch): > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ > > @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly > code. I did some tests and I think it should be correct, but maybe you > still find some improvements :) > > Besides that, I have some general questions/comments regarding your change: > > 1. In constantPool.hpp, why don't you declare the '_name_index' and > '_resolved_klass_index' fields with type 'jushort'? As far as I can > see, they can only hold 16-bit values anyway. It would also save you > some space and several asserts (e.g. in unresolved_klass_at_put(): > > > 274 assert((name_index & 0xffff0000) == 0, "must be"); > 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); > > 2. What do you mean by: > > 106 // ... will be changed to support compressed pointers > 107 Array* _resolved_klasses; > > 3. Why don't we need the call to "release_tag_at_put()" in > "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int > class_index, Klass* k)" is used from > "ClassFileParser::fill_instance_klass() and before your change that > function used the previous version of "klass_at_put(int class_index, > Klass* k)" which did call "release_tag_at_put()". > > 4. In ConstantPool::copy_entry_to() you've changed the behavior for > tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, > JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was > copied to the new constant pool if one existed but now you always only > copy a class_index to the new constant pool (even if a resolved klass > existed). Is that OK? E.g. won't this lead to a new resolving for the > new constant pool and will this have performance impacts or other side > effects? For redefinition, I don't think there would be a noticeable performance impact to leaving the class in the unresolved state in the new constant pool. The interpreter calls into the vm for ldc anyway each time (should file a bug and fix that), and if the class is loaded it will find it in an (unlocked, I believe) system dictionary lookup. The checkcast and instanceof cases might be more expensive though. If redefinition performance becomes an issue again for customers, we can consider optimizing this by copying the class over to the new constant pool klasses array but we're trying to move away from the constant pool merging code long term. We have other redefinition enhancements that we're considering for jdk11. thanks, Coleen > > Thanks again for doing this nice change and best regards, > Volker > > > On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >> Hi Lois, >> >> I have updated the patch to include your comments, and fixes the handling of >> anonymous classes. I also added some more comments regarding the >> _temp_resolved_klass_index: >> >> (delta from last webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >> >> (full webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >> >> Thanks >> - Ioi >> >> >> On 4/15/17 2:31 AM, Lois Foltan wrote: >>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>> >>>> >>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>> HI Lois, >>>>> >>>>> Thanks for the review. Please see my comments in-line. >>>>> >>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> Looks really good. A couple of comments: >>>>>> >>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>> surrounding anonymous classes. Coleen and I discussed earlier today and I >>>>>> came away from that discussion with the idea that the only classes being >>>>>> patched currently are anonymous classes. >>>>> Line 5676 ... >>>>> >>>>> 5676 if (is_anonymous()) { >>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>> index >>>>> 5678 } >>>>> >>>>> corresponds to >>>>> >>>>> 5361 ik->set_name(_class_name); >>>>> 5362 >>>>> 5363 if (is_anonymous()) { >>>>> 5364 // I am well known to myself >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> 5366 } >>>>> >>>>> Even though the class is "anonymous", it actually has a name. ik->name() >>>>> probably is part of the constant pool, but I am not 100% sure. Also, I would >>>>> need to search the constant pool to find the index for ik->name(). So I just >>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>> append ik->name() to the end of the constant pool. >>>>> >>>>> "Anonymous" actually means "the class cannot be looked up by name in the >>>>> SystemDictionary". I think we need a better terminology :-) >>>>> >>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>> modify the comments to the following: >>>> >>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>> // anonymous class itself. If this class needs to refer to its own >>>> methods or >>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>> reference >>>> // _this_class_index. However, because this class is anonymous (it's >>>> // not stored in SystemDictionary), _this_class_index cannot be >>>> resolved >>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>> lookup. >>>> // Therefore, we must eagerly resolve _this_class_index now. >>>> >>>> So, Lois is right. Line 5676 is not necessary. I will revise the code to >>>> do the "eager resolution" without using ClassFileParser::patch_class. I'll >>>> post the updated code later. >>> >>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>> updated webrev. >>> Lois >>> >>>> Thanks >>>> - Ioi >>>> >>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>> java/lang/Class on line #5684. >>>>> 5683 Handle patch = cp_patch_at(i); >>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>> java_lang_Class::is_instance(patch())) { >>>>> 5685 // We need to append the names of the patched classes to >>>>> the end of the constant pool, >>>>> 5686 // because a patched class may have a Utf8 name that's >>>>> not already included in the >>>>> 5687 // original constant pool. >>>>> 5688 // >>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>> patch a Utf8, a String, or a Class. >>>>> 5690 // At this point, we don't know the tag for index i yet, >>>>> because we haven't parsed the >>>>> 5691 // constant pool. So we can only assume the worst -- >>>>> every String is used to patch a Class. >>>>> 5692 _max_num_patched_klasses ++; >>>>> >>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>> either Class or String could be treated as a Klass: >>>>> >>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>> 725 int index, >>>>> 726 Handle patch, >>>>> 727 TRAPS) { >>>>> ... >>>>> 732 switch (cp->tag_at(index).value()) { >>>>> 733 >>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>> 735 // Patching a class means pre-resolving it. >>>>> 736 // The name in the constant pool is ignored. >>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>> 739 "Illegal class patch at %d in class file >>>>> %s", >>>>> 740 index, CHECK); >>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>> 742 patch_class(cp, index, k, k->name()); >>>>> 743 } else { >>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>> 745 "Illegal class patch at %d in class file >>>>> %s", >>>>> 746 index, CHECK); >>>>> 747 Symbol* const name = java_lang_String::as_symbol(patch(), >>>>> CHECK); >>>>> 748 patch_class(cp, index, NULL, name); >>>>> 749 } >>>>> 750 break; >>>>> 751 } >>>>> >>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>> >>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>> ik->name() is in the cp_patch array. >>>>> >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> >>>>>> Also why not do this calculation in the rewriter or is that too late? >>>>>> >>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and the >>>>> associated tags array is allocated (both of which are fixed size, and cannot >>>>> be expanded), which is way before the rewriter is run. At this point, we >>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to make >>>>> a worst-case estimate on how long the CP/tags should be. >>>>> >>>>>> * line #5677, 5692 - a nit but I think the convention is to not have a >>>>>> space after the variable name and between the post increment operator. >>>>>> >>>>> Fixed. >>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>> really is not so much invalid as temporary for class redefinition purposes, >>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please consider >>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to rename >>>>>> the field or not, please add a one line comment ahead of >>>>>> ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses >>>>>> this currently. >>>>>> >>>>> Good idea. Will do. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Great change, thanks! >>>>>> Lois >>>>>> >>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>> >>>>>>> In addition to your requests, I made these changes: >>>>>>> >>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>> >>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>> int name_index = kslot.name_index(); >>>>>>> >>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick this >>>>>>> function over MetaspaceObj::is_shared(). >>>>>>> >>>>>>> [3] Massaged the CDS region size set-up code a little to pass internal >>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time picking >>>>>>> the "right" sizes, as this code will be obsoleted soon with JDK-8072061 >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>> >>>>>>>> This looks really good! >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>> >>>>>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>>>>> + Array* tags = >>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // >>>>>>>> tags->at(_length) is flags() >>>>>>>> >>>>>>>> >>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>> >>>>>>>> + Klass** adr = >>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>> resolved >>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>> + // hardware store ordering here. >>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>> + return k; >>>>>>>> >>>>>>>> The comment still refers to the switch between Symbol* and Klass* in >>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>> >>>>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>>>> >>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header file >>>>>>>> (so it's inlined) and have all the places where you get name_index use this >>>>>>>> function? So we don't have to know in multiple places that >>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about what the >>>>>>>> new format of the constant pool entry is {name_index, resolved_klass_index}. >>>>>>>> I'm happy to see this work nearing completion! The "constant" pool should >>>>>>>> be constant! thanks, Coleen >>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>> Hi,please review the following change >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from >>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool >>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + >>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. In >>>>>>>>> addition: + I moved _resolved_references and _reference_map from >>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no longer >>>>>>>>> modified for shared ConstantPools. As a result, none of the fields in >>>>>>>>> shared ConstantPools are modified at run time, so we can move them into the >>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results show no >>>>>>>>> performance regression, despite the extra level of indirection, which has a >>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 and >>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support ports. For the >>>>>>>>> aarch64, ppc and s390 ports, I have added some code without testing (it's >>>>>>>>> probably incomplete) - Port owners, please check if my patch work for you, >>>>>>>>> and I can incorporate your changes in my push. Alternatively, you can wait >>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, and I will >>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>> From david.holmes at oracle.com Sun Apr 23 11:45:56 2017 From: david.holmes at oracle.com (David Holmes) Date: Sun, 23 Apr 2017 21:45:56 +1000 Subject: RFR: 8179181: Cleanup BitMap search API In-Reply-To: <21181218-7956-411D-8AE3-46C6184B26AE@oracle.com> References: <09D5CEBD-41FA-4570-9CD2-9591FD501CBD@oracle.com> <1bbe2dc3-a848-b76d-9308-2d5e10eff411@oracle.com> <21181218-7956-411D-8AE3-46C6184B26AE@oracle.com> Message-ID: On 23/04/2017 2:49 PM, Kim Barrett wrote: >> On Apr 22, 2017, at 10:47 PM, David Holmes wrote: >> >> Hi Kim, >> >> On 23/04/2017 3:03 AM, Kim Barrett wrote: >>> Please review this small simplification of the BitMap search API. The >>> change consists of dropping the _inline suffix from some function >>> names, and removing the now redundant (and also declared inline) >>> forwarding wrappers. >> >> That was one seriously confused API! Presumably the intent was to have inlined and non-lined variants, but as you note everything was inline anyway. :( > > I particularly liked the "Non-inline versionsof the above.? comment, and yet, there they are in the .inline.hpp? Been that way since day 1 when they were integrated as part of the initial G1 open sourcing: JDK-6711316 David >> Cleanup looks good. > > Thanks. > >> >> Thanks, >> David >> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8179181 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8179181/hotspot.00/ >>> >>> Testing: >>> JPRT > > From kim.barrett at oracle.com Sun Apr 23 17:02:03 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 23 Apr 2017 13:02:03 -0400 Subject: RFR: 8179181: Cleanup BitMap search API In-Reply-To: <698dc93e-7f14-b758-84de-a07c759439df@oracle.com> References: <09D5CEBD-41FA-4570-9CD2-9591FD501CBD@oracle.com> <1bbe2dc3-a848-b76d-9308-2d5e10eff411@oracle.com> <21181218-7956-411D-8AE3-46C6184B26AE@oracle.com> <698dc93e-7f14-b758-84de-a07c759439df@oracle.com> Message-ID: <3C591DFC-40BB-4A2F-869E-DAD6C7172B20@oracle.com> > On Apr 23, 2017, at 4:13 AM, coleen.phillimore at oracle.com wrote: > > Looks good. Second review. Thanks. From martin.doerr at sap.com Mon Apr 24 09:10:20 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 24 Apr 2017 09:10:20 +0000 Subject: Optimizing byte reverse code for int value In-Reply-To: References: <622fa3e77da546dfb5155a1e4afacd7c@sap.com>, , <362a21f4-277c-c3f3-f7f0-08b55c8b2b0b@redhat.com> <89abbea5-9998-2e4d-62d3-e1f3e9bbd1d5@redhat.com> Message-ID: <2e13a32b56cd4d9f89758f4042602e9a@sap.com> Hi Michihiro, please note that I?m not a jdk8u reviewer. However, I have taken a quick look and I have the following concerns: 1. I think it?s incorrect for Big Endian. 2. The pattern can also match for an unaligned 4 byte address which would break platforms like SPARC. 3. I couldn?t see checks for shift amount and masks. Best regards, Martin From: Michihiro Horie [mailto:HORIE at jp.ibm.com] Sent: Freitag, 21. April 2017 18:18 To: Doerr, Martin Cc: aph at redhat.com; Hiroshi H Horii ; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker ; Gustavo Bueno Romero Subject: RE: Optimizing byte reverse code for int value Would you review following change for jdk8? Webrev: http://cr.openjdk.java.net/~horii/8178294/webrev.01/ Our byte-reverse optimization now works in shared code. I tested it with jtreg on x86, ppc64, and ppc64le. Best regards, -- Michihiro, IBM Research - Tokyo ----- Original message ----- From: "Doerr, Martin" > To: Michihiro Horie/Japan/IBM at IBMJP Cc: "aph at redhat.com" >, Hiroshi H Horii/Japan/IBM at IBMJP, "hotspot-dev at openjdk.java.net" >, "ppc-aix-port-dev at openjdk.java.net" >, "Simonis, Volker" > Subject: RE: Optimizing byte reverse code for int value Date: Wed, Apr 12, 2017 12:13 AM Hi Michihiro, thanks for the quick reply. I think Andrew?s idea is to optimize in the shared code instead of the platform backends. I haven?t thought about where this could be done. Or would it be possible to backport jdk (especially Unsafe) changes? If the required changes are small enough and we don?t have to touch any public interface, this might be an option, too. We?ll appreciate if you take care of the new match rules for PPC64. Thanks a lot. Best regards, Martin From: Michihiro Horie [mailto:HORIE at jp.ibm.com] Sent: Dienstag, 11. April 2017 16:55 To: Doerr, Martin > Cc: aph at redhat.com; Hiroshi H Horii >; hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker > Subject: RE: Optimizing byte reverse code for int value Andrew, Martin, Thanks a lot for your helpful feedback. >Have you considered it as a generic optimization for all processors? We would support all processors for our byte-reverse optimization to make it generic. Currently, I just finished adding match rules for little endian and big endian on PPC64, and am testing it in AIX. >In addition, I noticed that we don't have match rules which exploit byte reverse load/store instructions on PPC64. We would like to handle adding match rules for byte reverse load/store instructions on PPC64 for JDK10 if you would not mind. Would it be fine with you? Best regards, -- Michihiro, IBM Research - Tokyo ----- Original message ----- From: "Doerr, Martin" > To: Andrew Haley >, Michihiro Horie/Japan/IBM at IBMJP Cc: "Simonis, Volker" >, "ppc-aix-port-dev at openjdk.java.net" >, "hotspot-dev at openjdk.java.net" >, Hiroshi H Horii/Japan/IBM at IBMJP Subject: RE: Optimizing byte reverse code for int value Date: Tue, Apr 11, 2017 10:44 PM Hi Andrew, thank you for your helpful comments. I fully agree with you. In addition, I noticed that we don't have match rules which exploit byte reverse load/store instructions on PPC64. SPARC already has them: match(Set dst (ReverseBytesI/L/US/S (LoadI src))); match(Set dst (StoreI dst (ReverseBytesI/L/US/S src))); I think we should add them for jdk10. They should be used when the platform endianness doesn't match the bigEndian parameter in Unsafe methods. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Andrew Haley Sent: Dienstag, 11. April 2017 13:02 To: Michihiro Horie > Cc: Simonis, Volker >; ppc-aix-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; Hiroshi H Horii > Subject: Re: Optimizing byte reverse code for int value On 11/04/17 11:36, Michihiro Horie wrote: > Thank you very much for letting us know Unsafe.getIntUnaligned is available in > JDK9. I do agree we should fix Java source code. > We think our byte-reverse optimization would still work on jdk8u as Hiroshi > mentioned. Would you agree on this point? I do, but I do not agree that this patch should necessarily be done in the PowerPC-specific back end. Have you considered it as a generic optimization for all processors? Andrew. From ioi.lam at oracle.com Mon Apr 24 12:18:06 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 24 Apr 2017 20:18:06 +0800 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> Message-ID: <58FDECFE.5060105@oracle.com> Hi Volker, On 4/21/17 12:02 AM, Volker Simonis wrote: > Hi Ioi, > > thanks once again for considering our ports! Please find the required > additions for ppc64/s390x in the following webrew (which is based upon > your latest v03 patch): > > http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ Thanks for the patch. I will integrate it and post an updated webrev. > @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly > code. I did some tests and I think it should be correct, but maybe you > still find some improvements :) > > Besides that, I have some general questions/comments regarding your change: > > 1. In constantPool.hpp, why don't you declare the '_name_index' and > '_resolved_klass_index' fields with type 'jushort'? As far as I can > see, they can only hold 16-bit values anyway. It would also save you > some space and several asserts (e.g. in unresolved_klass_at_put(): > > > 274 assert((name_index & 0xffff0000) == 0, "must be"); > 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); I think the HotSpot convention is to use ints as parameter and return types, for values that are actually 16-bits or less, like here in constantPool.hpp: void field_at_put(int which, int class_index, int name_and_type_index) { tag_at_put(which, JVM_CONSTANT_Fieldref); *int_at_addr(which) = ((jint) name_and_type_index<<16) | class_index; } I am not sure what the reasons are. It could be that the parameters usually need to be computed arithmetically, and it's much easier for the caller of the method to use ints -- otherwise you will get lots of compiler warnings which would force you to use lots of casting, resulting in code that's hard to read and probably incorrect. > 2. What do you mean by: > > 106 // ... will be changed to support compressed pointers > 107 Array* _resolved_klasses; Sorry the comment isn't very clear. How about this? 106 // Consider using an array of compressed klass pointers to // save space on 64-bit platforms. 107 Array* _resolved_klasses; > 3. Why don't we need the call to "release_tag_at_put()" in > "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int > class_index, Klass* k)" is used from > "ClassFileParser::fill_instance_klass() and before your change that > function used the previous version of "klass_at_put(int class_index, > Klass* k)" which did call "release_tag_at_put()". Good catch. I'll add the following, because the class is now resolved. release_tag_at_put(class_index, JVM_CONSTANT_UnresolvedClass); > 4. In ConstantPool::copy_entry_to() you've changed the behavior for > tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, > JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was > copied to the new constant pool if one existed but now you always only > copy a class_index to the new constant pool (even if a resolved klass > existed). Is that OK? E.g. won't this lead to a new resolving for the > new constant pool and will this have performance impacts or other side > effects? I think Coleen has answered this in a separate mail :-) Thanks - Ioi > Thanks again for doing this nice change and best regards, > Volker > > > On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >> Hi Lois, >> >> I have updated the patch to include your comments, and fixes the handling of >> anonymous classes. I also added some more comments regarding the >> _temp_resolved_klass_index: >> >> (delta from last webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >> >> (full webrev) >> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >> >> Thanks >> - Ioi >> >> >> On 4/15/17 2:31 AM, Lois Foltan wrote: >>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>> >>>> >>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>> HI Lois, >>>>> >>>>> Thanks for the review. Please see my comments in-line. >>>>> >>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> Looks really good. A couple of comments: >>>>>> >>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>> surrounding anonymous classes. Coleen and I discussed earlier today and I >>>>>> came away from that discussion with the idea that the only classes being >>>>>> patched currently are anonymous classes. >>>>> Line 5676 ... >>>>> >>>>> 5676 if (is_anonymous()) { >>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>> index >>>>> 5678 } >>>>> >>>>> corresponds to >>>>> >>>>> 5361 ik->set_name(_class_name); >>>>> 5362 >>>>> 5363 if (is_anonymous()) { >>>>> 5364 // I am well known to myself >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> 5366 } >>>>> >>>>> Even though the class is "anonymous", it actually has a name. ik->name() >>>>> probably is part of the constant pool, but I am not 100% sure. Also, I would >>>>> need to search the constant pool to find the index for ik->name(). So I just >>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>> append ik->name() to the end of the constant pool. >>>>> >>>>> "Anonymous" actually means "the class cannot be looked up by name in the >>>>> SystemDictionary". I think we need a better terminology :-) >>>>> >>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>> modify the comments to the following: >>>> >>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>> // anonymous class itself. If this class needs to refer to its own >>>> methods or >>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>> reference >>>> // _this_class_index. However, because this class is anonymous (it's >>>> // not stored in SystemDictionary), _this_class_index cannot be >>>> resolved >>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>> lookup. >>>> // Therefore, we must eagerly resolve _this_class_index now. >>>> >>>> So, Lois is right. Line 5676 is not necessary. I will revise the code to >>>> do the "eager resolution" without using ClassFileParser::patch_class. I'll >>>> post the updated code later. >>> >>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>> updated webrev. >>> Lois >>> >>>> Thanks >>>> - Ioi >>>> >>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>> java/lang/Class on line #5684. >>>>> 5683 Handle patch = cp_patch_at(i); >>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>> java_lang_Class::is_instance(patch())) { >>>>> 5685 // We need to append the names of the patched classes to >>>>> the end of the constant pool, >>>>> 5686 // because a patched class may have a Utf8 name that's >>>>> not already included in the >>>>> 5687 // original constant pool. >>>>> 5688 // >>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>> patch a Utf8, a String, or a Class. >>>>> 5690 // At this point, we don't know the tag for index i yet, >>>>> because we haven't parsed the >>>>> 5691 // constant pool. So we can only assume the worst -- >>>>> every String is used to patch a Class. >>>>> 5692 _max_num_patched_klasses ++; >>>>> >>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>> either Class or String could be treated as a Klass: >>>>> >>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>> 725 int index, >>>>> 726 Handle patch, >>>>> 727 TRAPS) { >>>>> ... >>>>> 732 switch (cp->tag_at(index).value()) { >>>>> 733 >>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>> 735 // Patching a class means pre-resolving it. >>>>> 736 // The name in the constant pool is ignored. >>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>> 739 "Illegal class patch at %d in class file >>>>> %s", >>>>> 740 index, CHECK); >>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>> 742 patch_class(cp, index, k, k->name()); >>>>> 743 } else { >>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>> 745 "Illegal class patch at %d in class file >>>>> %s", >>>>> 746 index, CHECK); >>>>> 747 Symbol* const name = java_lang_String::as_symbol(patch(), >>>>> CHECK); >>>>> 748 patch_class(cp, index, NULL, name); >>>>> 749 } >>>>> 750 break; >>>>> 751 } >>>>> >>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>> >>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>> ik->name() is in the cp_patch array. >>>>> >>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>> ik->name()); // eagerly resolve >>>>> >>>>>> Also why not do this calculation in the rewriter or is that too late? >>>>>> >>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and the >>>>> associated tags array is allocated (both of which are fixed size, and cannot >>>>> be expanded), which is way before the rewriter is run. At this point, we >>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to make >>>>> a worst-case estimate on how long the CP/tags should be. >>>>> >>>>>> * line #5677, 5692 - a nit but I think the convention is to not have a >>>>>> space after the variable name and between the post increment operator. >>>>>> >>>>> Fixed. >>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>> really is not so much invalid as temporary for class redefinition purposes, >>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please consider >>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to rename >>>>>> the field or not, please add a one line comment ahead of >>>>>> ConstantPool::temp_unresolved_klass_at_put that only class redefinition uses >>>>>> this currently. >>>>>> >>>>> Good idea. Will do. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Great change, thanks! >>>>>> Lois >>>>>> >>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>> >>>>>>> In addition to your requests, I made these changes: >>>>>>> >>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>> >>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>> int name_index = kslot.name_index(); >>>>>>> >>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick this >>>>>>> function over MetaspaceObj::is_shared(). >>>>>>> >>>>>>> [3] Massaged the CDS region size set-up code a little to pass internal >>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time picking >>>>>>> the "right" sizes, as this code will be obsoleted soon with JDK-8072061 >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>> >>>>>>>> This looks really good! >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>> >>>>>>>> + // Add one extra element to tags for storing ConstantPool::flags(). >>>>>>>> + Array* tags = >>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, "invariant"); // >>>>>>>> tags->at(_length) is flags() >>>>>>>> >>>>>>>> >>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>> >>>>>>>> + Klass** adr = >>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>> resolved >>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>> + // hardware store ordering here. >>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>> + return k; >>>>>>>> >>>>>>>> The comment still refers to the switch between Symbol* and Klass* in >>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>> >>>>>>>> + int name_index = extract_high_short_from_int(*int_at_addr(which)); >>>>>>>> >>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header file >>>>>>>> (so it's inlined) and have all the places where you get name_index use this >>>>>>>> function? So we don't have to know in multiple places that >>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about what the >>>>>>>> new format of the constant pool entry is {name_index, resolved_klass_index}. >>>>>>>> I'm happy to see this work nearing completion! The "constant" pool should >>>>>>>> be constant! thanks, Coleen >>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>> Hi,please review the following change >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the Klass from >>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the ConstantPool >>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>> ConstantPool::klass_at(i) finds the at this->_resolved_klasses->at(i) + >>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. In >>>>>>>>> addition: + I moved _resolved_references and _reference_map from >>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no longer >>>>>>>>> modified for shared ConstantPools. As a result, none of the fields in >>>>>>>>> shared ConstantPools are modified at run time, so we can move them into the >>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results show no >>>>>>>>> performance regression, despite the extra level of indirection, which has a >>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 and >>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support ports. For the >>>>>>>>> aarch64, ppc and s390 ports, I have added some code without testing (it's >>>>>>>>> probably incomplete) - Port owners, please check if my patch work for you, >>>>>>>>> and I can incorporate your changes in my push. Alternatively, you can wait >>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, and I will >>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>> From shade at redhat.com Mon Apr 24 12:53:26 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 24 Apr 2017 14:53:26 +0200 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics Message-ID: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> Hi, See https://bugs.openjdk.java.net/browse/JDK-8169061. Our current Atomic code looks like this: // Adding a lock prefix to an instruction on MP machine #define LOCK_IF_MP(mp) "cmp $0, " #mp "; je 1f; lock; 1: " inline jint Atomic::add (jint add_value, volatile jint* dest) { jint addend = add_value; int mp = os::is_MP(); __asm__ volatile ( LOCK_IF_MP(%3) "xaddl %0,(%2)" : "=r" (addend) : "0" (addend), "r" (dest), "r" (mp) : "cc", "memory"); return addend + add_value; } Notice the LOCK_IF_MP part, which means we have flag reads, additional branch, etc. on the hot path. I would like us to consider dropping runtime os::is_MP checks from Atomics. The history of the original change predates OpenJDK, but I think it was deemed a plausible optimization in mostly-uniprocessor world. Today, this penalizes multi-core platforms without a good reason. Proposed change for jdk10/hs: http://cr.openjdk.java.net/~shade/8169061/webrev.01/ It unconditionally does lock prefix on all atomics. Our interest for doing this is GC performance work. For example, in Shenandoah, native CASes are very frequently used for bitmap manipulation during marking, updating forwarding pointers for evacuating objects, updating references in the heap. We have measured around 1-2% GC time improvement with this change. I fully expect it to benefit G1 too, because it also does CASes on bitmaps. Testing: shenandoah tests, hotspot_runtime on Linux x86_64 Putting this via the build-test system on other platforms would be appreciated! Thanks, -Aleksey From aph at redhat.com Mon Apr 24 13:06:38 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 24 Apr 2017 14:06:38 +0100 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> Message-ID: On 24/04/17 13:53, Aleksey Shipilev wrote: > Notice the LOCK_IF_MP part, which means we have flag reads, additional branch, > etc. on the hot path. I would like us to consider dropping runtime os::is_MP > checks from Atomics. The history of the original change predates OpenJDK, but I > think it was deemed a plausible optimization in mostly-uniprocessor world. > Today, this penalizes multi-core platforms without a good reason. I'm sure that you are right. I (almost) completely ripped this out of AArch64 too. My reasoning was more to do with containers and the cloud. Processors can be dynamically added and removed, and this was enough to make me shiver. Having said that, there are still usages of is_MP in shared code, and I see that I've missed removing it in one place in the AArch64 sources. Andrew. From mikael.gerdin at oracle.com Mon Apr 24 14:10:55 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 24 Apr 2017 16:10:55 +0200 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> Message-ID: <569111e9-d4a8-5c25-998d-1c2c3e2b62b0@oracle.com> Hi Aleksey, On 2017-04-24 14:53, Aleksey Shipilev wrote: > Hi, > > See https://bugs.openjdk.java.net/browse/JDK-8169061. > > Our current Atomic code looks like this: > > // Adding a lock prefix to an instruction on MP machine > #define LOCK_IF_MP(mp) "cmp $0, " #mp "; je 1f; lock; 1: " > > inline jint Atomic::add (jint add_value, volatile jint* dest) { > jint addend = add_value; > int mp = os::is_MP(); > __asm__ volatile ( LOCK_IF_MP(%3) "xaddl %0,(%2)" > : "=r" (addend) > : "0" (addend), "r" (dest), "r" (mp) > : "cc", "memory"); > return addend + add_value; > } > > Notice the LOCK_IF_MP part, which means we have flag reads, additional branch, > etc. on the hot path. I would like us to consider dropping runtime os::is_MP > checks from Atomics. The history of the original change predates OpenJDK, but I > think it was deemed a plausible optimization in mostly-uniprocessor world. > Today, this penalizes multi-core platforms without a good reason. > > Proposed change for jdk10/hs: > http://cr.openjdk.java.net/~shade/8169061/webrev.01/ I think this change makes a lot of sense. I'm not familiar enough with the details of the Atomic class to give you a thorough code review but I want to state that I support this change. /Mikael > > It unconditionally does lock prefix on all atomics. > > Our interest for doing this is GC performance work. For example, in Shenandoah, > native CASes are very frequently used for bitmap manipulation during marking, > updating forwarding pointers for evacuating objects, updating references in the > heap. We have measured around 1-2% GC time improvement with this change. I fully > expect it to benefit G1 too, because it also does CASes on bitmaps. > > Testing: shenandoah tests, hotspot_runtime on Linux x86_64 > > Putting this via the build-test system on other platforms would be appreciated! > > Thanks, > -Aleksey > From volker.simonis at gmail.com Mon Apr 24 17:08:41 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 24 Apr 2017 19:08:41 +0200 Subject: [10] RFR (S) 8175813: PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: <58EEAF7B.6020708@linux.vnet.ibm.com> References: <58C1AE06.9060609@linux.vnet.ibm.com> <58EEAF7B.6020708@linux.vnet.ibm.com> Message-ID: Hi Gustavo, thanks for addressing this problem and sorry for my late reply. I think this is a good change which definitely improves the situation for uncommon NUMA configurations without changing the handling for common topologies. It would be great if somebody could run this trough JPRT, but as Gustavo mentioned, I don't expect any regressions. @Igor: I think you've been the original author of the NUMA-aware allocator port to Linux (i.e. "6684395: Port NUMA-aware allocator to linux"). If you could find some spare minutes to take a look at this change, your comment would be very much appreciated :) Following some minor comments from me: - in os::numa_get_groups_num() you now use numa_num_configured_nodes() to get the actual number of configured nodes. This is good and certainly an improvement over the previous implementation. However, the man page for numa_num_configured_nodes() mentions that the returned count may contain currently disabled nodes. Do we currently handle disabled nodes? What will be the consequence if we would use such a disabled node (e.g. mbind() warnings)? - the same question applies to the usage of Linux::isnode_in_configured_nodes() within os::numa_get_leaf_groups(). Does isnode_in_configured_nodes() (i.e. the node set defined by 'numa_all_nodes_ptr' take into account the disabled nodes or not? Can this be a potential problem (i.e. if we use a disabled node). - I'd like to suggest renaming the 'index' part of the following variables and functions to 'nindex' ('node_index' is probably to long) in the following code, to emphasize that we have node indexes pointing to actual, not always consecutive node numbers: 2879 // Create an index -> node mapping, since nodes are not always consecutive 2880 _index_to_node = new (ResourceObj::C_HEAP, mtInternal) GrowableArray(0, true); 2881 rebuild_index_to_node_map(); - can you please wrap the following one-line else statement into curly braces (it's more readable and we usually do it that way in HotSpot although there are no formal style guidelines :) 2953 } else 2954 // Current node is already a configured node. 2955 closest_node = index_to_node()->at(i); - in os::Linux::rebuild_cpu_to_node_map(), if you set 'closest_distance' to INT_MAX at the beginning of the loop, you can later avoid the check for '|| !closest_distance'. Also, according to the man page, numa_distance() returns 0 if it can not determine the distance. So with the above change, the condition on line 2974 should read: 2947 if (distance && distance < closest_distance) { Finally, and not directly related to your change, I'd suggest the following clean-ups: - remove the usage of 'NCPUS = 32768' in os::Linux::rebuild_cpu_to_node_map(). The comment on that line is unclear to me and probably related to an older version/problem of libnuma? I think we should simply use numa_allocate_cpumask()/numa_free_cpumask() instead. - we still use the NUMA version 1 function prototypes (e.g. "numa_node_to_cpus(int node, unsigned long *buffer, int buffer_len)" instead of "numa_node_to_cpus(int node, struct bitmask *mask)", but also "numa_interleave_memory()" and maybe others). I think we should switch all prototypes to the new NUMA version 2 interface which you've already used for the new functions which you've added. That said, I think these changes all require libnuma 2.0 (see os::Linux::libnuma_dlsym). So before starting this, you should make sure that libnuma 2.0 is available on all platforms to which you'd like to down-port this change. For jdk10 we could definitely do it, for jdk9 probably also, for jdk8 I'm not so sure. Regards, Volker On Thu, Apr 13, 2017 at 12:51 AM, Gustavo Romero wrote: > Hi, > > Any update on it? > > Thank you. > > Regards, > Gustavo > > On 09-03-2017 16:33, Gustavo Romero wrote: >> Hi, >> >> Could the following webrev be reviewed please? >> >> It improves the numa node detection when non-consecutive or memory-less nodes >> exist in the system. >> >> webrev: http://cr.openjdk.java.net/~gromero/8175813/v2/ >> bug : https://bugs.openjdk.java.net/browse/JDK-8175813 >> >> Currently, although no problem exists when the JVM detects numa nodes that are >> consecutive and have memory, for example in a numa topology like: >> >> available: 2 nodes (0-1) >> node 0 cpus: 0 8 16 24 32 >> node 0 size: 65258 MB >> node 0 free: 34 MB >> node 1 cpus: 40 48 56 64 72 >> node 1 size: 65320 MB >> node 1 free: 150 MB >> node distances: >> node 0 1 >> 0: 10 20 >> 1: 20 10, >> >> it fails on detecting numa nodes to be used in the Parallel GC in a numa >> topology like: >> >> available: 4 nodes (0-1,16-17) >> node 0 cpus: 0 8 16 24 32 >> node 0 size: 130706 MB >> node 0 free: 7729 MB >> node 1 cpus: 40 48 56 64 72 >> node 1 size: 0 MB >> node 1 free: 0 MB >> node 16 cpus: 80 88 96 104 112 >> node 16 size: 130630 MB >> node 16 free: 5282 MB >> node 17 cpus: 120 128 136 144 152 >> node 17 size: 0 MB >> node 17 free: 0 MB >> node distances: >> node 0 1 16 17 >> 0: 10 20 40 40 >> 1: 20 10 40 40 >> 16: 40 40 10 20 >> 17: 40 40 20 10, >> >> where node 16 is not consecutive in relation to 1 and also nodes 1 and 17 have >> no memory. >> >> If a topology like that exists, os::numa_make_local() will receive a local group >> id as a hint that is not available in the system to be bound (it will receive >> all nodes from 0 to 17), causing a proliferation of "mbind: Invalid argument" >> messages: >> >> http://cr.openjdk.java.net/~gromero/logs/jdk10_pristine.log >> >> That change improves the detection by making the JVM numa API aware of the >> existence of numa nodes that are non-consecutive from 0 to the highest node >> number and also of nodes that might be memory-less nodes, i.e. that might not >> be, in libnuma terms, a configured node. Hence just the configured nodes will >> be available: >> >> http://cr.openjdk.java.net/~gromero/logs/jdk10_numa_patched.log >> >> The change has no effect on numa topologies were the problem does not occur, >> i.e. no change in the number of nodes and no change in the cpu to node map. On >> numa topologies where memory-less nodes exist (like in the last example above), >> cpus from a memory-less node won't be able to bind locally so they are mapped >> to the closest node, otherwise they would be not associate to any node and >> MutableNUMASpace::cas_allocate() would pick a node randomly, compromising the >> performance. >> >> I found no regressions on x64 for the following numa topology: >> >> available: 2 nodes (0-1) >> node 0 cpus: 0 1 2 3 8 9 10 11 >> node 0 size: 24102 MB >> node 0 free: 19806 MB >> node 1 cpus: 4 5 6 7 12 13 14 15 >> node 1 size: 24190 MB >> node 1 free: 21951 MB >> node distances: >> node 0 1 >> 0: 10 21 >> 1: 21 10 >> >> I understand that fixing the current numa detection is a prerequisite to enable >> UseNUMA by the default [1] and to extend the numa-aware allocation to the G1 GC [2]. >> >> Thank you. >> >> >> Best regards, >> Gustavo >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8046153 (JEP 163: Enable NUMA Mode by Default When Appropriate) >> [2] https://bugs.openjdk.java.net/browse/JDK-8046147 (JEP 157: G1 GC: NUMA-Aware Allocation) >> > From daniel.smith at oracle.com Mon Apr 24 21:26:52 2017 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 24 Apr 2017 15:26:52 -0600 Subject: Call for Speakers -- 2017 JVM Language Summit Message-ID: CALL FOR SPEAKERS -- JVM LANGUAGE SUMMIT, JULY-AUGUST 2017 We are pleased to announce the 2017 JVM Language Summit to be held at Oracle's Santa Clara campus on July 31-August 2, 2017. Registration is now open for speaker submissions and will remain open through May 22, 2017. There is no registration fee for speakers. A limited number of early registration slots are also available for regular attendees. The JVM Language Summit is an open technical collaboration among language designers, compiler writers, tool builders, runtime engineers, and VM architects. We will share our experiences as creators of both the JVM and programming languages for the JVM. We also welcome non-JVM developers of similar technologies to attend or speak on their runtime, VM, or language of choice. Presentations will be recorded and made available to the public. This event is being organized by language and JVM engineers -- no marketers involved! So bring your slide rules and be prepared for some seriously geeky discussions. Format The summit is held in a single classroom-style room to support direct communication between participants. About 100-120 attendees are expected. The schedule consists of a single track of traditional presentations (about 6 each day) interspersed with less-formal multitrack "workshop" discussion groups (2-4 each day) and, possibly, impromptu "lightning talks." Workshops will be less structured than in the past, favoring an open discussion format with only a small amount of prepared material. Thus, rather than collecting workshop abstracts from speakers, we're asking each registrant to suggest a few topics of interest. After choosing the most popular topics, we'll ask some registrants if they'd like to act as discussion leaders. Instructions for Speaker Registration If you'd like to give a presentation, please register as a Speaker and include a detailed abstract. Speaker registration will remain open through May 22. There is no fee. See below for help preparing your abstract and talk. You will be notified about whether your proposal has been accepted; if not, you will be able to register as a regular attendee. For a successful speaker submission, please note the following: - All talks should be deeply technical, given by designers and implementors to designers and implementors. We all speak bytecode here! - Each talk, we hope and expect, will inform the audience, in detail, about the state of the art of language design or implementation on the JVM, or will explore the present and future capabilities of the JVM itself. (Some will do so indirectly by discussing non-JVM technologies.) - Know your audience: attendees may not be likely to ever use your specific language or tool, but could learn something from your interactions with the JVM. A broad goal of the summit is to inspire us to work together on JVM-based technologies that enable a rich ecosystem at higher layers. To register: register.jvmlangsummit.com For further information: jvmlangsummit.com Questions: inquire2017 at jvmlangsummit.com From volker.simonis at gmail.com Tue Apr 25 08:57:58 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 25 Apr 2017 10:57:58 +0200 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> <15bf57e1-a53b-07c1-1216-deacdb0d8231@oracle.com> Message-ID: Hi Ram, while this sounds interesting, I wonder how this plays together with NUMA and Large page support. I understand that these are different concepts, but in the end it all bails down tot he fact that memory access is not uniform and we have different "kinds" of memory. It seems to me that this fact is currently not very well handled in HotSpot and needs some general redesign. There are for example two JEPs [1,2] about improving the NUMA support in general and in G1. One of the problems is that NUMA support doesn't play well together with Large/Huge page support. I think your proposal must be evaluated in the broader context of enhancing the VM and GC for non-uniform memory architectures. Otherwise it would be yet another point fix which doesn't plays well together with other features like NUMA and LargePages. Thanks, Volker [1] https://bugs.openjdk.java.net/browse/JDK-8046153 (JEP 163: Enable NUMA Mode by Default When Appropriate) [2] https://bugs.openjdk.java.net/browse/JDK-8046147 (JEP 157: G1 GC: NUMA-Aware Allocation) On Wed, Apr 19, 2017 at 4:04 PM, Ram Krishnan wrote: > Many thanks David. > > Thanks, > Ramki > > On Tue, Apr 18, 2017 at 11:08 PM, David Holmes > wrote: > >> On 19/04/2017 11:38 AM, Ram Krishnan wrote: >> >>> Hi David, >>> >>> Many thanks, please find attached text version of document for temporary >>> hosting. >>> >> >> Hosted at: http://cr.openjdk.java.net/~dholmes/JEP-cache-partitioning- >> v1.txt >> >> David >> >> >>> Thanks, >>> Ramki >>> >>> On Tue, Apr 18, 2017 at 5:42 PM, David Holmes >> > wrote: >>> >>> Hi Ramki, >>> >>> On 19/04/2017 8:27 AM, Ram Krishnan wrote: >>> >>> Hi David, >>> >>> Thanks for the clarification. >>> >>> I have signed the OCA and mailed it to >>> oracle-ca_us(at)oracle.com >>> . Any help to expedite processing would be >>> much >>> appreciated. >>> >>> >>> Can't help with that I'm afraid. :) >>> >>> We are seeing promising POC results (details in the google doc) >>> for this >>> proposal -- would really appreciate your help in moving this >>> forward. >>> >>> >>> If you email me a text/html version of the document I can host it on >>> cr.openjdk.java.net temporarily. For >>> this to become a JEP you will need a sponsor with the necessary >>> OpenJDK credentials. >>> >>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html >>> >>> >>> Cheers, >>> David >>> >>> >>> Thanks, >>> Ramki >>> >>> On Tue, Apr 18, 2017 at 1:55 PM, David Holmes >>> >>> >> >> wrote: >>> >>> Hi Ramki, >>> >>> On 19/04/2017 12:34 AM, Ram Krishnan wrote: >>> >>> Please find detailed proposal below, looking forward to >>> your >>> comments. >>> >>> "Minimize application tail latency using >>> cache-partitioning-aware G1GC" -- >>> >>> https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbB >>> ZTclOWyg0arhuycXN94/edit >>> >> BZTclOWyg0arhuycXN94/edit> >>> >>> >> BZTclOWyg0arhuycXN94/edit >>> >> BZTclOWyg0arhuycXN94/edit>> >>> >>> >>> All contributions to OpenJDK need to be hosted on OpenJDK >>> infrastructure not on external systems like the above. >>> >>> Also I can not see you listed as an OCA signatory. Are you an >>> OpenJDK contributor? >>> >>> Thanks, >>> David >>> ----- >>> >>> Thanks, >>> Ramki >>> >>> On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels >>> >>> >> >>> wrote: >>> >>> Maybe it would be better to concentrate the processor >>> optimizations on >>> accessors and barrriers without introducing a >>> completely new GC >>> architecture. I can imagine that especially in the >>> area of >>> NUMA, TLAB, huge >>> pages, cache consistency and possibly MMX extensions >>> there >>> is some >>> potential. >>> >>> Abandoning the global STW - while it seems like a >>> pretty >>> powerful change - >>> is I guess not a good starter exercise. Especially >>> since it >>> is not only a >>> question of mutator threads. >>> >>> Gruss >>> Bernd >>> -- >>> http://bernd.eckenfels.net >>> ------------------------------ >>> *From:* hotspot-gc-dev >>> >> >>> >> >> on >>> behalf of Ram Krishnan >> >>> >> >> >>> *Sent:* Friday, April 14, 2017 6:36:27 AM >>> *To:* Asif Qamar; Andrew Haley; >>> hotspot-gc-dev at openjdk.java.net >>> >>> >> > >>> *Subject:* Re: linux os processor optimizations for >>> OpenJDK GC >>> performance enhancement >>> >>> Thanks Andrew. >>> >>> >>Surely there is: a thread could have its TLAB >>> allocated >>> from a region >>> >>> local to that socket (or core), and the GC >>> thread >>> for that region >>> could run on the same socket. It only works >>> for >>> young gen, but that's >>> a lot of the problem. >>> >>> >>> A clarification -- does the TLAB allocation apply to >>> tenured >>> space also? >>> If not, the above would work only for young gen >>> cases where >>> there is no >>> promotion to tenured right? >>> >>> Thanks, >>> Ramki >>> >>> On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan >>> >>> >> >>> wrote: >>> >>> >>> ---------- Forwarded message ---------- >>> From: >>> >>> Andrew Haley >> >> >> >>> Date: Thu, Apr 13, 2017 at 9:52 AM >>> Subject: Re: linux os processor optimizations for >>> OpenJDK GC performance >>> enhancement >>> To: >>> >>> hotspot-gc-dev at openjdk.java.net >>> >>> >> > >>> >>> >>> On 13/04/17 16:33, Kim Barrett wrote: >>> >>> An application thread may touch memory in any >>> region; there is no >>> notion of a thread being "scoped" to a >>> specific set >>> of regions. While >>> it might happen that a thread would only touch >>> regions not being >>> worked on by the collector, there is no a >>> priori way >>> to know that. >>> >>> >>> >>> Surely there is: a thread could have its TLAB >>> allocated >>> from a region >>> local to that socket (or core), and the GC >>> thread for >>> that region >>> could run on the same socket. It only works for >>> young >>> gen, but that's >>> a lot of the problem. >>> >>> Andrew. >>> >>> >>> >>> >>> -- >>> Thanks, >>> Ramki >>> >>> >>> >>> >>> -- >>> Thanks, >>> Ramki >>> >>> >>> >>> >>> >>> >>> >>> -- >>> Thanks, >>> Ramki >>> >>> >>> >>> >>> -- >>> Thanks, >>> Ramki >>> >> > > > -- > Thanks, > Ramki From erik.helin at oracle.com Tue Apr 25 09:37:48 2017 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 25 Apr 2017 11:37:48 +0200 Subject: RFR: 8179266: Add test/gc/g1/humongousObjects/TestHeapCounters.java to ProblemList.txt Message-ID: <1da8bc62-951f-8525-5d50-0936afee1e62@oracle.com> Hi all, the test hotspot/test/gc/g1/humongousObjects/TestHeapCounters.java fails intermittently, see https://bugs.openjdk.java.net/browse/JDK-8178918. I have added the test to hotspot/test/ProblemList.txt until the problem is fixed in order to avoid intermittent failures in hotspot_tier1. Webrev: http://cr.openjdk.java.net/~ehelin/8179266/00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8179266 Testing: Test is *not* run when executing: $ jtreg -conc:8 -exclude:hotspot/test/ProblemList.txt -jdk:build/linux-x64/images/jdk/ hotspot/test/:hotspot_tier1_gc_1 Thanks, Erik From mikael.gerdin at oracle.com Tue Apr 25 09:52:21 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 25 Apr 2017 11:52:21 +0200 Subject: RFR: 8179266: Add test/gc/g1/humongousObjects/TestHeapCounters.java to ProblemList.txt In-Reply-To: <1da8bc62-951f-8525-5d50-0936afee1e62@oracle.com> References: <1da8bc62-951f-8525-5d50-0936afee1e62@oracle.com> Message-ID: Hi Erik, On 2017-04-25 11:37, Erik Helin wrote: > Hi all, > > the test hotspot/test/gc/g1/humongousObjects/TestHeapCounters.java fails > intermittently, see https://bugs.openjdk.java.net/browse/JDK-8178918. I > have added the test to hotspot/test/ProblemList.txt until the problem is > fixed in order to avoid intermittent failures in hotspot_tier1. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8179266/00/ Looks good. /Mikael > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8179266 > > Testing: > Test is *not* run when executing: > $ jtreg -conc:8 -exclude:hotspot/test/ProblemList.txt > -jdk:build/linux-x64/images/jdk/ hotspot/test/:hotspot_tier1_gc_1 > > Thanks, > Erik From stefan.johansson at oracle.com Tue Apr 25 09:53:41 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 25 Apr 2017 11:53:41 +0200 Subject: RFR: 8179266: Add test/gc/g1/humongousObjects/TestHeapCounters.java to ProblemList.txt In-Reply-To: References: <1da8bc62-951f-8525-5d50-0936afee1e62@oracle.com> Message-ID: <540e09a7-4874-c106-3501-eed88340d1e2@oracle.com> On 2017-04-25 11:52, Mikael Gerdin wrote: > Hi Erik, > > On 2017-04-25 11:37, Erik Helin wrote: >> Hi all, >> >> the test hotspot/test/gc/g1/humongousObjects/TestHeapCounters.java fails >> intermittently, see https://bugs.openjdk.java.net/browse/JDK-8178918. I >> have added the test to hotspot/test/ProblemList.txt until the problem is >> fixed in order to avoid intermittent failures in hotspot_tier1. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8179266/00/ > > Looks good. +1 > /Mikael > >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8179266 >> >> Testing: >> Test is *not* run when executing: >> $ jtreg -conc:8 -exclude:hotspot/test/ProblemList.txt >> -jdk:build/linux-x64/images/jdk/ hotspot/test/:hotspot_tier1_gc_1 >> >> Thanks, >> Erik From shade at redhat.com Tue Apr 25 10:47:05 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 25 Apr 2017 12:47:05 +0200 Subject: RFR(XL/M) : 8178788: wrap JCStress test suite as jtreg tests In-Reply-To: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> References: <9A2C94EA-89A3-4C75-9D3C-51E058BD8A1D@oracle.com> Message-ID: <342c3748-616f-8a7d-74f4-3ce929b1e0dc@redhat.com> On 04/19/2017 12:12 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html >> 69903 lines changed: 69903 ins; 0 del; 0 mod; > (69524 lines are generated) > > Hi all, > > could you please review this patch which adds a jtreg test wrapper for > jcstress test suite and jtreg tests which run jsctress tests thru this > wrapper? > > webrev: http://cr.openjdk.java.net/~iignatyev//8178788/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8178788 testing: TL;DR: This patch introduces more problems than it solves. Just run the jcstress tests-all JAR against the tested runtime. Wrapping jcstress tests with jtreg defies the purpose of jcstress harness -- that is, running lots of tests as fast as it possibly could without affecting testing quality. For example, by cleverly reusing VMs between the tests, using Whitebox to deoptimize without restarting the VMs, etc. It really wastes CPU time to run each test in isolation. Also, it does not "automatically" work, which defies "easy to run" goal: Caused by: java.io.FileNotFoundException: Couldn't automatically resolve dependency for jcstress-tests-all , revision 0.3 Please specify the location using jdk.test.lib.artifacts.jcstress-tests-all at jdk.test.lib.artifacts.DefaultArtifactManager.resolve(DefaultArtifactManager.java:37) at jdk.test.lib.artifacts.ArtifactResolver.resolve(ArtifactResolver.java:54) at applications.jcstress.JcstressRunner.pathToArtifact(JcstressRunner.java:53) ... 8 more Okay, brilliant! How do I configure this, if I run "make test"? CONF=linux-x86_64-normal-server-release LOG=info make test TEST="hotspot_all" -Aleksey From volker.simonis at gmail.com Tue Apr 25 12:37:03 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 25 Apr 2017 14:37:03 +0200 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Message-ID: Hi Kim, thanks for implementing the AIX part of the change! I've compiled and tested it and it works fine. Although we currently have no 32-bit port on AIX I think you could leave the 32-bit variant in for reference (and to help the brave developer who starts the 32-bit version of the AIX port :) Thank you and best regards, Volker On Fri, Apr 21, 2017 at 7:41 PM, Kim Barrett wrote: >> On Apr 21, 2017, at 1:54 AM, Chris Plummer wrote: >> >> On 4/19/17 11:17 PM, Kim Barrett wrote: >>> Please review this addition of the count_trailing_zeros function. >>> >>> [?] >>> As part of reviewing this change, please feel free to suggest >>> alternative ways to structure the code. I'm not completely happy with >>> the way I've done it, but alternatives I've tried seemed worse. >> Are you talking about the #if/#elseif structure and the ////// delimiters. I'd suggest replacing the ////// with a block comment that stands out. Something like: >> >> /************** >> * GCC Support >> **************/ > > Sure, that makes things stand out a bit more. > > Updated webrev (only comment changes in count_trailing_zeros.hpp) > http://cr.openjdk.java.net/~kbarrett/8179004/hotspot.01/ > > What I'm looking for is whether this TARGET_COMPILER dispatch with the > corresponding code is okay, or whether some other structure would be > preferred. > > Some alternatives I considered were: > > * Always #include OS_CPU_INCLUDE(count_trailing_zeros). But that > proliforates files which will be duplicates (possibly duplicates > by #include of some shared toolchain-related include). > > * Dispatch to #include "utilities/count_trailing_zeros_.hpp" > similarly to what globalDefinitions.hpp does. This adds 4 small > files, which seems a little excessive. > > * Put these toolchain-specific definitions in the corresponding > globalDefintions_.hpp. But the #include of those files by > globalDefinitions.hpp happens very early in that file, before things > like the uintx type are defined. Also, presently debug.hpp can't > be #include'd by globalDefinitions.hpp, so the latter does similar > things "by hand". (I think that's something that ought to be fixed.) > >> Otherwise it looks fine to me, but I'm no expert in this area. The testing looks satisfactory and the bitmap results you are getting are reason enough to add this functionality. > > Thanks. Do you want to be counted as a reviewer? > From volker.simonis at gmail.com Tue Apr 25 15:06:22 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 25 Apr 2017 17:06:22 +0200 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <58FDECFE.5060105@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> <58FDECFE.5060105@oracle.com> Message-ID: On Mon, Apr 24, 2017 at 2:18 PM, Ioi Lam wrote: > Hi Volker, > > > On 4/21/17 12:02 AM, Volker Simonis wrote: >> >> Hi Ioi, >> >> thanks once again for considering our ports! Please find the required >> additions for ppc64/s390x in the following webrew (which is based upon >> your latest v03 patch): >> >> http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ > > Thanks for the patch. I will integrate it and post an updated webrev. >> >> @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly >> code. I did some tests and I think it should be correct, but maybe you >> still find some improvements :) >> >> Besides that, I have some general questions/comments regarding your >> change: >> >> 1. In constantPool.hpp, why don't you declare the '_name_index' and >> '_resolved_klass_index' fields with type 'jushort'? As far as I can >> see, they can only hold 16-bit values anyway. It would also save you >> some space and several asserts (e.g. in unresolved_klass_at_put(): >> >> >> 274 assert((name_index & 0xffff0000) == 0, "must be"); >> 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); > > > I think the HotSpot convention is to use ints as parameter and return types, > for values that are actually 16-bits or less, like here in constantPool.hpp: > > void field_at_put(int which, int class_index, int name_and_type_index) { > tag_at_put(which, JVM_CONSTANT_Fieldref); > *int_at_addr(which) = ((jint) name_and_type_index<<16) | class_index; > } > > I am not sure what the reasons are. It could be that the parameters usually > need to be computed arithmetically, and it's much easier for the caller of > the method to use ints -- otherwise you will get lots of compiler warnings > which would force you to use lots of casting, resulting in code that's hard > to read and probably incorrect. > OK, but you could still use shorts in the the object to save space, although I'm not sure how much that will save in total. But if nobody else cares, I'm fine with the current version. >> 2. What do you mean by: >> >> 106 // ... will be changed to support compressed pointers >> 107 Array* _resolved_klasses; > > > Sorry the comment isn't very clear. How about this? > > 106 // Consider using an array of compressed klass pointers to > // save space on 64-bit platforms. > 107 Array* _resolved_klasses; > Sorry I still didn't get it? Do you mean you want to use array of "narrowKlass" (i.e. unsigned int)? But using compressed class pointers is a runtime decision while this is a compile time decision. >> 3. Why don't we need the call to "release_tag_at_put()" in >> "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int >> class_index, Klass* k)" is used from >> "ClassFileParser::fill_instance_klass() and before your change that >> function used the previous version of "klass_at_put(int class_index, >> Klass* k)" which did call "release_tag_at_put()". > > > Good catch. I'll add the following, because the class is now resolved. > > release_tag_at_put(class_index, JVM_CONSTANT_UnresolvedClass); >> >> 4. In ConstantPool::copy_entry_to() you've changed the behavior for >> tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, >> JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was >> copied to the new constant pool if one existed but now you always only >> copy a class_index to the new constant pool (even if a resolved klass >> existed). Is that OK? E.g. won't this lead to a new resolving for the >> new constant pool and will this have performance impacts or other side >> effects? > > I think Coleen has answered this in a separate mail :-) > > Thanks > - Ioi > >> Thanks again for doing this nice change and best regards, >> Volker >> >> >> On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >>> >>> Hi Lois, >>> >>> I have updated the patch to include your comments, and fixes the handling >>> of >>> anonymous classes. I also added some more comments regarding the >>> _temp_resolved_klass_index: >>> >>> (delta from last webrev) >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >>> >>> (full webrev) >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >>> >>> Thanks >>> - Ioi >>> >>> >>> On 4/15/17 2:31 AM, Lois Foltan wrote: >>>> >>>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>>> >>>>> >>>>> >>>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>>> >>>>>> HI Lois, >>>>>> >>>>>> Thanks for the review. Please see my comments in-line. >>>>>> >>>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>>> >>>>>>> Hi Ioi, >>>>>>> >>>>>>> Looks really good. A couple of comments: >>>>>>> >>>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>>> surrounding anonymous classes. Coleen and I discussed earlier today >>>>>>> and I >>>>>>> came away from that discussion with the idea that the only classes >>>>>>> being >>>>>>> patched currently are anonymous classes. >>>>>> >>>>>> Line 5676 ... >>>>>> >>>>>> 5676 if (is_anonymous()) { >>>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>>> index >>>>>> 5678 } >>>>>> >>>>>> corresponds to >>>>>> >>>>>> 5361 ik->set_name(_class_name); >>>>>> 5362 >>>>>> 5363 if (is_anonymous()) { >>>>>> 5364 // I am well known to myself >>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>> ik->name()); // eagerly resolve >>>>>> 5366 } >>>>>> >>>>>> Even though the class is "anonymous", it actually has a name. >>>>>> ik->name() >>>>>> probably is part of the constant pool, but I am not 100% sure. Also, I >>>>>> would >>>>>> need to search the constant pool to find the index for ik->name(). So >>>>>> I just >>>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>>> append ik->name() to the end of the constant pool. >>>>>> >>>>>> "Anonymous" actually means "the class cannot be looked up by name in >>>>>> the >>>>>> SystemDictionary". I think we need a better terminology :-) >>>>>> >>>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>>> modify the comments to the following: >>>>> >>>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>>> // anonymous class itself. If this class needs to refer to its own >>>>> methods or >>>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>>> reference >>>>> // _this_class_index. However, because this class is anonymous >>>>> (it's >>>>> // not stored in SystemDictionary), _this_class_index cannot be >>>>> resolved >>>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>>> lookup. >>>>> // Therefore, we must eagerly resolve _this_class_index now. >>>>> >>>>> So, Lois is right. Line 5676 is not necessary. I will revise the code >>>>> to >>>>> do the "eager resolution" without using ClassFileParser::patch_class. >>>>> I'll >>>>> post the updated code later. >>>> >>>> >>>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>>> updated webrev. >>>> Lois >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>>> java/lang/Class on line #5684. >>>>>> >>>>>> 5683 Handle patch = cp_patch_at(i); >>>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>>> java_lang_Class::is_instance(patch())) { >>>>>> 5685 // We need to append the names of the patched classes >>>>>> to >>>>>> the end of the constant pool, >>>>>> 5686 // because a patched class may have a Utf8 name that's >>>>>> not already included in the >>>>>> 5687 // original constant pool. >>>>>> 5688 // >>>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>>> patch a Utf8, a String, or a Class. >>>>>> 5690 // At this point, we don't know the tag for index i >>>>>> yet, >>>>>> because we haven't parsed the >>>>>> 5691 // constant pool. So we can only assume the worst -- >>>>>> every String is used to patch a Class. >>>>>> 5692 _max_num_patched_klasses ++; >>>>>> >>>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>>> either Class or String could be treated as a Klass: >>>>>> >>>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>>> 725 int index, >>>>>> 726 Handle patch, >>>>>> 727 TRAPS) { >>>>>> ... >>>>>> 732 switch (cp->tag_at(index).value()) { >>>>>> 733 >>>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>>> 735 // Patching a class means pre-resolving it. >>>>>> 736 // The name in the constant pool is ignored. >>>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>>> 739 "Illegal class patch at %d in class >>>>>> file >>>>>> %s", >>>>>> 740 index, CHECK); >>>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>>> 742 patch_class(cp, index, k, k->name()); >>>>>> 743 } else { >>>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>>> 745 "Illegal class patch at %d in class >>>>>> file >>>>>> %s", >>>>>> 746 index, CHECK); >>>>>> 747 Symbol* const name = >>>>>> java_lang_String::as_symbol(patch(), >>>>>> CHECK); >>>>>> 748 patch_class(cp, index, NULL, name); >>>>>> 749 } >>>>>> 750 break; >>>>>> 751 } >>>>>> >>>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>>> >>>>>> >>>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>>> ik->name() is in the cp_patch array. >>>>>> >>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>> ik->name()); // eagerly resolve >>>>>> >>>>>>> Also why not do this calculation in the rewriter or is that too >>>>>>> late? >>>>>>> >>>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and >>>>>> the >>>>>> associated tags array is allocated (both of which are fixed size, and >>>>>> cannot >>>>>> be expanded), which is way before the rewriter is run. At this point, >>>>>> we >>>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to >>>>>> make >>>>>> a worst-case estimate on how long the CP/tags should be. >>>>>> >>>>>>> * line #5677, 5692 - a nit but I think the convention is to not have >>>>>>> a >>>>>>> space after the variable name and between the post increment >>>>>>> operator. >>>>>>> >>>>>> Fixed. >>>>>>> >>>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>>> really is not so much invalid as temporary for class redefinition >>>>>>> purposes, >>>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please >>>>>>> consider >>>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to >>>>>>> rename >>>>>>> the field or not, please add a one line comment ahead of >>>>>>> ConstantPool::temp_unresolved_klass_at_put that only class >>>>>>> redefinition uses >>>>>>> this currently. >>>>>>> >>>>>> Good idea. Will do. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>>> Great change, thanks! >>>>>>> Lois >>>>>>> >>>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>>> >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>>> >>>>>>>> In addition to your requests, I made these changes: >>>>>>>> >>>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>>> >>>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>>> int name_index = kslot.name_index(); >>>>>>>> >>>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick >>>>>>>> this >>>>>>>> function over MetaspaceObj::is_shared(). >>>>>>>> >>>>>>>> [3] Massaged the CDS region size set-up code a little to pass >>>>>>>> internal >>>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time >>>>>>>> picking >>>>>>>> the "right" sizes, as this code will be obsoleted soon with >>>>>>>> JDK-8072061 >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> This looks really good! >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>>> >>>>>>>>> + // Add one extra element to tags for storing >>>>>>>>> ConstantPool::flags(). >>>>>>>>> + Array* tags = >>>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, >>>>>>>>> "invariant"); // >>>>>>>>> tags->at(_length) is flags() >>>>>>>>> >>>>>>>>> >>>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>>> >>>>>>>>> + Klass** adr = >>>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>>> resolved >>>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>>> + // hardware store ordering here. >>>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>>> + return k; >>>>>>>>> >>>>>>>>> The comment still refers to the switch between Symbol* and Klass* >>>>>>>>> in >>>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>>> >>>>>>>>> + int name_index = >>>>>>>>> extract_high_short_from_int(*int_at_addr(which)); >>>>>>>>> >>>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header >>>>>>>>> file >>>>>>>>> (so it's inlined) and have all the places where you get name_index >>>>>>>>> use this >>>>>>>>> function? So we don't have to know in multiple places that >>>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about >>>>>>>>> what the >>>>>>>>> new format of the constant pool entry is {name_index, >>>>>>>>> resolved_klass_index}. >>>>>>>>> I'm happy to see this work nearing completion! The "constant" pool >>>>>>>>> should >>>>>>>>> be constant! thanks, Coleen >>>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>>> >>>>>>>>>> Hi,please review the following change >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>>>>>>>>> Klass from >>>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the >>>>>>>>>> ConstantPool >>>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>>> ConstantPool::klass_at(i) finds the at >>>>>>>>>> this->_resolved_klasses->at(i) + >>>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. >>>>>>>>>> In >>>>>>>>>> addition: + I moved _resolved_references and _reference_map >>>>>>>>>> from >>>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no >>>>>>>>>> longer >>>>>>>>>> modified for shared ConstantPools. As a result, none of the >>>>>>>>>> fields in >>>>>>>>>> shared ConstantPools are modified at run time, so we can move them >>>>>>>>>> into the >>>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results >>>>>>>>>> show no >>>>>>>>>> performance regression, despite the extra level of indirection, >>>>>>>>>> which has a >>>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 >>>>>>>>>> and >>>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support >>>>>>>>>> ports. For the >>>>>>>>>> aarch64, ppc and s390 ports, I have added some code without >>>>>>>>>> testing (it's >>>>>>>>>> probably incomplete) - Port owners, please check if my patch >>>>>>>>>> work for you, >>>>>>>>>> and I can incorporate your changes in my push. Alternatively, you >>>>>>>>>> can wait >>>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, >>>>>>>>>> and I will >>>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>>> >>>>>>>> > From ramkri123 at gmail.com Tue Apr 25 17:27:33 2017 From: ramkri123 at gmail.com (Ram Krishnan) Date: Tue, 25 Apr 2017 10:27:33 -0700 Subject: linux os processor optimizations for OpenJDK GC performance enhancement In-Reply-To: References: <610247D0-6B75-4535-88E6-7F80C83258ED@oracle.com> <58d3cf9f-0e0c-56d1-f8fe-ad9c614373cb@oracle.com> <15bf57e1-a53b-07c1-1216-deacdb0d8231@oracle.com> Message-ID: Hi Volker, This proposal is complementary to Large page and NUMA support. Please find below the typical processor memory access hierarchy and the role of each feature and other discussion points. *Typical Processor Memory Access Hierarchy* *Translation Lookaside Buffer (TLB)* -- Role of TLB is to translate from virtual to physical memory address in HW -- Large page support makes sure that TLB entries are not exhausted and thus lead to better performance -- Note: There are two TLB accesses in the case JVM is running on a hardware virtualized platform such as KVM, VMware etc. *Cache Hierarchy -- Scope of this proposal* *System Memory* -- Modern multi-socket machines typically have Non-uniform memory access (NUMA), with not all memory equidistant from each socket. -- NUMA support makes sure that memory local to a socket is used to the extent possible and thus lead to better performance. *Your concern with NUMA <-> large page interaction* I can see your concern with JEP [1] with the following remark about NUMA <-> large page interaction >>When using large pages, where multiple regions map to the same physical page, things get a >>bit complicated. For now, we will finesse this by disabling NUMA optimizations as soon as >>the page size exceeds some small multiple of region size (say 4), and deal with the more >>general case in a separate later phase. *Other ways to handle NUMA* Use Linux numactl -- https://linux.die.net/man/8/numactl -- "numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for command and inherited by all of its children. In addition it can set persistent policy for shared memory segments or files." NUMA topology awareness (leveraging linux numactl) is supported by orchestration systems such as OpenStack, Kubernetes etc. http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-topology-awareness-in-openstack-compute/ ) The caveats are 1) JVM should not require resources more than what a single socket can provide in terms of CPU, Memory and PCIe I/O 2) There may be resource fragmentation depending on the JVM resource request pattern. These are typically not a problem in modern server class CPUs. Hope this clarifies. Thanks, Ramki On Tue, Apr 25, 2017 at 1:57 AM, Volker Simonis wrote: > Hi Ram, > > while this sounds interesting, I wonder how this plays together with > NUMA and Large page support. I understand that these are different > concepts, but in the end it all bails down tot he fact that memory > access is not uniform and we have different "kinds" of memory. It > seems to me that this fact is currently not very well handled in > HotSpot and needs some general redesign. There are for example two > JEPs [1,2] about improving the NUMA support in general and in G1. One > of the problems is that NUMA support doesn't play well together with > Large/Huge page support. > > I think your proposal must be evaluated in the broader context of > enhancing the VM and GC for non-uniform memory architectures. > Otherwise it would be yet another point fix which doesn't plays well > together with other features like NUMA and LargePages. > > Thanks, > Volker > > [1] https://bugs.openjdk.java.net/browse/JDK-8046153 (JEP 163: Enable > NUMA Mode by Default When Appropriate) > [2] https://bugs.openjdk.java.net/browse/JDK-8046147 (JEP 157: G1 GC: > NUMA-Aware Allocation) > > On Wed, Apr 19, 2017 at 4:04 PM, Ram Krishnan wrote: > > Many thanks David. > > > > Thanks, > > Ramki > > > > On Tue, Apr 18, 2017 at 11:08 PM, David Holmes > > wrote: > > > >> On 19/04/2017 11:38 AM, Ram Krishnan wrote: > >> > >>> Hi David, > >>> > >>> Many thanks, please find attached text version of document for > temporary > >>> hosting. > >>> > >> > >> Hosted at: http://cr.openjdk.java.net/~dholmes/JEP-cache-partitioning- > >> v1.txt > >> > >> David > >> > >> > >>> Thanks, > >>> Ramki > >>> > >>> On Tue, Apr 18, 2017 at 5:42 PM, David Holmes >>> > wrote: > >>> > >>> Hi Ramki, > >>> > >>> On 19/04/2017 8:27 AM, Ram Krishnan wrote: > >>> > >>> Hi David, > >>> > >>> Thanks for the clarification. > >>> > >>> I have signed the OCA and mailed it to > >>> oracle-ca_us(at)oracle.com > >>> . Any help to expedite processing would be > >>> much > >>> appreciated. > >>> > >>> > >>> Can't help with that I'm afraid. :) > >>> > >>> We are seeing promising POC results (details in the google doc) > >>> for this > >>> proposal -- would really appreciate your help in moving this > >>> forward. > >>> > >>> > >>> If you email me a text/html version of the document I can host it > on > >>> cr.openjdk.java.net temporarily. For > >>> this to become a JEP you will need a sponsor with the necessary > >>> OpenJDK credentials. > >>> > >>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > >>> > >>> > >>> Cheers, > >>> David > >>> > >>> > >>> Thanks, > >>> Ramki > >>> > >>> On Tue, Apr 18, 2017 at 1:55 PM, David Holmes > >>> > >>> >>> >> wrote: > >>> > >>> Hi Ramki, > >>> > >>> On 19/04/2017 12:34 AM, Ram Krishnan wrote: > >>> > >>> Please find detailed proposal below, looking forward to > >>> your > >>> comments. > >>> > >>> "Minimize application tail latency using > >>> cache-partitioning-aware G1GC" -- > >>> > >>> https://docs.google.com/document/d/1rPMG4XUiE7cUEOogW1z5tBbB > >>> ZTclOWyg0arhuycXN94/edit > >>> >>> BZTclOWyg0arhuycXN94/edit> > >>> > >>> >>> BZTclOWyg0arhuycXN94/edit > >>> >>> BZTclOWyg0arhuycXN94/edit>> > >>> > >>> > >>> All contributions to OpenJDK need to be hosted on OpenJDK > >>> infrastructure not on external systems like the above. > >>> > >>> Also I can not see you listed as an OCA signatory. Are you > an > >>> OpenJDK contributor? > >>> > >>> Thanks, > >>> David > >>> ----- > >>> > >>> Thanks, > >>> Ramki > >>> > >>> On Thu, Apr 13, 2017 at 11:04 PM, Bernd Eckenfels > >>> > > >>> >>> > >>> wrote: > >>> > >>> Maybe it would be better to concentrate the > processor > >>> optimizations on > >>> accessors and barrriers without introducing a > >>> completely new GC > >>> architecture. I can imagine that especially in the > >>> area of > >>> NUMA, TLAB, huge > >>> pages, cache consistency and possibly MMX > extensions > >>> there > >>> is some > >>> potential. > >>> > >>> Abandoning the global STW - while it seems like a > >>> pretty > >>> powerful change - > >>> is I guess not a good starter exercise. Especially > >>> since it > >>> is not only a > >>> question of mutator threads. > >>> > >>> Gruss > >>> Bernd > >>> -- > >>> http://bernd.eckenfels.net > >>> ------------------------------ > >>> *From:* hotspot-gc-dev > >>> >>> > >>> >>> >> on > >>> behalf of Ram Krishnan >>> > >>> >>> >> > >>> *Sent:* Friday, April 14, 2017 6:36:27 AM > >>> *To:* Asif Qamar; Andrew Haley; > >>> hotspot-gc-dev at openjdk.java.net > >>> > >>> >>> > > >>> *Subject:* Re: linux os processor optimizations for > >>> OpenJDK GC > >>> performance enhancement > >>> > >>> Thanks Andrew. > >>> > >>> >>Surely there is: a thread could have its TLAB > >>> allocated > >>> from a region > >>> > >>> local to that socket (or core), and the GC > >>> thread > >>> for that region > >>> could run on the same socket. It only > works > >>> for > >>> young gen, but that's > >>> a lot of the problem. > >>> > >>> > >>> A clarification -- does the TLAB allocation apply > to > >>> tenured > >>> space also? > >>> If not, the above would work only for young gen > >>> cases where > >>> there is no > >>> promotion to tenured right? > >>> > >>> Thanks, > >>> Ramki > >>> > >>> On Thu, Apr 13, 2017 at 12:55 PM, Ram Krishnan > >>> > >>> >> > >>> wrote: > >>> > >>> > >>> ---------- Forwarded message ---------- > >>> From: > >>> > >>> Andrew Haley >>> >>> >> > >>> Date: Thu, Apr 13, 2017 at 9:52 AM > >>> Subject: Re: linux os processor optimizations > for > >>> OpenJDK GC performance > >>> enhancement > >>> To: > >>> > >>> hotspot-gc-dev at openjdk.java.net > >>> > >>> >>> > > >>> > >>> > >>> On 13/04/17 16:33, Kim Barrett wrote: > >>> > >>> An application thread may touch memory in > any > >>> region; there is no > >>> notion of a thread being "scoped" to a > >>> specific set > >>> of regions. While > >>> it might happen that a thread would only > touch > >>> regions not being > >>> worked on by the collector, there is no a > >>> priori way > >>> to know that. > >>> > >>> > >>> > >>> Surely there is: a thread could have its TLAB > >>> allocated > >>> from a region > >>> local to that socket (or core), and the GC > >>> thread for > >>> that region > >>> could run on the same socket. It only works > for > >>> young > >>> gen, but that's > >>> a lot of the problem. > >>> > >>> Andrew. > >>> > >>> > >>> > >>> > >>> -- > >>> Thanks, > >>> Ramki > >>> > >>> > >>> > >>> > >>> -- > >>> Thanks, > >>> Ramki > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> -- > >>> Thanks, > >>> Ramki > >>> > >>> > >>> > >>> > >>> -- > >>> Thanks, > >>> Ramki > >>> > >> > > > > > > -- > > Thanks, > > Ramki > -- Thanks, Ramki From kim.barrett at oracle.com Tue Apr 25 18:50:39 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 25 Apr 2017 14:50:39 -0400 Subject: RFR: 8179004: Add an efficient implementation of the "count trailing zeros" operation In-Reply-To: References: <6B50E7BF-A0CC-4100-A332-9E0EC8054C39@oracle.com> Message-ID: <5BBA1AEB-22CD-4E8B-B563-7FAF1F5ABF5B@oracle.com> > On Apr 25, 2017, at 8:37 AM, Volker Simonis wrote: > > Hi Kim, > > thanks for implementing the AIX part of the change! > I've compiled and tested it and it works fine. > Although we currently have no 32-bit port on AIX I think you could > leave the 32-bit variant in for reference (and to help the brave > developer who starts the 32-bit version of the AIX port :) > > Thank you and best regards, > Volker > Thanks. From david.holmes at oracle.com Wed Apr 26 01:17:32 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Apr 2017 11:17:32 +1000 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> Message-ID: Hi Aleksey, This looks good to me. I agree with assuming always MP for these atomic ops. We should also look at what other os::is_MP() checks should be deleted (different CR of course). Small nit: atomic_solaris_x86.hpp - when you deleted text the indentation of some declarations changed eg: jbyte _Atomic_cmpxchg_byte(jbyte exchange_value, volatile jbyte* dest, jbyte compare_value); should be jbyte _Atomic_cmpxchg_byte(jbyte exchange_value, volatile jbyte* dest, jbyte compare_value); Running it through JPRT. Thanks, David On 24/04/2017 10:53 PM, Aleksey Shipilev wrote: > Hi, > > See https://bugs.openjdk.java.net/browse/JDK-8169061. > > Our current Atomic code looks like this: > > // Adding a lock prefix to an instruction on MP machine > #define LOCK_IF_MP(mp) "cmp $0, " #mp "; je 1f; lock; 1: " > > inline jint Atomic::add (jint add_value, volatile jint* dest) { > jint addend = add_value; > int mp = os::is_MP(); > __asm__ volatile ( LOCK_IF_MP(%3) "xaddl %0,(%2)" > : "=r" (addend) > : "0" (addend), "r" (dest), "r" (mp) > : "cc", "memory"); > return addend + add_value; > } > > Notice the LOCK_IF_MP part, which means we have flag reads, additional branch, > etc. on the hot path. I would like us to consider dropping runtime os::is_MP > checks from Atomics. The history of the original change predates OpenJDK, but I > think it was deemed a plausible optimization in mostly-uniprocessor world. > Today, this penalizes multi-core platforms without a good reason. > > Proposed change for jdk10/hs: > http://cr.openjdk.java.net/~shade/8169061/webrev.01/ > > It unconditionally does lock prefix on all atomics. > > Our interest for doing this is GC performance work. For example, in Shenandoah, > native CASes are very frequently used for bitmap manipulation during marking, > updating forwarding pointers for evacuating objects, updating references in the > heap. We have measured around 1-2% GC time improvement with this change. I fully > expect it to benefit G1 too, because it also does CASes on bitmaps. > > Testing: shenandoah tests, hotspot_runtime on Linux x86_64 > > Putting this via the build-test system on other platforms would be appreciated! > > Thanks, > -Aleksey > From gromero at linux.vnet.ibm.com Wed Apr 26 02:49:36 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 25 Apr 2017 23:49:36 -0300 Subject: [10] RFR (S) 8175813: PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: References: <58C1AE06.9060609@linux.vnet.ibm.com> <58EEAF7B.6020708@linux.vnet.ibm.com> Message-ID: <59000AC0.7050507@linux.vnet.ibm.com> Dear Volker, On 24-04-2017 14:08, Volker Simonis wrote: > Hi Gustavo, > > thanks for addressing this problem and sorry for my late reply. I > think this is a good change which definitely improves the situation > for uncommon NUMA configurations without changing the handling for > common topologies. Thanks a lot for reviewing the change! > It would be great if somebody could run this trough JPRT, but as > Gustavo mentioned, I don't expect any regressions. > > @Igor: I think you've been the original author of the NUMA-aware > allocator port to Linux (i.e. "6684395: Port NUMA-aware allocator to > linux"). If you could find some spare minutes to take a look at this > change, your comment would be very much appreciated :) > > Following some minor comments from me: > > - in os::numa_get_groups_num() you now use numa_num_configured_nodes() > to get the actual number of configured nodes. This is good and > certainly an improvement over the previous implementation. However, > the man page for numa_num_configured_nodes() mentions that the > returned count may contain currently disabled nodes. Do we currently > handle disabled nodes? What will be the consequence if we would use > such a disabled node (e.g. mbind() warnings)? In [1] 'numa_memnode_ptr' is set to keep a list of *just nodes with memory in found in /sys/devices/system/node/* Hence numa_num_configured_nodes() just returns the number of nodes in 'numa_memnode_ptr' [2], thus just returns the number of nodes with memory in the system. To the best of my knowledge there is no system configuration on Linux/PPC64 that could match such a notion of "disabled nodes" as it appears in libnuma's manual. If it is enabled, it's in that dir and just the ones with memory will be taken into account. If it's disabled (somehow), it's not in the dir, so won't be taken into account (i.e. no mbind() tried against it). On Power it's possible to have a numa node without memory (memory-less node, a case covered in this change), a numa node without cpus at all but with memory (a configured node anyway, so a case already covered) but to disable a specific numa node so it does not appear in /sys/devices/system/node/* it's only possible from the inners of the control module. Or other rare condition not invisible / adjustable from the OS. Also I'm not aware of a case where a node is in this dir but is at the same time flagged as something like "disabled". There are cpu/memory hotplugs, but that does not change numa nodes status AFAIK. [1] https://github.com/numactl/numactl/blob/master/libnuma.c#L334-L347 [2] https://github.com/numactl/numactl/blob/master/libnuma.c#L614-L618 > - the same question applies to the usage of > Linux::isnode_in_configured_nodes() within os::numa_get_leaf_groups(). > Does isnode_in_configured_nodes() (i.e. the node set defined by > 'numa_all_nodes_ptr' take into account the disabled nodes or not? Can > this be a potential problem (i.e. if we use a disabled node). On the meaning of "disabled nodes", it's the same case as above, so to the best of knowledge it's not a potential problem. Anyway 'numa_all_nodes_ptr' just includes the configured nodes (with memory), i.e. "all nodes on which the calling task may allocate memory". It's exactly the same pointer returned by numa_get_membind() v2 [3] which: "returns the mask of nodes from which memory can currently be allocated" and that is used, for example, in "numactl --show" to show nodes from where memory can be allocated [4, 5]. [3] https://github.com/numactl/numactl/blob/master/libnuma.c#L1147 [4] https://github.com/numactl/numactl/blob/master/numactl.c#L144 [5] https://github.com/numactl/numactl/blob/master/numactl.c#L177 > - I'd like to suggest renaming the 'index' part of the following > variables and functions to 'nindex' ('node_index' is probably to long) > in the following code, to emphasize that we have node indexes pointing > to actual, not always consecutive node numbers: > > 2879 // Create an index -> node mapping, since nodes are not > always consecutive > 2880 _index_to_node = new (ResourceObj::C_HEAP, mtInternal) > GrowableArray(0, true); > 2881 rebuild_index_to_node_map(); Simple change but much better to read indeed. Done. > - can you please wrap the following one-line else statement into curly > braces (it's more readable and we usually do it that way in HotSpot > although there are no formal style guidelines :) > > 2953 } else > 2954 // Current node is already a configured node. > 2955 closest_node = index_to_node()->at(i); Done. > - in os::Linux::rebuild_cpu_to_node_map(), if you set > 'closest_distance' to INT_MAX at the beginning of the loop, you can > later avoid the check for '|| !closest_distance'. Also, according to > the man page, numa_distance() returns 0 if it can not determine the > distance. So with the above change, the condition on line 2974 should > read: > > 2947 if (distance && distance < closest_distance) { > Sure, much better to set the initial condition as distant as possible and adjust to a closer one bit by bit improving the if condition. Done. > Finally, and not directly related to your change, I'd suggest the > following clean-ups: > > - remove the usage of 'NCPUS = 32768' in > os::Linux::rebuild_cpu_to_node_map(). The comment on that line is > unclear to me and probably related to an older version/problem of > libnuma? I think we should simply use > numa_allocate_cpumask()/numa_free_cpumask() instead. > > - we still use the NUMA version 1 function prototypes (e.g. > "numa_node_to_cpus(int node, unsigned long *buffer, int buffer_len)" > instead of "numa_node_to_cpus(int node, struct bitmask *mask)", but > also "numa_interleave_memory()" and maybe others). I think we should > switch all prototypes to the new NUMA version 2 interface which you've > already used for the new functions which you've added. I agree. Could I open a new bug to address these clean-ups? > That said, I think these changes all require libnuma 2.0 (see > os::Linux::libnuma_dlsym). So before starting this, you should make > sure that libnuma 2.0 is available on all platforms to which you'd > like to down-port this change. For jdk10 we could definitely do it, > for jdk9 probably also, for jdk8 I'm not so sure. libnuma v1 last release dates back to 2008, but any idea how could I check that for sure since it's on shared code? new webrev: http://cr.openjdk.java.net/~gromero/8175813/v3/ Thank you! Best regards, Gustavo > Regards, > Volker > > On Thu, Apr 13, 2017 at 12:51 AM, Gustavo Romero > wrote: >> Hi, >> >> Any update on it? >> >> Thank you. >> >> Regards, >> Gustavo >> >> On 09-03-2017 16:33, Gustavo Romero wrote: >>> Hi, >>> >>> Could the following webrev be reviewed please? >>> >>> It improves the numa node detection when non-consecutive or memory-less nodes >>> exist in the system. >>> >>> webrev: http://cr.openjdk.java.net/~gromero/8175813/v2/ >>> bug : https://bugs.openjdk.java.net/browse/JDK-8175813 >>> >>> Currently, although no problem exists when the JVM detects numa nodes that are >>> consecutive and have memory, for example in a numa topology like: >>> >>> available: 2 nodes (0-1) >>> node 0 cpus: 0 8 16 24 32 >>> node 0 size: 65258 MB >>> node 0 free: 34 MB >>> node 1 cpus: 40 48 56 64 72 >>> node 1 size: 65320 MB >>> node 1 free: 150 MB >>> node distances: >>> node 0 1 >>> 0: 10 20 >>> 1: 20 10, >>> >>> it fails on detecting numa nodes to be used in the Parallel GC in a numa >>> topology like: >>> >>> available: 4 nodes (0-1,16-17) >>> node 0 cpus: 0 8 16 24 32 >>> node 0 size: 130706 MB >>> node 0 free: 7729 MB >>> node 1 cpus: 40 48 56 64 72 >>> node 1 size: 0 MB >>> node 1 free: 0 MB >>> node 16 cpus: 80 88 96 104 112 >>> node 16 size: 130630 MB >>> node 16 free: 5282 MB >>> node 17 cpus: 120 128 136 144 152 >>> node 17 size: 0 MB >>> node 17 free: 0 MB >>> node distances: >>> node 0 1 16 17 >>> 0: 10 20 40 40 >>> 1: 20 10 40 40 >>> 16: 40 40 10 20 >>> 17: 40 40 20 10, >>> >>> where node 16 is not consecutive in relation to 1 and also nodes 1 and 17 have >>> no memory. >>> >>> If a topology like that exists, os::numa_make_local() will receive a local group >>> id as a hint that is not available in the system to be bound (it will receive >>> all nodes from 0 to 17), causing a proliferation of "mbind: Invalid argument" >>> messages: >>> >>> http://cr.openjdk.java.net/~gromero/logs/jdk10_pristine.log >>> >>> That change improves the detection by making the JVM numa API aware of the >>> existence of numa nodes that are non-consecutive from 0 to the highest node >>> number and also of nodes that might be memory-less nodes, i.e. that might not >>> be, in libnuma terms, a configured node. Hence just the configured nodes will >>> be available: >>> >>> http://cr.openjdk.java.net/~gromero/logs/jdk10_numa_patched.log >>> >>> The change has no effect on numa topologies were the problem does not occur, >>> i.e. no change in the number of nodes and no change in the cpu to node map. On >>> numa topologies where memory-less nodes exist (like in the last example above), >>> cpus from a memory-less node won't be able to bind locally so they are mapped >>> to the closest node, otherwise they would be not associate to any node and >>> MutableNUMASpace::cas_allocate() would pick a node randomly, compromising the >>> performance. >>> >>> I found no regressions on x64 for the following numa topology: >>> >>> available: 2 nodes (0-1) >>> node 0 cpus: 0 1 2 3 8 9 10 11 >>> node 0 size: 24102 MB >>> node 0 free: 19806 MB >>> node 1 cpus: 4 5 6 7 12 13 14 15 >>> node 1 size: 24190 MB >>> node 1 free: 21951 MB >>> node distances: >>> node 0 1 >>> 0: 10 21 >>> 1: 21 10 >>> >>> I understand that fixing the current numa detection is a prerequisite to enable >>> UseNUMA by the default [1] and to extend the numa-aware allocation to the G1 GC [2]. >>> >>> Thank you. >>> >>> >>> Best regards, >>> Gustavo >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8046153 (JEP 163: Enable NUMA Mode by Default When Appropriate) >>> [2] https://bugs.openjdk.java.net/browse/JDK-8046147 (JEP 157: G1 GC: NUMA-Aware Allocation) >>> >> > From david.holmes at oracle.com Wed Apr 26 03:29:58 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Apr 2017 13:29:58 +1000 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> Message-ID: <9a61cfb1-3f4d-07f1-1436-32e9b54a1e48@oracle.com> One oversight in atomic_bsd_x86.hpp: 118 inline void Atomic::inc_ptr(volatile intptr_t* dest) { 119 __asm__ __volatile__ ( "lock addq $1,(%0)" 120 : 121 : "r" (dest), "r" (mp) Forgot to delete "mp" reference. Also copyright years need updating. Thanks, David On 26/04/2017 11:17 AM, David Holmes wrote: > Hi Aleksey, > > This looks good to me. I agree with assuming always MP for these atomic > ops. We should also look at what other os::is_MP() checks should be > deleted (different CR of course). > > Small nit: atomic_solaris_x86.hpp - when you deleted text the > indentation of some declarations changed eg: > > jbyte _Atomic_cmpxchg_byte(jbyte exchange_value, volatile jbyte* dest, > jbyte compare_value); > > should be > > jbyte _Atomic_cmpxchg_byte(jbyte exchange_value, volatile jbyte* dest, > jbyte compare_value); > > Running it through JPRT. > > Thanks, > David > > On 24/04/2017 10:53 PM, Aleksey Shipilev wrote: >> Hi, >> >> See https://bugs.openjdk.java.net/browse/JDK-8169061. >> >> Our current Atomic code looks like this: >> >> // Adding a lock prefix to an instruction on MP machine >> #define LOCK_IF_MP(mp) "cmp $0, " #mp "; je 1f; lock; 1: " >> >> inline jint Atomic::add (jint add_value, volatile jint* dest) { >> jint addend = add_value; >> int mp = os::is_MP(); >> __asm__ volatile ( LOCK_IF_MP(%3) "xaddl %0,(%2)" >> : "=r" (addend) >> : "0" (addend), "r" (dest), "r" (mp) >> : "cc", "memory"); >> return addend + add_value; >> } >> >> Notice the LOCK_IF_MP part, which means we have flag reads, additional >> branch, >> etc. on the hot path. I would like us to consider dropping runtime >> os::is_MP >> checks from Atomics. The history of the original change predates >> OpenJDK, but I >> think it was deemed a plausible optimization in mostly-uniprocessor >> world. >> Today, this penalizes multi-core platforms without a good reason. >> >> Proposed change for jdk10/hs: >> http://cr.openjdk.java.net/~shade/8169061/webrev.01/ >> >> It unconditionally does lock prefix on all atomics. >> >> Our interest for doing this is GC performance work. For example, in >> Shenandoah, >> native CASes are very frequently used for bitmap manipulation during >> marking, >> updating forwarding pointers for evacuating objects, updating >> references in the >> heap. We have measured around 1-2% GC time improvement with this >> change. I fully >> expect it to benefit G1 too, because it also does CASes on bitmaps. >> >> Testing: shenandoah tests, hotspot_runtime on Linux x86_64 >> >> Putting this via the build-test system on other platforms would be >> appreciated! >> >> Thanks, >> -Aleksey >> From shafi.s.ahmad at oracle.com Wed Apr 26 05:32:00 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 25 Apr 2017 22:32:00 -0700 (PDT) Subject: [8u] RFR for JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to jdk8u-dev Message-ID: Hi, Please review the backport of bug: "JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking" to jdk8u-dev Please note that this is not a clean backport due to - A) Method remove_handle(jobject h) is not present in jdk8u and looks it is related to module so I haven't merged associated change to jdk8u. B) File moduleEntry.cpp is not present in jdk8u repo, so I have ignore its changes. Webrev: http://cr.openjdk.java.net/~shshahma/8168914/webrev.00/ jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8168914 Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/606c35b6fac5 Test: Run jprt and jtreg is in progress. Regards, Shafi From shade at redhat.com Wed Apr 26 06:36:02 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 26 Apr 2017 08:36:02 +0200 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: <9a61cfb1-3f4d-07f1-1436-32e9b54a1e48@oracle.com> References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> <9a61cfb1-3f4d-07f1-1436-32e9b54a1e48@oracle.com> Message-ID: On 04/26/2017 05:29 AM, David Holmes wrote: > One oversight in atomic_bsd_x86.hpp: > > 118 inline void Atomic::inc_ptr(volatile intptr_t* dest) { > 119 __asm__ __volatile__ ( "lock addq $1,(%0)" > 120 : > 121 : "r" (dest), "r" (mp) > > Forgot to delete "mp" reference. > > Also copyright years need updating. Right, thanks for running it! I think I captured everything in here: http://cr.openjdk.java.net/~shade/8169061/webrev.02/ Thanks, -Aleksey From david.holmes at oracle.com Wed Apr 26 06:38:37 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Apr 2017 16:38:37 +1000 Subject: [8u] RFR for JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to jdk8u-dev In-Reply-To: References: Message-ID: Hi Shafi, On 26/04/2017 3:32 PM, Shafi Ahmad wrote: > Hi, > > Please review the backport of bug: "JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking" to jdk8u-dev > > Please note that this is not a clean backport due to - > A) Method remove_handle(jobject h) is not present in jdk8u and looks it is related to module so I haven't merged associated change to jdk8u. Without remove_handle_unsafe there is no use of "contains" so all of the code in the block: 138 #ifdef ASSERT 139 class VerifyContainsOopClosure : public OopClosure { ... 167 #endif is dead code and can be deleted. Otherwise the backported code looks okay to me. Thanks, David ----- > B) File moduleEntry.cpp is not present in jdk8u repo, so I have ignore its changes. > > > Webrev: http://cr.openjdk.java.net/~shshahma/8168914/webrev.00/ > jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8168914 > Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/606c35b6fac5 > > Test: Run jprt and jtreg is in progress. > > Regards, > Shafi > From david.holmes at oracle.com Wed Apr 26 06:42:55 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Apr 2017 16:42:55 +1000 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> <9a61cfb1-3f4d-07f1-1436-32e9b54a1e48@oracle.com> Message-ID: Looks fine to me! Thanks, David ----- On 26/04/2017 4:36 PM, Aleksey Shipilev wrote: > On 04/26/2017 05:29 AM, David Holmes wrote: >> One oversight in atomic_bsd_x86.hpp: >> >> 118 inline void Atomic::inc_ptr(volatile intptr_t* dest) { >> 119 __asm__ __volatile__ ( "lock addq $1,(%0)" >> 120 : >> 121 : "r" (dest), "r" (mp) >> >> Forgot to delete "mp" reference. >> >> Also copyright years need updating. > > Right, thanks for running it! > > I think I captured everything in here: > http://cr.openjdk.java.net/~shade/8169061/webrev.02/ > > Thanks, > -Aleksey > From shade at redhat.com Wed Apr 26 07:08:36 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 26 Apr 2017 09:08:36 +0200 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> <9a61cfb1-3f4d-07f1-1436-32e9b54a1e48@oracle.com> Message-ID: <872289a9-8810-6936-eda2-48bb7968df5d@redhat.com> Thanks! Would you mind sponsoring it? Changeset: http://cr.openjdk.java.net/~shade/8169061/hotspot.changeset -Aleksey On 04/26/2017 08:42 AM, David Holmes wrote: > Looks fine to me! > > Thanks, > David > ----- > > On 26/04/2017 4:36 PM, Aleksey Shipilev wrote: >> On 04/26/2017 05:29 AM, David Holmes wrote: >>> One oversight in atomic_bsd_x86.hpp: >>> >>> 118 inline void Atomic::inc_ptr(volatile intptr_t* dest) { >>> 119 __asm__ __volatile__ ( "lock addq $1,(%0)" >>> 120 : >>> 121 : "r" (dest), "r" (mp) >>> >>> Forgot to delete "mp" reference. >>> >>> Also copyright years need updating. >> >> Right, thanks for running it! >> >> I think I captured everything in here: >> http://cr.openjdk.java.net/~shade/8169061/webrev.02/ >> >> Thanks, >> -Aleksey >> From david.holmes at oracle.com Wed Apr 26 07:16:43 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Apr 2017 17:16:43 +1000 Subject: RFR (M) 8169061: Drop os::is_MP checks from Atomics In-Reply-To: <872289a9-8810-6936-eda2-48bb7968df5d@redhat.com> References: <146a65cb-56ab-2d2f-9367-88dde8fea567@redhat.com> <9a61cfb1-3f4d-07f1-1436-32e9b54a1e48@oracle.com> <872289a9-8810-6936-eda2-48bb7968df5d@redhat.com> Message-ID: On 26/04/2017 5:08 PM, Aleksey Shipilev wrote: > Thanks! Would you mind sponsoring it? No problem. David > Changeset: > http://cr.openjdk.java.net/~shade/8169061/hotspot.changeset > > -Aleksey > > On 04/26/2017 08:42 AM, David Holmes wrote: >> Looks fine to me! >> >> Thanks, >> David >> ----- >> >> On 26/04/2017 4:36 PM, Aleksey Shipilev wrote: >>> On 04/26/2017 05:29 AM, David Holmes wrote: >>>> One oversight in atomic_bsd_x86.hpp: >>>> >>>> 118 inline void Atomic::inc_ptr(volatile intptr_t* dest) { >>>> 119 __asm__ __volatile__ ( "lock addq $1,(%0)" >>>> 120 : >>>> 121 : "r" (dest), "r" (mp) >>>> >>>> Forgot to delete "mp" reference. >>>> >>>> Also copyright years need updating. >>> >>> Right, thanks for running it! >>> >>> I think I captured everything in here: >>> http://cr.openjdk.java.net/~shade/8169061/webrev.02/ >>> >>> Thanks, >>> -Aleksey >>> > > From shafi.s.ahmad at oracle.com Wed Apr 26 07:43:07 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 26 Apr 2017 00:43:07 -0700 (PDT) Subject: [8u] RFR for JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to jdk8u-dev In-Reply-To: References: Message-ID: <2bbedc3e-95c4-4bfa-ab96-477b56b7a7ac@default> Hi David, Thank you for the review comment. I have deleted the redundant code from the cpp and hpp file under #ifdef ASSERT ... #endif Please find updated webrev - http://cr.openjdk.java.net/~shshahma/8168914/webrev.01/ Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, April 26, 2017 12:09 PM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Cc: Coleen Phillimore ; Thomas Schatzl > ; Stefan Karlsson > > Subject: Re: [8u] RFR for JDK-8168914: Crash in > ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to > jdk8u-dev > > Hi Shafi, > > On 26/04/2017 3:32 PM, Shafi Ahmad wrote: > > Hi, > > > > Please review the backport of bug: "JDK-8168914: Crash in > > ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking" to > > jdk8u-dev > > > > Please note that this is not a clean backport due to - > > A) Method remove_handle(jobject h) is not present in jdk8u and looks it is > related to module so I haven't merged associated change to jdk8u. > > Without remove_handle_unsafe there is no use of "contains" so all of the > code in the block: > > 138 #ifdef ASSERT > 139 class VerifyContainsOopClosure : public OopClosure { > ... > 167 #endif > > is dead code and can be deleted. > > Otherwise the backported code looks okay to me. > > Thanks, > David > ----- > > > B) File moduleEntry.cpp is not present in jdk8u repo, so I have ignore its > changes. > > > > > > Webrev: http://cr.openjdk.java.net/~shshahma/8168914/webrev.00/ > > jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8168914 > > Original patch pushed to jdk9: > > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/606c35b6fac5 > > > > Test: Run jprt and jtreg is in progress. > > > > Regards, > > Shafi > > From shafi.s.ahmad at oracle.com Wed Apr 26 08:23:26 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 26 Apr 2017 01:23:26 -0700 (PDT) Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' Message-ID: Hi, Please review the one line trivial change for the fix of bug 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' Summary: I have initialized the uninitialized variable 'imp' with "unknown" as CPU implementation . jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 webrev link: http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ Testing: run jprt Regards, Shafi From shade at redhat.com Wed Apr 26 08:27:48 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 26 Apr 2017 10:27:48 +0200 Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' In-Reply-To: References: Message-ID: On 04/26/2017 10:23 AM, Shafi Ahmad wrote: > jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > webrev link: http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ Obviously correct. Looks good. -Aleksey From erik.helin at oracle.com Wed Apr 26 10:13:32 2017 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 26 Apr 2017 12:13:32 +0200 Subject: [8u] RFR for JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to jdk8u-dev In-Reply-To: <2bbedc3e-95c4-4bfa-ab96-477b56b7a7ac@default> References: <2bbedc3e-95c4-4bfa-ab96-477b56b7a7ac@default> Message-ID: <8f906b83-3f73-ee2b-dc02-0abdb414359c@oracle.com> On 04/26/2017 09:43 AM, Shafi Ahmad wrote: > Hi David, > > Thank you for the review comment. > > I have deleted the redundant code from the cpp and hpp file under #ifdef ASSERT ... #endif > Please find updated webrev - http://cr.openjdk.java.net/~shshahma/8168914/webrev.01/ Looks good to me, Reviewed (I was the original author of the patch). Thanks for taking care of the backport! Erik > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, April 26, 2017 12:09 PM >> To: Shafi Ahmad ; hotspot- >> dev at openjdk.java.net >> Cc: Coleen Phillimore ; Thomas Schatzl >> ; Stefan Karlsson >> >> Subject: Re: [8u] RFR for JDK-8168914: Crash in >> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to >> jdk8u-dev >> >> Hi Shafi, >> >> On 26/04/2017 3:32 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please review the backport of bug: "JDK-8168914: Crash in >>> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking" to >>> jdk8u-dev >>> >>> Please note that this is not a clean backport due to - >>> A) Method remove_handle(jobject h) is not present in jdk8u and looks it is >> related to module so I haven't merged associated change to jdk8u. >> >> Without remove_handle_unsafe there is no use of "contains" so all of the >> code in the block: >> >> 138 #ifdef ASSERT >> 139 class VerifyContainsOopClosure : public OopClosure { >> ... >> 167 #endif >> >> is dead code and can be deleted. >> >> Otherwise the backported code looks okay to me. >> >> Thanks, >> David >> ----- >> >>> B) File moduleEntry.cpp is not present in jdk8u repo, so I have ignore its >> changes. >>> >>> >>> Webrev: http://cr.openjdk.java.net/~shshahma/8168914/webrev.00/ >>> jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8168914 >>> Original patch pushed to jdk9: >>> http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/606c35b6fac5 >>> >>> Test: Run jprt and jtreg is in progress. >>> >>> Regards, >>> Shafi >>> From shafi.s.ahmad at oracle.com Wed Apr 26 10:18:20 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 26 Apr 2017 03:18:20 -0700 (PDT) Subject: [8u] RFR for JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to jdk8u-dev In-Reply-To: <8f906b83-3f73-ee2b-dc02-0abdb414359c@oracle.com> References: <2bbedc3e-95c4-4bfa-ab96-477b56b7a7ac@default> <8f906b83-3f73-ee2b-dc02-0abdb414359c@oracle.com> Message-ID: <58118486-274d-494e-b4e1-8425955b9308@default> Thank you Eric for reviewing it. Regards, Shafi > -----Original Message----- > From: Erik Helin > Sent: Wednesday, April 26, 2017 3:44 PM > To: Shafi Ahmad ; David Holmes > ; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8168914: Crash in > ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to > jdk8u-dev > > On 04/26/2017 09:43 AM, Shafi Ahmad wrote: > > Hi David, > > > > Thank you for the review comment. > > > > I have deleted the redundant code from the cpp and hpp file under > > #ifdef ASSERT ... #endif Please find updated webrev - > > http://cr.openjdk.java.net/~shshahma/8168914/webrev.01/ > > Looks good to me, Reviewed (I was the original author of the patch). > Thanks for taking care of the backport! > > Erik > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Wednesday, April 26, 2017 12:09 PM > >> To: Shafi Ahmad ; hotspot- > >> dev at openjdk.java.net > >> Cc: Coleen Phillimore ; Thomas Schatzl > >> ; Stefan Karlsson > >> > >> Subject: Re: [8u] RFR for JDK-8168914: Crash in > >> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to > >> jdk8u-dev > >> > >> Hi Shafi, > >> > >> On 26/04/2017 3:32 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please review the backport of bug: "JDK-8168914: Crash in > >>> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking" > >>> to jdk8u-dev > >>> > >>> Please note that this is not a clean backport due to - > >>> A) Method remove_handle(jobject h) is not present in jdk8u and > >>> looks it is > >> related to module so I haven't merged associated change to jdk8u. > >> > >> Without remove_handle_unsafe there is no use of "contains" so all of > >> the code in the block: > >> > >> 138 #ifdef ASSERT > >> 139 class VerifyContainsOopClosure : public OopClosure { > >> ... > >> 167 #endif > >> > >> is dead code and can be deleted. > >> > >> Otherwise the backported code looks okay to me. > >> > >> Thanks, > >> David > >> ----- > >> > >>> B) File moduleEntry.cpp is not present in jdk8u repo, so I have > >>> ignore its > >> changes. > >>> > >>> > >>> Webrev: http://cr.openjdk.java.net/~shshahma/8168914/webrev.00/ > >>> jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8168914 > >>> Original patch pushed to jdk9: > >>> http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/606c35b6fac5 > >>> > >>> Test: Run jprt and jtreg is in progress. > >>> > >>> Regards, > >>> Shafi > >>> From david.holmes at oracle.com Wed Apr 26 12:05:27 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Apr 2017 22:05:27 +1000 Subject: [8u] RFR for JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to jdk8u-dev In-Reply-To: <2bbedc3e-95c4-4bfa-ab96-477b56b7a7ac@default> References: <2bbedc3e-95c4-4bfa-ab96-477b56b7a7ac@default> Message-ID: <1a484356-9d88-73b4-90fd-41f19c9f1af4@oracle.com> Looks good! Thanks, David On 26/04/2017 5:43 PM, Shafi Ahmad wrote: > Hi David, > > Thank you for the review comment. > > I have deleted the redundant code from the cpp and hpp file under #ifdef ASSERT ... #endif > Please find updated webrev - http://cr.openjdk.java.net/~shshahma/8168914/webrev.01/ > > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, April 26, 2017 12:09 PM >> To: Shafi Ahmad ; hotspot- >> dev at openjdk.java.net >> Cc: Coleen Phillimore ; Thomas Schatzl >> ; Stefan Karlsson >> >> Subject: Re: [8u] RFR for JDK-8168914: Crash in >> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to >> jdk8u-dev >> >> Hi Shafi, >> >> On 26/04/2017 3:32 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please review the backport of bug: "JDK-8168914: Crash in >>> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking" to >>> jdk8u-dev >>> >>> Please note that this is not a clean backport due to - >>> A) Method remove_handle(jobject h) is not present in jdk8u and looks it is >> related to module so I haven't merged associated change to jdk8u. >> >> Without remove_handle_unsafe there is no use of "contains" so all of the >> code in the block: >> >> 138 #ifdef ASSERT >> 139 class VerifyContainsOopClosure : public OopClosure { >> ... >> 167 #endif >> >> is dead code and can be deleted. >> >> Otherwise the backported code looks okay to me. >> >> Thanks, >> David >> ----- >> >>> B) File moduleEntry.cpp is not present in jdk8u repo, so I have ignore its >> changes. >>> >>> >>> Webrev: http://cr.openjdk.java.net/~shshahma/8168914/webrev.00/ >>> jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8168914 >>> Original patch pushed to jdk9: >>> http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/606c35b6fac5 >>> >>> Test: Run jprt and jtreg is in progress. >>> >>> Regards, >>> Shafi >>> From shafi.s.ahmad at oracle.com Wed Apr 26 12:08:17 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 26 Apr 2017 05:08:17 -0700 (PDT) Subject: [8u] RFR for JDK-8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to jdk8u-dev In-Reply-To: <1a484356-9d88-73b4-90fd-41f19c9f1af4@oracle.com> References: <2bbedc3e-95c4-4bfa-ab96-477b56b7a7ac@default> <1a484356-9d88-73b4-90fd-41f19c9f1af4@oracle.com> Message-ID: <983d095c-aa97-4f96-a41c-fe8117bf4723@default> Hi David, Thank you for the review. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, April 26, 2017 5:35 PM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Cc: Coleen Phillimore ; Thomas Schatzl > ; Stefan Karlsson > > Subject: Re: [8u] RFR for JDK-8168914: Crash in > ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to > jdk8u-dev > > Looks good! > > Thanks, > David > > On 26/04/2017 5:43 PM, Shafi Ahmad wrote: > > Hi David, > > > > Thank you for the review comment. > > > > I have deleted the redundant code from the cpp and hpp file under > > #ifdef ASSERT ... #endif Please find updated webrev - > > http://cr.openjdk.java.net/~shshahma/8168914/webrev.01/ > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Wednesday, April 26, 2017 12:09 PM > >> To: Shafi Ahmad ; hotspot- > >> dev at openjdk.java.net > >> Cc: Coleen Phillimore ; Thomas Schatzl > >> ; Stefan Karlsson > >> > >> Subject: Re: [8u] RFR for JDK-8168914: Crash in > >> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking? to > >> jdk8u-dev > >> > >> Hi Shafi, > >> > >> On 26/04/2017 3:32 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please review the backport of bug: "JDK-8168914: Crash in > >>> ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking" > >>> to jdk8u-dev > >>> > >>> Please note that this is not a clean backport due to - > >>> A) Method remove_handle(jobject h) is not present in jdk8u and > >>> looks it is > >> related to module so I haven't merged associated change to jdk8u. > >> > >> Without remove_handle_unsafe there is no use of "contains" so all of > >> the code in the block: > >> > >> 138 #ifdef ASSERT > >> 139 class VerifyContainsOopClosure : public OopClosure { > >> ... > >> 167 #endif > >> > >> is dead code and can be deleted. > >> > >> Otherwise the backported code looks okay to me. > >> > >> Thanks, > >> David > >> ----- > >> > >>> B) File moduleEntry.cpp is not present in jdk8u repo, so I have > >>> ignore its > >> changes. > >>> > >>> > >>> Webrev: http://cr.openjdk.java.net/~shshahma/8168914/webrev.00/ > >>> jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8168914 > >>> Original patch pushed to jdk9: > >>> http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/606c35b6fac5 > >>> > >>> Test: Run jprt and jtreg is in progress. > >>> > >>> Regards, > >>> Shafi > >>> From shafi.s.ahmad at oracle.com Wed Apr 26 12:21:06 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 26 Apr 2017 05:21:06 -0700 (PDT) Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' In-Reply-To: References: Message-ID: Hi Aleksey, Thank you for the review. Regards, Shafi > -----Original Message----- > From: Aleksey Shipilev [mailto:shade at redhat.com] > Sent: Wednesday, April 26, 2017 1:58 PM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: [10] RFR for 'JDK-8177958: Possible uninitialized char* in > vm_version_solaris_sparc.cpp' > > On 04/26/2017 10:23 AM, Shafi Ahmad wrote: > > jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > > webrev link: http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ > > Obviously correct. Looks good. > > -Aleksey > > From kevin.walls at oracle.com Wed Apr 26 12:24:36 2017 From: kevin.walls at oracle.com (Kevin Walls) Date: Wed, 26 Apr 2017 13:24:36 +0100 Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' In-Reply-To: References: Message-ID: Thanks Shafi, looks good to me. 8-) -- Kevin On 26/04/2017 09:23, Shafi Ahmad wrote: > Hi, > > Please review the one line trivial change for the fix of bug 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' > > Summary: > I have initialized the uninitialized variable 'imp' with "unknown" as CPU implementation . > > jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > webrev link: http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ > > Testing: run jprt > > Regards, > Shafi From david.holmes at oracle.com Wed Apr 26 12:33:52 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Apr 2017 22:33:52 +1000 Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' In-Reply-To: References: Message-ID: <31e8f7ce-e6f7-1c79-9b2c-41c6ae9b2904@oracle.com> Hi Shafi, On 26/04/2017 6:23 PM, Shafi Ahmad wrote: > Hi, > > Please review the one line trivial change for the fix of bug 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' > > Summary: > I have initialized the uninitialized variable 'imp' with "unknown" as CPU implementation . That fixes the immediate problem - thanks. However the basic problem that kstat failures go unreported/logged persists. If the assert is triggered all we will see is "Unknown CPU implementation unknown" - which is not very enlightening. Perhaps a future enhancement ... David > jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > webrev link: http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ > > Testing: run jprt > > Regards, > Shafi > From shafi.s.ahmad at oracle.com Wed Apr 26 12:44:44 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 26 Apr 2017 05:44:44 -0700 (PDT) Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' In-Reply-To: <31e8f7ce-e6f7-1c79-9b2c-41c6ae9b2904@oracle.com> References: <31e8f7ce-e6f7-1c79-9b2c-41c6ae9b2904@oracle.com> Message-ID: <55c67eb5-901b-4ecb-ad0d-7106ab96c412@default> Thank you David and Kevin. Other possible change may be like - diff -r 762465099d93 src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp --- a/src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp Sat Apr 22 00:21:28 2017 +0000 +++ b/src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp Sun Apr 23 23:49:42 2017 -0700 @@ -404,7 +404,7 @@ // is available to us as well Sysinfo cpu_info(SI_CPUBRAND); bool use_solaris_12_api = cpu_info.valid(); - const char* impl; + const char* impl = "Unknown"; int impl_m = 0; if (use_solaris_12_api) { impl = cpu_info.value(); @@ -431,7 +431,7 @@ kstat_close(kc); } } - assert(impl_m != 0, "Unknown CPU implementation %s", impl); + assert(impl_m != 0, "%s CPU implementation", impl); features |= impl_m; bool is_sun4v = (features & sun4v_m) != 0; After the above change we will lose the old message format. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, April 26, 2017 6:04 PM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: [10] RFR for 'JDK-8177958: Possible uninitialized char* in > vm_version_solaris_sparc.cpp' > > Hi Shafi, > > On 26/04/2017 6:23 PM, Shafi Ahmad wrote: > > Hi, > > > > Please review the one line trivial change for the fix of bug 'JDK-8177958: > Possible uninitialized char* in vm_version_solaris_sparc.cpp' > > > > Summary: > > I have initialized the uninitialized variable 'imp' with "unknown" as CPU > implementation . > > That fixes the immediate problem - thanks. > > However the basic problem that kstat failures go unreported/logged persists. > If the assert is triggered all we will see is "Unknown CPU implementation > unknown" - which is not very enlightening. Perhaps a future enhancement ... > > David > > > jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > > webrev link: http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ > > > > Testing: run jprt > > > > Regards, > > Shafi > > From david.holmes at oracle.com Wed Apr 26 20:50:16 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Apr 2017 06:50:16 +1000 Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' In-Reply-To: <55c67eb5-901b-4ecb-ad0d-7106ab96c412@default> References: <31e8f7ce-e6f7-1c79-9b2c-41c6ae9b2904@oracle.com> <55c67eb5-901b-4ecb-ad0d-7106ab96c412@default> Message-ID: <1cd0cdaa-1dd4-81a9-3411-c2d83fb549a1@oracle.com> On 26/04/2017 10:44 PM, Shafi Ahmad wrote: > Thank you David and Kevin. > > Other possible change may be like - > > diff -r 762465099d93 src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp > --- a/src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp Sat Apr 22 00:21:28 2017 +0000 > +++ b/src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp Sun Apr 23 23:49:42 2017 -0700 > @@ -404,7 +404,7 @@ > // is available to us as well > Sysinfo cpu_info(SI_CPUBRAND); > bool use_solaris_12_api = cpu_info.valid(); > - const char* impl; > + const char* impl = "Unknown"; > int impl_m = 0; > if (use_solaris_12_api) { > impl = cpu_info.value(); > @@ -431,7 +431,7 @@ > kstat_close(kc); > } > } > - assert(impl_m != 0, "Unknown CPU implementation %s", impl); > + assert(impl_m != 0, "%s CPU implementation", impl); I would change to: assert(impl_m != 0, "Unrecognized CPU implementation: %s", impl); That assertion is checking the recognition of the impl string, not whether we obtained an impl string in the first place. Thanks, David > features |= impl_m; > > bool is_sun4v = (features & sun4v_m) != 0; > > After the above change we will lose the old message format. > > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, April 26, 2017 6:04 PM >> To: Shafi Ahmad ; hotspot- >> dev at openjdk.java.net >> Subject: Re: [10] RFR for 'JDK-8177958: Possible uninitialized char* in >> vm_version_solaris_sparc.cpp' >> >> Hi Shafi, >> >> On 26/04/2017 6:23 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please review the one line trivial change for the fix of bug 'JDK-8177958: >> Possible uninitialized char* in vm_version_solaris_sparc.cpp' >>> >>> Summary: >>> I have initialized the uninitialized variable 'imp' with "unknown" as CPU >> implementation . >> >> That fixes the immediate problem - thanks. >> >> However the basic problem that kstat failures go unreported/logged persists. >> If the assert is triggered all we will see is "Unknown CPU implementation >> unknown" - which is not very enlightening. Perhaps a future enhancement ... >> >> David >> >>> jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 >>> webrev link: http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ >>> >>> Testing: run jprt >>> >>> Regards, >>> Shafi >>> From rkennke at redhat.com Wed Apr 26 20:58:05 2017 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 26 Apr 2017 22:58:05 +0200 Subject: RFR: JDK-8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' Message-ID: <23550b76-2f10-bb4f-315d-4b2137ad796e@redhat.com> I posted this before: http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/024889.html And just realized it's not been fixed yet. Maybe now would be a good time to include it? I believe this changeset address all the issues mentioned in the above discussions. http://cr.openjdk.java.net/~rkennke/8167659/webrev.00/ Testing: jcstress -m sanity, specjvm (cannot currently run jtreg tests because I've no idea how to run the jcstress-jtreg tests..) Roman From shafi.s.ahmad at oracle.com Thu Apr 27 04:27:34 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 26 Apr 2017 21:27:34 -0700 (PDT) Subject: [10] RFR for 'JDK-8177958: Possible uninitialized char* in vm_version_solaris_sparc.cpp' In-Reply-To: <1cd0cdaa-1dd4-81a9-3411-c2d83fb549a1@oracle.com> References: <31e8f7ce-e6f7-1c79-9b2c-41c6ae9b2904@oracle.com> <55c67eb5-901b-4ecb-ad0d-7106ab96c412@default> <1cd0cdaa-1dd4-81a9-3411-c2d83fb549a1@oracle.com> Message-ID: <1645fdd7-ab1b-41a0-97f4-69c7127c27f6@default> Thank you David. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Thursday, April 27, 2017 2:20 AM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: [10] RFR for 'JDK-8177958: Possible uninitialized char* in > vm_version_solaris_sparc.cpp' > > On 26/04/2017 10:44 PM, Shafi Ahmad wrote: > > Thank you David and Kevin. > > > > Other possible change may be like - > > > > diff -r 762465099d93 > src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp > > --- a/src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp Sat > Apr 22 00:21:28 2017 +0000 > > +++ b/src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp Sun > Apr 23 23:49:42 2017 -0700 > > @@ -404,7 +404,7 @@ > > // is available to us as well > > Sysinfo cpu_info(SI_CPUBRAND); > > bool use_solaris_12_api = cpu_info.valid(); > > - const char* impl; > > + const char* impl = "Unknown"; > > int impl_m = 0; > > if (use_solaris_12_api) { > > impl = cpu_info.value(); > > @@ -431,7 +431,7 @@ > > kstat_close(kc); > > } > > } > > - assert(impl_m != 0, "Unknown CPU implementation %s", impl); > > + assert(impl_m != 0, "%s CPU implementation", impl); > > I would change to: > > assert(impl_m != 0, "Unrecognized CPU implementation: %s", impl); > > That assertion is checking the recognition of the impl string, not whether we > obtained an impl string in the first place. > > Thanks, > David > > > features |= impl_m; > > > > bool is_sun4v = (features & sun4v_m) != 0; > > > > After the above change we will lose the old message format. > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Wednesday, April 26, 2017 6:04 PM > >> To: Shafi Ahmad ; hotspot- > >> dev at openjdk.java.net > >> Subject: Re: [10] RFR for 'JDK-8177958: Possible uninitialized char* > >> in vm_version_solaris_sparc.cpp' > >> > >> Hi Shafi, > >> > >> On 26/04/2017 6:23 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please review the one line trivial change for the fix of bug 'JDK-8177958: > >> Possible uninitialized char* in vm_version_solaris_sparc.cpp' > >>> > >>> Summary: > >>> I have initialized the uninitialized variable 'imp' with "unknown" > >>> as CPU > >> implementation . > >> > >> That fixes the immediate problem - thanks. > >> > >> However the basic problem that kstat failures go unreported/logged > persists. > >> If the assert is triggered all we will see is "Unknown CPU > >> implementation unknown" - which is not very enlightening. Perhaps a > future enhancement ... > >> > >> David > >> > >>> jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8177958 > >>> webrev link: > http://cr.openjdk.java.net/~shshahma/8177958/webrev.00/ > >>> > >>> Testing: run jprt > >>> > >>> Regards, > >>> Shafi > >>> From doug.simon at oracle.com Thu Apr 27 14:47:53 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 27 Apr 2017 16:47:53 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> Message-ID: <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> > On 21 Apr 2017, at 13:46, Doug Simon wrote: > > There has been some discussion about whether we want to update Graal in the JDK at this late stage. The main (only?) risk is a regression in the AOT tool. > > If we don't update Graal from upstream, then the qualified exports from JVMCI to jdk.internal.vm.compiler cannot be removed in JDK 9. Note that in addition to updating Graal to remove the qualified exports, there would also need to be changes in the relevant make files to add --add-exports options when compiling Graal and jaotc as they use the dynamically exported JVMCI packages. > > I have an updated hotspot patch that adapts Graal to the JVMCI API changes: > > http://cr.openjdk.java.net/~dnsimon/8177845/hotspot.02/ > > Note that this patch does not include the changes removing use of JDK internal API from Graal. Cherry picking those upstream Graal changes would be more work than simply doing a complete update from upstream Graal. > > As I see it, there are 2 options here: > > 1. Go with the current webrev (including hotspot.02 patch) and live with the qualified exports. > 2. Go with the current webrev (including hotspot.02 patch) and create a follow up bug to update Graal from upstream, perform the relevant make file changes and remove the qualified exports. I made a new webrev[1] that implements option 1.5 ;-) The changes added since the first webrev[2] are: - Cherry picked changes from upstream Graal that remove use of JDK internals. - The jdk.internal.vm.ci.enabled system property is set to true in arguments.cpp[3] iff EnableJVMCI is true and this property is checked in all the public methods in jdk.vm.ci.services. - The jdk.vm.ci.services package is (once again) only exported to jdk.internal.vm.compiler based on advice from the jigsaw team: "We reviewed the unqualified export jdk.vm.ci.services from jdk.internal.vm.ci module. This brings jdk.internal.vm.ci to be resolved by default that of course may resolve additional modules that provides the services that JVMCI uses. In addition. JVMCI is meant to be used (only) when -XX:+EnableJVMCI is specified but now it?s defined by default. An internal module should only have qualified exports as a design principle. The Lab Graal will have the same module name, jdk.internal.vm.compiler. The advise is to keep it as qualified export `exports jdk.vm.ci.services to jdk.internal.vm.compiler` and remove all other qualified exports as we discussed." - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. Unfortunately there needs to be one --add-exports option per qualified export target as combining them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) breaks the make process: Launcher-jdk.aot.gmk:31: *** missing separator. Stop. make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. The latest webrev has been tested against upstream Graal, the closed AOT tests and jprt. -Doug [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 [2] http://cr.openjdk.java.net/~dnsimon/8177845 [3] http://cr.openjdk.java.net/~dnsimon/8177845.02/hotspot/src/share/vm/runtime/arguments.cpp.udiff.html [4] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html > >> On 20 Apr 2017, at 20:50, Doug Simon wrote: >> >> I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an >> exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg >> tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. >> Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to >> the version compatible with the JVMCI API changes. >> >> In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these >> tests better isolates them from parts of the runtime they are not aiming to test. >> >> org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) >> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) >> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) >> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) >> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) >> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) >> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) >> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) >> at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) >> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:563) >> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >> at java.base/java.lang.Thread.run(Thread.java:844) >> Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) >> ... 26 more >> >> -Doug >> >>> On 19 Apr 2017, at 23:26, Doug Simon wrote: >>> >>> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: >>> >>> 1. JVMCIServiceLocator.getProvider(Class) is now protected >>> 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission >>> 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK >>> >>> -Doug >>> >>>> On 19 Apr 2017, at 23:12, Doug Simon wrote: >>>> >>>>> >>>>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>>>> >>>>>> >>>>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>>>> >>>>>>> >>>>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>>>> >>>>>>> >>>>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>>>> >>>>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>>>> >>>>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>>>> >>>>>>> It's unqualified and no restriction in this change. >>>>>> >>>>>> The public methods currently in jdk.vm.ci.services are: >>>>>> >>>>>> 1. JVMCIServiceLocator.getProvider(Class) >>>>>> 2. JVMCIServiceLocator.getProviders(Class) >>>>>> 3. Services.initializeJVMCI() >>>>>> 4. Services.getSavedProperties() >>>>>> 5. Services.exportJVMCITo(Class) >>>>>> 6. Services.load(Class) >>>>>> 7. Services.loadSingle(Class, boolean) >>>>>> >>>>>> 1 should be made protected. I'll update the webrev with this change. >>>>> >>>>> Good. >>>>> >>>>>> >>>>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>>>> >>>>> Good. >>>>> >>>>>> >>>>>> 3 is harmless from a security perspective in my opinion. >>>>> >>>>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >>>> >>>> Vladimir, can you please bring this to the attention of the relevant engineer. >>>> >>>>>> >>>>>> 4 checks for JVMCIPermission. >>>>> >>>>> Ok. >>>>> >>>>>> >>>>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>>>> >>>>> About this, will this Graal update happen for JDK 9? >>>> >>>> Yes. >>>> >>>>> It?s awfully late in the cycle... >>>> >>>> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >>>> >>>> -Doug >>> >> > From Alan.Bateman at oracle.com Thu Apr 27 15:40:54 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 27 Apr 2017 16:40:54 +0100 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> Message-ID: <18ba3d88-1fa0-7d89-d57b-043cea141722@oracle.com> On 27/04/2017 15:47, Doug Simon wrote: > : > > - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. > Unfortunately there needs to be one --add-exports option per qualified export target as combining > them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) > breaks the make process: > > Launcher-jdk.aot.gmk:31: *** missing separator. Stop. > make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. > `--add-exports` is accumulative so what you have is okay. Alternatively the build has CommaList to convert space separated lists into comma separated lists so I would expect this should work: --add-exports=jdk.internal.vm.ci/jdk.vm.ci.amd64=$(call CommaList, jdk.internal.vm.compiler jdk.aot) -Alan From doug.simon at oracle.com Thu Apr 27 15:42:56 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 27 Apr 2017 17:42:56 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <18ba3d88-1fa0-7d89-d57b-043cea141722@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> <18ba3d88-1fa0-7d89-d57b-043cea141722@oracle.com> Message-ID: <4E03F554-1596-497B-A972-469782AAB12C@oracle.com> > On 27 Apr 2017, at 17:40, Alan Bateman wrote: > > On 27/04/2017 15:47, Doug Simon wrote: > >> : >> >> - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. >> Unfortunately there needs to be one --add-exports option per qualified export target as combining >> them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) >> breaks the make process: >> >> Launcher-jdk.aot.gmk:31: *** missing separator. Stop. >> make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. >> > `--add-exports` is accumulative so what you have is okay. Yes, it seems to work so I'm going to leave it as is. > Alternatively the build has CommaList to convert space separated lists into comma separated lists so I would expect this should work: > > --add-exports=jdk.internal.vm.ci/jdk.vm.ci.amd64=$(call CommaList, jdk.internal.vm.compiler jdk.aot) Thanks. Good to know in the future. -Doug From cthalinger at twitter.com Thu Apr 27 16:44:09 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Apr 2017 09:44:09 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> Message-ID: Thanks for doing this. I have a hard time following what?s happening :-) src/jdk.internal.vm.compiler/.mx.graal/suite.py + "jdklibraries" : { + "JVMCI_SERVICES" : { + "path" : "lib/jvmci-services.jar", + "sourcePath" : "lib/jvmci-services.src.zip", + "optional" : False, + "jdkStandardizedSince" : "9", + "module" : "jdk.internal.vm.ci" + }, + "JVMCI_API" : { + "path" : "lib/jvmci/jvmci-api.jar", + "sourcePath" : "lib/jvmci/jvmci-api.src.zip", + "dependencies" : [ + "JVMCI_SERVICES", + ], + "optional" : False, + "jdkStandardizedSince" : "9", + "module" : "jdk.internal.vm.ci" + }, + "JVMCI_HOTSPOT" : { + "path" : "lib/jvmci/jvmci-hotspot.jar", + "sourcePath" : "lib/jvmci/jvmci-hotspot.src.zip", + "dependencies" : [ + "JVMCI_API", + ], + "optional" : False, + "jdkStandardizedSince" : "9", + "module" : "jdk.internal.vm.ci" + }, + }, Can you explain to me why we need this? I don?t think these files are built in 9. src/jdk.internal.vm.ci/share/classes/module-info.java Why can we remove the exports to jdk.internal.vm.compiler? Because of this in JVMCIServiceLocator: + ReflectionAccessJDK.openJVMCITo(getClass()); ? > On Apr 27, 2017, at 7:47 AM, Doug Simon wrote: > >> >> On 21 Apr 2017, at 13:46, Doug Simon wrote: >> >> There has been some discussion about whether we want to update Graal in the JDK at this late stage. The main (only?) risk is a regression in the AOT tool. >> >> If we don't update Graal from upstream, then the qualified exports from JVMCI to jdk.internal.vm.compiler cannot be removed in JDK 9. Note that in addition to updating Graal to remove the qualified exports, there would also need to be changes in the relevant make files to add --add-exports options when compiling Graal and jaotc as they use the dynamically exported JVMCI packages. >> >> I have an updated hotspot patch that adapts Graal to the JVMCI API changes: >> >> http://cr.openjdk.java.net/~dnsimon/8177845/hotspot.02/ >> >> Note that this patch does not include the changes removing use of JDK internal API from Graal. Cherry picking those upstream Graal changes would be more work than simply doing a complete update from upstream Graal. >> >> As I see it, there are 2 options here: >> >> 1. Go with the current webrev (including hotspot.02 patch) and live with the qualified exports. >> 2. Go with the current webrev (including hotspot.02 patch) and create a follow up bug to update Graal from upstream, perform the relevant make file changes and remove the qualified exports. > > I made a new webrev[1] that implements option 1.5 ;-) The changes added since the first webrev[2] are: > > - Cherry picked changes from upstream Graal that remove use of JDK internals. > > - The jdk.internal.vm.ci.enabled system property is set to true in arguments.cpp[3] iff EnableJVMCI is true > and this property is checked in all the public methods in jdk.vm.ci.services. > > - The jdk.vm.ci.services package is (once again) only exported to jdk.internal.vm.compiler based on > advice from the jigsaw team: > > "We reviewed the unqualified export jdk.vm.ci.services from jdk.internal.vm.ci module. This brings > jdk.internal.vm.ci to be resolved by default that of course may resolve additional modules that > provides the services that JVMCI uses. In addition. JVMCI is meant to be used (only) when > -XX:+EnableJVMCI is specified but now it?s defined by default. > > An internal module should only have qualified exports as a design principle. The Lab Graal will > have the same module name, jdk.internal.vm.compiler. The advise is to keep it as qualified export > `exports jdk.vm.ci.services to jdk.internal.vm.compiler` and remove all other qualified exports as > we discussed." > > - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. > Unfortunately there needs to be one --add-exports option per qualified export target as combining > them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) > breaks the make process: > > Launcher-jdk.aot.gmk:31: *** missing separator. Stop. > make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. > > The latest webrev has been tested against upstream Graal, the closed AOT tests and jprt. > > -Doug > > [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 > [2] http://cr.openjdk.java.net/~dnsimon/8177845 > [3] http://cr.openjdk.java.net/~dnsimon/8177845.02/hotspot/src/share/vm/runtime/arguments.cpp.udiff.html > [4] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html > >> >>> On 20 Apr 2017, at 20:50, Doug Simon wrote: >>> >>> I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an >>> exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg >>> tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. >>> Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to >>> the version compatible with the JVMCI API changes. >>> >>> In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these >>> tests better isolates them from parts of the runtime they are not aiming to test. >>> >>> org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) >>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) >>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) >>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) >>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) >>> at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) >>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) >>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.base/java.lang.reflect.Method.invoke(Method.java:563) >>> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >>> at java.base/java.lang.Thread.run(Thread.java:844) >>> Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) >>> ... 26 more >>> >>> -Doug >>> >>>> On 19 Apr 2017, at 23:26, Doug Simon wrote: >>>> >>>> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: >>>> >>>> 1. JVMCIServiceLocator.getProvider(Class) is now protected >>>> 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission >>>> 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK >>>> >>>> -Doug >>>> >>>>> On 19 Apr 2017, at 23:12, Doug Simon wrote: >>>>> >>>>>> >>>>>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>>>>> >>>>>>> >>>>>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>>>>> >>>>>>>> >>>>>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>>>>> >>>>>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>>>>> >>>>>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>>>>> >>>>>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>>>>> >>>>>>>> It's unqualified and no restriction in this change. >>>>>>> >>>>>>> The public methods currently in jdk.vm.ci.services are: >>>>>>> >>>>>>> 1. JVMCIServiceLocator.getProvider(Class) >>>>>>> 2. JVMCIServiceLocator.getProviders(Class) >>>>>>> 3. Services.initializeJVMCI() >>>>>>> 4. Services.getSavedProperties() >>>>>>> 5. Services.exportJVMCITo(Class) >>>>>>> 6. Services.load(Class) >>>>>>> 7. Services.loadSingle(Class, boolean) >>>>>>> >>>>>>> 1 should be made protected. I'll update the webrev with this change. >>>>>> >>>>>> Good. >>>>>> >>>>>>> >>>>>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>>>>> >>>>>> Good. >>>>>> >>>>>>> >>>>>>> 3 is harmless from a security perspective in my opinion. >>>>>> >>>>>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >>>>> >>>>> Vladimir, can you please bring this to the attention of the relevant engineer. >>>>> >>>>>>> >>>>>>> 4 checks for JVMCIPermission. >>>>>> >>>>>> Ok. >>>>>> >>>>>>> >>>>>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>>>>> >>>>>> About this, will this Graal update happen for JDK 9? >>>>> >>>>> Yes. >>>>> >>>>>> It?s awfully late in the cycle... >>>>> >>>>> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >>>>> >>>>> -Doug From doug.simon at oracle.com Thu Apr 27 18:51:43 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 27 Apr 2017 20:51:43 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> Message-ID: <70D315B0-38E8-444C-AC89-897263638D4F@oracle.com> > On 27 Apr 2017, at 18:44, Christian Thalinger wrote: > > Thanks for doing this. I have a hard time following what?s happening :-) > > src/jdk.internal.vm.compiler/.mx.graal/suite.py > > + "jdklibraries" : { > + "JVMCI_SERVICES" : { > + "path" : "lib/jvmci-services.jar", > + "sourcePath" : "lib/jvmci-services.src.zip", > + "optional" : False, > + "jdkStandardizedSince" : "9", > + "module" : "jdk.internal.vm.ci" > + }, > + "JVMCI_API" : { > + "path" : "lib/jvmci/jvmci-api.jar", > + "sourcePath" : "lib/jvmci/jvmci-api.src.zip", > + "dependencies" : [ > + "JVMCI_SERVICES", > + ], > + "optional" : False, > + "jdkStandardizedSince" : "9", > + "module" : "jdk.internal.vm.ci" > + }, > + "JVMCI_HOTSPOT" : { > + "path" : "lib/jvmci/jvmci-hotspot.jar", > + "sourcePath" : "lib/jvmci/jvmci-hotspot.src.zip", > + "dependencies" : [ > + "JVMCI_API", > + ], > + "optional" : False, > + "jdkStandardizedSince" : "9", > + "module" : "jdk.internal.vm.ci" > + }, > + }, > > Can you explain to me why we need this? I don?t think these files are built in 9. This simply allows `mx eclipseinit` to work when wanting to edit the JDK Graal sources in Eclipse. As you state, this is completely by-passed by the JDK make system. > > src/jdk.internal.vm.ci/share/classes/module-info.java > > Why can we remove the exports to jdk.internal.vm.compiler? Because of this in JVMCIServiceLocator: > > + ReflectionAccessJDK.openJVMCITo(getClass()); Yes. And the --add-exports in CompileJavaModules.gmk[1] and Launcher-jdk.aot.gmk[2]. It also means these VM options are required in upstream Graal when running the Graal junit tests. -Doug [1] http://cr.openjdk.java.net/~dnsimon/8177845.02/make/make/CompileJavaModules.gmk.udiff.html [2] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html > > ? > >> On Apr 27, 2017, at 7:47 AM, Doug Simon wrote: >> >>> >>> On 21 Apr 2017, at 13:46, Doug Simon wrote: >>> >>> There has been some discussion about whether we want to update Graal in the JDK at this late stage. The main (only?) risk is a regression in the AOT tool. >>> >>> If we don't update Graal from upstream, then the qualified exports from JVMCI to jdk.internal.vm.compiler cannot be removed in JDK 9. Note that in addition to updating Graal to remove the qualified exports, there would also need to be changes in the relevant make files to add --add-exports options when compiling Graal and jaotc as they use the dynamically exported JVMCI packages. >>> >>> I have an updated hotspot patch that adapts Graal to the JVMCI API changes: >>> >>> http://cr.openjdk.java.net/~dnsimon/8177845/hotspot.02/ >>> >>> Note that this patch does not include the changes removing use of JDK internal API from Graal. Cherry picking those upstream Graal changes would be more work than simply doing a complete update from upstream Graal. >>> >>> As I see it, there are 2 options here: >>> >>> 1. Go with the current webrev (including hotspot.02 patch) and live with the qualified exports. >>> 2. Go with the current webrev (including hotspot.02 patch) and create a follow up bug to update Graal from upstream, perform the relevant make file changes and remove the qualified exports. >> >> I made a new webrev[1] that implements option 1.5 ;-) The changes added since the first webrev[2] are: >> >> - Cherry picked changes from upstream Graal that remove use of JDK internals. >> >> - The jdk.internal.vm.ci.enabled system property is set to true in arguments.cpp[3] iff EnableJVMCI is true >> and this property is checked in all the public methods in jdk.vm.ci.services. >> >> - The jdk.vm.ci.services package is (once again) only exported to jdk.internal.vm.compiler based on >> advice from the jigsaw team: >> >> "We reviewed the unqualified export jdk.vm.ci.services from jdk.internal.vm.ci module. This brings >> jdk.internal.vm.ci to be resolved by default that of course may resolve additional modules that >> provides the services that JVMCI uses. In addition. JVMCI is meant to be used (only) when >> -XX:+EnableJVMCI is specified but now it?s defined by default. >> >> An internal module should only have qualified exports as a design principle. The Lab Graal will >> have the same module name, jdk.internal.vm.compiler. The advise is to keep it as qualified export >> `exports jdk.vm.ci.services to jdk.internal.vm.compiler` and remove all other qualified exports as >> we discussed." >> >> - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. >> Unfortunately there needs to be one --add-exports option per qualified export target as combining >> them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) >> breaks the make process: >> >> Launcher-jdk.aot.gmk:31: *** missing separator. Stop. >> make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. >> >> The latest webrev has been tested against upstream Graal, the closed AOT tests and jprt. >> >> -Doug >> >> [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 >> [2] http://cr.openjdk.java.net/~dnsimon/8177845 >> [3] http://cr.openjdk.java.net/~dnsimon/8177845.02/hotspot/src/share/vm/runtime/arguments.cpp.udiff.html >> [4] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html >> >>> >>>> On 20 Apr 2017, at 20:50, Doug Simon wrote: >>>> >>>> I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an >>>> exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg >>>> tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. >>>> Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to >>>> the version compatible with the JVMCI API changes. >>>> >>>> In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these >>>> tests better isolates them from parts of the runtime they are not aiming to test. >>>> >>>> org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) >>>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) >>>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) >>>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) >>>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) >>>> at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) >>>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) >>>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.base/java.lang.reflect.Method.invoke(Method.java:563) >>>> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >>>> at java.base/java.lang.Thread.run(Thread.java:844) >>>> Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) >>>> ... 26 more >>>> >>>> -Doug >>>> >>>>> On 19 Apr 2017, at 23:26, Doug Simon wrote: >>>>> >>>>> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: >>>>> >>>>> 1. JVMCIServiceLocator.getProvider(Class) is now protected >>>>> 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission >>>>> 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK >>>>> >>>>> -Doug >>>>> >>>>>> On 19 Apr 2017, at 23:12, Doug Simon wrote: >>>>>> >>>>>>> >>>>>>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>>>>>> >>>>>>>> >>>>>>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>>>>>> >>>>>>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>>>>>> >>>>>>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>>>>>> >>>>>>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>>>>>> >>>>>>>>> It's unqualified and no restriction in this change. >>>>>>>> >>>>>>>> The public methods currently in jdk.vm.ci.services are: >>>>>>>> >>>>>>>> 1. JVMCIServiceLocator.getProvider(Class) >>>>>>>> 2. JVMCIServiceLocator.getProviders(Class) >>>>>>>> 3. Services.initializeJVMCI() >>>>>>>> 4. Services.getSavedProperties() >>>>>>>> 5. Services.exportJVMCITo(Class) >>>>>>>> 6. Services.load(Class) >>>>>>>> 7. Services.loadSingle(Class, boolean) >>>>>>>> >>>>>>>> 1 should be made protected. I'll update the webrev with this change. >>>>>>> >>>>>>> Good. >>>>>>> >>>>>>>> >>>>>>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>>>>>> >>>>>>> Good. >>>>>>> >>>>>>>> >>>>>>>> 3 is harmless from a security perspective in my opinion. >>>>>>> >>>>>>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >>>>>> >>>>>> Vladimir, can you please bring this to the attention of the relevant engineer. >>>>>> >>>>>>>> >>>>>>>> 4 checks for JVMCIPermission. >>>>>>> >>>>>>> Ok. >>>>>>> >>>>>>>> >>>>>>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>>>>>> >>>>>>> About this, will this Graal update happen for JDK 9? >>>>>> >>>>>> Yes. >>>>>> >>>>>>> It?s awfully late in the cycle... >>>>>> >>>>>> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >>>>>> >>>>>> -Doug > From cthalinger at twitter.com Thu Apr 27 18:53:29 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Apr 2017 11:53:29 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <70D315B0-38E8-444C-AC89-897263638D4F@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> <70D315B0-38E8-444C-AC89-897263638D4F@oracle.com> Message-ID: <0FCF026F-AB78-4D1C-857B-667FB7FA7FCA@twitter.com> > On Apr 27, 2017, at 11:51 AM, Doug Simon wrote: > >> >> On 27 Apr 2017, at 18:44, Christian Thalinger wrote: >> >> Thanks for doing this. I have a hard time following what?s happening :-) >> >> src/jdk.internal.vm.compiler/.mx.graal/suite.py >> >> + "jdklibraries" : { >> + "JVMCI_SERVICES" : { >> + "path" : "lib/jvmci-services.jar", >> + "sourcePath" : "lib/jvmci-services.src.zip", >> + "optional" : False, >> + "jdkStandardizedSince" : "9", >> + "module" : "jdk.internal.vm.ci" >> + }, >> + "JVMCI_API" : { >> + "path" : "lib/jvmci/jvmci-api.jar", >> + "sourcePath" : "lib/jvmci/jvmci-api.src.zip", >> + "dependencies" : [ >> + "JVMCI_SERVICES", >> + ], >> + "optional" : False, >> + "jdkStandardizedSince" : "9", >> + "module" : "jdk.internal.vm.ci" >> + }, >> + "JVMCI_HOTSPOT" : { >> + "path" : "lib/jvmci/jvmci-hotspot.jar", >> + "sourcePath" : "lib/jvmci/jvmci-hotspot.src.zip", >> + "dependencies" : [ >> + "JVMCI_API", >> + ], >> + "optional" : False, >> + "jdkStandardizedSince" : "9", >> + "module" : "jdk.internal.vm.ci" >> + }, >> + }, >> >> Can you explain to me why we need this? I don?t think these files are built in 9. > > This simply allows `mx eclipseinit` to work when wanting to edit the JDK Graal sources in Eclipse. As you state, this is completely by-passed by the JDK make system. Got it. > >> >> src/jdk.internal.vm.ci/share/classes/module-info.java >> >> Why can we remove the exports to jdk.internal.vm.compiler? Because of this in JVMCIServiceLocator: >> >> + ReflectionAccessJDK.openJVMCITo(getClass()); > > Yes. And the --add-exports in CompileJavaModules.gmk[1] and Launcher-jdk.aot.gmk[2]. It also means these VM options are required in upstream Graal when running the Graal junit tests. Ok, thanks. > > -Doug > > [1] http://cr.openjdk.java.net/~dnsimon/8177845.02/make/make/CompileJavaModules.gmk.udiff.html > [2] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html > >> >> ? >> >>> On Apr 27, 2017, at 7:47 AM, Doug Simon wrote: >>> >>>> >>>> On 21 Apr 2017, at 13:46, Doug Simon wrote: >>>> >>>> There has been some discussion about whether we want to update Graal in the JDK at this late stage. The main (only?) risk is a regression in the AOT tool. >>>> >>>> If we don't update Graal from upstream, then the qualified exports from JVMCI to jdk.internal.vm.compiler cannot be removed in JDK 9. Note that in addition to updating Graal to remove the qualified exports, there would also need to be changes in the relevant make files to add --add-exports options when compiling Graal and jaotc as they use the dynamically exported JVMCI packages. >>>> >>>> I have an updated hotspot patch that adapts Graal to the JVMCI API changes: >>>> >>>> http://cr.openjdk.java.net/~dnsimon/8177845/hotspot.02/ >>>> >>>> Note that this patch does not include the changes removing use of JDK internal API from Graal. Cherry picking those upstream Graal changes would be more work than simply doing a complete update from upstream Graal. >>>> >>>> As I see it, there are 2 options here: >>>> >>>> 1. Go with the current webrev (including hotspot.02 patch) and live with the qualified exports. >>>> 2. Go with the current webrev (including hotspot.02 patch) and create a follow up bug to update Graal from upstream, perform the relevant make file changes and remove the qualified exports. >>> >>> I made a new webrev[1] that implements option 1.5 ;-) The changes added since the first webrev[2] are: >>> >>> - Cherry picked changes from upstream Graal that remove use of JDK internals. >>> >>> - The jdk.internal.vm.ci.enabled system property is set to true in arguments.cpp[3] iff EnableJVMCI is true >>> and this property is checked in all the public methods in jdk.vm.ci.services. >>> >>> - The jdk.vm.ci.services package is (once again) only exported to jdk.internal.vm.compiler based on >>> advice from the jigsaw team: >>> >>> "We reviewed the unqualified export jdk.vm.ci.services from jdk.internal.vm.ci module. This brings >>> jdk.internal.vm.ci to be resolved by default that of course may resolve additional modules that >>> provides the services that JVMCI uses. In addition. JVMCI is meant to be used (only) when >>> -XX:+EnableJVMCI is specified but now it?s defined by default. >>> >>> An internal module should only have qualified exports as a design principle. The Lab Graal will >>> have the same module name, jdk.internal.vm.compiler. The advise is to keep it as qualified export >>> `exports jdk.vm.ci.services to jdk.internal.vm.compiler` and remove all other qualified exports as >>> we discussed." >>> >>> - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. >>> Unfortunately there needs to be one --add-exports option per qualified export target as combining >>> them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) >>> breaks the make process: >>> >>> Launcher-jdk.aot.gmk:31: *** missing separator. Stop. >>> make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. >>> >>> The latest webrev has been tested against upstream Graal, the closed AOT tests and jprt. >>> >>> -Doug >>> >>> [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 >>> [2] http://cr.openjdk.java.net/~dnsimon/8177845 >>> [3] http://cr.openjdk.java.net/~dnsimon/8177845.02/hotspot/src/share/vm/runtime/arguments.cpp.udiff.html >>> [4] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html >>> >>>> >>>>> On 20 Apr 2017, at 20:50, Doug Simon wrote: >>>>> >>>>> I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an >>>>> exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg >>>>> tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. >>>>> Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to >>>>> the version compatible with the JVMCI API changes. >>>>> >>>>> In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these >>>>> tests better isolates them from parts of the runtime they are not aiming to test. >>>>> >>>>> org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) >>>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) >>>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) >>>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) >>>>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) >>>>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) >>>>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) >>>>> at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) >>>>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) >>>>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>> at java.base/java.lang.reflect.Method.invoke(Method.java:563) >>>>> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >>>>> at java.base/java.lang.Thread.run(Thread.java:844) >>>>> Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) >>>>> ... 26 more >>>>> >>>>> -Doug >>>>> >>>>>> On 19 Apr 2017, at 23:26, Doug Simon wrote: >>>>>> >>>>>> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: >>>>>> >>>>>> 1. JVMCIServiceLocator.getProvider(Class) is now protected >>>>>> 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission >>>>>> 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK >>>>>> >>>>>> -Doug >>>>>> >>>>>>> On 19 Apr 2017, at 23:12, Doug Simon wrote: >>>>>>> >>>>>>>> >>>>>>>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>>>>>>> >>>>>>>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>>>>>>> >>>>>>>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>>>>>>> >>>>>>>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>>>>>>> >>>>>>>>>> It's unqualified and no restriction in this change. >>>>>>>>> >>>>>>>>> The public methods currently in jdk.vm.ci.services are: >>>>>>>>> >>>>>>>>> 1. JVMCIServiceLocator.getProvider(Class) >>>>>>>>> 2. JVMCIServiceLocator.getProviders(Class) >>>>>>>>> 3. Services.initializeJVMCI() >>>>>>>>> 4. Services.getSavedProperties() >>>>>>>>> 5. Services.exportJVMCITo(Class) >>>>>>>>> 6. Services.load(Class) >>>>>>>>> 7. Services.loadSingle(Class, boolean) >>>>>>>>> >>>>>>>>> 1 should be made protected. I'll update the webrev with this change. >>>>>>>> >>>>>>>> Good. >>>>>>>> >>>>>>>>> >>>>>>>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>>>>>>> >>>>>>>> Good. >>>>>>>> >>>>>>>>> >>>>>>>>> 3 is harmless from a security perspective in my opinion. >>>>>>>> >>>>>>>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >>>>>>> >>>>>>> Vladimir, can you please bring this to the attention of the relevant engineer. >>>>>>> >>>>>>>>> >>>>>>>>> 4 checks for JVMCIPermission. >>>>>>>> >>>>>>>> Ok. >>>>>>>> >>>>>>>>> >>>>>>>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>>>>>>> >>>>>>>> About this, will this Graal update happen for JDK 9? >>>>>>> >>>>>>> Yes. >>>>>>> >>>>>>>> It?s awfully late in the cycle... >>>>>>> >>>>>>> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >>>>>>> >>>>>>> -Doug From vladimir.kozlov at oracle.com Thu Apr 27 19:15:19 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Apr 2017 12:15:19 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> Message-ID: make/CompileJavaModules.gmk Please, adjust comment for jdk.aot: ># Don't use Indy strings concatenation to have good JVMCI startup performance. --- <# Don't use Indy strings concatenation to have good JAOTC startup performance. Should we also add jdk.vm.ci.aarch64 and jdk.vm.ci.sparc exports for AOT? It is not needed for JDK 9 but we will support them in a future and analyze build failures is painful. Same in make/launcher/Launcher-jdk.aot.gmk. I don't see changes in hotspot/make/CompileTools.gmk and hotspot/make/gensrc/Gensrc-jdk.internal.vm.compiler.gmk (which do annotation processor build): http://hg.openjdk.java.net/jdk9/dev/hotspot/file/f1cca489e9c6/make/gensrc/Gensrc-jdk.internal.vm.compiler.gmk I assume you don't need to change anything there. Right? Overall changes looks good to me. thanks, Vladimir On 4/27/17 7:47 AM, Doug Simon wrote: > >> On 21 Apr 2017, at 13:46, Doug Simon wrote: >> >> There has been some discussion about whether we want to update Graal in the JDK at this late stage. The main (only?) risk is a regression in the AOT tool. >> >> If we don't update Graal from upstream, then the qualified exports from JVMCI to jdk.internal.vm.compiler cannot be removed in JDK 9. Note that in addition to updating Graal to remove the qualified exports, there would also need to be changes in the relevant make files to add --add-exports options when compiling Graal and jaotc as they use the dynamically exported JVMCI packages. >> >> I have an updated hotspot patch that adapts Graal to the JVMCI API changes: >> >> http://cr.openjdk.java.net/~dnsimon/8177845/hotspot.02/ >> >> Note that this patch does not include the changes removing use of JDK internal API from Graal. Cherry picking those upstream Graal changes would be more work than simply doing a complete update from upstream Graal. >> >> As I see it, there are 2 options here: >> >> 1. Go with the current webrev (including hotspot.02 patch) and live with the qualified exports. >> 2. Go with the current webrev (including hotspot.02 patch) and create a follow up bug to update Graal from upstream, perform the relevant make file changes and remove the qualified exports. > > I made a new webrev[1] that implements option 1.5 ;-) The changes added since the first webrev[2] are: > > - Cherry picked changes from upstream Graal that remove use of JDK internals. > > - The jdk.internal.vm.ci.enabled system property is set to true in arguments.cpp[3] iff EnableJVMCI is true > and this property is checked in all the public methods in jdk.vm.ci.services. > > - The jdk.vm.ci.services package is (once again) only exported to jdk.internal.vm.compiler based on > advice from the jigsaw team: > > "We reviewed the unqualified export jdk.vm.ci.services from jdk.internal.vm.ci module. This brings > jdk.internal.vm.ci to be resolved by default that of course may resolve additional modules that > provides the services that JVMCI uses. In addition. JVMCI is meant to be used (only) when > -XX:+EnableJVMCI is specified but now it?s defined by default. > > An internal module should only have qualified exports as a design principle. The Lab Graal will > have the same module name, jdk.internal.vm.compiler. The advise is to keep it as qualified export > `exports jdk.vm.ci.services to jdk.internal.vm.compiler` and remove all other qualified exports as > we discussed." > > - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. > Unfortunately there needs to be one --add-exports option per qualified export target as combining > them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) > breaks the make process: > > Launcher-jdk.aot.gmk:31: *** missing separator. Stop. > make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. > > The latest webrev has been tested against upstream Graal, the closed AOT tests and jprt. > > -Doug > > [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 > [2] http://cr.openjdk.java.net/~dnsimon/8177845 > [3] http://cr.openjdk.java.net/~dnsimon/8177845.02/hotspot/src/share/vm/runtime/arguments.cpp.udiff.html > [4] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html > >> >>> On 20 Apr 2017, at 20:50, Doug Simon wrote: >>> >>> I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an >>> exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg >>> tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. >>> Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to >>> the version compatible with the JVMCI API changes. >>> >>> In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these >>> tests better isolates them from parts of the runtime they are not aiming to test. >>> >>> org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) >>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) >>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) >>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) >>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) >>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) >>> at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) >>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) >>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.base/java.lang.reflect.Method.invoke(Method.java:563) >>> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >>> at java.base/java.lang.Thread.run(Thread.java:844) >>> Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) >>> ... 26 more >>> >>> -Doug >>> >>>> On 19 Apr 2017, at 23:26, Doug Simon wrote: >>>> >>>> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: >>>> >>>> 1. JVMCIServiceLocator.getProvider(Class) is now protected >>>> 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission >>>> 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK >>>> >>>> -Doug >>>> >>>>> On 19 Apr 2017, at 23:12, Doug Simon wrote: >>>>> >>>>>> >>>>>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>>>>> >>>>>>> >>>>>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>>>>> >>>>>>>> >>>>>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>>>>> >>>>>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>>>>> >>>>>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>>>>> >>>>>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>>>>> >>>>>>>> It's unqualified and no restriction in this change. >>>>>>> >>>>>>> The public methods currently in jdk.vm.ci.services are: >>>>>>> >>>>>>> 1. JVMCIServiceLocator.getProvider(Class) >>>>>>> 2. JVMCIServiceLocator.getProviders(Class) >>>>>>> 3. Services.initializeJVMCI() >>>>>>> 4. Services.getSavedProperties() >>>>>>> 5. Services.exportJVMCITo(Class) >>>>>>> 6. Services.load(Class) >>>>>>> 7. Services.loadSingle(Class, boolean) >>>>>>> >>>>>>> 1 should be made protected. I'll update the webrev with this change. >>>>>> >>>>>> Good. >>>>>> >>>>>>> >>>>>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>>>>> >>>>>> Good. >>>>>> >>>>>>> >>>>>>> 3 is harmless from a security perspective in my opinion. >>>>>> >>>>>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >>>>> >>>>> Vladimir, can you please bring this to the attention of the relevant engineer. >>>>> >>>>>>> >>>>>>> 4 checks for JVMCIPermission. >>>>>> >>>>>> Ok. >>>>>> >>>>>>> >>>>>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>>>>> >>>>>> About this, will this Graal update happen for JDK 9? >>>>> >>>>> Yes. >>>>> >>>>>> It?s awfully late in the cycle... >>>>> >>>>> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >>>>> >>>>> -Doug >>>> >>> >> > From doug.simon at oracle.com Thu Apr 27 19:34:45 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 27 Apr 2017 21:34:45 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> Message-ID: <8B0AC7A0-33E5-4D19-A36A-505807720A4D@oracle.com> > On 27 Apr 2017, at 21:15, Vladimir Kozlov wrote: > > make/CompileJavaModules.gmk Please, adjust comment for jdk.aot: > > ># Don't use Indy strings concatenation to have good JVMCI startup performance. > --- > <# Don't use Indy strings concatenation to have good JAOTC startup performance. > > Should we also add jdk.vm.ci.aarch64 and jdk.vm.ci.sparc exports for AOT? It is not needed for JDK 9 but we will support them in a future and analyze build failures is painful. > > Same in make/launcher/Launcher-jdk.aot.gmk. Good point. I made it the same set of exports as for jdk.internal.vm.compiler (in both locations[1][2]). > I don't see changes in hotspot/make/CompileTools.gmk and hotspot/make/gensrc/Gensrc-jdk.internal.vm.compiler.gmk (which do annotation processor build): > > http://hg.openjdk.java.net/jdk9/dev/hotspot/file/f1cca489e9c6/make/gensrc/Gensrc-jdk.internal.vm.compiler.gmk > > I assume you don't need to change anything there. Right? I presume not as well. These sources are compiled as Java 8 sources where modules are not a concept. > Overall changes looks good to me. Thanks for the review! I'll run the Graal and AOT tests once more and then integrate this change. -Doug [1] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html [2] http://cr.openjdk.java.net/~dnsimon/8177845.02/make/make/CompileJavaModules.gmk.udiff.html > On 4/27/17 7:47 AM, Doug Simon wrote: >> >>> On 21 Apr 2017, at 13:46, Doug Simon wrote: >>> >>> There has been some discussion about whether we want to update Graal in the JDK at this late stage. The main (only?) risk is a regression in the AOT tool. >>> >>> If we don't update Graal from upstream, then the qualified exports from JVMCI to jdk.internal.vm.compiler cannot be removed in JDK 9. Note that in addition to updating Graal to remove the qualified exports, there would also need to be changes in the relevant make files to add --add-exports options when compiling Graal and jaotc as they use the dynamically exported JVMCI packages. >>> >>> I have an updated hotspot patch that adapts Graal to the JVMCI API changes: >>> >>> http://cr.openjdk.java.net/~dnsimon/8177845/hotspot.02/ >>> >>> Note that this patch does not include the changes removing use of JDK internal API from Graal. Cherry picking those upstream Graal changes would be more work than simply doing a complete update from upstream Graal. >>> >>> As I see it, there are 2 options here: >>> >>> 1. Go with the current webrev (including hotspot.02 patch) and live with the qualified exports. >>> 2. Go with the current webrev (including hotspot.02 patch) and create a follow up bug to update Graal from upstream, perform the relevant make file changes and remove the qualified exports. >> >> I made a new webrev[1] that implements option 1.5 ;-) The changes added since the first webrev[2] are: >> >> - Cherry picked changes from upstream Graal that remove use of JDK internals. >> >> - The jdk.internal.vm.ci.enabled system property is set to true in arguments.cpp[3] iff EnableJVMCI is true >> and this property is checked in all the public methods in jdk.vm.ci.services. >> >> - The jdk.vm.ci.services package is (once again) only exported to jdk.internal.vm.compiler based on >> advice from the jigsaw team: >> >> "We reviewed the unqualified export jdk.vm.ci.services from jdk.internal.vm.ci module. This brings >> jdk.internal.vm.ci to be resolved by default that of course may resolve additional modules that >> provides the services that JVMCI uses. In addition. JVMCI is meant to be used (only) when >> -XX:+EnableJVMCI is specified but now it?s defined by default. >> >> An internal module should only have qualified exports as a design principle. The Lab Graal will >> have the same module name, jdk.internal.vm.compiler. The advise is to keep it as qualified export >> `exports jdk.vm.ci.services to jdk.internal.vm.compiler` and remove all other qualified exports as >> we discussed." >> >> - The jaotc launcher now needs to explicitly export JVMCI and jdk.internal.vm.compiler to jdk.aot[4]. >> Unfortunately there needs to be one --add-exports option per qualified export target as combining >> them with a comma (e.g., --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,jdk.aot) >> breaks the make process: >> >> Launcher-jdk.aot.gmk:31: *** missing separator. Stop. >> make/Main.gmk:232: recipe for target 'jdk.aot-launchers' failed. >> >> The latest webrev has been tested against upstream Graal, the closed AOT tests and jprt. >> >> -Doug >> >> [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 >> [2] http://cr.openjdk.java.net/~dnsimon/8177845 >> [3] http://cr.openjdk.java.net/~dnsimon/8177845.02/hotspot/src/share/vm/runtime/arguments.cpp.udiff.html >> [4] http://cr.openjdk.java.net/~dnsimon/8177845.02/jdk/make/launcher/Launcher-jdk.aot.gmk.udiff.html >> >>> >>>> On 20 Apr 2017, at 20:50, Doug Simon wrote: >>>> >>>> I've had to update the webrev again to support selection of a "null" compiler (i.e. one that raises an >>>> exception upon a compilation request) and added -Djvmci.Compiler=null to a large number of JVMCI jtreg >>>> tests to prevent Graal being selected and initialized by the JVMCI compiler auto-selection mechanism. >>>> Initializing Graal will currently fail with errors (see stack trace below) until Graal is updated to >>>> the version compatible with the JVMCI API changes. >>>> >>>> In addition to resolving the compatibility issue, explicitly selecting the "null" compiler for these >>>> tests better isolates them from parts of the runtime they are not aiming to test. >>>> >>>> org.graalvm.compiler.debug.GraalError: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:217) >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:138) >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:95) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:104) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:290) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:65) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:73) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:83) >>>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>> at jdk.internal.vm.ci/jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:82) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfig.config(HotSpotVMConfig.java:41) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.getHolder(HotSpotResolvedJavaMethodImpl.java:92) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedJavaMethodImpl.fromMetaspace(HotSpotResolvedJavaMethodImpl.java:110) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asResolvedJavaMethod(Native Method) >>>> at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVMHelper.asResolvedJavaMethod(CompilerToVMHelper.java:185) >>>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:59) >>>> at compiler.jvmci.common.CTVMUtilities.getResolvedMethod(CTVMUtilities.java:64) >>>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.runSanityCorrectTest(AllocateCompileIdTest.java:125) >>>> at java.base/java.util.ArrayList.forEach(ArrayList.java:1378) >>>> at compiler.jvmci.compilerToVM.AllocateCompileIdTest.main(AllocateCompileIdTest.java:71) >>>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.base/java.lang.reflect.Method.invoke(Method.java:563) >>>> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >>>> at java.base/java.lang.Thread.run(Thread.java:844) >>>> Caused by: java.lang.ClassCastException: java.base/java.util.ImmutableCollections$MapN cannot be cast to java.base/java.util.Properties >>>> at jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.getSavedProperties(HotSpotGraalCompilerFactory.java:215) >>>> ... 26 more >>>> >>>> -Doug >>>> >>>>> On 19 Apr 2017, at 23:26, Doug Simon wrote: >>>>> >>>>> I've updated http://cr.openjdk.java.net/~dnsimon/8177845/hotspot/ with these changes: >>>>> >>>>> 1. JVMCIServiceLocator.getProvider(Class) is now protected >>>>> 2. JVMCIServiceLocator.getProviders(Class) now checks JVMCIPermission >>>>> 3. Rename: jdk.vm.ci.services.internal.JDK9 -> jdk.vm.ci.services.internal.ReflectionAccessJDK >>>>> >>>>> -Doug >>>>> >>>>>> On 19 Apr 2017, at 23:12, Doug Simon wrote: >>>>>> >>>>>>> >>>>>>> On 19 Apr 2017, at 21:40, Christian Thalinger wrote: >>>>>>> >>>>>>>> >>>>>>>> On Apr 19, 2017, at 9:27 AM, Doug Simon wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> On 19 Apr 2017, at 21:04, Mandy Chung wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Apr 19, 2017, at 11:55 AM, Christian Thalinger wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Apr 19, 2017, at 8:38 AM, Mandy Chung wrote: >>>>>>>>>>> >>>>>>>>>>> Since jdk.internal.vm.compiler becomes an upgradeable module, it is not hashed with java.base to allow it to be upgraded and there is no integrity check. Such qualified export will be granted to any module named jdk.internal.vm.compiler at runtime. The goal is for upgradeable modules not to use any internal APIs and eliminate the qualified exports. >>>>>>>>>>> >>>>>>>>>>> The main thing is that jdk.vm.ci.services API would need to be guarded if it?s used by non-Graal modules. >>>>>>>>>> >>>>>>>>>> This all makes sense but where is the restriction that only jdk.internal.vm.compiler can use jdk.vm.ci.services? >>>>>>>>> >>>>>>>>> It's unqualified and no restriction in this change. >>>>>>>> >>>>>>>> The public methods currently in jdk.vm.ci.services are: >>>>>>>> >>>>>>>> 1. JVMCIServiceLocator.getProvider(Class) >>>>>>>> 2. JVMCIServiceLocator.getProviders(Class) >>>>>>>> 3. Services.initializeJVMCI() >>>>>>>> 4. Services.getSavedProperties() >>>>>>>> 5. Services.exportJVMCITo(Class) >>>>>>>> 6. Services.load(Class) >>>>>>>> 7. Services.loadSingle(Class, boolean) >>>>>>>> >>>>>>>> 1 should be made protected. I'll update the webrev with this change. >>>>>>> >>>>>>> Good. >>>>>>> >>>>>>>> >>>>>>>> 2 should check for JVMCIPermission. I'll update the webrev with this change. >>>>>>> >>>>>>> Good. >>>>>>> >>>>>>>> >>>>>>>> 3 is harmless from a security perspective in my opinion. >>>>>>> >>>>>>> Would be good if one of Oracle?s security engineers could take a quick look just to be sure. >>>>>> >>>>>> Vladimir, can you please bring this to the attention of the relevant engineer. >>>>>> >>>>>>>> >>>>>>>> 4 checks for JVMCIPermission. >>>>>>> >>>>>>> Ok. >>>>>>> >>>>>>>> >>>>>>>> 5, 6 and 7 will be removed in a follow bug that updates Graal from upstream (and removes its usage of these methods). >>>>>>> >>>>>>> About this, will this Graal update happen for JDK 9? >>>>>> >>>>>> Yes. >>>>>> >>>>>>> It?s awfully late in the cycle... >>>>>> >>>>>> These are jigsaw related changes and I've been told jigsaw has an FC exception (although I don't exactly know what that is). >>>>>> >>>>>> -Doug >>>>> >>>> >>> >> From mandy.chung at oracle.com Thu Apr 27 21:24:44 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 27 Apr 2017 14:24:44 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> Message-ID: <450F2636-45EB-4551-A1D3-EFC0A34C59C5@oracle.com> > On Apr 27, 2017, at 7:47 AM, Doug Simon wrote: > > [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 I reviewed the top repo and jdk repo change. For hotspot change, I reviewed jdk.vm.ci.services/* and jdk.internal.vm.ci module-info.java. make/make/CompileJavaModules.gmk jdk/make/launcher/Launcher-jdk.aot.gmk Reading the original jdk.internal.vm.ci module-info.java [1], jdk.aot only accesses a small set of packages. e.g. it does not access jdk.vm.ci.aarch64, jdk.vm.ci.code.stack, jdk.vm.ci.common, etc. 115 jdk.internal.vm.compiler \ This should be removed from PLATFORM_MODULES list. $PLATFORM_MODULES already includes $UPGRADEABLE_MODULES. Otherwise, looks good to me. There are jdk tests verifying upgradeable modules and qualified exports. I suggest to run through JPRT -testset core options to catch any test failure. Mandy [1] http://cr.openjdk.java.net/~dnsimon/8177845.02/hotspot/src/jdk.internal.vm.ci/share/classes/module-info.java.sdiff.html From doug.simon at oracle.com Thu Apr 27 21:30:06 2017 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 27 Apr 2017 23:30:06 +0200 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <450F2636-45EB-4551-A1D3-EFC0A34C59C5@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> <450F2636-45EB-4551-A1D3-EFC0A34C59C5@oracle.com> Message-ID: <1512A408-8CEA-40FC-BDFC-1AA7B9A1312E@oracle.com> > On 27 Apr 2017, at 23:24, Mandy Chung wrote: > > >> On Apr 27, 2017, at 7:47 AM, Doug Simon wrote: >> >> [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 > > I reviewed the top repo and jdk repo change. For hotspot change, > I reviewed jdk.vm.ci.services/* and jdk.internal.vm.ci module-info.java. > > make/make/CompileJavaModules.gmk > jdk/make/launcher/Launcher-jdk.aot.gmk > Reading the original jdk.internal.vm.ci module-info.java [1], > jdk.aot only accesses a small set of packages. e.g. it does not > access jdk.vm.ci.aarch64, jdk.vm.ci.code.stack, jdk.vm.ci.common, etc. I know but I agree with Vladimir that there's no harm in exporting everything to jdk.aot to simplify continued development of this tool as support for other platforms is added. As I learnt putting this patch together, missing these exports when they're actually needed leads to very confusing compiler error messages. > 115 jdk.internal.vm.compiler \ > > This should be removed from PLATFORM_MODULES list. $PLATFORM_MODULES > already includes $UPGRADEABLE_MODULES. Will do. > Otherwise, looks good to me. Thanks for the review. > There are jdk tests verifying upgradeable modules and qualified exports. > I suggest to run through JPRT -testset core options to catch any test failure. Will do. -Doug From mandy.chung at oracle.com Thu Apr 27 22:04:41 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 27 Apr 2017 15:04:41 -0700 Subject: RFR: 8177845: Need a mechanism to load Graal In-Reply-To: <1512A408-8CEA-40FC-BDFC-1AA7B9A1312E@oracle.com> References: <9F8C5BD3-F7A6-4C66-B083-BDAD262BCFD7@oracle.com> <8C3D9D03-293B-41E2-BB47-D64086CBB218@twitter.com> <0647633E-32C7-4D4D-822E-14658DE5ECBF@oracle.com> <39D11403-8706-4014-BB68-56D24CA28634@oracle.com> <95E2B6BC-345D-4E66-947C-B6662565A405@oracle.com> <20D7FA70-7D7F-4F9D-B8BE-B5A9F24E618F@oracle.com> <9BF3C1FD-C963-4661-8442-DE06313565AE@oracle.com> <450F2636-45EB-4551-A1D3-EFC0A34C59C5@oracle.com> <1512A408-8CEA-40FC-BDFC-1AA7B9A1312E@oracle.com> Message-ID: <090FBD05-CAD2-424F-B447-AD28870F3F46@oracle.com> > On Apr 27, 2017, at 2:30 PM, Doug Simon wrote: > > >> On 27 Apr 2017, at 23:24, Mandy Chung wrote: >> >> >>> On Apr 27, 2017, at 7:47 AM, Doug Simon wrote: >>> >>> [1] http://cr.openjdk.java.net/~dnsimon/8177845.02 >> >> I reviewed the top repo and jdk repo change. For hotspot change, >> I reviewed jdk.vm.ci.services/* and jdk.internal.vm.ci module-info.java. >> >> make/make/CompileJavaModules.gmk >> jdk/make/launcher/Launcher-jdk.aot.gmk >> Reading the original jdk.internal.vm.ci module-info.java [1], >> jdk.aot only accesses a small set of packages. e.g. it does not >> access jdk.vm.ci.aarch64, jdk.vm.ci.code.stack, jdk.vm.ci.common, etc. > > I know but I agree with Vladimir that there's no harm in exporting everything to jdk.aot to simplify continued development of this tool as support for other platforms is added. As I learnt putting this patch together, missing these exports when they're actually needed leads to very confusing compiler error messages. Indeed the compiler error message is very confusing and ought to be improved (JDK-8179359). I have no issue to export these packages to jdk.aot, as you agree to do. Mandy From david.holmes at oracle.com Fri Apr 28 00:13:13 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Apr 2017 10:13:13 +1000 Subject: RFR 8u backport: 8179084: HotSpot VM fails to start when AggressiveHeap is set Message-ID: <7d25c02c-57d7-29a0-7a40-0e71a5bae406@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8179084 JDK 9 changeset: http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/f1cca489e9c6 webrev: http://cr.openjdk.java.net/~dholmes/8179084/webrev/ The functional fix in arguments.cpp is the same as in 9, just different line numbers. The test had to be adjusted for the 8u test environment and other differences between 8 and 9: - no @modules needed - @requires vm.gc.parallel not supported - test library location and package names are different - PrintFlagsFinal output is different so needed to change pattern to match The @requires was there because the test will only work on a VM which has a default GC that supports parallel GC. The only binaries we have in 8u that would fail this is a JRE/JDK that defaults to the MinimalVM. But the MinimalVM only supports Compact1 and Compact2, and the test uses APIs from Compact3 anyway - so I added the test to the needs_compact3 group in TEST.groups to deal with this limitation. Testing: manual JPRT (I added the test to hotspot_gc for testing purposes) Thanks, David From serguei.spitsyn at oracle.com Fri Apr 28 05:14:06 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 27 Apr 2017 22:14:06 -0700 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase Message-ID: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> Please, review the jdk 10 fix for the test enhancement: https://bugs.openjdk.java.net/browse/JDK-8172970 Webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ Summary: The task was to provide a test coverage for the JVMTI functions allowed during the start phase. It includes both enabling and disabling the can_generate_early_vmstart capability. Testing the JVMTI functions allowed in any case has not been targeted by this fix. Testing: New test is passed. Thanks, Serguei From david.holmes at oracle.com Fri Apr 28 06:11:26 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Apr 2017 16:11:26 +1000 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> Message-ID: Hi Serguei, On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: > Please, review the jdk 10 fix for the test enhancement: > https://bugs.openjdk.java.net/browse/JDK-8172970 > > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ Sorry but I can't quite figure out exactly what this test is doing. What is the overall call structure here? I was expecting to see a difference between things that can be called at early-start and those that can not - or are these all expected to work okay in either case? A few comments: 44 #define TranslateError(err) "JVMTI error" I don't see the point of the above. --- 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { The thread local functions use "long" as the datatype but that will only be 32-bit on 64-bit Windows. I think you need to use intptr_t for complete portability. --- 277 printf(" Filed declaring"); typo: filed -> field --- All your little wrapper functions make the JVMTI call and then call check_jvmti_error - but all that does is record if it passed or failed. If it failed you still continue with the next operation even if it relies on the success of the first one eg: 378 set_thread_local(jvmti, thread, exp_val); 379 act_val = get_thread_local(jvmti, cur_thread); and the sequences in print_method_info: 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); 229 check_jvmti_error(jvmti, "IsMethodNative", err); 230 printf(" Method is native: %d\n", is_native); 231 232 if (is_native == JNI_FALSE) { 233 err = (*jvmti)->GetMaxLocals(jvmti, method, &locals_max); The call at #233 may not be valid because the method actually is native but the IsMethodNative call failed for some reason. Thanks, David ----- > > > Summary: > The task was to provide a test coverage for the JVMTI functions > allowed during the start phase. > It includes both enabling and disabling the can_generate_early_vmstart > capability. > Testing the JVMTI functions allowed in any case has not been targeted > by this fix. > > Testing: > New test is passed. > > > Thanks, > Serguei From kim.barrett at oracle.com Fri Apr 28 06:11:56 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 28 Apr 2017 02:11:56 -0400 Subject: RFR 8u backport: 8179084: HotSpot VM fails to start when AggressiveHeap is set In-Reply-To: <7d25c02c-57d7-29a0-7a40-0e71a5bae406@oracle.com> References: <7d25c02c-57d7-29a0-7a40-0e71a5bae406@oracle.com> Message-ID: > On Apr 27, 2017, at 8:13 PM, David Holmes wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8179084 > JDK 9 changeset: http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/f1cca489e9c6 > > webrev: http://cr.openjdk.java.net/~dholmes/8179084/webrev/ > > The functional fix in arguments.cpp is the same as in 9, just different line numbers. > > The test had to be adjusted for the 8u test environment and other differences between 8 and 9: > - no @modules needed > - @requires vm.gc.parallel not supported > - test library location and package names are different > - PrintFlagsFinal output is different so needed to change pattern to match > > The @requires was there because the test will only work on a VM which has a default GC that supports parallel GC. The only binaries we have in 8u that would fail this is a JRE/JDK that defaults to the MinimalVM. But the MinimalVM only supports Compact1 and Compact2, and the test uses APIs from Compact3 anyway - so I added the test to the needs_compact3 group in TEST.groups to deal with this limitation. > > Testing: manual > JPRT (I added the test to hotspot_gc for testing purposes) > > Thanks, > David Looks good. From david.holmes at oracle.com Fri Apr 28 06:12:28 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Apr 2017 16:12:28 +1000 Subject: RFR 8u backport: 8179084: HotSpot VM fails to start when AggressiveHeap is set In-Reply-To: References: <7d25c02c-57d7-29a0-7a40-0e71a5bae406@oracle.com> Message-ID: <5abc7866-da7d-5e43-f716-ded1ce4e48fe@oracle.com> Thanks Kim! David On 28/04/2017 4:11 PM, Kim Barrett wrote: >> On Apr 27, 2017, at 8:13 PM, David Holmes wrote: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8179084 >> JDK 9 changeset: http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/f1cca489e9c6 >> >> webrev: http://cr.openjdk.java.net/~dholmes/8179084/webrev/ >> >> The functional fix in arguments.cpp is the same as in 9, just different line numbers. >> >> The test had to be adjusted for the 8u test environment and other differences between 8 and 9: >> - no @modules needed >> - @requires vm.gc.parallel not supported >> - test library location and package names are different >> - PrintFlagsFinal output is different so needed to change pattern to match >> >> The @requires was there because the test will only work on a VM which has a default GC that supports parallel GC. The only binaries we have in 8u that would fail this is a JRE/JDK that defaults to the MinimalVM. But the MinimalVM only supports Compact1 and Compact2, and the test uses APIs from Compact3 anyway - so I added the test to the needs_compact3 group in TEST.groups to deal with this limitation. >> >> Testing: manual >> JPRT (I added the test to hotspot_gc for testing purposes) >> >> Thanks, >> David > > Looks good. > From david.holmes at oracle.com Fri Apr 28 06:17:06 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Apr 2017 16:17:06 +1000 Subject: RFR 8u backport: 8179084: HotSpot VM fails to start when AggressiveHeap is set In-Reply-To: <5abc7866-da7d-5e43-f716-ded1ce4e48fe@oracle.com> References: <7d25c02c-57d7-29a0-7a40-0e71a5bae406@oracle.com> <5abc7866-da7d-5e43-f716-ded1ce4e48fe@oracle.com> Message-ID: Still need a 8u Reviewer to okay this too, please. Thanks, David On 28/04/2017 4:12 PM, David Holmes wrote: > Thanks Kim! > > David > > On 28/04/2017 4:11 PM, Kim Barrett wrote: >>> On Apr 27, 2017, at 8:13 PM, David Holmes >>> wrote: >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8179084 >>> JDK 9 changeset: >>> http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/f1cca489e9c6 >>> >>> webrev: http://cr.openjdk.java.net/~dholmes/8179084/webrev/ >>> >>> The functional fix in arguments.cpp is the same as in 9, just >>> different line numbers. >>> >>> The test had to be adjusted for the 8u test environment and other >>> differences between 8 and 9: >>> - no @modules needed >>> - @requires vm.gc.parallel not supported >>> - test library location and package names are different >>> - PrintFlagsFinal output is different so needed to change pattern to >>> match >>> >>> The @requires was there because the test will only work on a VM which >>> has a default GC that supports parallel GC. The only binaries we have >>> in 8u that would fail this is a JRE/JDK that defaults to the >>> MinimalVM. But the MinimalVM only supports Compact1 and Compact2, and >>> the test uses APIs from Compact3 anyway - so I added the test to the >>> needs_compact3 group in TEST.groups to deal with this limitation. >>> >>> Testing: manual >>> JPRT (I added the test to hotspot_gc for testing purposes) >>> >>> Thanks, >>> David >> >> Looks good. >> From stefan.karlsson at oracle.com Fri Apr 28 07:11:37 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 28 Apr 2017 09:11:37 +0200 Subject: RFR 8u backport: 8179084: HotSpot VM fails to start when AggressiveHeap is set In-Reply-To: References: <7d25c02c-57d7-29a0-7a40-0e71a5bae406@oracle.com> <5abc7866-da7d-5e43-f716-ded1ce4e48fe@oracle.com> Message-ID: <6060d294-9a5a-b328-53e5-d2bc7901d981@oracle.com> On 2017-04-28 08:17, David Holmes wrote: > Still need a 8u Reviewer to okay this too, please. Looks good. StefanK > > Thanks, > David > > On 28/04/2017 4:12 PM, David Holmes wrote: >> Thanks Kim! >> >> David >> >> On 28/04/2017 4:11 PM, Kim Barrett wrote: >>>> On Apr 27, 2017, at 8:13 PM, David Holmes >>>> wrote: >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8179084 >>>> JDK 9 changeset: >>>> http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/f1cca489e9c6 >>>> >>>> webrev: http://cr.openjdk.java.net/~dholmes/8179084/webrev/ >>>> >>>> The functional fix in arguments.cpp is the same as in 9, just >>>> different line numbers. >>>> >>>> The test had to be adjusted for the 8u test environment and other >>>> differences between 8 and 9: >>>> - no @modules needed >>>> - @requires vm.gc.parallel not supported >>>> - test library location and package names are different >>>> - PrintFlagsFinal output is different so needed to change pattern to >>>> match >>>> >>>> The @requires was there because the test will only work on a VM which >>>> has a default GC that supports parallel GC. The only binaries we have >>>> in 8u that would fail this is a JRE/JDK that defaults to the >>>> MinimalVM. But the MinimalVM only supports Compact1 and Compact2, and >>>> the test uses APIs from Compact3 anyway - so I added the test to the >>>> needs_compact3 group in TEST.groups to deal with this limitation. >>>> >>>> Testing: manual >>>> JPRT (I added the test to hotspot_gc for testing purposes) >>>> >>>> Thanks, >>>> David >>> >>> Looks good. >>> From david.holmes at oracle.com Fri Apr 28 07:19:01 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Apr 2017 17:19:01 +1000 Subject: RFR 8u backport: 8179084: HotSpot VM fails to start when AggressiveHeap is set In-Reply-To: <6060d294-9a5a-b328-53e5-d2bc7901d981@oracle.com> References: <7d25c02c-57d7-29a0-7a40-0e71a5bae406@oracle.com> <5abc7866-da7d-5e43-f716-ded1ce4e48fe@oracle.com> <6060d294-9a5a-b328-53e5-d2bc7901d981@oracle.com> Message-ID: <5ad62aa0-89aa-e1b5-1512-ba4d164eec8f@oracle.com> Many thanks Stefan! David On 28/04/2017 5:11 PM, Stefan Karlsson wrote: > On 2017-04-28 08:17, David Holmes wrote: >> Still need a 8u Reviewer to okay this too, please. > > Looks good. > > StefanK > >> >> Thanks, >> David >> >> On 28/04/2017 4:12 PM, David Holmes wrote: >>> Thanks Kim! >>> >>> David >>> >>> On 28/04/2017 4:11 PM, Kim Barrett wrote: >>>>> On Apr 27, 2017, at 8:13 PM, David Holmes >>>>> wrote: >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8179084 >>>>> JDK 9 changeset: >>>>> http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/f1cca489e9c6 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8179084/webrev/ >>>>> >>>>> The functional fix in arguments.cpp is the same as in 9, just >>>>> different line numbers. >>>>> >>>>> The test had to be adjusted for the 8u test environment and other >>>>> differences between 8 and 9: >>>>> - no @modules needed >>>>> - @requires vm.gc.parallel not supported >>>>> - test library location and package names are different >>>>> - PrintFlagsFinal output is different so needed to change pattern to >>>>> match >>>>> >>>>> The @requires was there because the test will only work on a VM which >>>>> has a default GC that supports parallel GC. The only binaries we have >>>>> in 8u that would fail this is a JRE/JDK that defaults to the >>>>> MinimalVM. But the MinimalVM only supports Compact1 and Compact2, and >>>>> the test uses APIs from Compact3 anyway - so I added the test to the >>>>> needs_compact3 group in TEST.groups to deal with this limitation. >>>>> >>>>> Testing: manual >>>>> JPRT (I added the test to hotspot_gc for testing purposes) >>>>> >>>>> Thanks, >>>>> David >>>> >>>> Looks good. >>>> From serguei.spitsyn at oracle.com Fri Apr 28 07:47:54 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 28 Apr 2017 00:47:54 -0700 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> Message-ID: Hi David, Thank you for looking at the test! On 4/27/17 23:11, David Holmes wrote: > Hi Serguei, > > On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the jdk 10 fix for the test enhancement: >> https://bugs.openjdk.java.net/browse/JDK-8172970 >> >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >> > > Sorry but I can't quite figure out exactly what this test is doing. > What is the overall call structure here? This is to make sure the functions allowed in the start and live phases work Ok. As the list of functions is pretty big the test does sanity checks that the functions do not crash nor return errors. > I was expecting to see a difference between things that can be called > at early-start and those that can not - or are these all expected to > work okay in either case? All these functions are expected to work okay in both cases. Of course, the main concern is the early start. But we have never had such coverage in the past so that the normal start phase needs to be covered too. > > A few comments: > > 44 #define TranslateError(err) "JVMTI error" > > I don't see the point of the above. Good catch - removed. It is a left over from another test that I used as initial template. > --- > > 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { > > The thread local functions use "long" as the datatype but that will > only be 32-bit on 64-bit Windows. I think you need to use intptr_t for > complete portability. The type long has the same format as the type void* which has to be portable even on win-32. But maybe I'm missing something. Anyway, I've replaced it with the intptr_t. > > --- > > 277 printf(" Filed declaring"); > > typo: filed -> field Good catch - fixed. > > --- > > All your little wrapper functions make the JVMTI call and then call > check_jvmti_error - but all that does is record if it passed or > failed. If it failed you still continue with the next operation even > if it relies on the success of the first one eg: > > 378 set_thread_local(jvmti, thread, exp_val); > 379 act_val = get_thread_local(jvmti, cur_thread); > > and the sequences in print_method_info: > > 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); > 229 check_jvmti_error(jvmti, "IsMethodNative", err); > 230 printf(" Method is native: %d\n", is_native); > 231 > 232 if (is_native == JNI_FALSE) { > 233 err = (*jvmti)->GetMaxLocals(jvmti, method, &locals_max); > > The call at #233 may not be valid because the method actually is > native but the IsMethodNative call failed for some reason. > It is intentional. I have done it as a last cleanup. The point is to simplify code by skipping all the extra checks if it does not lead to any fatal errors. The most important in such a case is that the static variable result is set to FAILED. It will cause the test to fail. Then there is no point to analyze the printed results if a JVMTI error reported before. If you insist, I will add back all the extra check to make sure all printed output is valid. Thanks, Serguei > Thanks, > David > ----- > >> >> >> Summary: >> The task was to provide a test coverage for the JVMTI functions >> allowed during the start phase. >> It includes both enabling and disabling the can_generate_early_vmstart >> capability. >> Testing the JVMTI functions allowed in any case has not been targeted >> by this fix. >> >> Testing: >> New test is passed. >> >> >> Thanks, >> Serguei From serguei.spitsyn at oracle.com Fri Apr 28 08:07:56 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 28 Apr 2017 01:07:56 -0700 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> Message-ID: <5ea66d4b-f1bb-1729-ba51-5bdcd5b3c470@oracle.com> The updated webrev is: http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.2/ I've re-arranged a little bit code in the ClassPrepare callback and the function test_class_functions(). Thanks, Serguei On 4/28/17 00:47, serguei.spitsyn at oracle.com wrote: > Hi David, > > Thank you for looking at the test! > > > On 4/27/17 23:11, David Holmes wrote: >> Hi Serguei, >> >> On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the jdk 10 fix for the test enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8172970 >>> >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >>> >> >> Sorry but I can't quite figure out exactly what this test is doing. >> What is the overall call structure here? > > This is to make sure the functions allowed in the start and live > phases work Ok. > As the list of functions is pretty big the test does sanity checks > that the functions do not crash nor return errors. > > >> I was expecting to see a difference between things that can be called >> at early-start and those that can not - or are these all expected to >> work okay in either case? > > All these functions are expected to work okay in both cases. > Of course, the main concern is the early start. > But we have never had such coverage in the past so that the normal > start phase needs to be covered too. > > >> >> A few comments: >> >> 44 #define TranslateError(err) "JVMTI error" >> >> I don't see the point of the above. > > Good catch - removed. > It is a left over from another test that I used as initial template. > > >> --- >> >> 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { >> >> The thread local functions use "long" as the datatype but that will >> only be 32-bit on 64-bit Windows. I think you need to use intptr_t >> for complete portability. > > The type long has the same format as the type void* which has to be > portable even on win-32. > But maybe I'm missing something. > Anyway, I've replaced it with the intptr_t. > > >> >> --- >> >> 277 printf(" Filed declaring"); >> >> typo: filed -> field > > > Good catch - fixed. > >> >> --- >> >> All your little wrapper functions make the JVMTI call and then call >> check_jvmti_error - but all that does is record if it passed or >> failed. If it failed you still continue with the next operation even >> if it relies on the success of the first one eg: >> >> 378 set_thread_local(jvmti, thread, exp_val); >> 379 act_val = get_thread_local(jvmti, cur_thread); >> >> and the sequences in print_method_info: >> >> 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); >> 229 check_jvmti_error(jvmti, "IsMethodNative", err); >> 230 printf(" Method is native: %d\n", is_native); >> 231 >> 232 if (is_native == JNI_FALSE) { >> 233 err = (*jvmti)->GetMaxLocals(jvmti, method, &locals_max); >> >> The call at #233 may not be valid because the method actually is >> native but the IsMethodNative call failed for some reason. >> > > It is intentional. I have done it as a last cleanup. > The point is to simplify code by skipping all the extra checks if it > does not lead to any fatal errors. > The most important in such a case is that the static variable result > is set to FAILED. > It will cause the test to fail. > Then there is no point to analyze the printed results if a JVMTI error > reported before. > > If you insist, I will add back all the extra check to make sure all > printed output is valid. > > > Thanks, > Serguei > >> Thanks, >> David >> ----- >> >>> >>> >>> Summary: >>> The task was to provide a test coverage for the JVMTI functions >>> allowed during the start phase. >>> It includes both enabling and disabling the >>> can_generate_early_vmstart >>> capability. >>> Testing the JVMTI functions allowed in any case has not been targeted >>> by this fix. >>> >>> Testing: >>> New test is passed. >>> >>> >>> Thanks, >>> Serguei > From serguei.spitsyn at oracle.com Fri Apr 28 08:54:53 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 28 Apr 2017 01:54:53 -0700 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> Message-ID: <8cbc7547-7e0f-e86e-baea-37a59788ce68@oracle.com> On 4/28/17 00:47, serguei.spitsyn at oracle.com wrote: > Hi David, > > Thank you for looking at the test! > > > On 4/27/17 23:11, David Holmes wrote: >> Hi Serguei, >> >> On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the jdk 10 fix for the test enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8172970 >>> >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >>> >> >> Sorry but I can't quite figure out exactly what this test is doing. >> What is the overall call structure here? > > This is to make sure the functions allowed in the start and live > phases work Ok. To be more clear: "during the start or live phase". Thanks, Serguei > As the list of functions is pretty big the test does sanity checks > that the functions do not crash nor return errors. > From ioi.lam at oracle.com Fri Apr 28 11:30:57 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 28 Apr 2017 04:30:57 -0700 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> <58FDECFE.5060105@oracle.com> Message-ID: <590327F1.7070200@oracle.com> On 4/25/17 8:06 AM, Volker Simonis wrote: > On Mon, Apr 24, 2017 at 2:18 PM, Ioi Lam wrote: >> Hi Volker, >> >> >> On 4/21/17 12:02 AM, Volker Simonis wrote: >>> Hi Ioi, >>> >>> thanks once again for considering our ports! Please find the required >>> additions for ppc64/s390x in the following webrew (which is based upon >>> your latest v03 patch): >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ >> Thanks for the patch. I will integrate it and post an updated webrev. >>> @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly >>> code. I did some tests and I think it should be correct, but maybe you >>> still find some improvements :) >>> >>> Besides that, I have some general questions/comments regarding your >>> change: >>> >>> 1. In constantPool.hpp, why don't you declare the '_name_index' and >>> '_resolved_klass_index' fields with type 'jushort'? As far as I can >>> see, they can only hold 16-bit values anyway. It would also save you >>> some space and several asserts (e.g. in unresolved_klass_at_put(): >>> >>> >>> 274 assert((name_index & 0xffff0000) == 0, "must be"); >>> 275 assert((resolved_klass_index & 0xffff0000) == 0, "must be"); >> >> I think the HotSpot convention is to use ints as parameter and return types, >> for values that are actually 16-bits or less, like here in constantPool.hpp: >> >> void field_at_put(int which, int class_index, int name_and_type_index) { >> tag_at_put(which, JVM_CONSTANT_Fieldref); >> *int_at_addr(which) = ((jint) name_and_type_index<<16) | class_index; >> } >> >> I am not sure what the reasons are. It could be that the parameters usually >> need to be computed arithmetically, and it's much easier for the caller of >> the method to use ints -- otherwise you will get lots of compiler warnings >> which would force you to use lots of casting, resulting in code that's hard >> to read and probably incorrect. >> > OK, but you could still use shorts in the the object to save space, > although I'm not sure how much that will save in total. But if nobody > else cares, I'm fine with the current version. The CPKlassSlot objects are stored only on the stack, so the savings is not worth the trouble of adding extract type casts. Inside the ConstantPool itself, the name_index and resolved_klass_index are stored as a pair of 16-bit values. >>> 2. What do you mean by: >>> >>> 106 // ... will be changed to support compressed pointers >>> 107 Array* _resolved_klasses; >> >> Sorry the comment isn't very clear. How about this? >> >> 106 // Consider using an array of compressed klass pointers to >> // save space on 64-bit platforms. >> 107 Array* _resolved_klasses; >> > Sorry I still didn't get it? Do you mean you want to use array of > "narrowKlass" (i.e. unsigned int)? But using compressed class pointers > is a runtime decision while this is a compile time decision. I haven't figured out how to do it yet :-) Most likely, it will be something like: union { Array* X; Array* Y; } _resolved_klasses; and you need to decide at run time whether to use X or Y. - Ioi >>> 3. Why don't we need the call to "release_tag_at_put()" in >>> "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int >>> class_index, Klass* k)" is used from >>> "ClassFileParser::fill_instance_klass() and before your change that >>> function used the previous version of "klass_at_put(int class_index, >>> Klass* k)" which did call "release_tag_at_put()". >> >> Good catch. I'll add the following, because the class is now resolved. >> >> release_tag_at_put(class_index, JVM_CONSTANT_UnresolvedClass); >>> 4. In ConstantPool::copy_entry_to() you've changed the behavior for >>> tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, >>> JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was >>> copied to the new constant pool if one existed but now you always only >>> copy a class_index to the new constant pool (even if a resolved klass >>> existed). Is that OK? E.g. won't this lead to a new resolving for the >>> new constant pool and will this have performance impacts or other side >>> effects? >> I think Coleen has answered this in a separate mail :-) >> >> Thanks >> - Ioi >> >>> Thanks again for doing this nice change and best regards, >>> Volker >>> >>> >>> On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >>>> Hi Lois, >>>> >>>> I have updated the patch to include your comments, and fixes the handling >>>> of >>>> anonymous classes. I also added some more comments regarding the >>>> _temp_resolved_klass_index: >>>> >>>> (delta from last webrev) >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >>>> >>>> (full webrev) >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 4/15/17 2:31 AM, Lois Foltan wrote: >>>>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>>>> >>>>>> >>>>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>>>> HI Lois, >>>>>>> >>>>>>> Thanks for the review. Please see my comments in-line. >>>>>>> >>>>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> Looks really good. A couple of comments: >>>>>>>> >>>>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>>>> surrounding anonymous classes. Coleen and I discussed earlier today >>>>>>>> and I >>>>>>>> came away from that discussion with the idea that the only classes >>>>>>>> being >>>>>>>> patched currently are anonymous classes. >>>>>>> Line 5676 ... >>>>>>> >>>>>>> 5676 if (is_anonymous()) { >>>>>>> 5677 _max_num_patched_klasses ++; // for patching the class >>>>>>> index >>>>>>> 5678 } >>>>>>> >>>>>>> corresponds to >>>>>>> >>>>>>> 5361 ik->set_name(_class_name); >>>>>>> 5362 >>>>>>> 5363 if (is_anonymous()) { >>>>>>> 5364 // I am well known to myself >>>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>>> ik->name()); // eagerly resolve >>>>>>> 5366 } >>>>>>> >>>>>>> Even though the class is "anonymous", it actually has a name. >>>>>>> ik->name() >>>>>>> probably is part of the constant pool, but I am not 100% sure. Also, I >>>>>>> would >>>>>>> need to search the constant pool to find the index for ik->name(). So >>>>>>> I just >>>>>>> got lazy here and use the same logic in ConstantPool::patch_class() to >>>>>>> append ik->name() to the end of the constant pool. >>>>>>> >>>>>>> "Anonymous" actually means "the class cannot be looked up by name in >>>>>>> the >>>>>>> SystemDictionary". I think we need a better terminology :-) >>>>>>> >>>>>> I finally realized why we need the "eagerly resolve" on line 5365. I'll >>>>>> modify the comments to the following: >>>>>> >>>>>> // _this_class_index is a CONSTANT_Class entry that refers to this >>>>>> // anonymous class itself. If this class needs to refer to its own >>>>>> methods or >>>>>> // fields, it would use a CONSTANT_MethodRef, etc, which would >>>>>> reference >>>>>> // _this_class_index. However, because this class is anonymous >>>>>> (it's >>>>>> // not stored in SystemDictionary), _this_class_index cannot be >>>>>> resolved >>>>>> // with ConstantPool::klass_at_impl, which does a SystemDictionary >>>>>> lookup. >>>>>> // Therefore, we must eagerly resolve _this_class_index now. >>>>>> >>>>>> So, Lois is right. Line 5676 is not necessary. I will revise the code >>>>>> to >>>>>> do the "eager resolution" without using ClassFileParser::patch_class. >>>>>> I'll >>>>>> post the updated code later. >>>>> >>>>> Thanks Ioi for studying this and explaining! Look forward to seeing the >>>>> updated webrev. >>>>> Lois >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>>>> So a bit confused as why the check on line #5676 and a check for a >>>>>>>> java/lang/Class on line #5684. >>>>>>> 5683 Handle patch = cp_patch_at(i); >>>>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>>>> java_lang_Class::is_instance(patch())) { >>>>>>> 5685 // We need to append the names of the patched classes >>>>>>> to >>>>>>> the end of the constant pool, >>>>>>> 5686 // because a patched class may have a Utf8 name that's >>>>>>> not already included in the >>>>>>> 5687 // original constant pool. >>>>>>> 5688 // >>>>>>> 5689 // Note that a String in cp_patch_at(i) may be used to >>>>>>> patch a Utf8, a String, or a Class. >>>>>>> 5690 // At this point, we don't know the tag for index i >>>>>>> yet, >>>>>>> because we haven't parsed the >>>>>>> 5691 // constant pool. So we can only assume the worst -- >>>>>>> every String is used to patch a Class. >>>>>>> 5692 _max_num_patched_klasses ++; >>>>>>> >>>>>>> Line 5684 checks for all objects in the cp_patch array. Later, when >>>>>>> ClassFileParser::patch_constant_pool() is called, any objects that are >>>>>>> either Class or String could be treated as a Klass: >>>>>>> >>>>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>>>> 725 int index, >>>>>>> 726 Handle patch, >>>>>>> 727 TRAPS) { >>>>>>> ... >>>>>>> 732 switch (cp->tag_at(index).value()) { >>>>>>> 733 >>>>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>>>> 735 // Patching a class means pre-resolving it. >>>>>>> 736 // The name in the constant pool is ignored. >>>>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>>>> 739 "Illegal class patch at %d in class >>>>>>> file >>>>>>> %s", >>>>>>> 740 index, CHECK); >>>>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>>>> 742 patch_class(cp, index, k, k->name()); >>>>>>> 743 } else { >>>>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>>>> 745 "Illegal class patch at %d in class >>>>>>> file >>>>>>> %s", >>>>>>> 746 index, CHECK); >>>>>>> 747 Symbol* const name = >>>>>>> java_lang_String::as_symbol(patch(), >>>>>>> CHECK); >>>>>>> 748 patch_class(cp, index, NULL, name); >>>>>>> 749 } >>>>>>> 750 break; >>>>>>> 751 } >>>>>>> >>>>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>>>> >>>>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>>>> ik->name() is in the cp_patch array. >>>>>>> >>>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>>> ik->name()); // eagerly resolve >>>>>>> >>>>>>>> Also why not do this calculation in the rewriter or is that too >>>>>>>> late? >>>>>>>> >>>>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool and >>>>>>> the >>>>>>> associated tags array is allocated (both of which are fixed size, and >>>>>>> cannot >>>>>>> be expanded), which is way before the rewriter is run. At this point, >>>>>>> we >>>>>>> don't know what cp->tag_at(index) is (line #732), so the code needs to >>>>>>> make >>>>>>> a worst-case estimate on how long the CP/tags should be. >>>>>>> >>>>>>>> * line #5677, 5692 - a nit but I think the convention is to not have >>>>>>>> a >>>>>>>> space after the variable name and between the post increment >>>>>>>> operator. >>>>>>>> >>>>>>> Fixed. >>>>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>>>> I understand the concept behind _invalid_resolved_klass_index, but it >>>>>>>> really is not so much invalid as temporary for class redefinition >>>>>>>> purposes, >>>>>>>> as you explain in ConstantPool::allocate_resolved_klasses. Please >>>>>>>> consider >>>>>>>> renaming to _temp_unresolved_klass_index. And whether you choose to >>>>>>>> rename >>>>>>>> the field or not, please add a one line comment ahead of >>>>>>>> ConstantPool::temp_unresolved_klass_at_put that only class >>>>>>>> redefinition uses >>>>>>>> this currently. >>>>>>>> >>>>>>> Good idea. Will do. >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>>> Great change, thanks! >>>>>>>> Lois >>>>>>>> >>>>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>>>> Hi Coleen, >>>>>>>>> >>>>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>>>> >>>>>>>>> In addition to your requests, I made these changes: >>>>>>>>> >>>>>>>>> [1] To consolidate the multiple extract_high/low code, I've added >>>>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>>>> >>>>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>>>> int name_index = kslot.name_index(); >>>>>>>>> >>>>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to pick >>>>>>>>> this >>>>>>>>> function over MetaspaceObj::is_shared(). >>>>>>>>> >>>>>>>>> [3] Massaged the CDS region size set-up code a little to pass >>>>>>>>> internal >>>>>>>>> tests, because RO/RW ratio has changed. I didn't spend too much time >>>>>>>>> picking >>>>>>>>> the "right" sizes, as this code will be obsoleted soon with >>>>>>>>> JDK-8072061 >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>>>> >>>>>>>>>> This looks really good! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>>>> >>>>>>>>>> + // Add one extra element to tags for storing >>>>>>>>>> ConstantPool::flags(). >>>>>>>>>> + Array* tags = >>>>>>>>>> MetadataFactory::new_writeable_array(loader_data, length+1, 0, >>>>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, >>>>>>>>>> "invariant"); // >>>>>>>>>> tags->at(_length) is flags() >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think this is left over, since _flags didn't get moved after all. >>>>>>>>>> >>>>>>>>>> + Klass** adr = >>>>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>>>> + // The interpreter assumes when the tag is stored, the klass is >>>>>>>>>> resolved >>>>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we need >>>>>>>>>> + // hardware store ordering here. >>>>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>>>> + return k; >>>>>>>>>> >>>>>>>>>> The comment still refers to the switch between Symbol* and Klass* >>>>>>>>>> in >>>>>>>>>> the constant pool. The entry in the Klass array should be NULL. >>>>>>>>>> >>>>>>>>>> + int name_index = >>>>>>>>>> extract_high_short_from_int(*int_at_addr(which)); >>>>>>>>>> >>>>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp header >>>>>>>>>> file >>>>>>>>>> (so it's inlined) and have all the places where you get name_index >>>>>>>>>> use this >>>>>>>>>> function? So we don't have to know in multiple places that >>>>>>>>>> extract_high_short_from_int() is where the name index is. And in >>>>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a comment about >>>>>>>>>> what the >>>>>>>>>> new format of the constant pool entry is {name_index, >>>>>>>>>> resolved_klass_index}. >>>>>>>>>> I'm happy to see this work nearing completion! The "constant" pool >>>>>>>>>> should >>>>>>>>>> be constant! thanks, Coleen >>>>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>>>> Hi,please review the following change >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds the >>>>>>>>>>> Klass from >>>>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the >>>>>>>>>>> ConstantPool >>>>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>>>> ConstantPool::klass_at(i) finds the at >>>>>>>>>>> this->_resolved_klasses->at(i) + >>>>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is modified. >>>>>>>>>>> In >>>>>>>>>>> addition: + I moved _resolved_references and _reference_map >>>>>>>>>>> from >>>>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no >>>>>>>>>>> longer >>>>>>>>>>> modified for shared ConstantPools. As a result, none of the >>>>>>>>>>> fields in >>>>>>>>>>> shared ConstantPools are modified at run time, so we can move them >>>>>>>>>>> into the >>>>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark results >>>>>>>>>>> show no >>>>>>>>>>> performance regression, despite the extra level of indirection, >>>>>>>>>>> which has a >>>>>>>>>>> negligible overhead for the interpreter. - RBT testing for tier2 >>>>>>>>>>> and >>>>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support >>>>>>>>>>> ports. For the >>>>>>>>>>> aarch64, ppc and s390 ports, I have added some code without >>>>>>>>>>> testing (it's >>>>>>>>>>> probably incomplete) - Port owners, please check if my patch >>>>>>>>>>> work for you, >>>>>>>>>>> and I can incorporate your changes in my push. Alternatively, you >>>>>>>>>>> can wait >>>>>>>>>>> for my push and provide fixes (if necessary) in a new changeset, >>>>>>>>>>> and I will >>>>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>>>> From david.holmes at oracle.com Fri Apr 28 11:42:18 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Apr 2017 21:42:18 +1000 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: <5ea66d4b-f1bb-1729-ba51-5bdcd5b3c470@oracle.com> References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> <5ea66d4b-f1bb-1729-ba51-5bdcd5b3c470@oracle.com> Message-ID: Hi Serguei, On 28/04/2017 6:07 PM, serguei.spitsyn at oracle.com wrote: > The updated webrev is: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.2/ Thanks for the updates (the issue with long is that on win64 it is only 32-bit while void* is 64-bit). I prefer to see fast-fail rather than potentially triggering cascading failures (check_jvmti_error could even call exit() I think). But let's see what others think - it's only a preference not a necessity. Thanks, David > > I've re-arranged a little bit code in the ClassPrepare callback and the > function test_class_functions(). > > Thanks, > Serguei > > > On 4/28/17 00:47, serguei.spitsyn at oracle.com wrote: >> Hi David, >> >> Thank you for looking at the test! >> >> >> On 4/27/17 23:11, David Holmes wrote: >>> Hi Serguei, >>> >>> On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the jdk 10 fix for the test enhancement: >>>> https://bugs.openjdk.java.net/browse/JDK-8172970 >>>> >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >>>> >>> >>> Sorry but I can't quite figure out exactly what this test is doing. >>> What is the overall call structure here? >> >> This is to make sure the functions allowed in the start and live >> phases work Ok. >> As the list of functions is pretty big the test does sanity checks >> that the functions do not crash nor return errors. >> >> >>> I was expecting to see a difference between things that can be called >>> at early-start and those that can not - or are these all expected to >>> work okay in either case? >> >> All these functions are expected to work okay in both cases. >> Of course, the main concern is the early start. >> But we have never had such coverage in the past so that the normal >> start phase needs to be covered too. >> >> >>> >>> A few comments: >>> >>> 44 #define TranslateError(err) "JVMTI error" >>> >>> I don't see the point of the above. >> >> Good catch - removed. >> It is a left over from another test that I used as initial template. >> >> >>> --- >>> >>> 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { >>> >>> The thread local functions use "long" as the datatype but that will >>> only be 32-bit on 64-bit Windows. I think you need to use intptr_t >>> for complete portability. >> >> The type long has the same format as the type void* which has to be >> portable even on win-32. >> But maybe I'm missing something. >> Anyway, I've replaced it with the intptr_t. >> >> >>> >>> --- >>> >>> 277 printf(" Filed declaring"); >>> >>> typo: filed -> field >> >> >> Good catch - fixed. >> >>> >>> --- >>> >>> All your little wrapper functions make the JVMTI call and then call >>> check_jvmti_error - but all that does is record if it passed or >>> failed. If it failed you still continue with the next operation even >>> if it relies on the success of the first one eg: >>> >>> 378 set_thread_local(jvmti, thread, exp_val); >>> 379 act_val = get_thread_local(jvmti, cur_thread); >>> >>> and the sequences in print_method_info: >>> >>> 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); >>> 229 check_jvmti_error(jvmti, "IsMethodNative", err); >>> 230 printf(" Method is native: %d\n", is_native); >>> 231 >>> 232 if (is_native == JNI_FALSE) { >>> 233 err = (*jvmti)->GetMaxLocals(jvmti, method, &locals_max); >>> >>> The call at #233 may not be valid because the method actually is >>> native but the IsMethodNative call failed for some reason. >>> >> >> It is intentional. I have done it as a last cleanup. >> The point is to simplify code by skipping all the extra checks if it >> does not lead to any fatal errors. >> The most important in such a case is that the static variable result >> is set to FAILED. >> It will cause the test to fail. >> Then there is no point to analyze the printed results if a JVMTI error >> reported before. >> >> If you insist, I will add back all the extra check to make sure all >> printed output is valid. >> >> >> Thanks, >> Serguei >> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> >>>> Summary: >>>> The task was to provide a test coverage for the JVMTI functions >>>> allowed during the start phase. >>>> It includes both enabling and disabling the >>>> can_generate_early_vmstart >>>> capability. >>>> Testing the JVMTI functions allowed in any case has not been targeted >>>> by this fix. >>>> >>>> Testing: >>>> New test is passed. >>>> >>>> >>>> Thanks, >>>> Serguei >> > From patric.hedlin at oracle.com Fri Apr 28 13:48:49 2017 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Fri, 28 Apr 2017 15:48:49 +0200 Subject: JDK10/RFR(L): 8172231: SPARC ISA/CPU feature detection is broken/insufficient (on Solaris). Message-ID: <6801aebd-eb60-6711-2859-0096818007eb@oracle.com> Dear all, I would like to ask for help to review the following change/update: Issue: https://bugs.openjdk.java.net/browse/JDK-8172231 Webrev: http://cr.openjdk.java.net/~neliasso/8172231/ 8172231: SPARC ISA/CPU feature detection is broken/insufficient (on Solaris). Updating SPARC feature/capability detection (incorporating changes from Martin Walsh). More complete set of features as provided by 'getisax(2)' interface, propagated via JVMCI. More robust hardware probing for additional features (up to Core S4). Removing support for old, pre Niagara, hardware. Removing support for old, pre 11.1, Solaris. Changed behaviour: Changing SPARC setup for AllocatePrefetchLines and AllocateInstancePrefetchLines such that they will (still) be doubled when cache-line size is small (32 bytes), but more moderately increased on new/contemporary hardware (inc >= 50%). Changing to default instruction fetch alignment based on derived caps. instead of relying on default/configuration values. The above changes also subsumes: 8035146: assert(is_T_family(features) == is_niagara(features), "Niagara should be T series") is incorrect 8054979: Remove unnecessary defines in SPARC's VM_Version::platform_features Rationale: Current hardware detection on Solaris/SPARC is not up to date with the "latest" (here, meaning commercially available server solutions, i.e. T7/M7). To facilitate improved use of the new hardware features provided (by Core S3&S4) these capabilities need to be recognised by the JVM. NOTE: This update is limited to Core S3&S4, i.e. not including Core S5. Proper Core S5 support will be added when regular testing and benchmarking resources are available, i.e. regular testing need to include M8 hardware. Caveat: This update will introduce some redundancies into the code base, features and definitions currently not used, as well as a (small) number of FIXMEs, addressed by subsequent bug or feature updates/patches. Fujitsu HW is treated very conservatively. Testing: Mostly tested on JDK9 (RBT/hotspot/comp). Only local testing on JDK10 (jtreg/hotspot). Benchmarking: Benchmark reports from a limited set of runs can be found at: http://aurora.se.oracle.com/performance/reporting/report/patric.hedlin.TvM.jbb05 http://aurora.se.oracle.com/performance/reporting/report/patric.hedlin.TvM.jvm08 http://aurora.se.oracle.com/performance/reporting/report/patric.hedlin.TvM.octane.plus (Limited availability of M7 hardware prevents complete suites/runs.) Best regards, Patric From serguei.spitsyn at oracle.com Fri Apr 28 17:34:15 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 28 Apr 2017 10:34:15 -0700 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> <5ea66d4b-f1bb-1729-ba51-5bdcd5b3c470@oracle.com> Message-ID: <1e443485-63d6-4473-8809-76f47a380436@oracle.com> Hi David, On 4/28/17 04:42, David Holmes wrote: > Hi Serguei, > > On 28/04/2017 6:07 PM, serguei.spitsyn at oracle.com wrote: >> The updated webrev is: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.2/ >> > > Thanks for the updates (the issue with long is that on win64 it is > only 32-bit while void* is 64-bit). Ok, thanks. Than you are right, using long on win64 is not compatible. > > I prefer to see fast-fail rather than potentially triggering cascading > failures (check_jvmti_error could even call exit() I think). But let's > see what others think - it's only a preference not a necessity. Ok, I'll consider call exit() as it would keep it simple. Thanks, Serguei > > Thanks, > David > >> >> I've re-arranged a little bit code in the ClassPrepare callback and the >> function test_class_functions(). >> >> Thanks, >> Serguei >> >> >> On 4/28/17 00:47, serguei.spitsyn at oracle.com wrote: >>> Hi David, >>> >>> Thank you for looking at the test! >>> >>> >>> On 4/27/17 23:11, David Holmes wrote: >>>> Hi Serguei, >>>> >>>> On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the jdk 10 fix for the test enhancement: >>>>> https://bugs.openjdk.java.net/browse/JDK-8172970 >>>>> >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >>>>> >>>>> >>>> >>>> Sorry but I can't quite figure out exactly what this test is doing. >>>> What is the overall call structure here? >>> >>> This is to make sure the functions allowed in the start and live >>> phases work Ok. >>> As the list of functions is pretty big the test does sanity checks >>> that the functions do not crash nor return errors. >>> >>> >>>> I was expecting to see a difference between things that can be called >>>> at early-start and those that can not - or are these all expected to >>>> work okay in either case? >>> >>> All these functions are expected to work okay in both cases. >>> Of course, the main concern is the early start. >>> But we have never had such coverage in the past so that the normal >>> start phase needs to be covered too. >>> >>> >>>> >>>> A few comments: >>>> >>>> 44 #define TranslateError(err) "JVMTI error" >>>> >>>> I don't see the point of the above. >>> >>> Good catch - removed. >>> It is a left over from another test that I used as initial template. >>> >>> >>>> --- >>>> >>>> 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { >>>> >>>> The thread local functions use "long" as the datatype but that will >>>> only be 32-bit on 64-bit Windows. I think you need to use intptr_t >>>> for complete portability. >>> >>> The type long has the same format as the type void* which has to be >>> portable even on win-32. >>> But maybe I'm missing something. >>> Anyway, I've replaced it with the intptr_t. >>> >>> >>>> >>>> --- >>>> >>>> 277 printf(" Filed declaring"); >>>> >>>> typo: filed -> field >>> >>> >>> Good catch - fixed. >>> >>>> >>>> --- >>>> >>>> All your little wrapper functions make the JVMTI call and then call >>>> check_jvmti_error - but all that does is record if it passed or >>>> failed. If it failed you still continue with the next operation even >>>> if it relies on the success of the first one eg: >>>> >>>> 378 set_thread_local(jvmti, thread, exp_val); >>>> 379 act_val = get_thread_local(jvmti, cur_thread); >>>> >>>> and the sequences in print_method_info: >>>> >>>> 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); >>>> 229 check_jvmti_error(jvmti, "IsMethodNative", err); >>>> 230 printf(" Method is native: %d\n", is_native); >>>> 231 >>>> 232 if (is_native == JNI_FALSE) { >>>> 233 err = (*jvmti)->GetMaxLocals(jvmti, method, &locals_max); >>>> >>>> The call at #233 may not be valid because the method actually is >>>> native but the IsMethodNative call failed for some reason. >>>> >>> >>> It is intentional. I have done it as a last cleanup. >>> The point is to simplify code by skipping all the extra checks if it >>> does not lead to any fatal errors. >>> The most important in such a case is that the static variable result >>> is set to FAILED. >>> It will cause the test to fail. >>> Then there is no point to analyze the printed results if a JVMTI error >>> reported before. >>> >>> If you insist, I will add back all the extra check to make sure all >>> printed output is valid. >>> >>> >>> Thanks, >>> Serguei >>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> >>>>> Summary: >>>>> The task was to provide a test coverage for the JVMTI functions >>>>> allowed during the start phase. >>>>> It includes both enabling and disabling the >>>>> can_generate_early_vmstart >>>>> capability. >>>>> Testing the JVMTI functions allowed in any case has not been >>>>> targeted >>>>> by this fix. >>>>> >>>>> Testing: >>>>> New test is passed. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>> >> From aph at redhat.com Fri Apr 28 19:28:11 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 28 Apr 2017 20:28:11 +0100 Subject: RFR: 8179444: AArch64: Put zero_words on a diet Message-ID: The code we generate for ClearArray in C2 is much too verbose. It looks like this: 0x000003ffad2213a4: cbz x11, 0x000003ffad22140c 0x000003ffad2213a8: tbz w10, #3, 0x000003ffad2213b4 0x000003ffad2213ac: str xzr, [x10],#8 0x000003ffad2213b0: sub x11, x11, #0x1 0x000003ffad2213b4: subs xscratch1, x11, #0x20 0x000003ffad2213b8: b.lt 0x000003ffad2213c0 0x000003ffad2213bc: bl Stub::zero_longs ; {external_word} 0x000003ffad2213c0: and xscratch1, x11, #0xe 0x000003ffad2213c4: sub x11, x11, xscratch1 0x000003ffad2213c8: add x10, x10, xscratch1, lsl #3 0x000003ffad2213cc: adr xscratch2, 0x000003ffad2213fc 0x000003ffad2213d0: sub xscratch2, xscratch2, xscratch1, lsl #1 0x000003ffad2213d4: br xscratch2 0x000003ffad2213d8: add x10, x10, #0x80 0x000003ffad2213dc: stp xzr, xzr, [x10,#-128] 0x000003ffad2213e0: stp xzr, xzr, [x10,#-112] 0x000003ffad2213e4: stp xzr, xzr, [x10,#-96] 0x000003ffad2213e8: stp xzr, xzr, [x10,#-80] 0x000003ffad2213ec: stp xzr, xzr, [x10,#-64] 0x000003ffad2213f0: stp xzr, xzr, [x10,#-48] 0x000003ffad2213f4: stp xzr, xzr, [x10,#-32] 0x000003ffad2213f8: stp xzr, xzr, [x10,#-16] 0x000003ffad2213fc: subs x11, x11, #0x10 0x000003ffad221400: b.ge 0x000003ffad2213d8 0x000003ffad221404: tbz w11, #0, 0x000003ffad22140c 0x000003ffad221408: str xzr, [x10],#8 This patch takes much of this code and puts it into a stub. The new version of ClearArray is: 0x000003ffad21088c: cmp x11, #0x8 0x000003ffad210890: b.lt 0x000003ffad210898 0x000003ffad210894: bl Stub::zero_blocks ; {runtime_call StubRoutines (2)} 0x000003ffad210898: and xscratch1, x11, #0x6 0x000003ffad21089c: adr xscratch2, 0x000003ffad2108b4 0x000003ffad2108a0: sub xscratch2, xscratch2, xscratch1, lsl #1 0x000003ffad2108a4: br xscratch2 0x000003ffad2108a8: stp xzr, xzr, [x10],#16 0x000003ffad2108ac: stp xzr, xzr, [x10],#16 0x000003ffad2108b0: stp xzr, xzr, [x10],#16 0x000003ffad2108b4: tbz w11, #0, 0x000003ffad2108bc 0x000003ffad2108b8: str xzr, [x10] The idea is to handle array sizes of 0-7 words inline, so small arrays are got out of the way very quickly, and handle anything larger in Stub::zero_blocks. I wanted to make sure that there is no significant loss of performance, and I have attached the results of the benchmark I used, which does no more than create an array of ints of various sizes. There are winners and losers, but nothing is changed by very much, and the code cache usage of each ClearArray goes down from 104 to 48 bytes. http://cr.openjdk.java.net/~aph/8179444/ OK? Andrew. Machine A: Before: Benchmark (size) Mode Cnt Score Error Units CreateArray.newArray 5 avgt 5 48.221 ? 3.185 ns/op CreateArray.newArray 7 avgt 5 48.853 ? 1.921 ns/op CreateArray.newArray 10 avgt 5 49.963 ? 2.240 ns/op CreateArray.newArray 15 avgt 5 52.538 ? 1.332 ns/op CreateArray.newArray 23 avgt 5 57.289 ? 1.120 ns/op CreateArray.newArray 34 avgt 5 67.091 ? 2.207 ns/op CreateArray.newArray 51 avgt 5 119.948 ? 1.839 ns/op CreateArray.newArray 77 avgt 5 101.851 ? 1.968 ns/op CreateArray.newArray 115 avgt 5 142.568 ? 3.621 ns/op CreateArray.newArray 173 avgt 5 180.204 ? 2.908 ns/op CreateArray.newArray 259 avgt 5 170.446 ? 6.083 ns/op CreateArray.newArray 389 avgt 5 231.124 ? 1.804 ns/op CreateArray.newArray 584 avgt 5 248.411 ? 0.438 ns/op CreateArray.newArray 876 avgt 5 241.776 ? 1.261 ns/op CreateArray.newArray 1314 avgt 5 383.609 ? 1.363 ns/op CreateArray.newArray 1971 avgt 5 483.217 ? 8.044 ns/op After: Benchmark (size) Mode Cnt Score Error Units CreateArray.newArray 5 avgt 5 47.256 ? 1.511 ns/op CreateArray.newArray 7 avgt 5 48.674 ? 1.046 ns/op CreateArray.newArray 10 avgt 5 50.915 ? 2.581 ns/op CreateArray.newArray 15 avgt 5 53.351 ? 6.562 ns/op CreateArray.newArray 23 avgt 5 56.746 ? 3.820 ns/op CreateArray.newArray 34 avgt 5 65.796 ? 3.357 ns/op CreateArray.newArray 51 avgt 5 119.825 ? 2.268 ns/op CreateArray.newArray 77 avgt 5 100.708 ? 1.647 ns/op CreateArray.newArray 115 avgt 5 135.210 ? 2.844 ns/op CreateArray.newArray 173 avgt 5 180.521 ? 1.373 ns/op CreateArray.newArray 259 avgt 5 160.899 ? 2.677 ns/op CreateArray.newArray 389 avgt 5 230.253 ? 1.412 ns/op CreateArray.newArray 584 avgt 5 249.173 ? 2.827 ns/op CreateArray.newArray 876 avgt 5 242.180 ? 0.991 ns/op CreateArray.newArray 1314 avgt 5 385.272 ? 1.872 ns/op CreateArray.newArray 1971 avgt 5 485.198 ? 3.196 ns/op Machine B: The timings for Machine B are very noisy with small array sizes, so it's hard to conclude very much, but I don't think there is any regression. Before: Benchmark (size) Mode Cnt Score Error Units CreateArray.newArray 5 avgt 5 89.209 ? 11.640 ns/op CreateArray.newArray 7 avgt 5 93.453 ? 2.113 ns/op CreateArray.newArray 10 avgt 5 93.388 ? 21.406 ns/op CreateArray.newArray 15 avgt 5 102.904 ? 23.075 ns/op CreateArray.newArray 23 avgt 5 117.167 ? 19.673 ns/op CreateArray.newArray 34 avgt 5 130.184 ? 1.042 ns/op CreateArray.newArray 51 avgt 5 132.981 ? 8.446 ns/op CreateArray.newArray 77 avgt 5 137.438 ? 5.723 ns/op CreateArray.newArray 115 avgt 5 135.289 ? 3.393 ns/op CreateArray.newArray 173 avgt 5 151.245 ? 8.469 ns/op CreateArray.newArray 259 avgt 5 157.292 ? 2.087 ns/op CreateArray.newArray 389 avgt 5 176.621 ? 3.741 ns/op CreateArray.newArray 584 avgt 5 200.957 ? 6.825 ns/op CreateArray.newArray 876 avgt 5 233.122 ? 3.508 ns/op CreateArray.newArray 1314 avgt 5 280.525 ? 5.696 ns/op CreateArray.newArray 1971 avgt 5 360.799 ? 8.859 ns/op After: Benchmark (size) Mode Cnt Score Error Units CreateArray.newArray 5 avgt 5 90.168 ? 4.363 ns/op CreateArray.newArray 7 avgt 5 88.221 ? 32.537 ns/op CreateArray.newArray 10 avgt 5 97.991 ? 1.778 ns/op CreateArray.newArray 15 avgt 5 102.441 ? 30.219 ns/op CreateArray.newArray 23 avgt 5 120.875 ? 11.074 ns/op CreateArray.newArray 34 avgt 5 130.916 ? 2.476 ns/op CreateArray.newArray 51 avgt 5 134.765 ? 10.002 ns/op CreateArray.newArray 77 avgt 5 138.228 ? 2.479 ns/op CreateArray.newArray 115 avgt 5 135.907 ? 1.025 ns/op CreateArray.newArray 173 avgt 5 150.318 ? 9.291 ns/op CreateArray.newArray 259 avgt 5 156.671 ? 2.023 ns/op CreateArray.newArray 389 avgt 5 175.735 ? 3.861 ns/op CreateArray.newArray 584 avgt 5 206.501 ? 9.117 ns/op CreateArray.newArray 876 avgt 5 233.676 ? 3.463 ns/op CreateArray.newArray 1314 avgt 5 280.259 ? 4.131 ns/op CreateArray.newArray 1971 avgt 5 360.037 ? 9.968 ns/op From serguei.spitsyn at oracle.com Sat Apr 29 00:13:56 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 28 Apr 2017 17:13:56 -0700 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: <1e443485-63d6-4473-8809-76f47a380436@oracle.com> References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> <5ea66d4b-f1bb-1729-ba51-5bdcd5b3c470@oracle.com> <1e443485-63d6-4473-8809-76f47a380436@oracle.com> Message-ID: Hi David, On 4/28/17 10:34, serguei.spitsyn at oracle.com wrote: > Hi David, > > > On 4/28/17 04:42, David Holmes wrote: >> Hi Serguei, >> >> On 28/04/2017 6:07 PM, serguei.spitsyn at oracle.com wrote: >>> The updated webrev is: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.2/ >>> >> >> Thanks for the updates (the issue with long is that on win64 it is >> only 32-bit while void* is 64-bit). > > Ok, thanks. > Than you are right, using long on win64 is not compatible. > >> >> I prefer to see fast-fail rather than potentially triggering >> cascading failures (check_jvmti_error could even call exit() I >> think). But let's see what others think - it's only a preference not >> a necessity. > > Ok, I'll consider call exit() as it would keep it simple. > New webrev version is: http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.3/ Thanks, Serguei > Thanks, > Serguei > >> >> Thanks, >> David >> >>> >>> I've re-arranged a little bit code in the ClassPrepare callback and the >>> function test_class_functions(). >>> >>> Thanks, >>> Serguei >>> >>> >>> On 4/28/17 00:47, serguei.spitsyn at oracle.com wrote: >>>> Hi David, >>>> >>>> Thank you for looking at the test! >>>> >>>> >>>> On 4/27/17 23:11, David Holmes wrote: >>>>> Hi Serguei, >>>>> >>>>> On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the jdk 10 fix for the test enhancement: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8172970 >>>>>> >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >>>>>> >>>>>> >>>>> >>>>> Sorry but I can't quite figure out exactly what this test is doing. >>>>> What is the overall call structure here? >>>> >>>> This is to make sure the functions allowed in the start and live >>>> phases work Ok. >>>> As the list of functions is pretty big the test does sanity checks >>>> that the functions do not crash nor return errors. >>>> >>>> >>>>> I was expecting to see a difference between things that can be called >>>>> at early-start and those that can not - or are these all expected to >>>>> work okay in either case? >>>> >>>> All these functions are expected to work okay in both cases. >>>> Of course, the main concern is the early start. >>>> But we have never had such coverage in the past so that the normal >>>> start phase needs to be covered too. >>>> >>>> >>>>> >>>>> A few comments: >>>>> >>>>> 44 #define TranslateError(err) "JVMTI error" >>>>> >>>>> I don't see the point of the above. >>>> >>>> Good catch - removed. >>>> It is a left over from another test that I used as initial template. >>>> >>>> >>>>> --- >>>>> >>>>> 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { >>>>> >>>>> The thread local functions use "long" as the datatype but that will >>>>> only be 32-bit on 64-bit Windows. I think you need to use intptr_t >>>>> for complete portability. >>>> >>>> The type long has the same format as the type void* which has to be >>>> portable even on win-32. >>>> But maybe I'm missing something. >>>> Anyway, I've replaced it with the intptr_t. >>>> >>>> >>>>> >>>>> --- >>>>> >>>>> 277 printf(" Filed declaring"); >>>>> >>>>> typo: filed -> field >>>> >>>> >>>> Good catch - fixed. >>>> >>>>> >>>>> --- >>>>> >>>>> All your little wrapper functions make the JVMTI call and then call >>>>> check_jvmti_error - but all that does is record if it passed or >>>>> failed. If it failed you still continue with the next operation even >>>>> if it relies on the success of the first one eg: >>>>> >>>>> 378 set_thread_local(jvmti, thread, exp_val); >>>>> 379 act_val = get_thread_local(jvmti, cur_thread); >>>>> >>>>> and the sequences in print_method_info: >>>>> >>>>> 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); >>>>> 229 check_jvmti_error(jvmti, "IsMethodNative", err); >>>>> 230 printf(" Method is native: %d\n", is_native); >>>>> 231 >>>>> 232 if (is_native == JNI_FALSE) { >>>>> 233 err = (*jvmti)->GetMaxLocals(jvmti, method, >>>>> &locals_max); >>>>> >>>>> The call at #233 may not be valid because the method actually is >>>>> native but the IsMethodNative call failed for some reason. >>>>> >>>> >>>> It is intentional. I have done it as a last cleanup. >>>> The point is to simplify code by skipping all the extra checks if it >>>> does not lead to any fatal errors. >>>> The most important in such a case is that the static variable result >>>> is set to FAILED. >>>> It will cause the test to fail. >>>> Then there is no point to analyze the printed results if a JVMTI error >>>> reported before. >>>> >>>> If you insist, I will add back all the extra check to make sure all >>>> printed output is valid. >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> >>>>>> Summary: >>>>>> The task was to provide a test coverage for the JVMTI functions >>>>>> allowed during the start phase. >>>>>> It includes both enabling and disabling the >>>>>> can_generate_early_vmstart >>>>>> capability. >>>>>> Testing the JVMTI functions allowed in any case has not been >>>>>> targeted >>>>>> by this fix. >>>>>> >>>>>> Testing: >>>>>> New test is passed. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>> >>> > From david.holmes at oracle.com Sat Apr 29 01:08:46 2017 From: david.holmes at oracle.com (David Holmes) Date: Sat, 29 Apr 2017 11:08:46 +1000 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> <5ea66d4b-f1bb-1729-ba51-5bdcd5b3c470@oracle.com> <1e443485-63d6-4473-8809-76f47a380436@oracle.com> Message-ID: <6d196b42-cb4f-b599-c25f-1fc3f66bcc2f@oracle.com> That looks fine to me Serguei! Thanks, David On 29/04/2017 10:13 AM, serguei.spitsyn at oracle.com wrote: > Hi David, > > > On 4/28/17 10:34, serguei.spitsyn at oracle.com wrote: >> Hi David, >> >> >> On 4/28/17 04:42, David Holmes wrote: >>> Hi Serguei, >>> >>> On 28/04/2017 6:07 PM, serguei.spitsyn at oracle.com wrote: >>>> The updated webrev is: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.2/ >>>> >>> >>> Thanks for the updates (the issue with long is that on win64 it is >>> only 32-bit while void* is 64-bit). >> >> Ok, thanks. >> Than you are right, using long on win64 is not compatible. >> >>> >>> I prefer to see fast-fail rather than potentially triggering >>> cascading failures (check_jvmti_error could even call exit() I >>> think). But let's see what others think - it's only a preference not >>> a necessity. >> >> Ok, I'll consider call exit() as it would keep it simple. >> > > New webrev version is: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.3/ > > > > Thanks, > Serguei > > >> Thanks, >> Serguei >> >>> >>> Thanks, >>> David >>> >>>> >>>> I've re-arranged a little bit code in the ClassPrepare callback and the >>>> function test_class_functions(). >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 4/28/17 00:47, serguei.spitsyn at oracle.com wrote: >>>>> Hi David, >>>>> >>>>> Thank you for looking at the test! >>>>> >>>>> >>>>> On 4/27/17 23:11, David Holmes wrote: >>>>>> Hi Serguei, >>>>>> >>>>>> On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Please, review the jdk 10 fix for the test enhancement: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8172970 >>>>>>> >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >>>>>>> >>>>>>> >>>>>> >>>>>> Sorry but I can't quite figure out exactly what this test is doing. >>>>>> What is the overall call structure here? >>>>> >>>>> This is to make sure the functions allowed in the start and live >>>>> phases work Ok. >>>>> As the list of functions is pretty big the test does sanity checks >>>>> that the functions do not crash nor return errors. >>>>> >>>>> >>>>>> I was expecting to see a difference between things that can be called >>>>>> at early-start and those that can not - or are these all expected to >>>>>> work okay in either case? >>>>> >>>>> All these functions are expected to work okay in both cases. >>>>> Of course, the main concern is the early start. >>>>> But we have never had such coverage in the past so that the normal >>>>> start phase needs to be covered too. >>>>> >>>>> >>>>>> >>>>>> A few comments: >>>>>> >>>>>> 44 #define TranslateError(err) "JVMTI error" >>>>>> >>>>>> I don't see the point of the above. >>>>> >>>>> Good catch - removed. >>>>> It is a left over from another test that I used as initial template. >>>>> >>>>> >>>>>> --- >>>>>> >>>>>> 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { >>>>>> >>>>>> The thread local functions use "long" as the datatype but that will >>>>>> only be 32-bit on 64-bit Windows. I think you need to use intptr_t >>>>>> for complete portability. >>>>> >>>>> The type long has the same format as the type void* which has to be >>>>> portable even on win-32. >>>>> But maybe I'm missing something. >>>>> Anyway, I've replaced it with the intptr_t. >>>>> >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> 277 printf(" Filed declaring"); >>>>>> >>>>>> typo: filed -> field >>>>> >>>>> >>>>> Good catch - fixed. >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> All your little wrapper functions make the JVMTI call and then call >>>>>> check_jvmti_error - but all that does is record if it passed or >>>>>> failed. If it failed you still continue with the next operation even >>>>>> if it relies on the success of the first one eg: >>>>>> >>>>>> 378 set_thread_local(jvmti, thread, exp_val); >>>>>> 379 act_val = get_thread_local(jvmti, cur_thread); >>>>>> >>>>>> and the sequences in print_method_info: >>>>>> >>>>>> 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); >>>>>> 229 check_jvmti_error(jvmti, "IsMethodNative", err); >>>>>> 230 printf(" Method is native: %d\n", is_native); >>>>>> 231 >>>>>> 232 if (is_native == JNI_FALSE) { >>>>>> 233 err = (*jvmti)->GetMaxLocals(jvmti, method, >>>>>> &locals_max); >>>>>> >>>>>> The call at #233 may not be valid because the method actually is >>>>>> native but the IsMethodNative call failed for some reason. >>>>>> >>>>> >>>>> It is intentional. I have done it as a last cleanup. >>>>> The point is to simplify code by skipping all the extra checks if it >>>>> does not lead to any fatal errors. >>>>> The most important in such a case is that the static variable result >>>>> is set to FAILED. >>>>> It will cause the test to fail. >>>>> Then there is no point to analyze the printed results if a JVMTI error >>>>> reported before. >>>>> >>>>> If you insist, I will add back all the extra check to make sure all >>>>> printed output is valid. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> >>>>>>> Summary: >>>>>>> The task was to provide a test coverage for the JVMTI functions >>>>>>> allowed during the start phase. >>>>>>> It includes both enabling and disabling the >>>>>>> can_generate_early_vmstart >>>>>>> capability. >>>>>>> Testing the JVMTI functions allowed in any case has not been >>>>>>> targeted >>>>>>> by this fix. >>>>>>> >>>>>>> Testing: >>>>>>> New test is passed. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>> >>>> >> > From serguei.spitsyn at oracle.com Sat Apr 29 04:33:04 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 28 Apr 2017 21:33:04 -0700 Subject: RFR(M): 8172970: TESTBUG: need test coverage for the JVMTI functions allowed in the start phase In-Reply-To: <6d196b42-cb4f-b599-c25f-1fc3f66bcc2f@oracle.com> References: <88938aee-0a83-99e9-1b95-6875173807a5@oracle.com> <5ea66d4b-f1bb-1729-ba51-5bdcd5b3c470@oracle.com> <1e443485-63d6-4473-8809-76f47a380436@oracle.com> <6d196b42-cb4f-b599-c25f-1fc3f66bcc2f@oracle.com> Message-ID: <3eb94f3b-142c-6cd6-3d15-a0ab840880dc@oracle.com> Thanks, David! Serguei On 4/28/17 18:08, David Holmes wrote: > That looks fine to me Serguei! > > Thanks, > David > > On 29/04/2017 10:13 AM, serguei.spitsyn at oracle.com wrote: >> Hi David, >> >> >> On 4/28/17 10:34, serguei.spitsyn at oracle.com wrote: >>> Hi David, >>> >>> >>> On 4/28/17 04:42, David Holmes wrote: >>>> Hi Serguei, >>>> >>>> On 28/04/2017 6:07 PM, serguei.spitsyn at oracle.com wrote: >>>>> The updated webrev is: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.2/ >>>>> >>>>> >>>> >>>> Thanks for the updates (the issue with long is that on win64 it is >>>> only 32-bit while void* is 64-bit). >>> >>> Ok, thanks. >>> Than you are right, using long on win64 is not compatible. >>> >>>> >>>> I prefer to see fast-fail rather than potentially triggering >>>> cascading failures (check_jvmti_error could even call exit() I >>>> think). But let's see what others think - it's only a preference not >>>> a necessity. >>> >>> Ok, I'll consider call exit() as it would keep it simple. >>> >> >> New webrev version is: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.3/ >> >> >> >> >> Thanks, >> Serguei >> >> >>> Thanks, >>> Serguei >>> >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> I've re-arranged a little bit code in the ClassPrepare callback >>>>> and the >>>>> function test_class_functions(). >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 4/28/17 00:47, serguei.spitsyn at oracle.com wrote: >>>>>> Hi David, >>>>>> >>>>>> Thank you for looking at the test! >>>>>> >>>>>> >>>>>> On 4/27/17 23:11, David Holmes wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> On 28/04/2017 3:14 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> Please, review the jdk 10 fix for the test enhancement: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8172970 >>>>>>>> >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8172970-start-phase.1/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Sorry but I can't quite figure out exactly what this test is doing. >>>>>>> What is the overall call structure here? >>>>>> >>>>>> This is to make sure the functions allowed in the start and live >>>>>> phases work Ok. >>>>>> As the list of functions is pretty big the test does sanity checks >>>>>> that the functions do not crash nor return errors. >>>>>> >>>>>> >>>>>>> I was expecting to see a difference between things that can be >>>>>>> called >>>>>>> at early-start and those that can not - or are these all >>>>>>> expected to >>>>>>> work okay in either case? >>>>>> >>>>>> All these functions are expected to work okay in both cases. >>>>>> Of course, the main concern is the early start. >>>>>> But we have never had such coverage in the past so that the normal >>>>>> start phase needs to be covered too. >>>>>> >>>>>> >>>>>>> >>>>>>> A few comments: >>>>>>> >>>>>>> 44 #define TranslateError(err) "JVMTI error" >>>>>>> >>>>>>> I don't see the point of the above. >>>>>> >>>>>> Good catch - removed. >>>>>> It is a left over from another test that I used as initial template. >>>>>> >>>>>> >>>>>>> --- >>>>>>> >>>>>>> 99 static long get_thread_local(jvmtiEnv *jvmti, jthread thread) { >>>>>>> >>>>>>> The thread local functions use "long" as the datatype but that will >>>>>>> only be 32-bit on 64-bit Windows. I think you need to use intptr_t >>>>>>> for complete portability. >>>>>> >>>>>> The type long has the same format as the type void* which has to be >>>>>> portable even on win-32. >>>>>> But maybe I'm missing something. >>>>>> Anyway, I've replaced it with the intptr_t. >>>>>> >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> 277 printf(" Filed declaring"); >>>>>>> >>>>>>> typo: filed -> field >>>>>> >>>>>> >>>>>> Good catch - fixed. >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> All your little wrapper functions make the JVMTI call and then call >>>>>>> check_jvmti_error - but all that does is record if it passed or >>>>>>> failed. If it failed you still continue with the next operation >>>>>>> even >>>>>>> if it relies on the success of the first one eg: >>>>>>> >>>>>>> 378 set_thread_local(jvmti, thread, exp_val); >>>>>>> 379 act_val = get_thread_local(jvmti, cur_thread); >>>>>>> >>>>>>> and the sequences in print_method_info: >>>>>>> >>>>>>> 228 err = (*jvmti)->IsMethodNative(jvmti, method, &is_native); >>>>>>> 229 check_jvmti_error(jvmti, "IsMethodNative", err); >>>>>>> 230 printf(" Method is native: %d\n", is_native); >>>>>>> 231 >>>>>>> 232 if (is_native == JNI_FALSE) { >>>>>>> 233 err = (*jvmti)->GetMaxLocals(jvmti, method, >>>>>>> &locals_max); >>>>>>> >>>>>>> The call at #233 may not be valid because the method actually is >>>>>>> native but the IsMethodNative call failed for some reason. >>>>>>> >>>>>> >>>>>> It is intentional. I have done it as a last cleanup. >>>>>> The point is to simplify code by skipping all the extra checks if it >>>>>> does not lead to any fatal errors. >>>>>> The most important in such a case is that the static variable result >>>>>> is set to FAILED. >>>>>> It will cause the test to fail. >>>>>> Then there is no point to analyze the printed results if a JVMTI >>>>>> error >>>>>> reported before. >>>>>> >>>>>> If you insist, I will add back all the extra check to make sure all >>>>>> printed output is valid. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Summary: >>>>>>>> The task was to provide a test coverage for the JVMTI functions >>>>>>>> allowed during the start phase. >>>>>>>> It includes both enabling and disabling the >>>>>>>> can_generate_early_vmstart >>>>>>>> capability. >>>>>>>> Testing the JVMTI functions allowed in any case has not been >>>>>>>> targeted >>>>>>>> by this fix. >>>>>>>> >>>>>>>> Testing: >>>>>>>> New test is passed. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>> >>>>> >>> >> From aph at redhat.com Sat Apr 29 13:07:14 2017 From: aph at redhat.com (Andrew Haley) Date: Sat, 29 Apr 2017 14:07:14 +0100 Subject: RFR: 8179444: AArch64: Put zero_words on a diet In-Reply-To: References: Message-ID: <6cac67bb-1002-78f9-81ba-deeec62ffe16@redhat.com> On 28/04/17 20:28, Andrew Haley wrote: > http://cr.openjdk.java.net/~aph/8179444/ I withdraw this patch. I've found some more dead wood to cut out. Andrew. From ioi.lam at oracle.com Sat Apr 29 15:30:06 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 29 Apr 2017 08:30:06 -0700 Subject: RFR (L) 8171392 Move Klass pointers outside of ConstantPool entries so ConstantPool can be read-only In-Reply-To: <590327F1.7070200@oracle.com> References: <58EC771B.9020202@oracle.com> <35e6276a-ddf1-9149-8588-acb4e13191f5@oracle.com> <58EF3D3A.6020903@oracle.com> <58F05EB5.10009@oracle.com> <58F0EB0E.60904@oracle.com> <58F33A4F.70104@oracle.com> <58FDECFE.5060105@oracle.com> <590327F1.7070200@oracle.com> Message-ID: <5904B17E.9090209@oracle.com> I've updated the patch to include Volker's ppc/s390 port as well as his comments. I've also included an updated patch (untested) for aarch64 for Andrew Haley to test: Full patch http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v04/ Delta from the previous version http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v04.delta/ Thanks - Ioi On 4/28/17 4:30 AM, Ioi Lam wrote: > > > On 4/25/17 8:06 AM, Volker Simonis wrote: >> On Mon, Apr 24, 2017 at 2:18 PM, Ioi Lam wrote: >>> Hi Volker, >>> >>> >>> On 4/21/17 12:02 AM, Volker Simonis wrote: >>>> Hi Ioi, >>>> >>>> thanks once again for considering our ports! Please find the required >>>> additions for ppc64/s390x in the following webrew (which is based upon >>>> your latest v03 patch): >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/2017/8171392_ppc64_s390x/ >>> Thanks for the patch. I will integrate it and post an updated webrev. >>>> @Martin/@Lucy: could you please have a look at my ppc64/s390x assembly >>>> code. I did some tests and I think it should be correct, but maybe you >>>> still find some improvements :) >>>> >>>> Besides that, I have some general questions/comments regarding your >>>> change: >>>> >>>> 1. In constantPool.hpp, why don't you declare the '_name_index' and >>>> '_resolved_klass_index' fields with type 'jushort'? As far as I can >>>> see, they can only hold 16-bit values anyway. It would also save you >>>> some space and several asserts (e.g. in unresolved_klass_at_put(): >>>> >>>> >>>> 274 assert((name_index & 0xffff0000) == 0, "must be"); >>>> 275 assert((resolved_klass_index & 0xffff0000) == 0, "must >>>> be"); >>> >>> I think the HotSpot convention is to use ints as parameter and >>> return types, >>> for values that are actually 16-bits or less, like here in >>> constantPool.hpp: >>> >>> void field_at_put(int which, int class_index, int >>> name_and_type_index) { >>> tag_at_put(which, JVM_CONSTANT_Fieldref); >>> *int_at_addr(which) = ((jint) name_and_type_index<<16) | >>> class_index; >>> } >>> >>> I am not sure what the reasons are. It could be that the parameters >>> usually >>> need to be computed arithmetically, and it's much easier for the >>> caller of >>> the method to use ints -- otherwise you will get lots of compiler >>> warnings >>> which would force you to use lots of casting, resulting in code >>> that's hard >>> to read and probably incorrect. >>> >> OK, but you could still use shorts in the the object to save space, >> although I'm not sure how much that will save in total. But if nobody >> else cares, I'm fine with the current version. > > The CPKlassSlot objects are stored only on the stack, so the savings > is not worth the trouble of adding extract type casts. > > Inside the ConstantPool itself, the name_index and > resolved_klass_index are stored as a pair of 16-bit values. > >>>> 2. What do you mean by: >>>> >>>> 106 // ... will be changed to support compressed pointers >>>> 107 Array* _resolved_klasses; >>> >>> Sorry the comment isn't very clear. How about this? >>> >>> 106 // Consider using an array of compressed klass pointers to >>> // save space on 64-bit platforms. >>> 107 Array* _resolved_klasses; >>> >> Sorry I still didn't get it? Do you mean you want to use array of >> "narrowKlass" (i.e. unsigned int)? But using compressed class pointers >> is a runtime decision while this is a compile time decision. > > I haven't figured out how to do it yet :-) > > Most likely, it will be something like: > > union { > Array* X; > Array* Y; > } _resolved_klasses; > > and you need to decide at run time whether to use X or Y. > > - Ioi >>>> 3. Why don't we need the call to "release_tag_at_put()" in >>>> "klass_at_put(int class_index, Klass* k)"? "klass_at_put(int >>>> class_index, Klass* k)" is used from >>>> "ClassFileParser::fill_instance_klass() and before your change that >>>> function used the previous version of "klass_at_put(int class_index, >>>> Klass* k)" which did call "release_tag_at_put()". >>> >>> Good catch. I'll add the following, because the class is now resolved. >>> >>> release_tag_at_put(class_index, JVM_CONSTANT_UnresolvedClass); >>>> 4. In ConstantPool::copy_entry_to() you've changed the behavior for >>>> tags JVM_CONSTANT_Class, JVM_CONSTANT_UnresolvedClass, >>>> JVM_CONSTANT_UnresolvedClassInError. Before, the resolved klass was >>>> copied to the new constant pool if one existed but now you always only >>>> copy a class_index to the new constant pool (even if a resolved klass >>>> existed). Is that OK? E.g. won't this lead to a new resolving for the >>>> new constant pool and will this have performance impacts or other side >>>> effects? >>> I think Coleen has answered this in a separate mail :-) >>> >>> Thanks >>> - Ioi >>> >>>> Thanks again for doing this nice change and best regards, >>>> Volker >>>> >>>> >>>> On Sun, Apr 16, 2017 at 11:33 AM, Ioi Lam wrote: >>>>> Hi Lois, >>>>> >>>>> I have updated the patch to include your comments, and fixes the >>>>> handling >>>>> of >>>>> anonymous classes. I also added some more comments regarding the >>>>> _temp_resolved_klass_index: >>>>> >>>>> (delta from last webrev) >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03.delta/ >>>>> >>>>> >>>>> (full webrev) >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v03/ >>>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 4/15/17 2:31 AM, Lois Foltan wrote: >>>>>> On 4/14/2017 11:30 AM, Ioi Lam wrote: >>>>>>> >>>>>>> >>>>>>> On 4/14/17 1:31 PM, Ioi Lam wrote: >>>>>>>> HI Lois, >>>>>>>> >>>>>>>> Thanks for the review. Please see my comments in-line. >>>>>>>> >>>>>>>> On 4/14/17 4:32 AM, Lois Foltan wrote: >>>>>>>>> Hi Ioi, >>>>>>>>> >>>>>>>>> Looks really good. A couple of comments: >>>>>>>>> >>>>>>>>> src/share/vm/classfile/classFileParser.cpp: >>>>>>>>> * line #5676 - I'm not sure I completely understand the logic >>>>>>>>> surrounding anonymous classes. Coleen and I discussed earlier >>>>>>>>> today >>>>>>>>> and I >>>>>>>>> came away from that discussion with the idea that the only >>>>>>>>> classes >>>>>>>>> being >>>>>>>>> patched currently are anonymous classes. >>>>>>>> Line 5676 ... >>>>>>>> >>>>>>>> 5676 if (is_anonymous()) { >>>>>>>> 5677 _max_num_patched_klasses ++; // for patching the >>>>>>>> class >>>>>>>> index >>>>>>>> 5678 } >>>>>>>> >>>>>>>> corresponds to >>>>>>>> >>>>>>>> 5361 ik->set_name(_class_name); >>>>>>>> 5362 >>>>>>>> 5363 if (is_anonymous()) { >>>>>>>> 5364 // I am well known to myself >>>>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>>>> ik->name()); // eagerly resolve >>>>>>>> 5366 } >>>>>>>> >>>>>>>> Even though the class is "anonymous", it actually has a name. >>>>>>>> ik->name() >>>>>>>> probably is part of the constant pool, but I am not 100% sure. >>>>>>>> Also, I >>>>>>>> would >>>>>>>> need to search the constant pool to find the index for >>>>>>>> ik->name(). So >>>>>>>> I just >>>>>>>> got lazy here and use the same logic in >>>>>>>> ConstantPool::patch_class() to >>>>>>>> append ik->name() to the end of the constant pool. >>>>>>>> >>>>>>>> "Anonymous" actually means "the class cannot be looked up by >>>>>>>> name in >>>>>>>> the >>>>>>>> SystemDictionary". I think we need a better terminology :-) >>>>>>>> >>>>>>> I finally realized why we need the "eagerly resolve" on line >>>>>>> 5365. I'll >>>>>>> modify the comments to the following: >>>>>>> >>>>>>> // _this_class_index is a CONSTANT_Class entry that refers >>>>>>> to this >>>>>>> // anonymous class itself. If this class needs to refer to >>>>>>> its own >>>>>>> methods or >>>>>>> // fields, it would use a CONSTANT_MethodRef, etc, which >>>>>>> would >>>>>>> reference >>>>>>> // _this_class_index. However, because this class is >>>>>>> anonymous >>>>>>> (it's >>>>>>> // not stored in SystemDictionary), _this_class_index >>>>>>> cannot be >>>>>>> resolved >>>>>>> // with ConstantPool::klass_at_impl, which does a >>>>>>> SystemDictionary >>>>>>> lookup. >>>>>>> // Therefore, we must eagerly resolve _this_class_index now. >>>>>>> >>>>>>> So, Lois is right. Line 5676 is not necessary. I will revise the >>>>>>> code >>>>>>> to >>>>>>> do the "eager resolution" without using >>>>>>> ClassFileParser::patch_class. >>>>>>> I'll >>>>>>> post the updated code later. >>>>>> >>>>>> Thanks Ioi for studying this and explaining! Look forward to >>>>>> seeing the >>>>>> updated webrev. >>>>>> Lois >>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>>>> So a bit confused as why the check on line #5676 and a check >>>>>>>>> for a >>>>>>>>> java/lang/Class on line #5684. >>>>>>>> 5683 Handle patch = cp_patch_at(i); >>>>>>>> 5684 if (java_lang_String::is_instance(patch()) || >>>>>>>> java_lang_Class::is_instance(patch())) { >>>>>>>> 5685 // We need to append the names of the patched >>>>>>>> classes >>>>>>>> to >>>>>>>> the end of the constant pool, >>>>>>>> 5686 // because a patched class may have a Utf8 name >>>>>>>> that's >>>>>>>> not already included in the >>>>>>>> 5687 // original constant pool. >>>>>>>> 5688 // >>>>>>>> 5689 // Note that a String in cp_patch_at(i) may be >>>>>>>> used to >>>>>>>> patch a Utf8, a String, or a Class. >>>>>>>> 5690 // At this point, we don't know the tag for index i >>>>>>>> yet, >>>>>>>> because we haven't parsed the >>>>>>>> 5691 // constant pool. So we can only assume the >>>>>>>> worst -- >>>>>>>> every String is used to patch a Class. >>>>>>>> 5692 _max_num_patched_klasses ++; >>>>>>>> >>>>>>>> Line 5684 checks for all objects in the cp_patch array. Later, >>>>>>>> when >>>>>>>> ClassFileParser::patch_constant_pool() is called, any objects >>>>>>>> that are >>>>>>>> either Class or String could be treated as a Klass: >>>>>>>> >>>>>>>> 724 void ClassFileParser::patch_constant_pool(ConstantPool* cp, >>>>>>>> 725 int index, >>>>>>>> 726 Handle patch, >>>>>>>> 727 TRAPS) { >>>>>>>> ... >>>>>>>> 732 switch (cp->tag_at(index).value()) { >>>>>>>> 733 >>>>>>>> 734 case JVM_CONSTANT_UnresolvedClass: { >>>>>>>> 735 // Patching a class means pre-resolving it. >>>>>>>> 736 // The name in the constant pool is ignored. >>>>>>>> 737 if (java_lang_Class::is_instance(patch())) { >>>>>>>> 738 guarantee_property(!java_lang_Class::is_primitive(patch()), >>>>>>>> 739 "Illegal class patch at %d in >>>>>>>> class >>>>>>>> file >>>>>>>> %s", >>>>>>>> 740 index, CHECK); >>>>>>>> 741 Klass* k = java_lang_Class::as_Klass(patch()); >>>>>>>> 742 patch_class(cp, index, k, k->name()); >>>>>>>> 743 } else { >>>>>>>> 744 guarantee_property(java_lang_String::is_instance(patch()), >>>>>>>> 745 "Illegal class patch at %d in >>>>>>>> class >>>>>>>> file >>>>>>>> %s", >>>>>>>> 746 index, CHECK); >>>>>>>> 747 Symbol* const name = >>>>>>>> java_lang_String::as_symbol(patch(), >>>>>>>> CHECK); >>>>>>>> 748 patch_class(cp, index, NULL, name); >>>>>>>> 749 } >>>>>>>> 750 break; >>>>>>>> 751 } >>>>>>>> >>>>>>>>> Could the is_anonymous() if statement be combined into the loop? >>>>>>>> >>>>>>>> I think the answer is no. At line 5365, there is no guarantee that >>>>>>>> ik->name() is in the cp_patch array. >>>>>>>> >>>>>>>> 5365 patch_class(ik->constants(), _this_class_index, ik, >>>>>>>> ik->name()); // eagerly resolve >>>>>>>> >>>>>>>>> Also why not do this calculation in the rewriter or is >>>>>>>>> that too >>>>>>>>> late? >>>>>>>>> >>>>>>>> Line 5676 and 5684 need to be executed BEFORE the constant pool >>>>>>>> and >>>>>>>> the >>>>>>>> associated tags array is allocated (both of which are fixed >>>>>>>> size, and >>>>>>>> cannot >>>>>>>> be expanded), which is way before the rewriter is run. At this >>>>>>>> point, >>>>>>>> we >>>>>>>> don't know what cp->tag_at(index) is (line #732), so the code >>>>>>>> needs to >>>>>>>> make >>>>>>>> a worst-case estimate on how long the CP/tags should be. >>>>>>>> >>>>>>>>> * line #5677, 5692 - a nit but I think the convention is to >>>>>>>>> not have >>>>>>>>> a >>>>>>>>> space after the variable name and between the post increment >>>>>>>>> operator. >>>>>>>>> >>>>>>>> Fixed. >>>>>>>>> src/share/vm/classfile/constantPool.hpp: >>>>>>>>> I understand the concept behind _invalid_resolved_klass_index, >>>>>>>>> but it >>>>>>>>> really is not so much invalid as temporary for class redefinition >>>>>>>>> purposes, >>>>>>>>> as you explain in ConstantPool::allocate_resolved_klasses. >>>>>>>>> Please >>>>>>>>> consider >>>>>>>>> renaming to _temp_unresolved_klass_index. And whether you >>>>>>>>> choose to >>>>>>>>> rename >>>>>>>>> the field or not, please add a one line comment ahead of >>>>>>>>> ConstantPool::temp_unresolved_klass_at_put that only class >>>>>>>>> redefinition uses >>>>>>>>> this currently. >>>>>>>>> >>>>>>>> Good idea. Will do. >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>>> Great change, thanks! >>>>>>>>> Lois >>>>>>>>> >>>>>>>>> On 4/13/2017 4:56 AM, Ioi Lam wrote: >>>>>>>>>> Hi Coleen, >>>>>>>>>> >>>>>>>>>> Thanks for the comments. Here's a delta from the last patch >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v02/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> In addition to your requests, I made these changes: >>>>>>>>>> >>>>>>>>>> [1] To consolidate the multiple extract_high/low code, I've >>>>>>>>>> added >>>>>>>>>> CPKlassSlot, so the code is cleaner: >>>>>>>>>> >>>>>>>>>> CPKlassSlot kslot = this_cp->klass_slot_at(which); >>>>>>>>>> int resolved_klass_index = kslot.resolved_klass_index(); >>>>>>>>>> int name_index = kslot.name_index(); >>>>>>>>>> >>>>>>>>>> [2] Renamed ConstantPool::is_shared_quick() to >>>>>>>>>> ConstantPool::is_shared(). The C++ compiler should be able to >>>>>>>>>> pick >>>>>>>>>> this >>>>>>>>>> function over MetaspaceObj::is_shared(). >>>>>>>>>> >>>>>>>>>> [3] Massaged the CDS region size set-up code a little to pass >>>>>>>>>> internal >>>>>>>>>> tests, because RO/RW ratio has changed. I didn't spend too >>>>>>>>>> much time >>>>>>>>>> picking >>>>>>>>>> the "right" sizes, as this code will be obsoleted soon with >>>>>>>>>> JDK-8072061 >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> On 4/13/17 6:40 AM, coleen.phillimore at oracle.com wrote: >>>>>>>>>>> >>>>>>>>>>> This looks really good! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/src/share/vm/oops/constantPool.cpp.udiff.html >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> + // Add one extra element to tags for storing >>>>>>>>>>> ConstantPool::flags(). >>>>>>>>>>> + Array* tags = >>>>>>>>>>> MetadataFactory::new_writeable_array(loader_data, >>>>>>>>>>> length+1, 0, >>>>>>>>>>> CHECK_NULL); ... + assert(tags->length()-1 == _length, >>>>>>>>>>> "invariant"); // >>>>>>>>>>> tags->at(_length) is flags() >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think this is left over, since _flags didn't get moved >>>>>>>>>>> after all. >>>>>>>>>>> >>>>>>>>>>> + Klass** adr = >>>>>>>>>>> this_cp->resolved_klasses()->adr_at(resolved_klass_index); >>>>>>>>>>> + OrderAccess::release_store_ptr((Klass* volatile *)adr, k); >>>>>>>>>>> + // The interpreter assumes when the tag is stored, the >>>>>>>>>>> klass is >>>>>>>>>>> resolved >>>>>>>>>>> + // and the Klass* is a klass rather than a Symbol*, so we >>>>>>>>>>> need >>>>>>>>>>> + // hardware store ordering here. >>>>>>>>>>> + this_cp->release_tag_at_put(which, JVM_CONSTANT_Class); >>>>>>>>>>> + return k; >>>>>>>>>>> >>>>>>>>>>> The comment still refers to the switch between Symbol* and >>>>>>>>>>> Klass* >>>>>>>>>>> in >>>>>>>>>>> the constant pool. The entry in the Klass array should be >>>>>>>>>>> NULL. >>>>>>>>>>> >>>>>>>>>>> + int name_index = >>>>>>>>>>> extract_high_short_from_int(*int_at_addr(which)); >>>>>>>>>>> >>>>>>>>>>> Can you put klass_name_index_at() in the constantPool.hpp >>>>>>>>>>> header >>>>>>>>>>> file >>>>>>>>>>> (so it's inlined) and have all the places where you get >>>>>>>>>>> name_index >>>>>>>>>>> use this >>>>>>>>>>> function? So we don't have to know in multiple places that >>>>>>>>>>> extract_high_short_from_int() is where the name index is. >>>>>>>>>>> And in >>>>>>>>>>> constantPool.hpp, for unresolved_klass_at_put() add a >>>>>>>>>>> comment about >>>>>>>>>>> what the >>>>>>>>>>> new format of the constant pool entry is {name_index, >>>>>>>>>>> resolved_klass_index}. >>>>>>>>>>> I'm happy to see this work nearing completion! The >>>>>>>>>>> "constant" pool >>>>>>>>>>> should >>>>>>>>>>> be constant! thanks, Coleen >>>>>>>>>>> On 4/11/17 2:26 AM, Ioi Lam wrote: >>>>>>>>>>>> Hi,please review the following change >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8171392 >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8171392_make_constantpool_read_only.v01/ >>>>>>>>>>>> >>>>>>>>>>>> *Summary:** * Before: + ConstantPool::klass_at(i) finds >>>>>>>>>>>> the >>>>>>>>>>>> Klass from >>>>>>>>>>>> the i-th slot of ConstantPool. + When a klass is resolved, the >>>>>>>>>>>> ConstantPool >>>>>>>>>>>> is modified to store the Klass pointer. After: + >>>>>>>>>>>> ConstantPool::klass_at(i) finds the at >>>>>>>>>>>> this->_resolved_klasses->at(i) + >>>>>>>>>>>> When a klass is resolved, _resolved_klasses->at(i) is >>>>>>>>>>>> modified. >>>>>>>>>>>> In >>>>>>>>>>>> addition: + I moved _resolved_references and >>>>>>>>>>>> _reference_map >>>>>>>>>>>> from >>>>>>>>>>>> ConstantPool to ConstantPoolCache. + Now _flags is no >>>>>>>>>>>> longer >>>>>>>>>>>> modified for shared ConstantPools. As a result, none of the >>>>>>>>>>>> fields in >>>>>>>>>>>> shared ConstantPools are modified at run time, so we can >>>>>>>>>>>> move them >>>>>>>>>>>> into the >>>>>>>>>>>> RO region in the CDS archive. *Testing:** * - Benchmark >>>>>>>>>>>> results >>>>>>>>>>>> show no >>>>>>>>>>>> performance regression, despite the extra level of >>>>>>>>>>>> indirection, >>>>>>>>>>>> which has a >>>>>>>>>>>> negligible overhead for the interpreter. - RBT testing >>>>>>>>>>>> for tier2 >>>>>>>>>>>> and >>>>>>>>>>>> tier3. *Ports:** * - I have tested only the Oracle-support >>>>>>>>>>>> ports. For the >>>>>>>>>>>> aarch64, ppc and s390 ports, I have added some code without >>>>>>>>>>>> testing (it's >>>>>>>>>>>> probably incomplete) - Port owners, please check if my patch >>>>>>>>>>>> work for you, >>>>>>>>>>>> and I can incorporate your changes in my push. >>>>>>>>>>>> Alternatively, you >>>>>>>>>>>> can wait >>>>>>>>>>>> for my push and provide fixes (if necessary) in a new >>>>>>>>>>>> changeset, >>>>>>>>>>>> and I will >>>>>>>>>>>> be happy to sponsor it. Thanks - Ioi >>>>>>>>>> >