From kim.barrett at oracle.com Thu Jan 19 05:31:01 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 19 Jan 2017 00:31:01 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles Message-ID: Please review this change to jweak to support G1. The problem being addressed is that, when using G1, dereferencing a jweak must ensure the obtained referent is ensured live, just as for other weak reference types. This has always been a requirement, and failure to do so appears to be a since-day1 G1 bug. There are two categories of places that need to address this issue: JNIHandles::resolve and friends, and returning a jobject from native code to Java. In both of these places the jobject whose contained oop is being extracted might really be a jweak. jweak is representationally indistinguishable from jobject; jweak is just a typedef for jobject. The documentation says a jweak can be used anywhere a jobject is permitted, making that type equivalence necessary for a C API. (A C++ API might be able to have distinct types via inheritance, though would still need a method for distinguishing them at runtime. But since JNI is a C API...). The basic approach being taken is to "tag" jweaks by setting the low order bit (recall that jweak == jobject == _jobject*), in order to distinguish them from non-jweak jobjects. This tagging is only being done when the selected GC needs the distinction, e.g. G1. This gives us a representational difference between jobject and jweak on which to base the different behavior, hiding that distinction behind the existing API. JNIHandles::resolve and friends are changed to unconditionally strip off any potential weak tag when translating from jobject to to oop*. That's cheaper than testing for the conditional use of tagging and then stripping. Also stripped when deleting JNI handles. TemplateInterpreterGenerator::generate_native_entry and SharedRuntime::generate_native_wrapper are changed to conditionally emit code to apply the G1 pre-barrier to the value obtained from a jobject result, when the result is tagged as a jweak, which only occurs when using G1. For arm32/arm64, this required moving the g1_write_barrier_pre definitions from InterpreterMacroAssembler to MacroAssembler. Also moved g1_write_barrier_post, for consistency. Note: I've not tested the aarch64 changes; I'll need someone with access to that platform to test. Note: I've not implemented this change for ppc or s390; I'll need someone else to deal with those platforms. (It's been a very long time since I wrote any ppc assembly code, and I've never seen an s390.) Note: I've done a minimal change for zero. Code elsewhere suggests zero doesn't support G1, and I followed that. And finally, JNIHandles::make_weak_global conditionally tags the result. CR: https://bugs.openjdk.java.net/browse/JDK-8166188 Webrev: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ In addition to the above webrev, I've also included separate webrevs for a test of this change, and for a whitebox extension needed for that test. The whitebox extension is my in-progress work on JDK-8169517, which is an enhancement targeted for JDK 10, and is not part of this change. Since the test depends on that whitebox extension, the test can't be included with the change either, and will be deferred to JDK 10. But I'm including webrevs for both here, for for reference by reviewers, and for testing on the platforms I don't have access to. http://cr.openjdk.java.net/~kbarrett/8166188/test.03/ http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ Testing: All hotspot jtreg tests run locally (Linux x86_64). In particular: - gc/TestReturnJNIWeak (new) - runtime/SameObject (uses NewWeakGlobalRef) Ad hoc hotspot nightly test on Oracle supported platforms. This includes some closed tests that use NewWeakGlobalRef. From mikael.gerdin at oracle.com Thu Jan 19 16:28:27 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 19 Jan 2017 17:28:27 +0100 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: <62da5dd7-c7c7-409f-7b22-018f6f5dd43c@oracle.com> Hi Kim, On 2017-01-19 06:31, Kim Barrett wrote: > Please review this change to jweak to support G1. > > The problem being addressed is that, when using G1, dereferencing a > jweak must ensure the obtained referent is ensured live, just as for > other weak reference types. This has always been a requirement, and > failure to do so appears to be a since-day1 G1 bug. > > There are two categories of places that need to address this issue: > JNIHandles::resolve and friends, and returning a jobject from native > code to Java. In both of these places the jobject whose contained oop > is being extracted might really be a jweak. jweak is > representationally indistinguishable from jobject; jweak is just a > typedef for jobject. The documentation says a jweak can be used > anywhere a jobject is permitted, making that type equivalence > necessary for a C API. (A C++ API might be able to have distinct > types via inheritance, though would still need a method for > distinguishing them at runtime. But since JNI is a C API...). > > The basic approach being taken is to "tag" jweaks by setting the low > order bit (recall that jweak == jobject == _jobject*), in order to > distinguish them from non-jweak jobjects. This tagging is only being > done when the selected GC needs the distinction, e.g. G1. This gives > us a representational difference between jobject and jweak on which to > base the different behavior, hiding that distinction behind the > existing API. > > JNIHandles::resolve and friends are changed to unconditionally strip > off any potential weak tag when translating from jobject to to oop*. > That's cheaper than testing for the conditional use of tagging and > then stripping. Also stripped when deleting JNI handles. > > TemplateInterpreterGenerator::generate_native_entry and > SharedRuntime::generate_native_wrapper are changed to conditionally > emit code to apply the G1 pre-barrier to the value obtained from a > jobject result, when the result is tagged as a jweak, which only > occurs when using G1. > > For arm32/arm64, this required moving the g1_write_barrier_pre > definitions from InterpreterMacroAssembler to MacroAssembler. Also > moved g1_write_barrier_post, for consistency. > > Note: I've not tested the aarch64 changes; I'll need someone with > access to that platform to test. > > Note: I've not implemented this change for ppc or s390; I'll need > someone else to deal with those platforms. (It's been a very long > time since I wrote any ppc assembly code, and I've never seen an > s390.) > > Note: I've done a minimal change for zero. Code elsewhere suggests > zero doesn't support G1, and I followed that. > > And finally, JNIHandles::make_weak_global conditionally tags the > result. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ I thought a bit about this issue today and unfortunately I've uncovered yet another location where a weak JNI handle needs to have its weak tag cleared. Consider the following JNI code snippet jobject o = <...>; jweak w = (*env)->NewWeakGlobalRef(env, o); jobject param = w; (*env)->CallStaticVoidMethod(env, c, id, param); jni_CallStaticVoidMethod creates a JNI_ArgumentPusherVaArg and passes that to jni_invoke_static in jni.cpp jni_invoke_static creates a JavaCallArguments object and passes that to the argument pusher and invokes args->iterate( Fingerprinter(method).fingerprint() ); iterate calls get_object() on the args pusher which is supposed to transfer the object parameter from the va_list in CallStaticVoidMethod to the JavaCallArguments object. The code to perform that transfer is this nasty piece of code: inline void get_object() { jobject l = va_arg(_ap, jobject); _arguments->push_oop(Handle((oop *)l, false)); } l here is our jweak and the code just does an unsafe cast to oop* and even fakes a VM Handle using a special constructor in the Handle class. In a debug build (or with -Xcheck:jni) this results in JavaCallArguments::verify catching this bad oop using a SignatureChekker::check_obj (sic) which uses this interesting code snippet to verify an oop: intptr_t v = _value[p]; if (v != 0 ) { size_t t = (size_t)v; bad = (t < (size_t)os::vm_page_size() ) || !Handle::raw_resolve((oop *)v)->is_oop_or_null(true); In a product build we will crash at some random point in the future. I've created a test to provoke this situation and put it at: http://cr.openjdk.java.net/~mgerdin/8166188-pass-arguments/webrev/ run the test with $ make test-hotspot-jtreg-native Sorry for not coming up with this sooner, I just realized this problem today. /Mikael > > In addition to the above webrev, I've also included separate webrevs > for a test of this change, and for a whitebox extension needed for > that test. The whitebox extension is my in-progress work on > JDK-8169517, which is an enhancement targeted for JDK 10, and is not > part of this change. Since the test depends on that whitebox > extension, the test can't be included with the change either, and will > be deferred to JDK 10. But I'm including webrevs for both here, for > for reference by reviewers, and for testing on the platforms I don't > have access to. > > http://cr.openjdk.java.net/~kbarrett/8166188/test.03/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ > > Testing: > All hotspot jtreg tests run locally (Linux x86_64). > In particular: > - gc/TestReturnJNIWeak (new) > - runtime/SameObject (uses NewWeakGlobalRef) > > Ad hoc hotspot nightly test on Oracle supported platforms. This > includes some closed tests that use NewWeakGlobalRef. > From kim.barrett at oracle.com Thu Jan 19 18:04:31 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 19 Jan 2017 13:04:31 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <62da5dd7-c7c7-409f-7b22-018f6f5dd43c@oracle.com> References: <62da5dd7-c7c7-409f-7b22-018f6f5dd43c@oracle.com> Message-ID: <29CF89D4-6FC9-4968-BC6B-F7598FB30116@oracle.com> > On Jan 19, 2017, at 11:28 AM, Mikael Gerdin wrote: > > Hi Kim, > > On 2017-01-19 06:31, Kim Barrett wrote: >> Please review this change to jweak to support G1. >> >> [?] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ > > I thought a bit about this issue today and unfortunately I've uncovered yet another location where a weak JNI handle needs to have its weak tag cleared. > Consider the following JNI code snippet > > [?] > I've created a test to provoke this situation and put it at: > http://cr.openjdk.java.net/~mgerdin/8166188-pass-arguments/webrev/ > run the test with > $ make test-hotspot-jtreg-native > > Sorry for not coming up with this sooner, I just realized this problem today. Well, I?m happy you found it before this went out the door. Despite looking for this sort of thing, I completely missed this one. I need to re-review casts to oop*. From martin.doerr at sap.com Mon Jan 23 18:46:42 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 23 Jan 2017 18:46:42 +0000 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <29CF89D4-6FC9-4968-BC6B-F7598FB30116@oracle.com> References: <62da5dd7-c7c7-409f-7b22-018f6f5dd43c@oracle.com> <29CF89D4-6FC9-4968-BC6B-F7598FB30116@oracle.com> Message-ID: <66867f1838714d17bf76324a513a568a@DEWDFE13DE02.global.corp.sap> Hi everybody, thanks for addressing this issue and for the webrev. While reading code, I was wondering if the result handler for oop types ever gets called (except on PPC64). Looks like it is only there to compare the pointer after the native call and do something special in this case instead of calling it. Wouldn't it be cleaner to use the result handler and take care of the G1 pre barrier there? Btw. PPC64 currently uses a different implementation which really calls the oop result handler. The native entry saves the handle to the frame's lresult field and sets the oop_temp to NULL. The oop result handler takes care of the unboxing. In addition frame::interpreter_frame_result should use JNIHandles::resolve to retrieve the oop (which is not yet implemented as it should be). Also, one has to make sure that the active handles don't get reset too early. This implementation doesn't force the oop to be live. GC may still set it to NULL if it decides to kill the jweak. I think PPC64 and the other platforms should do it the same way. The question is what's the better approach. Best regards, Martin -----Original Message----- From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett Sent: Donnerstag, 19. Januar 2017 19:05 To: Mikael Gerdin Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; hotspot-dev developers Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles > On Jan 19, 2017, at 11:28 AM, Mikael Gerdin wrote: > > Hi Kim, > > On 2017-01-19 06:31, Kim Barrett wrote: >> Please review this change to jweak to support G1. >> >> [...] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ > > I thought a bit about this issue today and unfortunately I've uncovered yet another location where a weak JNI handle needs to have its weak tag cleared. > Consider the following JNI code snippet > > [...] > I've created a test to provoke this situation and put it at: > http://cr.openjdk.java.net/~mgerdin/8166188-pass-arguments/webrev/ > run the test with > $ make test-hotspot-jtreg-native > > Sorry for not coming up with this sooner, I just realized this problem today. Well, I'm happy you found it before this went out the door. Despite looking for this sort of thing, I completely missed this one. I need to re-review casts to oop*. From per.liden at oracle.com Tue Jan 24 10:49:19 2017 From: per.liden at oracle.com (Per Liden) Date: Tue, 24 Jan 2017 11:49:19 +0100 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: <5ddce951-f85f-6633-d25b-5359aebe7114@oracle.com> Hi Kim, On 2017-01-19 06:31, Kim Barrett wrote: > Please review this change to jweak to support G1. > > The problem being addressed is that, when using G1, dereferencing a > jweak must ensure the obtained referent is ensured live, just as for > other weak reference types. This has always been a requirement, and > failure to do so appears to be a since-day1 G1 bug. > > There are two categories of places that need to address this issue: > JNIHandles::resolve and friends, and returning a jobject from native > code to Java. In both of these places the jobject whose contained oop > is being extracted might really be a jweak. jweak is > representationally indistinguishable from jobject; jweak is just a > typedef for jobject. The documentation says a jweak can be used > anywhere a jobject is permitted, making that type equivalence > necessary for a C API. (A C++ API might be able to have distinct > types via inheritance, though would still need a method for > distinguishing them at runtime. But since JNI is a C API...). > > The basic approach being taken is to "tag" jweaks by setting the low > order bit (recall that jweak == jobject == _jobject*), in order to > distinguish them from non-jweak jobjects. This tagging is only being > done when the selected GC needs the distinction, e.g. G1. This gives > us a representational difference between jobject and jweak on which to > base the different behavior, hiding that distinction behind the > existing API. I'd like to suggest that we always tag jweaks, regardless of which GC is used. I can't see any performance issues to be afraid of here, but I can see a number of advantages with a unified jweak representation/handling, like avoiding #ifdef INCLUDE_ALL_GCS noise and reducing the number of code paths to test/debug. cheers, Per > > JNIHandles::resolve and friends are changed to unconditionally strip > off any potential weak tag when translating from jobject to to oop*. > That's cheaper than testing for the conditional use of tagging and > then stripping. Also stripped when deleting JNI handles. > > TemplateInterpreterGenerator::generate_native_entry and > SharedRuntime::generate_native_wrapper are changed to conditionally > emit code to apply the G1 pre-barrier to the value obtained from a > jobject result, when the result is tagged as a jweak, which only > occurs when using G1. > > For arm32/arm64, this required moving the g1_write_barrier_pre > definitions from InterpreterMacroAssembler to MacroAssembler. Also > moved g1_write_barrier_post, for consistency. > > Note: I've not tested the aarch64 changes; I'll need someone with > access to that platform to test. > > Note: I've not implemented this change for ppc or s390; I'll need > someone else to deal with those platforms. (It's been a very long > time since I wrote any ppc assembly code, and I've never seen an > s390.) > > Note: I've done a minimal change for zero. Code elsewhere suggests > zero doesn't support G1, and I followed that. > > And finally, JNIHandles::make_weak_global conditionally tags the > result. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ > > In addition to the above webrev, I've also included separate webrevs > for a test of this change, and for a whitebox extension needed for > that test. The whitebox extension is my in-progress work on > JDK-8169517, which is an enhancement targeted for JDK 10, and is not > part of this change. Since the test depends on that whitebox > extension, the test can't be included with the change either, and will > be deferred to JDK 10. But I'm including webrevs for both here, for > for reference by reviewers, and for testing on the platforms I don't > have access to. > > http://cr.openjdk.java.net/~kbarrett/8166188/test.03/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ > > Testing: > All hotspot jtreg tests run locally (Linux x86_64). > In particular: > - gc/TestReturnJNIWeak (new) > - runtime/SameObject (uses NewWeakGlobalRef) > > Ad hoc hotspot nightly test on Oracle supported platforms. This > includes some closed tests that use NewWeakGlobalRef. > From mikael.gerdin at oracle.com Tue Jan 24 14:50:04 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 24 Jan 2017 15:50:04 +0100 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <66867f1838714d17bf76324a513a568a@DEWDFE13DE02.global.corp.sap> References: <62da5dd7-c7c7-409f-7b22-018f6f5dd43c@oracle.com> <29CF89D4-6FC9-4968-BC6B-F7598FB30116@oracle.com> <66867f1838714d17bf76324a513a568a@DEWDFE13DE02.global.corp.sap> Message-ID: <38bbbce8-9bcb-8f4a-cd94-fd14d6a08b4d@oracle.com> Hi Martin, Thanks for taking a look at this change. On 2017-01-23 19:46, Doerr, Martin wrote: > Hi everybody, > > thanks for addressing this issue and for the webrev. > > While reading code, I was wondering if the result handler for oop types ever gets called (except on PPC64). > Looks like it is only there to compare the pointer after the native call and do something special in this case instead of calling it. > > Wouldn't it be cleaner to use the result handler and take care of the G1 pre barrier there? Can't say I know much about the result handlers in general but I get the feeling that they are only there for the interpreter. Are the result handlers applied to the return values of compiled native wrappers as well? Thanks /Mikael > > Btw. PPC64 currently uses a different implementation which really calls the oop result handler. The native entry saves the handle to the frame's lresult field and sets the oop_temp to NULL. The oop result handler takes care of the unboxing. In addition frame::interpreter_frame_result should use JNIHandles::resolve to retrieve the oop (which is not yet implemented as it should be). Also, one has to make sure that the active handles don't get reset too early. > This implementation doesn't force the oop to be live. GC may still set it to NULL if it decides to kill the jweak. > > I think PPC64 and the other platforms should do it the same way. The question is what's the better approach. > > Best regards, > Martin > > > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > Sent: Donnerstag, 19. Januar 2017 19:05 > To: Mikael Gerdin > Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; hotspot-dev developers > Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles > >> On Jan 19, 2017, at 11:28 AM, Mikael Gerdin wrote: >> >> Hi Kim, >> >> On 2017-01-19 06:31, Kim Barrett wrote: >>> Please review this change to jweak to support G1. >>> >>> [...] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ >> >> I thought a bit about this issue today and unfortunately I've uncovered yet another location where a weak JNI handle needs to have its weak tag cleared. >> Consider the following JNI code snippet >> >> [...] >> I've created a test to provoke this situation and put it at: >> http://cr.openjdk.java.net/~mgerdin/8166188-pass-arguments/webrev/ >> run the test with >> $ make test-hotspot-jtreg-native >> >> Sorry for not coming up with this sooner, I just realized this problem today. > > Well, I'm happy you found it before this went out the door. Despite looking for this sort of thing, I completely missed this one. I need to re-review casts to oop*. > From martin.doerr at sap.com Tue Jan 24 17:18:56 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 24 Jan 2017 17:18:56 +0000 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <38bbbce8-9bcb-8f4a-cd94-fd14d6a08b4d@oracle.com> References: <62da5dd7-c7c7-409f-7b22-018f6f5dd43c@oracle.com> <29CF89D4-6FC9-4968-BC6B-F7598FB30116@oracle.com> <66867f1838714d17bf76324a513a568a@DEWDFE13DE02.global.corp.sap> <38bbbce8-9bcb-8f4a-cd94-fd14d6a08b4d@oracle.com> Message-ID: Hi Mikael, I found out how the oop result processing works on x86. I was kind of confused about it, yesterday. The oop case just has an additional piece of code which forces the oop to be alive and visible to GC by copying it from the (possibly weak) handle to the oop result field. Later, the result handler reloads it from there. The current PPC64 implementation doesn't need the additional piece of code, because the oop lives in the (possibly weak) handle until the result handler loads it from there. We need to decide if we keep the PPC64 implementation or not. Not sure if it's worth porting this diff to other platforms. Best regards, Martin -----Original Message----- From: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] Sent: Dienstag, 24. Januar 2017 15:50 To: Doerr, Martin ; Kim Barrett Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; hotspot-dev developers Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles Hi Martin, Thanks for taking a look at this change. On 2017-01-23 19:46, Doerr, Martin wrote: > Hi everybody, > > thanks for addressing this issue and for the webrev. > > While reading code, I was wondering if the result handler for oop types ever gets called (except on PPC64). > Looks like it is only there to compare the pointer after the native call and do something special in this case instead of calling it. > > Wouldn't it be cleaner to use the result handler and take care of the G1 pre barrier there? Can't say I know much about the result handlers in general but I get the feeling that they are only there for the interpreter. Are the result handlers applied to the return values of compiled native wrappers as well? Thanks /Mikael > > Btw. PPC64 currently uses a different implementation which really calls the oop result handler. The native entry saves the handle to the frame's lresult field and sets the oop_temp to NULL. The oop result handler takes care of the unboxing. In addition frame::interpreter_frame_result should use JNIHandles::resolve to retrieve the oop (which is not yet implemented as it should be). Also, one has to make sure that the active handles don't get reset too early. > This implementation doesn't force the oop to be live. GC may still set it to NULL if it decides to kill the jweak. > > I think PPC64 and the other platforms should do it the same way. The question is what's the better approach. > > Best regards, > Martin > > > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] > On Behalf Of Kim Barrett > Sent: Donnerstag, 19. Januar 2017 19:05 > To: Mikael Gerdin > Cc: s390x-port-dev at openjdk.java.net; > ppc-aix-port-dev at openjdk.java.net; hotspot-dev developers > > Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak > JNI handles > >> On Jan 19, 2017, at 11:28 AM, Mikael Gerdin wrote: >> >> Hi Kim, >> >> On 2017-01-19 06:31, Kim Barrett wrote: >>> Please review this change to jweak to support G1. >>> >>> [...] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ >> >> I thought a bit about this issue today and unfortunately I've uncovered yet another location where a weak JNI handle needs to have its weak tag cleared. >> Consider the following JNI code snippet >> >> [...] >> I've created a test to provoke this situation and put it at: >> http://cr.openjdk.java.net/~mgerdin/8166188-pass-arguments/webrev/ >> run the test with >> $ make test-hotspot-jtreg-native >> >> Sorry for not coming up with this sooner, I just realized this problem today. > > Well, I'm happy you found it before this went out the door. Despite looking for this sort of thing, I completely missed this one. I need to re-review casts to oop*. > From martin.doerr at sap.com Fri Jan 27 16:44:02 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 27 Jan 2017 16:44:02 +0000 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <62da5dd7-c7c7-409f-7b22-018f6f5dd43c@oracle.com> <29CF89D4-6FC9-4968-BC6B-F7598FB30116@oracle.com> <66867f1838714d17bf76324a513a568a@DEWDFE13DE02.global.corp.sap> <38bbbce8-9bcb-8f4a-cd94-fd14d6a08b4d@oracle.com> Message-ID: Hi Kim, Mikael and all, we have implemented the G1 barriers for PPC64 and s390 in this webrev: http://cr.openjdk.java.net/~mdoerr/8166188_s390_ppc64/webrev.00/ The change just adds the barrier code. s390's template interpreter looks pretty much like Oracle's platforms, while PPC64's one loads the oop from the handle inside of the result handler. I think making platform implementations more similar is rather a topic for JDK10. Thanks and best regards, Martin -----Original Message----- From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Doerr, Martin Sent: Dienstag, 24. Januar 2017 18:19 To: Mikael Gerdin ; Kim Barrett Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; hotspot-dev developers Subject: RE: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles Hi Mikael, I found out how the oop result processing works on x86. I was kind of confused about it, yesterday. The oop case just has an additional piece of code which forces the oop to be alive and visible to GC by copying it from the (possibly weak) handle to the oop result field. Later, the result handler reloads it from there. The current PPC64 implementation doesn't need the additional piece of code, because the oop lives in the (possibly weak) handle until the result handler loads it from there. We need to decide if we keep the PPC64 implementation or not. Not sure if it's worth porting this diff to other platforms. Best regards, Martin -----Original Message----- From: Mikael Gerdin [mailto:mikael.gerdin at oracle.com] Sent: Dienstag, 24. Januar 2017 15:50 To: Doerr, Martin ; Kim Barrett Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; hotspot-dev developers Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles Hi Martin, Thanks for taking a look at this change. On 2017-01-23 19:46, Doerr, Martin wrote: > Hi everybody, > > thanks for addressing this issue and for the webrev. > > While reading code, I was wondering if the result handler for oop types ever gets called (except on PPC64). > Looks like it is only there to compare the pointer after the native call and do something special in this case instead of calling it. > > Wouldn't it be cleaner to use the result handler and take care of the G1 pre barrier there? Can't say I know much about the result handlers in general but I get the feeling that they are only there for the interpreter. Are the result handlers applied to the return values of compiled native wrappers as well? Thanks /Mikael > > Btw. PPC64 currently uses a different implementation which really calls the oop result handler. The native entry saves the handle to the frame's lresult field and sets the oop_temp to NULL. The oop result handler takes care of the unboxing. In addition frame::interpreter_frame_result should use JNIHandles::resolve to retrieve the oop (which is not yet implemented as it should be). Also, one has to make sure that the active handles don't get reset too early. > This implementation doesn't force the oop to be live. GC may still set it to NULL if it decides to kill the jweak. > > I think PPC64 and the other platforms should do it the same way. The question is what's the better approach. > > Best regards, > Martin > > > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] > On Behalf Of Kim Barrett > Sent: Donnerstag, 19. Januar 2017 19:05 > To: Mikael Gerdin > Cc: s390x-port-dev at openjdk.java.net; > ppc-aix-port-dev at openjdk.java.net; hotspot-dev developers > > Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak > JNI handles > >> On Jan 19, 2017, at 11:28 AM, Mikael Gerdin wrote: >> >> Hi Kim, >> >> On 2017-01-19 06:31, Kim Barrett wrote: >>> Please review this change to jweak to support G1. >>> >>> [...] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ >> >> I thought a bit about this issue today and unfortunately I've uncovered yet another location where a weak JNI handle needs to have its weak tag cleared. >> Consider the following JNI code snippet >> >> [...] >> I've created a test to provoke this situation and put it at: >> http://cr.openjdk.java.net/~mgerdin/8166188-pass-arguments/webrev/ >> run the test with >> $ make test-hotspot-jtreg-native >> >> Sorry for not coming up with this sooner, I just realized this problem today. > > Well, I'm happy you found it before this went out the door. Despite looking for this sort of thing, I completely missed this one. I need to re-review casts to oop*. > From aph at redhat.com Fri Jan 27 18:00:43 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 27 Jan 2017 18:00:43 +0000 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: <9eef4368-6e2c-461a-63ac-69a42f467218@redhat.com> On 19/01/17 05:31, Kim Barrett wrote: > Please review this change to jweak to support G1. > > The problem being addressed is that, when using G1, dereferencing a > jweak must ensure the obtained referent is ensured live, just as for > other weak reference types. This has always been a requirement, and > failure to do so appears to be a since-day1 G1 bug. > > There are two categories of places that need to address this issue: > JNIHandles::resolve and friends, and returning a jobject from native > code to Java. In both of these places the jobject whose contained oop > is being extracted might really be a jweak. jweak is > representationally indistinguishable from jobject; jweak is just a > typedef for jobject. The documentation says a jweak can be used > anywhere a jobject is permitted, making that type equivalence > necessary for a C API. (A C++ API might be able to have distinct > types via inheritance, though would still need a method for > distinguishing them at runtime. But since JNI is a C API...). > > The basic approach being taken is to "tag" jweaks by setting the low > order bit (recall that jweak == jobject == _jobject*), in order to > distinguish them from non-jweak jobjects. This tagging is only being > done when the selected GC needs the distinction, e.g. G1. This gives > us a representational difference between jobject and jweak on which to > base the different behavior, hiding that distinction behind the > existing API. > > JNIHandles::resolve and friends are changed to unconditionally strip > off any potential weak tag when translating from jobject to to oop*. > That's cheaper than testing for the conditional use of tagging and > then stripping. Also stripped when deleting JNI handles. > > TemplateInterpreterGenerator::generate_native_entry and > SharedRuntime::generate_native_wrapper are changed to conditionally > emit code to apply the G1 pre-barrier to the value obtained from a > jobject result, when the result is tagged as a jweak, which only > occurs when using G1. > > For arm32/arm64, this required moving the g1_write_barrier_pre > definitions from InterpreterMacroAssembler to MacroAssembler. Also > moved g1_write_barrier_post, for consistency. > > Note: I've not tested the aarch64 changes; I'll need someone with > access to that platform to test. > > Note: I've not implemented this change for ppc or s390; I'll need > someone else to deal with those platforms. (It's been a very long > time since I wrote any ppc assembly code, and I've never seen an > s390.) > > Note: I've done a minimal change for zero. Code elsewhere suggests > zero doesn't support G1, and I followed that. > > And finally, JNIHandles::make_weak_global conditionally tags the > result. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.02/ > > In addition to the above webrev, I've also included separate webrevs > for a test of this change, and for a whitebox extension needed for > that test. The whitebox extension is my in-progress work on > JDK-8169517, which is an enhancement targeted for JDK 10, and is not > part of this change. Since the test depends on that whitebox > extension, the test can't be included with the change either, and will > be deferred to JDK 10. But I'm including webrevs for both here, for > for reference by reviewers, and for testing on the platforms I don't > have access to. > > http://cr.openjdk.java.net/~kbarrett/8166188/test.03/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ > > Testing: > All hotspot jtreg tests run locally (Linux x86_64). > In particular: > - gc/TestReturnJNIWeak (new) > - runtime/SameObject (uses NewWeakGlobalRef) > > Ad hoc hotspot nightly test on Oracle supported platforms. This > includes some closed tests that use NewWeakGlobalRef. > Looks OK except for a small typo. I've done very light testing, but this edge case is very hard to test anyway. Missing semicolon: @@ -1403,10 +1403,33 @@ __ adr(t, ExternalAddress(AbstractInterpreter::result_handler(T_OBJECT))); __ cmp(t, result_handler); __ br(Assembler::NE, no_oop); - // retrieve result + // Unbox oop result, e.g. JNIHandles::resolve result. __ pop(ltos); __ cbz(r0, store_result); - __ ldr(r0, Address(r0, 0)); +#if !INCLUDE_ALL_GCS + assert(!JNIHandles::need_tagged_jweaks(), "Unexpected configuration"); +#else + if (JNIHandles::need_tagged_jweaks()) { + Label not_weak; + STATIC_ASSERT(JNIHandles::weak_tag_mask == 1u); + __ tbz(r0, 0, not_weak); // Test for weak tag. + // Load from tagged jweak. + __ ldr(r0, Address(r0, -JNIHandles::weak_tag_value)); + // Generate the G1 pre-barrier code to log the jweak's value. + assert(UseG1GC, "Unsupported use of tagged jweaks"); + __ enter(); // Barrier may call runtime. + __ g1_write_barrier_pre(noreg /* obj */, + r0 /* pre_val */, + rthread /* thread */, + t /* tmp */, + true /* tosca_live */, + true /* expand_call */); + __ leave(); + __ b(store_result); + __ bind(not_weak); + } +#endif // INCLUDE_ALL_GCS + __ ldr(r0, Address(r0, 0)) // Get oop from box. ^ HERE __ bind(store_result); __ str(r0, Address(rfp, frame::interpreter_frame_oop_temp_offset*wordSize)); // keep stack depth as expected by pushing oop which will eventually be discarded