From aeubanks at google.com  Tue Jun  4 00:42:27 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 3 Jun 2019 17:42:27 -0700
Subject: [RFR] Cleanup and add lots of new tests
Message-ID: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>

webrev:
http://cr.openjdk.java.net/~aeubanks/tsanlotsoftests/webrev.00/index.html

Cleanup tests and add lots of new tests

No need for both a jtreg @summary and a class description.
Add tests that test all combinations of racy/non-racy, array/field,
bool/byte/char/double/float/int/long/short/String.
Add tests that test that synchronized blocks, and synchronized blocks with
an exception are properly handled.
Add test that static initializer block is not considered racy.
Use System.out.println() instead of System.out.format() so that we don't
have to specify the right format string.

From jcbeyler at google.com  Wed Jun  5 04:25:05 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Tue, 4 Jun 2019 21:25:05 -0700
Subject: [RFR] Cleanup and add lots of new tests
In-Reply-To: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>
References: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>
Message-ID: <CAF9BGBz=OCJaeiCS7Ezu3-FO5DjJxio45wAnF=H-yXV+ObKbTA@mail.gmail.com>

Hi Arthur,

Looks good to me; by curiosity, why do we allocate two elements for the
arrays?

Thanks,
Jc

On Mon, Jun 3, 2019 at 5:43 PM Arthur Eubanks <aeubanks at google.com> wrote:

> webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanlotsoftests/webrev.00/index.html
>
> Cleanup tests and add lots of new tests
>
> No need for both a jtreg @summary and a class description.
> Add tests that test all combinations of racy/non-racy, array/field,
> bool/byte/char/double/float/int/long/short/String.
> Add tests that test that synchronized blocks, and synchronized blocks with
> an exception are properly handled.
> Add test that static initializer block is not considered racy.
> Use System.out.println() instead of System.out.format() so that we don't
> have to specify the right format string.
>


-- 

Thanks,
Jc

From aeubanks at google.com  Wed Jun  5 16:02:08 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 5 Jun 2019 09:02:08 -0700
Subject: [RFR] Cleanup and add lots of new tests
In-Reply-To: <CAF9BGBz=OCJaeiCS7Ezu3-FO5DjJxio45wAnF=H-yXV+ObKbTA@mail.gmail.com>
References: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>
 <CAF9BGBz=OCJaeiCS7Ezu3-FO5DjJxio45wAnF=H-yXV+ObKbTA@mail.gmail.com>
Message-ID: <CAPW48spMf5hO9qm-eUc-mSLpAPfba69Md=P2n37-AD_sDjXW3w@mail.gmail.com>

>
> Looks good to me; by curiosity, why do we allocate two elements for the
> arrays?
>
I thought it'd be good if there was one element we accessed and one we
didn't. It almost certainly doesn't add any test coverage, but I kind of
liked that it was a little more complex than just a one element array.

From jcbeyler at google.com  Wed Jun  5 16:56:31 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Wed, 5 Jun 2019 09:56:31 -0700
Subject: [RFR] Cleanup and add lots of new tests
In-Reply-To: <CAPW48spMf5hO9qm-eUc-mSLpAPfba69Md=P2n37-AD_sDjXW3w@mail.gmail.com>
References: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>
 <CAF9BGBz=OCJaeiCS7Ezu3-FO5DjJxio45wAnF=H-yXV+ObKbTA@mail.gmail.com>
 <CAPW48spMf5hO9qm-eUc-mSLpAPfba69Md=P2n37-AD_sDjXW3w@mail.gmail.com>
Message-ID: <CAF9BGBxnHH9cCT_h8zUizh3CTy5p279NQec4xTBxkYw2GrxMiw@mail.gmail.com>

Fair enough. I think it adds really nothing from a TSAN point of view and I
don't think it changes much. The only difference I really see is that the
second element of the array is initialized by the main thread but no one
touches that element.

I let you decide if you want to leave it; I think it does not really make
it more complex from a TSAN point of view :)

LGTM either way and I don't need a webrev even if you do update it,
Jc

On Wed, Jun 5, 2019 at 9:02 AM Arthur Eubanks <aeubanks at google.com> wrote:

> Looks good to me; by curiosity, why do we allocate two elements for the
>> arrays?
>>
> I thought it'd be good if there was one element we accessed and one we
> didn't. It almost certainly doesn't add any test coverage, but I kind of
> liked that it was a little more complex than just a one element array.
>


-- 

Thanks,
Jc

From manc at google.com  Wed Jun  5 21:23:21 2019
From: manc at google.com (Man Cao)
Date: Wed, 5 Jun 2019 14:23:21 -0700
Subject: [RFR] Cleanup and add lots of new tests
In-Reply-To: <CAF9BGBxnHH9cCT_h8zUizh3CTy5p279NQec4xTBxkYw2GrxMiw@mail.gmail.com>
References: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>
 <CAF9BGBz=OCJaeiCS7Ezu3-FO5DjJxio45wAnF=H-yXV+ObKbTA@mail.gmail.com>
 <CAPW48spMf5hO9qm-eUc-mSLpAPfba69Md=P2n37-AD_sDjXW3w@mail.gmail.com>
 <CAF9BGBxnHH9cCT_h8zUizh3CTy5p279NQec4xTBxkYw2GrxMiw@mail.gmail.com>
Message-ID: <CA+w6HxZV-ndNhqZyOPS6knuu_fRc9cHMZb4fCmX689QB5xnH_g@mail.gmail.com>

Thanks for converting these tests!

For NonRacyStaticInitLoopTest.java, can we create a separate thread to read
from the static field? I think currently there is only one main thread that
initialize and read from the static field.

In NonRacySyncBlockExceptionLoopTest.java and
NonRacySyncBlockLoopTest.java, the @summary line could be more descriptive.

In NonRacySyncBlockLoopTest.java:
protected synchronized void run(int i) {
  synchronized (this) {
The run() method does not need to be synchronized, right?

For next code version, no need for a webrev, just inline an updated
NonRacyStaticInitLoopRunner
class in email would work.

-Man


On Wed, Jun 5, 2019 at 9:57 AM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Fair enough. I think it adds really nothing from a TSAN point of view and I
> don't think it changes much. The only difference I really see is that the
> second element of the array is initialized by the main thread but no one
> touches that element.
>
> I let you decide if you want to leave it; I think it does not really make
> it more complex from a TSAN point of view :)
>
> LGTM either way and I don't need a webrev even if you do update it,
> Jc
>
> On Wed, Jun 5, 2019 at 9:02 AM Arthur Eubanks <aeubanks at google.com> wrote:
>
> > Looks good to me; by curiosity, why do we allocate two elements for the
> >> arrays?
> >>
> > I thought it'd be good if there was one element we accessed and one we
> > didn't. It almost certainly doesn't add any test coverage, but I kind of
> > liked that it was a little more complex than just a one element array.
> >
>
>
> --
>
> Thanks,
> Jc
>

From aeubanks at google.com  Mon Jun 10 17:02:06 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 10 Jun 2019 10:02:06 -0700
Subject: [RFR] Cleanup and add lots of new tests
In-Reply-To: <CA+w6HxZV-ndNhqZyOPS6knuu_fRc9cHMZb4fCmX689QB5xnH_g@mail.gmail.com>
References: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>
 <CAF9BGBz=OCJaeiCS7Ezu3-FO5DjJxio45wAnF=H-yXV+ObKbTA@mail.gmail.com>
 <CAPW48spMf5hO9qm-eUc-mSLpAPfba69Md=P2n37-AD_sDjXW3w@mail.gmail.com>
 <CAF9BGBxnHH9cCT_h8zUizh3CTy5p279NQec4xTBxkYw2GrxMiw@mail.gmail.com>
 <CA+w6HxZV-ndNhqZyOPS6knuu_fRc9cHMZb4fCmX689QB5xnH_g@mail.gmail.com>
Message-ID: <CAPW48soP+En41cz+sYf95rGL_uRYo0+eW0zYU-8okZxzpFiF_Q@mail.gmail.com>

On Wed, Jun 5, 2019 at 2:23 PM Man Cao <manc at google.com> wrote:

> Thanks for converting these tests!
>
> For NonRacyStaticInitLoopTest.java, can we create a separate thread to
> read from the static field? I think currently there is only one main thread
> that initialize and read from the static field.
>
Does this look good?
class NonRacyStaticInitLoopRunner {
  private static int x;

  static {
    x = 5;
  }

  public static void main(String[] args) throws InterruptedException {
    Thread t =
        new Thread(
            () -> {
              x = 2;
            });
    t.start();
    t.join();
  }
}
>
>
> In NonRacySyncBlockExceptionLoopTest.java and
> NonRacySyncBlockLoopTest.java, the @summary line could be more descriptive.
>
Done

>
> In NonRacySyncBlockLoopTest.java:
> protected synchronized void run(int i) {
>   synchronized (this) {
> The run() method does not need to be synchronized, right?
>
Done

>
> For next code version, no need for a webrev, just inline an updated NonRacyStaticInitLoopRunner
> class in email would work.
>
> -Man
>

From manc at google.com  Mon Jun 10 17:27:57 2019
From: manc at google.com (Man Cao)
Date: Mon, 10 Jun 2019 10:27:57 -0700
Subject: [RFR] Cleanup and add lots of new tests
In-Reply-To: <CAPW48soP+En41cz+sYf95rGL_uRYo0+eW0zYU-8okZxzpFiF_Q@mail.gmail.com>
References: <CAPW48spDoQb51aTCCwoRLBz-QBXNgJ4umPmw8_LxNOua1maJtw@mail.gmail.com>
 <CAF9BGBz=OCJaeiCS7Ezu3-FO5DjJxio45wAnF=H-yXV+ObKbTA@mail.gmail.com>
 <CAPW48spMf5hO9qm-eUc-mSLpAPfba69Md=P2n37-AD_sDjXW3w@mail.gmail.com>
 <CAF9BGBxnHH9cCT_h8zUizh3CTy5p279NQec4xTBxkYw2GrxMiw@mail.gmail.com>
 <CA+w6HxZV-ndNhqZyOPS6knuu_fRc9cHMZb4fCmX689QB5xnH_g@mail.gmail.com>
 <CAPW48soP+En41cz+sYf95rGL_uRYo0+eW0zYU-8okZxzpFiF_Q@mail.gmail.com>
Message-ID: <CA+w6HxYTx7aoUyEbJ9gMpWKGYmk9T5_jBcteb9EJceU1h1Z3+w@mail.gmail.com>

Looks good.

-Man


On Mon, Jun 10, 2019 at 10:02 AM Arthur Eubanks <aeubanks at google.com> wrote:

>
>
> On Wed, Jun 5, 2019 at 2:23 PM Man Cao <manc at google.com> wrote:
>
>> Thanks for converting these tests!
>>
>> For NonRacyStaticInitLoopTest.java, can we create a separate thread to
>> read from the static field? I think currently there is only one main thread
>> that initialize and read from the static field.
>>
> Does this look good?
> class NonRacyStaticInitLoopRunner {
>   private static int x;
>
>   static {
>     x = 5;
>   }
>
>   public static void main(String[] args) throws InterruptedException {
>     Thread t =
>         new Thread(
>             () -> {
>               x = 2;
>             });
>     t.start();
>     t.join();
>   }
> }
>>
>>
>> In NonRacySyncBlockExceptionLoopTest.java and
>> NonRacySyncBlockLoopTest.java, the @summary line could be more descriptive.
>>
> Done
>
>>
>> In NonRacySyncBlockLoopTest.java:
>> protected synchronized void run(int i) {
>>   synchronized (this) {
>> The run() method does not need to be synchronized, right?
>>
> Done
>
>>
>> For next code version, no need for a webrev, just inline an updated NonRacyStaticInitLoopRunner
>> class in email would work.
>>
>> -Man
>>
>

From aeubanks at google.com  Mon Jun 10 17:48:26 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Mon, 10 Jun 2019 17:48:26 +0000
Subject: hg: tsan/dev: Cleanup tests and add lots of new tests
Message-ID: <201906101748.x5AHmQrs028446@aojmv0008.oracle.com>

Changeset: 7d807da8698c
Author:    aeubanks
Date:      2019-06-03 17:34 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/7d807da8698c

Cleanup tests and add lots of new tests

No need for both a jtreg @summary and a class description.
Add tests that test all combinations of racy/non-racy, array/field, bool/byte/char/double/float/int/long/short/String.
Add tests that test that synchronized blocks, and synchronized blocks with an exception are properly handled.
Add test that static initializer block is not considered racy.
Use System.out.println() instead of System.out.format() so that we don't have to specify the right format string.

! test/hotspot/jtreg/tsan/AbstractLoop.java
! test/hotspot/jtreg/tsan/JvmtiTaggerTest.java
+ test/hotspot/jtreg/tsan/NonRacyBooleanArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyBooleanMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyByteArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyByteMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyCharArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyCharMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyDoubleArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyDoubleMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyFloatArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyFloatMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyIntArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyLongArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyLongMemberLoopTest.java
! test/hotspot/jtreg/tsan/NonRacyNativeLoopNativeSyncTest.java
! test/hotspot/jtreg/tsan/NonRacyNativeLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyShortArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyShortMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyStaticInitLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyStringArrayLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacyStringMemberLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacySyncBlockExceptionLoopTest.java
+ test/hotspot/jtreg/tsan/NonRacySyncBlockLoopTest.java
! test/hotspot/jtreg/tsan/NonRawRacyNativeLoopTest.java
+ test/hotspot/jtreg/tsan/RacyBooleanArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyBooleanMemberLoopTest.java
+ test/hotspot/jtreg/tsan/RacyByteArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyByteMemberLoopTest.java
+ test/hotspot/jtreg/tsan/RacyCharArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyCharMemberLoopTest.java
+ test/hotspot/jtreg/tsan/RacyDoubleArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyDoubleMemberLoopTest.java
+ test/hotspot/jtreg/tsan/RacyFloatArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyFloatMemberLoopTest.java
+ test/hotspot/jtreg/tsan/RacyIntArrayLoopTest.java
! test/hotspot/jtreg/tsan/RacyIntMemberLoopTest.java
! test/hotspot/jtreg/tsan/RacyIntMemberNoJavaMemLoopTest.java
+ test/hotspot/jtreg/tsan/RacyLongArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyLongMemberLoopTest.java
! test/hotspot/jtreg/tsan/RacyNativeLoopTest.java
+ test/hotspot/jtreg/tsan/RacyShortArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyShortMemberLoopTest.java
+ test/hotspot/jtreg/tsan/RacyStringArrayLoopTest.java
+ test/hotspot/jtreg/tsan/RacyStringMemberLoopTest.java
! test/hotspot/jtreg/tsan/RawRacyNativeLoopTest.java


From aeubanks at google.com  Tue Jun 11 00:36:27 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 10 Jun 2019 17:36:27 -0700
Subject: [RFR] Ignore fields annotated with @LazyInit
Message-ID: <CAPW48sqNeEihaT=Ko1i1GDEp7cs_1ieofV2yARgwaFBiArhpBw@mail.gmail.com>

webrev:
http://cr.openjdk.java.net/~aeubanks/tsanfieldignore/webrev.00/index.html

This change introduces a new annotation,
java.util.concurrent.annotation.LazyInit, which causes TSAN to ignore races
on that field.
References are not simply ignored, but a release/acquire is performed on
them. This is so that any following accesses to its member variables are
not also reported as racy.

LazyInitReferenceLoopTest.java is still failing with weird stack trace
issues, but I'd like some feedback on this change anyway.

From manc at google.com  Wed Jun 12 23:22:56 2019
From: manc at google.com (Man Cao)
Date: Wed, 12 Jun 2019 16:22:56 -0700
Subject: [RFR] Ignore fields annotated with @LazyInit
In-Reply-To: <CAPW48sqNeEihaT=Ko1i1GDEp7cs_1ieofV2yARgwaFBiArhpBw@mail.gmail.com>
References: <CAPW48sqNeEihaT=Ko1i1GDEp7cs_1ieofV2yARgwaFBiArhpBw@mail.gmail.com>
Message-ID: <CA+w6Hxa-3G-r7n+xE+U=ibCC_Zz2Q51wDUDUUqq28GiUYxc2Yg@mail.gmail.com>

Looks good. One minor comment is the VM symbol
java_util_concurrent_atomic_annotation_LazyInit, can we remove "atomic_"
from the name?
No need for a new webrev for this change.

-Man


On Mon, Jun 10, 2019 at 5:37 PM Arthur Eubanks <aeubanks at google.com> wrote:

> webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanfieldignore/webrev.00/index.html
>
> This change introduces a new annotation,
> java.util.concurrent.annotation.LazyInit, which causes TSAN to ignore races
> on that field.
> References are not simply ignored, but a release/acquire is performed on
> them. This is so that any following accesses to its member variables are
> not also reported as racy.
>
> LazyInitReferenceLoopTest.java is still failing with weird stack trace
> issues, but I'd like some feedback on this change anyway.
>

From aeubanks at google.com  Fri Jun 21 23:35:04 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Fri, 21 Jun 2019 16:35:04 -0700
Subject: [RFR]: Disable CDS when TSAN is on
Message-ID: <CAPW48sooA--+CzKfTKzmtLdLwERVBCwcjSjcNNamTP3Ar5UPJw@mail.gmail.com>

Webrev: http://cr.openjdk.java.net/~aeubanks/tsancds/webrev.00/index.html

Disable CDS when TSAN is on

We eagerly allocate jmethodIDs in ClassFileParser::fill_instance_klass().
ClassFileParser::fill_instance_klass() is never called on classes in a
shared archive.

This helps with an upcoming test in a future change.

From manc at google.com  Fri Jun 21 23:40:06 2019
From: manc at google.com (Man Cao)
Date: Fri, 21 Jun 2019 16:40:06 -0700
Subject: [RFR]: Disable CDS when TSAN is on
In-Reply-To: <CAPW48sooA--+CzKfTKzmtLdLwERVBCwcjSjcNNamTP3Ar5UPJw@mail.gmail.com>
References: <CAPW48sooA--+CzKfTKzmtLdLwERVBCwcjSjcNNamTP3Ar5UPJw@mail.gmail.com>
Message-ID: <CA+w6HxY_qAh0cJUMOmCdVZdgnsK0uwd+8vDwyfRPcq7Ke99j-g@mail.gmail.com>

Looks good.

-Man


On Fri, Jun 21, 2019 at 4:35 PM Arthur Eubanks <aeubanks at google.com> wrote:

> Webrev: http://cr.openjdk.java.net/~aeubanks/tsancds/webrev.00/index.html
>
> Disable CDS when TSAN is on
>
> We eagerly allocate jmethodIDs in ClassFileParser::fill_instance_klass().
> ClassFileParser::fill_instance_klass() is never called on classes in a
> shared archive.
>
> This helps with an upcoming test in a future change.
>

From jcbeyler at google.com  Sat Jun 22 20:58:16 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Sat, 22 Jun 2019 13:58:16 -0700
Subject: [RFR]: Disable CDS when TSAN is on
In-Reply-To: <CA+w6HxY_qAh0cJUMOmCdVZdgnsK0uwd+8vDwyfRPcq7Ke99j-g@mail.gmail.com>
References: <CAPW48sooA--+CzKfTKzmtLdLwERVBCwcjSjcNNamTP3Ar5UPJw@mail.gmail.com>
 <CA+w6HxY_qAh0cJUMOmCdVZdgnsK0uwd+8vDwyfRPcq7Ke99j-g@mail.gmail.com>
Message-ID: <CAF9BGBx6fcj6CjrYyk1aOUPzzWsrZFL1BxozG2yrHbhCwiGwpw@mail.gmail.com>

LGTM as well :-)
Jc

On Fri, Jun 21, 2019 at 4:40 PM Man Cao <manc at google.com> wrote:

> Looks good.
>
> -Man
>
>
> On Fri, Jun 21, 2019 at 4:35 PM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
> > Webrev:
> http://cr.openjdk.java.net/~aeubanks/tsancds/webrev.00/index.html
> >
> > Disable CDS when TSAN is on
> >
> > We eagerly allocate jmethodIDs in ClassFileParser::fill_instance_klass().
> > ClassFileParser::fill_instance_klass() is never called on classes in a
> > shared archive.
> >
> > This helps with an upcoming test in a future change.
> >
>


-- 

Thanks,
Jc

From aeubanks at google.com  Sat Jun 22 21:11:54 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Sat, 22 Jun 2019 21:11:54 +0000
Subject: hg: tsan/dev: Disable CDS when TSAN is on
Message-ID: <201906222111.x5MLBsNt004733@aojmv0008.oracle.com>

Changeset: d14a0240bb69
Author:    aeubanks
Date:      2019-06-21 16:14 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/d14a0240bb69

Disable CDS when TSAN is on

We eagerly allocate jmethodIDs in ClassFileParser::fill_instance_klass().
ClassFileParser::fill_instance_klass() is never called on classes in a shared archive.

This fixes a test coming in a future change.

! src/hotspot/share/runtime/arguments.cpp


From manc at google.com  Sun Jun 23 00:42:29 2019
From: manc at google.com (Man Cao)
Date: Sat, 22 Jun 2019 17:42:29 -0700
Subject: RFR (XS): Disable stack alignment assertion
Message-ID: <CA+w6HxaQoJtEt-Vy_Ob_U-zWuDJo8Z34S6rHN0=wFAHBG-pxLw@mail.gmail.com>

Hi,

Can I have reviews for this?
https://cr.openjdk.java.net/~manc/tsan20190621/webrev.00/
Comment out stack alignment assertion in os::verify_stack_alignment().
Currently Clang build fails under fastdebug due to this assertion failure.
Note that our internal JDK11 also commented out this assertion.

-Man

From aeubanks at google.com  Sun Jun 23 06:00:15 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Sat, 22 Jun 2019 23:00:15 -0700
Subject: RFR (XS): Disable stack alignment assertion
In-Reply-To: <CA+w6HxaQoJtEt-Vy_Ob_U-zWuDJo8Z34S6rHN0=wFAHBG-pxLw@mail.gmail.com>
References: <CA+w6HxaQoJtEt-Vy_Ob_U-zWuDJo8Z34S6rHN0=wFAHBG-pxLw@mail.gmail.com>
Message-ID: <CAPW48sqNHwUvk_67p=jKT15ABCS0KoPmcT0eL6MXr3xhs5UPpQ@mail.gmail.com>

LGTM

On Sat, Jun 22, 2019 at 5:43 PM Man Cao <manc at google.com> wrote:

> Hi,
>
> Can I have reviews for this?
> https://cr.openjdk.java.net/~manc/tsan20190621/webrev.00/
> Comment out stack alignment assertion in os::verify_stack_alignment().
> Currently Clang build fails under fastdebug due to this assertion failure.
> Note that our internal JDK11 also commented out this assertion.
>
> -Man
>

From aeubanks at google.com  Sun Jun 23 21:12:04 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Sun, 23 Jun 2019 14:12:04 -0700
Subject: [RFR]: Suppress some String races
Message-ID: <CAPW48srJud9yHqwOQXvnbXypbZ0xHNSqkAX5YKhW27-Y2Y7OvA@mail.gmail.com>

Webrev: http://cr.openjdk.java.net/~aeubanks/tsanstring/webrev.00/index.html

Some common String operations are racy, e.g. hashCode or concatenation.
These races should be suppressed, since they are in the JDK.
This change adds a test that calls hashCode() on a const String and does
local String concatenation.

From jcbeyler at google.com  Mon Jun 24 16:57:23 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Mon, 24 Jun 2019 09:57:23 -0700
Subject: RFR (XS): Disable stack alignment assertion
In-Reply-To: <CAPW48sqNHwUvk_67p=jKT15ABCS0KoPmcT0eL6MXr3xhs5UPpQ@mail.gmail.com>
References: <CA+w6HxaQoJtEt-Vy_Ob_U-zWuDJo8Z34S6rHN0=wFAHBG-pxLw@mail.gmail.com>
 <CAPW48sqNHwUvk_67p=jKT15ABCS0KoPmcT0eL6MXr3xhs5UPpQ@mail.gmail.com>
Message-ID: <CAF9BGBys1n48FQAgDnD6GspAewKBXa+J0LPOpTfQMwsXCnF-iA@mail.gmail.com>

LGTM :)

On Sat, Jun 22, 2019 at 11:00 PM Arthur Eubanks <aeubanks at google.com> wrote:

> LGTM
>
> On Sat, Jun 22, 2019 at 5:43 PM Man Cao <manc at google.com> wrote:
>
> > Hi,
> >
> > Can I have reviews for this?
> > https://cr.openjdk.java.net/~manc/tsan20190621/webrev.00/
> > Comment out stack alignment assertion in os::verify_stack_alignment().
> > Currently Clang build fails under fastdebug due to this assertion
> failure.
> > Note that our internal JDK11 also commented out this assertion.
> >
> > -Man
> >
>


-- 

Thanks,
Jc

From jcbeyler at google.com  Mon Jun 24 16:58:11 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Mon, 24 Jun 2019 09:58:11 -0700
Subject: [RFR]: Suppress some String races
In-Reply-To: <CAPW48srJud9yHqwOQXvnbXypbZ0xHNSqkAX5YKhW27-Y2Y7OvA@mail.gmail.com>
References: <CAPW48srJud9yHqwOQXvnbXypbZ0xHNSqkAX5YKhW27-Y2Y7OvA@mail.gmail.com>
Message-ID: <CAF9BGBz1NLVzFU7uECWpCfN1JDFwPsrkPN3wRet50S0vU95aGg@mail.gmail.com>

LGTM,
Jc

On Sun, Jun 23, 2019 at 2:12 PM Arthur Eubanks <aeubanks at google.com> wrote:

> Webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanstring/webrev.00/index.html
>
> Some common String operations are racy, e.g. hashCode or concatenation.
> These races should be suppressed, since they are in the JDK.
> This change adds a test that calls hashCode() on a const String and does
> local String concatenation.
>


-- 

Thanks,
Jc

From manc at google.com  Mon Jun 24 16:59:30 2019
From: manc at google.com (manc at google.com)
Date: Mon, 24 Jun 2019 16:59:30 +0000
Subject: hg: tsan/dev: Comment out stack alignment assertion in
 os::verify_stack_alignment()
Message-ID: <201906241659.x5OGxUa2022522@aojmv0008.oracle.com>

Changeset: bb39e1956052
Author:    manc
Date:      2019-06-22 17:36 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/bb39e1956052

Comment out stack alignment assertion in os::verify_stack_alignment()

! src/hotspot/os_cpu/linux_x86/os_linux_x86.cpp


From manc at google.com  Mon Jun 24 18:31:04 2019
From: manc at google.com (Man Cao)
Date: Mon, 24 Jun 2019 11:31:04 -0700
Subject: [RFR]: Suppress some String races
In-Reply-To: <CAF9BGBz1NLVzFU7uECWpCfN1JDFwPsrkPN3wRet50S0vU95aGg@mail.gmail.com>
References: <CAPW48srJud9yHqwOQXvnbXypbZ0xHNSqkAX5YKhW27-Y2Y7OvA@mail.gmail.com>
 <CAF9BGBz1NLVzFU7uECWpCfN1JDFwPsrkPN3wRet50S0vU95aGg@mail.gmail.com>
Message-ID: <CA+w6HxbyuYwgvVDwVg=2dby08p6nVdA3Ubbs4_W_3-DyBoCfww@mail.gmail.com>

Can we change the comment in the string content to C/C++ comment?
Also, please clarify the comments like the following:
"# SoftReference.timestamp\n"
"# String.hash\n"
"# java.lang.StringLatin1.charAt\n"

-Man


On Mon, Jun 24, 2019 at 9:58 AM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> LGTM,
> Jc
>
> On Sun, Jun 23, 2019 at 2:12 PM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
> > Webrev:
> > http://cr.openjdk.java.net/~aeubanks/tsanstring/webrev.00/index.html
> >
> > Some common String operations are racy, e.g. hashCode or concatenation.
> > These races should be suppressed, since they are in the JDK.
> > This change adds a test that calls hashCode() on a const String and does
> > local String concatenation.
> >
>
>
> --
>
> Thanks,
> Jc
>

From aeubanks at google.com  Mon Jun 24 20:46:04 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 24 Jun 2019 13:46:04 -0700
Subject: [RFR]: Suppress some String races
In-Reply-To: <CA+w6HxbyuYwgvVDwVg=2dby08p6nVdA3Ubbs4_W_3-DyBoCfww@mail.gmail.com>
References: <CAPW48srJud9yHqwOQXvnbXypbZ0xHNSqkAX5YKhW27-Y2Y7OvA@mail.gmail.com>
 <CAF9BGBz1NLVzFU7uECWpCfN1JDFwPsrkPN3wRet50S0vU95aGg@mail.gmail.com>
 <CA+w6HxbyuYwgvVDwVg=2dby08p6nVdA3Ubbs4_W_3-DyBoCfww@mail.gmail.com>
Message-ID: <CAPW48spB38WfVRdCoa7fmQFnEaq79f4FdyYbypdjsWDbRX=M9w@mail.gmail.com>

PTAL: http://cr.openjdk.java.net/~aeubanks/tsanstring/webrev.01/index.html
Also changed suppression to suppress java.lang.invoke anywhere in the stack
trace. This removes most of the other suppression lines.

On Mon, Jun 24, 2019 at 11:31 AM Man Cao <manc at google.com> wrote:

> Can we change the comment in the string content to C/C++ comment?
> Also, please clarify the comments like the following:
> "# SoftReference.timestamp\n"
> "# String.hash\n"
> "# java.lang.StringLatin1.charAt\n"
>
> -Man
>
>
> On Mon, Jun 24, 2019 at 9:58 AM Jean Christophe Beyler <
> jcbeyler at google.com> wrote:
>
>> LGTM,
>> Jc
>>
>> On Sun, Jun 23, 2019 at 2:12 PM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>> > Webrev:
>> > http://cr.openjdk.java.net/~aeubanks/tsanstring/webrev.00/index.html
>> >
>> > Some common String operations are racy, e.g. hashCode or concatenation.
>> > These races should be suppressed, since they are in the JDK.
>> > This change adds a test that calls hashCode() on a const String and does
>> > local String concatenation.
>> >
>>
>>
>> --
>>
>> Thanks,
>> Jc
>>
>

From aeubanks at google.com  Mon Jun 24 21:11:07 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 24 Jun 2019 14:11:07 -0700
Subject: [RFR] Ignore fields annotated with @LazyInit
In-Reply-To: <CA+w6Hxa-3G-r7n+xE+U=ibCC_Zz2Q51wDUDUUqq28GiUYxc2Yg@mail.gmail.com>
References: <CAPW48sqNeEihaT=Ko1i1GDEp7cs_1ieofV2yARgwaFBiArhpBw@mail.gmail.com>
 <CA+w6Hxa-3G-r7n+xE+U=ibCC_Zz2Q51wDUDUUqq28GiUYxc2Yg@mail.gmail.com>
Message-ID: <CAPW48sph6QP9u69XFmKCW5RLpux7qjksjPvMJhh5HJhj1TxnKw@mail.gmail.com>

>
> Looks good. One minor comment is the VM symbol
> java_util_concurrent_atomic_annotation_LazyInit, can we remove "atomic_"
> from the name?
>
Done.

New webrev:
http://cr.openjdk.java.net/~aeubanks/tsanfieldignore/webrev.01/index.html

From manc at google.com  Mon Jun 24 21:14:44 2019
From: manc at google.com (Man Cao)
Date: Mon, 24 Jun 2019 14:14:44 -0700
Subject: [RFR]: Suppress some String races
In-Reply-To: <CAPW48spB38WfVRdCoa7fmQFnEaq79f4FdyYbypdjsWDbRX=M9w@mail.gmail.com>
References: <CAPW48srJud9yHqwOQXvnbXypbZ0xHNSqkAX5YKhW27-Y2Y7OvA@mail.gmail.com>
 <CAF9BGBz1NLVzFU7uECWpCfN1JDFwPsrkPN3wRet50S0vU95aGg@mail.gmail.com>
 <CA+w6HxbyuYwgvVDwVg=2dby08p6nVdA3Ubbs4_W_3-DyBoCfww@mail.gmail.com>
 <CAPW48spB38WfVRdCoa7fmQFnEaq79f4FdyYbypdjsWDbRX=M9w@mail.gmail.com>
Message-ID: <CA+w6HxY0qMTiW5DKy2Kx-n-+OPbND3Pfz=0oh3FOty6NYNcZmA@mail.gmail.com>

Looks good.

-Man


On Mon, Jun 24, 2019 at 1:46 PM Arthur Eubanks <aeubanks at google.com> wrote:

> PTAL: http://cr.openjdk.java.net/~aeubanks/tsanstring/webrev.01/index.html
> Also changed suppression to suppress java.lang.invoke anywhere in the
> stack trace. This removes most of the other suppression lines.
>
> On Mon, Jun 24, 2019 at 11:31 AM Man Cao <manc at google.com> wrote:
>
>> Can we change the comment in the string content to C/C++ comment?
>> Also, please clarify the comments like the following:
>> "# SoftReference.timestamp\n"
>> "# String.hash\n"
>> "# java.lang.StringLatin1.charAt\n"
>>
>> -Man
>>
>>
>> On Mon, Jun 24, 2019 at 9:58 AM Jean Christophe Beyler <
>> jcbeyler at google.com> wrote:
>>
>>> LGTM,
>>> Jc
>>>
>>> On Sun, Jun 23, 2019 at 2:12 PM Arthur Eubanks <aeubanks at google.com>
>>> wrote:
>>>
>>> > Webrev:
>>> > http://cr.openjdk.java.net/~aeubanks/tsanstring/webrev.00/index.html
>>> >
>>> > Some common String operations are racy, e.g. hashCode or concatenation.
>>> > These races should be suppressed, since they are in the JDK.
>>> > This change adds a test that calls hashCode() on a const String and
>>> does
>>> > local String concatenation.
>>> >
>>>
>>>
>>> --
>>>
>>> Thanks,
>>> Jc
>>>
>>

From aeubanks at google.com  Mon Jun 24 21:20:11 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Mon, 24 Jun 2019 21:20:11 +0000
Subject: hg: tsan/dev: Suppress some String races
Message-ID: <201906242120.x5OLKCRT023671@aojmv0008.oracle.com>

Changeset: d2f03281e21d
Author:    aeubanks
Date:      2019-06-23 14:08 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/d2f03281e21d

Suppress some String races

Some common String operations are racy, e.g. hashCode or concatenation.
These races should be suppressed. Since they are in the JDK.
Add a test that calls hashCode() on a const String, and does local String concatenation.

! src/java.base/share/native/launcher/main.c
+ test/hotspot/jtreg/tsan/StringOperationsLoopTest.java


From manc at google.com  Tue Jun 25 00:51:09 2019
From: manc at google.com (Man Cao)
Date: Mon, 24 Jun 2019 17:51:09 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
Message-ID: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>

Hi,

Can I have reviews for the support for allocation and garbage collection?
https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/

There is no major algorithmic or logic change. Most difference from our
internal JDK8 version is renaming and code movement.
Below is a map of renaming:
GoogleLockMap -> TsanOopMap
GoogleLockMapImpl -> TsanOopMapImpl
GoogleOopMap -> TsanOopSizeMap
GoogleOopBucket -> TsanOopBucket
GoogleTsanOopMap_lock -> TsanOopMap_lock
GoogleLockMap::do_weak_oops() -> TsanOopMap::weak_oops_do()
BitMap -> CHeapBitMap
TraceCPUTime -> GCTraceCPUTime
VALUE_OBJ_CLASS_SPEC was removed in JDK-8173070

A notable change is that the call to TsanOopMap::weak_oops_do() is a new
serial phase in WeakProcessor, instead of instrumenting various collectors
and ReferenceProcessor.

Added a test that allocates and writes to byte arrays that are larger than
the heap size, so garbage collection must have happened. The test fails
without this patch.
Another change to TsanRunner.java is to use
jdk.test.lib.management.InputArguments to pass all VM options.
The existing way of using "test.java.opts" system property does not work
for VM options in "@run" tag, and does not contain VM options
from VM_OPTIONS environment variable such as from "make test
JTREG="VM_OPTIONS=-XX:+Foo"".

-Man

From jcbeyler at google.com  Tue Jun 25 22:42:03 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Tue, 25 Jun 2019 15:42:03 -0700
Subject: [RFR] Ignore fields annotated with @LazyInit
In-Reply-To: <CAPW48sph6QP9u69XFmKCW5RLpux7qjksjPvMJhh5HJhj1TxnKw@mail.gmail.com>
References: <CAPW48sqNeEihaT=Ko1i1GDEp7cs_1ieofV2yARgwaFBiArhpBw@mail.gmail.com>
 <CA+w6Hxa-3G-r7n+xE+U=ibCC_Zz2Q51wDUDUUqq28GiUYxc2Yg@mail.gmail.com>
 <CAPW48sph6QP9u69XFmKCW5RLpux7qjksjPvMJhh5HJhj1TxnKw@mail.gmail.com>
Message-ID: <CAF9BGByWoDTHjacWT1VLDFL6dH8UCo1NiENFa1PaD5mMjbgtpQ@mail.gmail.com>

Looks good to me too,

My only comment would be here:
http://cr.openjdk.java.net/~aeubanks/tsanfieldignore/webrev.01/src/hotspot/share/interpreter/interpreterRuntime.cpp.udiff.html

This could be:


+  bool is_tsan_ignore = false;+#if INCLUDE_TSAN+  is_tsan_ignore =
info.access_flags().is_stable() ||
info.access_flags().is_tsan_ignore();+#endif  // INCLUDE_TSAN+

No need of a new webrev either way :)

Jc


On Mon, Jun 24, 2019 at 2:11 PM Arthur Eubanks <aeubanks at google.com> wrote:

> >
> > Looks good. One minor comment is the VM symbol
> > java_util_concurrent_atomic_annotation_LazyInit, can we remove "atomic_"
> > from the name?
> >
> Done.
>
> New webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanfieldignore/webrev.01/index.html
>


-- 

Thanks,
Jc

From aeubanks at google.com  Wed Jun 26 15:46:57 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 26 Jun 2019 08:46:57 -0700
Subject: [RFR] Fix clobbered address passed to TSAN
Message-ID: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>

webrev:
http://cr.openjdk.java.net/~aeubanks/tsanarrayclobber/webrev.00/index.html
(ignore the wrong description, webrev.ksh is weird, it's good locally)

Fix clobbered address passed to TSAN

access_load_at() sometimes clobbered the address we passed to TSAN in
tsan_observe_load_or_store().

Do the TSAN instrumentation first, then do the load. Initially the order
was access_load_at(), then TSAN instrumentation. Field accesses can behave
as volatile, and I wanted arrays to match up wi
th the field TSAN instrumentation/actual load order. But it doesn't really
matter for array loads since they can never be volatile.
Added some asserts for fastdebug that the oop is valid and that the
field/element offset is within the oop size. The size check fails without
the TSAN instrumentation/actual load reordering becau
se the offset register was clobbered.

From aeubanks at google.com  Wed Jun 26 15:48:16 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 26 Jun 2019 08:48:16 -0700
Subject: [RFR] Ignore fields annotated with @LazyInit
In-Reply-To: <CAF9BGByWoDTHjacWT1VLDFL6dH8UCo1NiENFa1PaD5mMjbgtpQ@mail.gmail.com>
References: <CAPW48sqNeEihaT=Ko1i1GDEp7cs_1ieofV2yARgwaFBiArhpBw@mail.gmail.com>
 <CA+w6Hxa-3G-r7n+xE+U=ibCC_Zz2Q51wDUDUUqq28GiUYxc2Yg@mail.gmail.com>
 <CAPW48sph6QP9u69XFmKCW5RLpux7qjksjPvMJhh5HJhj1TxnKw@mail.gmail.com>
 <CAF9BGByWoDTHjacWT1VLDFL6dH8UCo1NiENFa1PaD5mMjbgtpQ@mail.gmail.com>
Message-ID: <CAPW48sp0cRsm757DyF8gtntsdmag2f3exXRFP6duF6dX_zdNdw@mail.gmail.com>

On Tue, Jun 25, 2019 at 3:42 PM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Looks good to me too,
>
> My only comment would be here:
>
> http://cr.openjdk.java.net/~aeubanks/tsanfieldignore/webrev.01/src/hotspot/share/interpreter/interpreterRuntime.cpp.udiff.html
>
> This could be:
>
>
> +  bool is_tsan_ignore = false;+#if INCLUDE_TSAN+  is_tsan_ignore = info.access_flags().is_stable() || info.access_flags().is_tsan_ignore();+#endif  // INCLUDE_TSAN+
>
> No need of a new webrev either way :)
>
> Jc
>
>
> Done.

From aeubanks at google.com  Wed Jun 26 16:10:23 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 26 Jun 2019 09:10:23 -0700
Subject: [RFR]: Suppress j.u.c
Message-ID: <CAPW48spbacLe5OEsz897JNFZvZgrtBna+X52_gPkRjf2c16ssA@mail.gmail.com>

webrev:
http://cr.openjdk.java.net/~aeubanks/tsansuppressjuc/webrev.00/index.html

Suppress races in j.u.c.ConcurrentHashMap
It's not worth trying to understand j.u.c., assume it's good.

These are appearing in some tests.

From manc at google.com  Wed Jun 26 17:54:24 2019
From: manc at google.com (Man Cao)
Date: Wed, 26 Jun 2019 10:54:24 -0700
Subject: [RFR]: Suppress j.u.c
In-Reply-To: <CAPW48spbacLe5OEsz897JNFZvZgrtBna+X52_gPkRjf2c16ssA@mail.gmail.com>
References: <CAPW48spbacLe5OEsz897JNFZvZgrtBna+X52_gPkRjf2c16ssA@mail.gmail.com>
Message-ID: <CA+w6Hxa9EK_ZLk-brktxU_bwHXPuP=ibZEh+Yc4N-+hSWchdUw@mail.gmail.com>

Looks good. But I'd improve the comment to something like "suppress known,
benign races in j.c.u". No need for another webrev.

-Man


On Wed, Jun 26, 2019 at 9:11 AM Arthur Eubanks <aeubanks at google.com> wrote:

> webrev:
> http://cr.openjdk.java.net/~aeubanks/tsansuppressjuc/webrev.00/index.html
>
> Suppress races in j.u.c.ConcurrentHashMap
> It's not worth trying to understand j.u.c., assume it's good.
>
> These are appearing in some tests.
>

From aeubanks at google.com  Wed Jun 26 18:08:47 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 26 Jun 2019 11:08:47 -0700
Subject: [RFR]: Suppress j.u.c
In-Reply-To: <CA+w6Hxa9EK_ZLk-brktxU_bwHXPuP=ibZEh+Yc4N-+hSWchdUw@mail.gmail.com>
References: <CAPW48spbacLe5OEsz897JNFZvZgrtBna+X52_gPkRjf2c16ssA@mail.gmail.com>
 <CA+w6Hxa9EK_ZLk-brktxU_bwHXPuP=ibZEh+Yc4N-+hSWchdUw@mail.gmail.com>
Message-ID: <CAPW48sqt5+Uf1BE+J03H38YnAj+jsyayPtkr8vPJderBBW5X-g@mail.gmail.com>

On Wed, Jun 26, 2019 at 10:54 AM Man Cao <manc at google.com> wrote:

> Looks good. But I'd improve the comment to something like "suppress known,
> benign races in j.c.u". No need for another webrev.
>
Done

From jcbeyler at google.com  Wed Jun 26 23:00:41 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Wed, 26 Jun 2019 16:00:41 -0700
Subject: [RFR]: Suppress j.u.c
In-Reply-To: <CAPW48sqt5+Uf1BE+J03H38YnAj+jsyayPtkr8vPJderBBW5X-g@mail.gmail.com>
References: <CAPW48spbacLe5OEsz897JNFZvZgrtBna+X52_gPkRjf2c16ssA@mail.gmail.com>
 <CA+w6Hxa9EK_ZLk-brktxU_bwHXPuP=ibZEh+Yc4N-+hSWchdUw@mail.gmail.com>
 <CAPW48sqt5+Uf1BE+J03H38YnAj+jsyayPtkr8vPJderBBW5X-g@mail.gmail.com>
Message-ID: <CAF9BGBzvRt6Ga4Y1sj2gmg9rZy1Q3iBWeRz4f3P8NRuB7o0GWw@mail.gmail.com>

LGTM,
Jc

On Wed, Jun 26, 2019 at 11:09 AM Arthur Eubanks <aeubanks at google.com> wrote:

> On Wed, Jun 26, 2019 at 10:54 AM Man Cao <manc at google.com> wrote:
>
> > Looks good. But I'd improve the comment to something like "suppress
> known,
> > benign races in j.c.u". No need for another webrev.
> >
> Done
>


-- 

Thanks,
Jc

From jcbeyler at google.com  Wed Jun 26 23:06:39 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Wed, 26 Jun 2019 16:06:39 -0700
Subject: [RFR] Fix clobbered address passed to TSAN
In-Reply-To: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>
References: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>
Message-ID: <CAF9BGBx-bEb34ZmebJvabdw6Y_bzRsh11GL20O-WPcQgmMVeog@mail.gmail.com>

Hi Arthur,

LGTM,  I was curious if the DEBUG code you added made sense to go upstream
to help diagnose issues or was this a failure that is only provoked by
TSAN's instrumentation before this patch and can never happen otherwise?

Thanks,
Jc

On Wed, Jun 26, 2019 at 8:47 AM Arthur Eubanks <aeubanks at google.com> wrote:

> webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanarrayclobber/webrev.00/index.html
> (ignore the wrong description, webrev.ksh is weird, it's good locally)
>
> Fix clobbered address passed to TSAN
>
> access_load_at() sometimes clobbered the address we passed to TSAN in
> tsan_observe_load_or_store().
>
> Do the TSAN instrumentation first, then do the load. Initially the order
> was access_load_at(), then TSAN instrumentation. Field accesses can behave
> as volatile, and I wanted arrays to match up wi
> th the field TSAN instrumentation/actual load order. But it doesn't really
> matter for array loads since they can never be volatile.
> Added some asserts for fastdebug that the oop is valid and that the
> field/element offset is within the oop size. The size check fails without
> the TSAN instrumentation/actual load reordering becau
> se the offset register was clobbered.
>


-- 

Thanks,
Jc

From manc at google.com  Wed Jun 26 23:19:57 2019
From: manc at google.com (Man Cao)
Date: Wed, 26 Jun 2019 16:19:57 -0700
Subject: [RFR] Fix clobbered address passed to TSAN
In-Reply-To: <CAF9BGBx-bEb34ZmebJvabdw6Y_bzRsh11GL20O-WPcQgmMVeog@mail.gmail.com>
References: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>
 <CAF9BGBx-bEb34ZmebJvabdw6Y_bzRsh11GL20O-WPcQgmMVeog@mail.gmail.com>
Message-ID: <CA+w6HxZPzzBfa8OsBDB0WedDiA0+ZOHywJF5hQpFVDPkxLZfRg@mail.gmail.com>

Offline chat with Arthur, for passing parameter to
SharedRuntime::verify_oop_index, it might be better to do the following
instead, so it accounts for field._disp:
__ leaq(c_rarg0, field);
__ subq(c_rarg0, field.base());

And why does it need "__ get_method(c_rarg2)", as c_rarg2 seems unused?

-Man


On Wed, Jun 26, 2019 at 4:07 PM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Hi Arthur,
>
> LGTM,  I was curious if the DEBUG code you added made sense to go upstream
> to help diagnose issues or was this a failure that is only provoked by
> TSAN's instrumentation before this patch and can never happen otherwise?
>
> Thanks,
> Jc
>
> On Wed, Jun 26, 2019 at 8:47 AM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
> > webrev:
> >
> http://cr.openjdk.java.net/~aeubanks/tsanarrayclobber/webrev.00/index.html
> > (ignore the wrong description, webrev.ksh is weird, it's good locally)
> >
> > Fix clobbered address passed to TSAN
> >
> > access_load_at() sometimes clobbered the address we passed to TSAN in
> > tsan_observe_load_or_store().
> >
> > Do the TSAN instrumentation first, then do the load. Initially the order
> > was access_load_at(), then TSAN instrumentation. Field accesses can
> behave
> > as volatile, and I wanted arrays to match up wi
> > th the field TSAN instrumentation/actual load order. But it doesn't
> really
> > matter for array loads since they can never be volatile.
> > Added some asserts for fastdebug that the oop is valid and that the
> > field/element offset is within the oop size. The size check fails without
> > the TSAN instrumentation/actual load reordering becau
> > se the offset register was clobbered.
> >
>
>
> --
>
> Thanks,
> Jc
>

From aeubanks at google.com  Wed Jun 26 23:57:18 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Wed, 26 Jun 2019 23:57:18 +0000
Subject: hg: tsan/dev: Suppress races in j.u.c.ConcurrentHashMap
Message-ID: <201906262357.x5QNvIc8010667@aojmv0008.oracle.com>

Changeset: b239596bc200
Author:    aeubanks
Date:      2019-06-26 09:07 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/b239596bc200

Suppress races in j.u.c.ConcurrentHashMap

These are appearing and it's not worth trying to understand j.u.c.

! src/java.base/share/native/launcher/main.c


From aeubanks at google.com  Thu Jun 27 00:04:55 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Thu, 27 Jun 2019 00:04:55 +0000
Subject: hg: tsan/dev: Don't reported races on fields marked @LazyInit
Message-ID: <201906270004.x5R04uQV012361@aojmv0008.oracle.com>

Changeset: c6fef9bb816b
Author:    aeubanks
Date:      2019-05-30 15:37 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/c6fef9bb816b

Don't reported races on fields marked @LazyInit

This change introduces a new annotation, java.util.concurrent.annotation.LazyInit, which causes TSAN to ignore races on that field.
References are not simply ignored, but a release/acquire is performed on them. This is so that any following accesses to its member variables are not also reported as racy.

! src/hotspot/cpu/x86/templateTable_x86.cpp
! src/hotspot/cpu/x86/templateTable_x86.hpp
! src/hotspot/share/classfile/classFileParser.cpp
! src/hotspot/share/classfile/vmSymbols.hpp
! src/hotspot/share/interpreter/interpreterRuntime.cpp
! src/hotspot/share/oops/cpCache.cpp
! src/hotspot/share/oops/cpCache.hpp
! src/hotspot/share/oops/fieldInfo.hpp
! src/hotspot/share/utilities/accessFlags.hpp
+ src/java.base/share/classes/java/util/concurrent/annotation/LazyInit.java
! src/java.base/share/classes/module-info.java
+ test/hotspot/jtreg/tsan/LazyInitLoopTest.java
+ test/hotspot/jtreg/tsan/LazyInitReferenceLoopTest.java


From aeubanks at google.com  Thu Jun 27 00:36:30 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 26 Jun 2019 17:36:30 -0700
Subject: [RFR] Fix clobbered address passed to TSAN
In-Reply-To: <CA+w6HxZPzzBfa8OsBDB0WedDiA0+ZOHywJF5hQpFVDPkxLZfRg@mail.gmail.com>
References: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>
 <CAF9BGBx-bEb34ZmebJvabdw6Y_bzRsh11GL20O-WPcQgmMVeog@mail.gmail.com>
 <CA+w6HxZPzzBfa8OsBDB0WedDiA0+ZOHywJF5hQpFVDPkxLZfRg@mail.gmail.com>
Message-ID: <CAPW48sopQROZHcVuW-8MZ68Bx8i43v5-pzVFyqT46M3mwXZmWA@mail.gmail.com>

On Wed, Jun 26, 2019 at 4:20 PM Man Cao <manc at google.com> wrote:

> Offline chat with Arthur, for passing parameter to
> SharedRuntime::verify_oop_index, it might be better to do the following
> instead, so it accounts for field._disp:
> __ leaq(c_rarg0, field);
> __ subq(c_rarg0, field.base());
>
Done.

>
> And why does it need "__ get_method(c_rarg2)", as c_rarg2 seems unused?
>
Sorry, that's from my debugging, removed.

>
> -Man
>
>
> On Wed, Jun 26, 2019 at 4:07 PM Jean Christophe Beyler <
> jcbeyler at google.com> wrote:
>
>> Hi Arthur,
>>
>> LGTM,  I was curious if the DEBUG code you added made sense to go upstream
>> to help diagnose issues or was this a failure that is only provoked by
>> TSAN's instrumentation before this patch and can never happen otherwise?
>>
> I suppose it's possible to add this to upstream. It's something we'd
probably have to put it behind the VerifyOops flag since I think it
significantly slow down execution time. And I think the code around
loading/storing from an array/field in interpreter code is pretty stable :P

>
>> Thanks,
>> Jc
>>
>
New webrev:
http://cr.openjdk.java.net/~aeubanks/tsanarrayclobber/webrev.01/index.html

From manc at google.com  Thu Jun 27 01:36:54 2019
From: manc at google.com (Man Cao)
Date: Wed, 26 Jun 2019 18:36:54 -0700
Subject: [RFR] Fix clobbered address passed to TSAN
In-Reply-To: <CAPW48sopQROZHcVuW-8MZ68Bx8i43v5-pzVFyqT46M3mwXZmWA@mail.gmail.com>
References: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>
 <CAF9BGBx-bEb34ZmebJvabdw6Y_bzRsh11GL20O-WPcQgmMVeog@mail.gmail.com>
 <CA+w6HxZPzzBfa8OsBDB0WedDiA0+ZOHywJF5hQpFVDPkxLZfRg@mail.gmail.com>
 <CAPW48sopQROZHcVuW-8MZ68Bx8i43v5-pzVFyqT46M3mwXZmWA@mail.gmail.com>
Message-ID: <CA+w6HxbWJowvD1WgJ-nn2wdgOZXDAs1P96GDtESqDKn2M9r4-Q@mail.gmail.com>

I see the instrumentation for stores (iastore, lastore, ...) is moved from
pre-barrier to post-barrier. Is that intended?

A minor issue is that the comment for TSAN in TemplateTable::aaload() could
be updated.

If the update is just for the above two issues, no need for another webrev
and looks good.

However, I'm now a bit skeptical if TSAN read instrumentation in
TemplateTable::getfield_or_static() are still correct, as they all follow a
call to access_load_at(), which could invoke GC's read barrier and clobber
unknown number of registers. Our internal JDK8 TSAN works because there
were no read barrier in JDK8. Fortunately we at least have the
verify_oop_index() for checking now.

-Man



On Wed, Jun 26, 2019 at 5:36 PM Arthur Eubanks <aeubanks at google.com> wrote:

>
>
> On Wed, Jun 26, 2019 at 4:20 PM Man Cao <manc at google.com> wrote:
>
>> Offline chat with Arthur, for passing parameter to
>> SharedRuntime::verify_oop_index, it might be better to do the following
>> instead, so it accounts for field._disp:
>> __ leaq(c_rarg0, field);
>> __ subq(c_rarg0, field.base());
>>
> Done.
>
>>
>> And why does it need "__ get_method(c_rarg2)", as c_rarg2 seems unused?
>>
> Sorry, that's from my debugging, removed.
>
>>
>> -Man
>>
>>
>> On Wed, Jun 26, 2019 at 4:07 PM Jean Christophe Beyler <
>> jcbeyler at google.com> wrote:
>>
>>> Hi Arthur,
>>>
>>> LGTM,  I was curious if the DEBUG code you added made sense to go
>>> upstream
>>> to help diagnose issues or was this a failure that is only provoked by
>>> TSAN's instrumentation before this patch and can never happen otherwise?
>>>
>> I suppose it's possible to add this to upstream. It's something we'd
> probably have to put it behind the VerifyOops flag since I think it
> significantly slow down execution time. And I think the code around
> loading/storing from an array/field in interpreter code is pretty stable :P
>
>>
>>> Thanks,
>>> Jc
>>>
>>
> New webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanarrayclobber/webrev.01/index.html
>
>

From aeubanks at google.com  Thu Jun 27 15:36:52 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Thu, 27 Jun 2019 08:36:52 -0700
Subject: [RFR] Fix clobbered address passed to TSAN
In-Reply-To: <CA+w6HxbWJowvD1WgJ-nn2wdgOZXDAs1P96GDtESqDKn2M9r4-Q@mail.gmail.com>
References: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>
 <CAF9BGBx-bEb34ZmebJvabdw6Y_bzRsh11GL20O-WPcQgmMVeog@mail.gmail.com>
 <CA+w6HxZPzzBfa8OsBDB0WedDiA0+ZOHywJF5hQpFVDPkxLZfRg@mail.gmail.com>
 <CAPW48sopQROZHcVuW-8MZ68Bx8i43v5-pzVFyqT46M3mwXZmWA@mail.gmail.com>
 <CA+w6HxbWJowvD1WgJ-nn2wdgOZXDAs1P96GDtESqDKn2M9r4-Q@mail.gmail.com>
Message-ID: <CAPW48soYbzJ-SR=ozzKZG7kNhACnJ77FOhSk8WNubbrnt5ZnMw@mail.gmail.com>

On Wed, Jun 26, 2019 at 6:37 PM Man Cao <manc at google.com> wrote:

> I see the instrumentation for stores (iastore, lastore, ...) is moved from
> pre-barrier to post-barrier. Is that intended?
>
Yes, just to be consistent and lower the risk of clobbered registers there
as well.

>
> A minor issue is that the comment for TSAN in TemplateTable::aaload()
> could be updated.
>
Removed the comment since now it applies to all loads/stores.

>
> If the update is just for the above two issues, no need for another webrev
> and looks good.
>

> However, I'm now a bit skeptical if TSAN read instrumentation in
> TemplateTable::getfield_or_static() are still correct, as they all follow a
> call to access_load_at(), which could invoke GC's read barrier and clobber
> unknown number of registers. Our internal JDK8 TSAN works because there
> were no read barrier in JDK8. Fortunately we at least have the
> verify_oop_index() for checking now.
>
Good point. I'm not sure exactly how `noreg` works, but as you said,
verify_oop_index() isn't complaining.

From aeubanks at google.com  Thu Jun 27 17:24:09 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Thu, 27 Jun 2019 10:24:09 -0700
Subject: [RFR] Fix clobbered address passed to TSAN
In-Reply-To: <CAPW48soYbzJ-SR=ozzKZG7kNhACnJ77FOhSk8WNubbrnt5ZnMw@mail.gmail.com>
References: <CAPW48soo84f=AQ7NCBMyL=kiedE8qb7yRPkJ=29D0k=kbuVfhQ@mail.gmail.com>
 <CAF9BGBx-bEb34ZmebJvabdw6Y_bzRsh11GL20O-WPcQgmMVeog@mail.gmail.com>
 <CA+w6HxZPzzBfa8OsBDB0WedDiA0+ZOHywJF5hQpFVDPkxLZfRg@mail.gmail.com>
 <CAPW48sopQROZHcVuW-8MZ68Bx8i43v5-pzVFyqT46M3mwXZmWA@mail.gmail.com>
 <CA+w6HxbWJowvD1WgJ-nn2wdgOZXDAs1P96GDtESqDKn2M9r4-Q@mail.gmail.com>
 <CAPW48soYbzJ-SR=ozzKZG7kNhACnJ77FOhSk8WNubbrnt5ZnMw@mail.gmail.com>
Message-ID: <CAPW48sqFsZGekxUMQbVEZ+__QDJ5g=h3PHZzoY0qF-qGsODqwA@mail.gmail.com>

On Thu, Jun 27, 2019 at 8:36 AM Arthur Eubanks <aeubanks at google.com> wrote:

>
>
> On Wed, Jun 26, 2019 at 6:37 PM Man Cao <manc at google.com> wrote:
>
>> I see the instrumentation for stores (iastore, lastore, ...) is moved
>> from pre-barrier to post-barrier. Is that intended?
>>
> Yes, just to be consistent and lower the risk of clobbered registers there
> as well.
>
Sorry, I misread your comment. These should be post-barrier, reverted.

>
>> A minor issue is that the comment for TSAN in TemplateTable::aaload()
>> could be updated.
>>
> Removed the comment since now it applies to all loads/stores.
>
>>
>> If the update is just for the above two issues, no need for another
>> webrev and looks good.
>>
>
>> However, I'm now a bit skeptical if TSAN read instrumentation in
>> TemplateTable::getfield_or_static() are still correct, as they all follow a
>> call to access_load_at(), which could invoke GC's read barrier and clobber
>> unknown number of registers. Our internal JDK8 TSAN works because there
>> were no read barrier in JDK8. Fortunately we at least have the
>> verify_oop_index() for checking now.
>>
> Good point. I'm not sure exactly how `noreg` works, but as you said,
> verify_oop_index() isn't complaining.
>

From aeubanks at google.com  Thu Jun 27 17:45:19 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Thu, 27 Jun 2019 17:45:19 +0000
Subject: hg: tsan/dev: Fix clobbered address passed to TSAN
Message-ID: <201906271745.x5RHjKaR018561@aojmv0008.oracle.com>

Changeset: 2d331264b87c
Author:    aeubanks
Date:      2019-06-26 08:42 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/2d331264b87c

Fix clobbered address passed to TSAN

access_load_at() sometimes clobbered the address we passed to TSAN in tsan_observe_load_or_store().
Do the TSAN instrumentation first, then do the load. Initially the order was access_load_at(), then TSAN instrumentation. Field accesses can behave as volatile, and I wanted arrays to match up with the field TSAN instrumentation/actual load order. But it doesn't really matter for array loads since they can never be volatile.

Added some asserts for fastdebug that the oop is valid and that the field/element offset is within the oop size. The size check fails without the TSAN instrumentation/actual load reordering because the offset register was clobbered.

! src/hotspot/cpu/x86/templateTable_x86.cpp
! src/hotspot/share/runtime/sharedRuntime.cpp
! src/hotspot/share/runtime/sharedRuntime.hpp


From jcbeyler at google.com  Thu Jun 27 21:48:32 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Thu, 27 Jun 2019 14:48:32 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
Message-ID: <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>

Hi Man,

https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
// Note Well.
->
// NOTE:

https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
-> I would put a note somewhere that if you change the 40 for HEAP_SIZE you
should change the Xms/Xmx... (I imagined we could create the Xms/Xmx with
string concatenation but that seems overkill)

https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
   - I find TsanOopBucket is not the right name, it might be TsanOop

  -  120       do {
 121         h = hash(h);
 122         bucket = &buckets[h % _size];
 123       } while (bucket->has_oop() && bucket->get_oop() != o);

This could be put in a helper method because done twice, no?

- print_map is not used, is it?
- oopMap should be oop_map no?

I admit I have not finished looking at rebuild_oops_map, it is a bit big
;-) could we break it up?

Thanks,
Jc

From aeubanks at google.com  Fri Jun 28 00:45:50 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Thu, 27 Jun 2019 17:45:50 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
 <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
Message-ID: <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>

Could you change the comment in tsanOopMap.hpp
  39 // 1. add() is only passed a live oop.
  40 // 2. add() must be thread-safe wrt itself.
"add()" -> "add_*()"


Another comment in tsanExternalDecls.hpp which seems inaccurate
+  // Called lazily when an oop should start to be tracked by Tsan.
It seems like we're calling __tsan_java_alloc() eagerly, not lazily.


tsanOopMap.cpp:

We fixed this, comment is wrong:
 448 // TODO: We never call __tsan_java_fini; we should.
And right above, maybe we could deallocate where we call __tsan_java_fini?
 447 // TODO: We never deallocate the oopMap; we should.

Should this instead be an assert and we do the check for ThreadSanitizer in
the caller?
 456   if (!ThreadSanitizer) return;


I also need more time to review rebuild_oops_map() :).

On Thu, Jun 27, 2019 at 2:49 PM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Hi Man,
>
>
> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
> // Note Well.
> ->
> // NOTE:
>
>
> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
> -> I would put a note somewhere that if you change the 40 for HEAP_SIZE you
> should change the Xms/Xmx... (I imagined we could create the Xms/Xmx with
> string concatenation but that seems overkill)
>
>
> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
>    - I find TsanOopBucket is not the right name, it might be TsanOop
>
>   -  120       do {
>  121         h = hash(h);
>  122         bucket = &buckets[h % _size];
>  123       } while (bucket->has_oop() && bucket->get_oop() != o);
>
> This could be put in a helper method because done twice, no?
>
> - print_map is not used, is it?
> - oopMap should be oop_map no?
>
> I admit I have not finished looking at rebuild_oops_map, it is a bit big
> ;-) could we break it up?
>
> Thanks,
> Jc
>

From aeubanks at google.com  Fri Jun 28 18:34:20 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Fri, 28 Jun 2019 11:34:20 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
 <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
 <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>
Message-ID: <CAPW48srT_8YiJosgSkaz8eX+LsVouCVmh2SG3Zz9WWLa594BSw@mail.gmail.com>

I wonder if for the part in TsanOopSizeMap::rebuild_oops_map where we tell
TSAN about moved objects, instead of repeatedly doing passes until
everything is moved (which as mentioned in the comments has a worst case of
O(N^2)), we create a new fake heap area, for each oop we pretend to move it
to the new fake heap area in a contiguous chunk of memory, then pretend to
move each oop from the new fake area to the final location after the GC
move. That would be O(N) and would (I think) simplify the code.
TSAN <https://clang.llvm.org/docs/ThreadSanitizer.html> already has 5-10x
memory overhead.

On Thu, Jun 27, 2019 at 5:45 PM Arthur Eubanks <aeubanks at google.com> wrote:

> Could you change the comment in tsanOopMap.hpp
>   39 // 1. add() is only passed a live oop.
>   40 // 2. add() must be thread-safe wrt itself.
> "add()" -> "add_*()"
>
>
> Another comment in tsanExternalDecls.hpp which seems inaccurate
> +  // Called lazily when an oop should start to be tracked by Tsan.
> It seems like we're calling __tsan_java_alloc() eagerly, not lazily.
>
>
> tsanOopMap.cpp:
>
> We fixed this, comment is wrong:
>  448 // TODO: We never call __tsan_java_fini; we should.
> And right above, maybe we could deallocate where we call __tsan_java_fini?
>  447 // TODO: We never deallocate the oopMap; we should.
>
> Should this instead be an assert and we do the check for ThreadSanitizer
> in the caller?
>  456   if (!ThreadSanitizer) return;
>
>
> I also need more time to review rebuild_oops_map() :).
>
> On Thu, Jun 27, 2019 at 2:49 PM Jean Christophe Beyler <
> jcbeyler at google.com> wrote:
>
>> Hi Man,
>>
>>
>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
>> // Note Well.
>> ->
>> // NOTE:
>>
>>
>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
>> -> I would put a note somewhere that if you change the 40 for HEAP_SIZE
>> you
>> should change the Xms/Xmx... (I imagined we could create the Xms/Xmx with
>> string concatenation but that seems overkill)
>>
>>
>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
>>    - I find TsanOopBucket is not the right name, it might be TsanOop
>>
>>   -  120       do {
>>  121         h = hash(h);
>>  122         bucket = &buckets[h % _size];
>>  123       } while (bucket->has_oop() && bucket->get_oop() != o);
>>
>> This could be put in a helper method because done twice, no?
>>
>> - print_map is not used, is it?
>> - oopMap should be oop_map no?
>>
>> I admit I have not finished looking at rebuild_oops_map, it is a bit big
>> ;-) could we break it up?
>>
>> Thanks,
>> Jc
>>
>

From manc at google.com  Fri Jun 28 19:33:57 2019
From: manc at google.com (Man Cao)
Date: Fri, 28 Jun 2019 12:33:57 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CAPW48srT_8YiJosgSkaz8eX+LsVouCVmh2SG3Zz9WWLa594BSw@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
 <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
 <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>
 <CAPW48srT_8YiJosgSkaz8eX+LsVouCVmh2SG3Zz9WWLa594BSw@mail.gmail.com>
Message-ID: <CA+w6HxYCPLFuUm=YF42MJvp8jqegQUyooG3rCVRBvCixsY+ZRQ@mail.gmail.com>

I don't think it's worth it to change the algorithm for
TsanOopSizeMap::rebuild_oops_map at this point. If want to improve
performance, we should first think for better ways for tracking
allocations, or parallelize TsanOopSizeMap::rebuild_oops_map. I'm happy to
do other cleanup and refactoring, though.

-Man


On Fri, Jun 28, 2019 at 11:34 AM Arthur Eubanks <aeubanks at google.com> wrote:

> I wonder if for the part in TsanOopSizeMap::rebuild_oops_map where we tell
> TSAN about moved objects, instead of repeatedly doing passes until
> everything is moved (which as mentioned in the comments has a worst case of
> O(N^2)), we create a new fake heap area, for each oop we pretend to move it
> to the new fake heap area in a contiguous chunk of memory, then pretend to
> move each oop from the new fake area to the final location after the GC
> move. That would be O(N) and would (I think) simplify the code.
> TSAN <https://clang.llvm.org/docs/ThreadSanitizer.html> already has 5-10x
> memory overhead.
>
> On Thu, Jun 27, 2019 at 5:45 PM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
>> Could you change the comment in tsanOopMap.hpp
>>   39 // 1. add() is only passed a live oop.
>>   40 // 2. add() must be thread-safe wrt itself.
>> "add()" -> "add_*()"
>>
>>
>> Another comment in tsanExternalDecls.hpp which seems inaccurate
>> +  // Called lazily when an oop should start to be tracked by Tsan.
>> It seems like we're calling __tsan_java_alloc() eagerly, not lazily.
>>
>>
>> tsanOopMap.cpp:
>>
>> We fixed this, comment is wrong:
>>  448 // TODO: We never call __tsan_java_fini; we should.
>> And right above, maybe we could deallocate where we call __tsan_java_fini?
>>  447 // TODO: We never deallocate the oopMap; we should.
>>
>> Should this instead be an assert and we do the check for ThreadSanitizer
>> in the caller?
>>  456   if (!ThreadSanitizer) return;
>>
>>
>> I also need more time to review rebuild_oops_map() :).
>>
>> On Thu, Jun 27, 2019 at 2:49 PM Jean Christophe Beyler <
>> jcbeyler at google.com> wrote:
>>
>>> Hi Man,
>>>
>>>
>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
>>> // Note Well.
>>> ->
>>> // NOTE:
>>>
>>>
>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
>>> -> I would put a note somewhere that if you change the 40 for HEAP_SIZE
>>> you
>>> should change the Xms/Xmx... (I imagined we could create the Xms/Xmx with
>>> string concatenation but that seems overkill)
>>>
>>>
>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
>>>    - I find TsanOopBucket is not the right name, it might be TsanOop
>>>
>>>   -  120       do {
>>>  121         h = hash(h);
>>>  122         bucket = &buckets[h % _size];
>>>  123       } while (bucket->has_oop() && bucket->get_oop() != o);
>>>
>>> This could be put in a helper method because done twice, no?
>>>
>>> - print_map is not used, is it?
>>> - oopMap should be oop_map no?
>>>
>>> I admit I have not finished looking at rebuild_oops_map, it is a bit big
>>> ;-) could we break it up?
>>>
>>> Thanks,
>>> Jc
>>>
>>

From manc at google.com  Sat Jun 29 02:00:19 2019
From: manc at google.com (Man Cao)
Date: Fri, 28 Jun 2019 19:00:19 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CA+w6HxYCPLFuUm=YF42MJvp8jqegQUyooG3rCVRBvCixsY+ZRQ@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
 <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
 <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>
 <CAPW48srT_8YiJosgSkaz8eX+LsVouCVmh2SG3Zz9WWLa594BSw@mail.gmail.com>
 <CA+w6HxYCPLFuUm=YF42MJvp8jqegQUyooG3rCVRBvCixsY+ZRQ@mail.gmail.com>
Message-ID: <CA+w6HxbC9QamrLJo2uGyZYc8je40QDmtHhAbW_PLec-tMS5nUA@mail.gmail.com>

I've addressed all cleanup/refactoring comments except this one:

> Should this instead be an assert and we do the check for ThreadSanitizer
in the caller?
>  456   if (!ThreadSanitizer) return;
It would be ugly to make WeakProcessor conditionally run
TsanOopMap::weak_oops_do(), due to its use of function pointers.
JvmtiExport::weak_oops_do and Jfr::weak_oops_do have similar checks that
guards their entire execution.

I also updated comment at the top of tsanOopMap.hpp, as it tracks all Java
objects instead of only lock objects, and fixed a few compiler warnings.
Also removed indention for "private:" / "public:" keywords, per latest
OpenJDK style.

https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00-01.diff/
https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.01/

-Man


On Fri, Jun 28, 2019 at 12:33 PM Man Cao <manc at google.com> wrote:

> I don't think it's worth it to change the algorithm for
> TsanOopSizeMap::rebuild_oops_map at this point. If want to improve
> performance, we should first think for better ways for tracking
> allocations, or parallelize TsanOopSizeMap::rebuild_oops_map. I'm happy to
> do other cleanup and refactoring, though.
>
> -Man
>
>
> On Fri, Jun 28, 2019 at 11:34 AM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
>> I wonder if for the part in TsanOopSizeMap::rebuild_oops_map where we
>> tell TSAN about moved objects, instead of repeatedly doing passes until
>> everything is moved (which as mentioned in the comments has a worst case of
>> O(N^2)), we create a new fake heap area, for each oop we pretend to move it
>> to the new fake heap area in a contiguous chunk of memory, then pretend to
>> move each oop from the new fake area to the final location after the GC
>> move. That would be O(N) and would (I think) simplify the code.
>> TSAN <https://clang.llvm.org/docs/ThreadSanitizer.html> already has
>> 5-10x memory overhead.
>>
>> On Thu, Jun 27, 2019 at 5:45 PM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>>> Could you change the comment in tsanOopMap.hpp
>>>   39 // 1. add() is only passed a live oop.
>>>   40 // 2. add() must be thread-safe wrt itself.
>>> "add()" -> "add_*()"
>>>
>>>
>>> Another comment in tsanExternalDecls.hpp which seems inaccurate
>>> +  // Called lazily when an oop should start to be tracked by Tsan.
>>> It seems like we're calling __tsan_java_alloc() eagerly, not lazily.
>>>
>>>
>>> tsanOopMap.cpp:
>>>
>>> We fixed this, comment is wrong:
>>>  448 // TODO: We never call __tsan_java_fini; we should.
>>> And right above, maybe we could deallocate where we call
>>> __tsan_java_fini?
>>>  447 // TODO: We never deallocate the oopMap; we should.
>>>
>>> Should this instead be an assert and we do the check for ThreadSanitizer
>>> in the caller?
>>>  456   if (!ThreadSanitizer) return;
>>>
>>>
>>> I also need more time to review rebuild_oops_map() :).
>>>
>>> On Thu, Jun 27, 2019 at 2:49 PM Jean Christophe Beyler <
>>> jcbeyler at google.com> wrote:
>>>
>>>> Hi Man,
>>>>
>>>>
>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
>>>> // Note Well.
>>>> ->
>>>> // NOTE:
>>>>
>>>>
>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
>>>> -> I would put a note somewhere that if you change the 40 for HEAP_SIZE
>>>> you
>>>> should change the Xms/Xmx... (I imagined we could create the Xms/Xmx
>>>> with
>>>> string concatenation but that seems overkill)
>>>>
>>>>
>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
>>>>    - I find TsanOopBucket is not the right name, it might be TsanOop
>>>>
>>>>   -  120       do {
>>>>  121         h = hash(h);
>>>>  122         bucket = &buckets[h % _size];
>>>>  123       } while (bucket->has_oop() && bucket->get_oop() != o);
>>>>
>>>> This could be put in a helper method because done twice, no?
>>>>
>>>> - print_map is not used, is it?
>>>> - oopMap should be oop_map no?
>>>>
>>>> I admit I have not finished looking at rebuild_oops_map, it is a bit big
>>>> ;-) could we break it up?
>>>>
>>>> Thanks,
>>>> Jc
>>>>
>>>