From jcbeyler at google.com  Mon Jul  1 16:09:43 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Mon, 1 Jul 2019 09:09:43 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CA+w6HxbC9QamrLJo2uGyZYc8je40QDmtHhAbW_PLec-tMS5nUA@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
 <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
 <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>
 <CAPW48srT_8YiJosgSkaz8eX+LsVouCVmh2SG3Zz9WWLa594BSw@mail.gmail.com>
 <CA+w6HxYCPLFuUm=YF42MJvp8jqegQUyooG3rCVRBvCixsY+ZRQ@mail.gmail.com>
 <CA+w6HxbC9QamrLJo2uGyZYc8je40QDmtHhAbW_PLec-tMS5nUA@mail.gmail.com>
Message-ID: <CAF9BGByNBJX2GvHZYury4s9kVdQGSGj3EsJ1uUJEVhKcqEP9bA@mail.gmail.com>

Hi Man,

Small nits:

buckets -> should be _buckets
+    TsanOopSizeMap* newMap = new TsanOopSizeMap(map_size / 2); -> should
be new_map

No need for a new webrev for me, it looks good,
Jc

On Fri, Jun 28, 2019 at 7:00 PM Man Cao <manc at google.com> wrote:

> I've addressed all cleanup/refactoring comments except this one:
>
> > Should this instead be an assert and we do the check for ThreadSanitizer
> in the caller?
> >  456   if (!ThreadSanitizer) return;
> It would be ugly to make WeakProcessor conditionally run
> TsanOopMap::weak_oops_do(), due to its use of function pointers.
> JvmtiExport::weak_oops_do and Jfr::weak_oops_do have similar checks that
> guards their entire execution.
>
> I also updated comment at the top of tsanOopMap.hpp, as it tracks all Java
> objects instead of only lock objects, and fixed a few compiler warnings.
> Also removed indention for "private:" / "public:" keywords, per latest
> OpenJDK style.
>
> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00-01.diff/
> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.01/
>
> -Man
>
>
> On Fri, Jun 28, 2019 at 12:33 PM Man Cao <manc at google.com> wrote:
>
>> I don't think it's worth it to change the algorithm for
>> TsanOopSizeMap::rebuild_oops_map at this point. If want to improve
>> performance, we should first think for better ways for tracking
>> allocations, or parallelize TsanOopSizeMap::rebuild_oops_map. I'm happy to
>> do other cleanup and refactoring, though.
>>
>> -Man
>>
>>
>> On Fri, Jun 28, 2019 at 11:34 AM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>>> I wonder if for the part in TsanOopSizeMap::rebuild_oops_map where we
>>> tell TSAN about moved objects, instead of repeatedly doing passes until
>>> everything is moved (which as mentioned in the comments has a worst case of
>>> O(N^2)), we create a new fake heap area, for each oop we pretend to move it
>>> to the new fake heap area in a contiguous chunk of memory, then pretend to
>>> move each oop from the new fake area to the final location after the GC
>>> move. That would be O(N) and would (I think) simplify the code.
>>> TSAN <https://clang.llvm.org/docs/ThreadSanitizer.html> already has
>>> 5-10x memory overhead.
>>>
>>> On Thu, Jun 27, 2019 at 5:45 PM Arthur Eubanks <aeubanks at google.com>
>>> wrote:
>>>
>>>> Could you change the comment in tsanOopMap.hpp
>>>>   39 // 1. add() is only passed a live oop.
>>>>   40 // 2. add() must be thread-safe wrt itself.
>>>> "add()" -> "add_*()"
>>>>
>>>>
>>>> Another comment in tsanExternalDecls.hpp which seems inaccurate
>>>> +  // Called lazily when an oop should start to be tracked by Tsan.
>>>> It seems like we're calling __tsan_java_alloc() eagerly, not lazily.
>>>>
>>>>
>>>> tsanOopMap.cpp:
>>>>
>>>> We fixed this, comment is wrong:
>>>>  448 // TODO: We never call __tsan_java_fini; we should.
>>>> And right above, maybe we could deallocate where we call
>>>> __tsan_java_fini?
>>>>  447 // TODO: We never deallocate the oopMap; we should.
>>>>
>>>> Should this instead be an assert and we do the check for
>>>> ThreadSanitizer in the caller?
>>>>  456   if (!ThreadSanitizer) return;
>>>>
>>>>
>>>> I also need more time to review rebuild_oops_map() :).
>>>>
>>>> On Thu, Jun 27, 2019 at 2:49 PM Jean Christophe Beyler <
>>>> jcbeyler at google.com> wrote:
>>>>
>>>>> Hi Man,
>>>>>
>>>>>
>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
>>>>> // Note Well.
>>>>> ->
>>>>> // NOTE:
>>>>>
>>>>>
>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
>>>>> -> I would put a note somewhere that if you change the 40 for
>>>>> HEAP_SIZE you
>>>>> should change the Xms/Xmx... (I imagined we could create the Xms/Xmx
>>>>> with
>>>>> string concatenation but that seems overkill)
>>>>>
>>>>>
>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
>>>>>    - I find TsanOopBucket is not the right name, it might be TsanOop
>>>>>
>>>>>   -  120       do {
>>>>>  121         h = hash(h);
>>>>>  122         bucket = &buckets[h % _size];
>>>>>  123       } while (bucket->has_oop() && bucket->get_oop() != o);
>>>>>
>>>>> This could be put in a helper method because done twice, no?
>>>>>
>>>>> - print_map is not used, is it?
>>>>> - oopMap should be oop_map no?
>>>>>
>>>>> I admit I have not finished looking at rebuild_oops_map, it is a bit
>>>>> big
>>>>> ;-) could we break it up?
>>>>>
>>>>> Thanks,
>>>>> Jc
>>>>>
>>>>

-- 

Thanks,
Jc

From aeubanks at google.com  Mon Jul  1 17:39:03 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 1 Jul 2019 10:39:03 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CAF9BGByNBJX2GvHZYury4s9kVdQGSGj3EsJ1uUJEVhKcqEP9bA@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
 <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
 <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>
 <CAPW48srT_8YiJosgSkaz8eX+LsVouCVmh2SG3Zz9WWLa594BSw@mail.gmail.com>
 <CA+w6HxYCPLFuUm=YF42MJvp8jqegQUyooG3rCVRBvCixsY+ZRQ@mail.gmail.com>
 <CA+w6HxbC9QamrLJo2uGyZYc8je40QDmtHhAbW_PLec-tMS5nUA@mail.gmail.com>
 <CAF9BGByNBJX2GvHZYury4s9kVdQGSGj3EsJ1uUJEVhKcqEP9bA@mail.gmail.com>
Message-ID: <CAPW48so5j5cYty=G0rf6c37NvA8-tjoGK+s+PugjKuq90F4KqQ@mail.gmail.com>

          // Update source and target ranges.
          source_low = MIN2(source_low, (char *)source_obj);
          source_high = MAX2(source_high, (char *)source_obj +
obj_size_bytes);
          target_low = MIN2(target_low, (char *)target_obj);
          target_high = MAX2(target_high, (char *)target_obj +
obj_size_bytes);
          if (target_obj < source_obj) {
            ++(*n_downward_moves);
          }
          // Append to the moves list.
          PendingMove move = {(char *)source_obj, (char *)target_obj,
                              obj_size_bytes};

(source_obj + obj_size_bytes) and (target_obj + obj_size_bytes) can be
replaced with move.source_end()/target_end(), right? Then might as well
replace (source_obj) with move.source_begin() and replace (target_obj) with
move.target_begin().

Otherwise LGTM, no need for new webrev.

On Mon, Jul 1, 2019 at 9:09 AM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Hi Man,
>
> Small nits:
>
> buckets -> should be _buckets
> +    TsanOopSizeMap* newMap = new TsanOopSizeMap(map_size / 2); -> should
> be new_map
>
> No need for a new webrev for me, it looks good,
> Jc
>
> On Fri, Jun 28, 2019 at 7:00 PM Man Cao <manc at google.com> wrote:
>
>> I've addressed all cleanup/refactoring comments except this one:
>>
>> > Should this instead be an assert and we do the check for
>> ThreadSanitizer in the caller?
>> >  456   if (!ThreadSanitizer) return;
>> It would be ugly to make WeakProcessor conditionally run
>> TsanOopMap::weak_oops_do(), due to its use of function pointers.
>> JvmtiExport::weak_oops_do and Jfr::weak_oops_do have similar checks that
>> guards their entire execution.
>>
>> I also updated comment at the top of tsanOopMap.hpp, as it tracks all
>> Java objects instead of only lock objects, and fixed a few compiler
>> warnings.
>> Also removed indention for "private:" / "public:" keywords, per latest
>> OpenJDK style.
>>
>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00-01.diff/
>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.01/
>>
>> -Man
>>
>>
>> On Fri, Jun 28, 2019 at 12:33 PM Man Cao <manc at google.com> wrote:
>>
>>> I don't think it's worth it to change the algorithm for
>>> TsanOopSizeMap::rebuild_oops_map at this point. If want to improve
>>> performance, we should first think for better ways for tracking
>>> allocations, or parallelize TsanOopSizeMap::rebuild_oops_map. I'm happy to
>>> do other cleanup and refactoring, though.
>>>
>>> -Man
>>>
>>>
>>> On Fri, Jun 28, 2019 at 11:34 AM Arthur Eubanks <aeubanks at google.com>
>>> wrote:
>>>
>>>> I wonder if for the part in TsanOopSizeMap::rebuild_oops_map where we
>>>> tell TSAN about moved objects, instead of repeatedly doing passes until
>>>> everything is moved (which as mentioned in the comments has a worst case of
>>>> O(N^2)), we create a new fake heap area, for each oop we pretend to move it
>>>> to the new fake heap area in a contiguous chunk of memory, then pretend to
>>>> move each oop from the new fake area to the final location after the GC
>>>> move. That would be O(N) and would (I think) simplify the code.
>>>> TSAN <https://clang.llvm.org/docs/ThreadSanitizer.html> already has
>>>> 5-10x memory overhead.
>>>>
>>>> On Thu, Jun 27, 2019 at 5:45 PM Arthur Eubanks <aeubanks at google.com>
>>>> wrote:
>>>>
>>>>> Could you change the comment in tsanOopMap.hpp
>>>>>   39 // 1. add() is only passed a live oop.
>>>>>   40 // 2. add() must be thread-safe wrt itself.
>>>>> "add()" -> "add_*()"
>>>>>
>>>>>
>>>>> Another comment in tsanExternalDecls.hpp which seems inaccurate
>>>>> +  // Called lazily when an oop should start to be tracked by Tsan.
>>>>> It seems like we're calling __tsan_java_alloc() eagerly, not lazily.
>>>>>
>>>>>
>>>>> tsanOopMap.cpp:
>>>>>
>>>>> We fixed this, comment is wrong:
>>>>>  448 // TODO: We never call __tsan_java_fini; we should.
>>>>> And right above, maybe we could deallocate where we call
>>>>> __tsan_java_fini?
>>>>>  447 // TODO: We never deallocate the oopMap; we should.
>>>>>
>>>>> Should this instead be an assert and we do the check for
>>>>> ThreadSanitizer in the caller?
>>>>>  456   if (!ThreadSanitizer) return;
>>>>>
>>>>>
>>>>> I also need more time to review rebuild_oops_map() :).
>>>>>
>>>>> On Thu, Jun 27, 2019 at 2:49 PM Jean Christophe Beyler <
>>>>> jcbeyler at google.com> wrote:
>>>>>
>>>>>> Hi Man,
>>>>>>
>>>>>>
>>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
>>>>>> // Note Well.
>>>>>> ->
>>>>>> // NOTE:
>>>>>>
>>>>>>
>>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
>>>>>> -> I would put a note somewhere that if you change the 40 for
>>>>>> HEAP_SIZE you
>>>>>> should change the Xms/Xmx... (I imagined we could create the Xms/Xmx
>>>>>> with
>>>>>> string concatenation but that seems overkill)
>>>>>>
>>>>>>
>>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
>>>>>>    - I find TsanOopBucket is not the right name, it might be TsanOop
>>>>>>
>>>>>>   -  120       do {
>>>>>>  121         h = hash(h);
>>>>>>  122         bucket = &buckets[h % _size];
>>>>>>  123       } while (bucket->has_oop() && bucket->get_oop() != o);
>>>>>>
>>>>>> This could be put in a helper method because done twice, no?
>>>>>>
>>>>>> - print_map is not used, is it?
>>>>>> - oopMap should be oop_map no?
>>>>>>
>>>>>> I admit I have not finished looking at rebuild_oops_map, it is a bit
>>>>>> big
>>>>>> ;-) could we break it up?
>>>>>>
>>>>>> Thanks,
>>>>>> Jc
>>>>>>
>>>>>
>
> --
>
> Thanks,
> Jc
>

From manc at google.com  Mon Jul  1 18:16:39 2019
From: manc at google.com (Man Cao)
Date: Mon, 1 Jul 2019 11:16:39 -0700
Subject: RFR (L): Add TSAN instrumentation for allocation and garbage
 collection
In-Reply-To: <CAPW48so5j5cYty=G0rf6c37NvA8-tjoGK+s+PugjKuq90F4KqQ@mail.gmail.com>
References: <CA+w6HxbT2OpAqv2EKqxRYWsOw+m8S+EZcz7u5bdiXty_d9c4eg@mail.gmail.com>
 <CAF9BGBwRWhPyUe0UEUN3p0aYOjhuuJEmqTaW6-dXEh7Q1h1rNA@mail.gmail.com>
 <CAPW48srk1-UygLBOqbN_irR5tEjdaMv-nQb5K82Jamfxgn80vw@mail.gmail.com>
 <CAPW48srT_8YiJosgSkaz8eX+LsVouCVmh2SG3Zz9WWLa594BSw@mail.gmail.com>
 <CA+w6HxYCPLFuUm=YF42MJvp8jqegQUyooG3rCVRBvCixsY+ZRQ@mail.gmail.com>
 <CA+w6HxbC9QamrLJo2uGyZYc8je40QDmtHhAbW_PLec-tMS5nUA@mail.gmail.com>
 <CAF9BGByNBJX2GvHZYury4s9kVdQGSGj3EsJ1uUJEVhKcqEP9bA@mail.gmail.com>
 <CAPW48so5j5cYty=G0rf6c37NvA8-tjoGK+s+PugjKuq90F4KqQ@mail.gmail.com>
Message-ID: <CA+w6HxYxB24wjjADWmOJbkEeUtekmz7jDk2mXSJCzb4M-f04wA@mail.gmail.com>

All done. Thanks for the reviews.

-Man


On Mon, Jul 1, 2019 at 10:39 AM Arthur Eubanks <aeubanks at google.com> wrote:

>           // Update source and target ranges.
>           source_low = MIN2(source_low, (char *)source_obj);
>           source_high = MAX2(source_high, (char *)source_obj +
> obj_size_bytes);
>           target_low = MIN2(target_low, (char *)target_obj);
>           target_high = MAX2(target_high, (char *)target_obj +
> obj_size_bytes);
>           if (target_obj < source_obj) {
>             ++(*n_downward_moves);
>           }
>           // Append to the moves list.
>           PendingMove move = {(char *)source_obj, (char *)target_obj,
>                               obj_size_bytes};
>
> (source_obj + obj_size_bytes) and (target_obj + obj_size_bytes) can be
> replaced with move.source_end()/target_end(), right? Then might as well
> replace (source_obj) with move.source_begin() and replace (target_obj) with
> move.target_begin().
>
> Otherwise LGTM, no need for new webrev.
>
> On Mon, Jul 1, 2019 at 9:09 AM Jean Christophe Beyler <jcbeyler at google.com>
> wrote:
>
>> Hi Man,
>>
>> Small nits:
>>
>> buckets -> should be _buckets
>> +    TsanOopSizeMap* newMap = new TsanOopSizeMap(map_size / 2); -> should
>> be new_map
>>
>> No need for a new webrev for me, it looks good,
>> Jc
>>
>> On Fri, Jun 28, 2019 at 7:00 PM Man Cao <manc at google.com> wrote:
>>
>>> I've addressed all cleanup/refactoring comments except this one:
>>>
>>> > Should this instead be an assert and we do the check for
>>> ThreadSanitizer in the caller?
>>> >  456   if (!ThreadSanitizer) return;
>>> It would be ugly to make WeakProcessor conditionally run
>>> TsanOopMap::weak_oops_do(), due to its use of function pointers.
>>> JvmtiExport::weak_oops_do and Jfr::weak_oops_do have similar checks that
>>> guards their entire execution.
>>>
>>> I also updated comment at the top of tsanOopMap.hpp, as it tracks all
>>> Java objects instead of only lock objects, and fixed a few compiler
>>> warnings.
>>> Also removed indention for "private:" / "public:" keywords, per latest
>>> OpenJDK style.
>>>
>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00-01.diff/
>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.01/
>>>
>>> -Man
>>>
>>>
>>> On Fri, Jun 28, 2019 at 12:33 PM Man Cao <manc at google.com> wrote:
>>>
>>>> I don't think it's worth it to change the algorithm for
>>>> TsanOopSizeMap::rebuild_oops_map at this point. If want to improve
>>>> performance, we should first think for better ways for tracking
>>>> allocations, or parallelize TsanOopSizeMap::rebuild_oops_map. I'm happy to
>>>> do other cleanup and refactoring, though.
>>>>
>>>> -Man
>>>>
>>>>
>>>> On Fri, Jun 28, 2019 at 11:34 AM Arthur Eubanks <aeubanks at google.com>
>>>> wrote:
>>>>
>>>>> I wonder if for the part in TsanOopSizeMap::rebuild_oops_map where we
>>>>> tell TSAN about moved objects, instead of repeatedly doing passes until
>>>>> everything is moved (which as mentioned in the comments has a worst case of
>>>>> O(N^2)), we create a new fake heap area, for each oop we pretend to move it
>>>>> to the new fake heap area in a contiguous chunk of memory, then pretend to
>>>>> move each oop from the new fake area to the final location after the GC
>>>>> move. That would be O(N) and would (I think) simplify the code.
>>>>> TSAN <https://clang.llvm.org/docs/ThreadSanitizer.html> already has
>>>>> 5-10x memory overhead.
>>>>>
>>>>> On Thu, Jun 27, 2019 at 5:45 PM Arthur Eubanks <aeubanks at google.com>
>>>>> wrote:
>>>>>
>>>>>> Could you change the comment in tsanOopMap.hpp
>>>>>>   39 // 1. add() is only passed a live oop.
>>>>>>   40 // 2. add() must be thread-safe wrt itself.
>>>>>> "add()" -> "add_*()"
>>>>>>
>>>>>>
>>>>>> Another comment in tsanExternalDecls.hpp which seems inaccurate
>>>>>> +  // Called lazily when an oop should start to be tracked by Tsan.
>>>>>> It seems like we're calling __tsan_java_alloc() eagerly, not lazily.
>>>>>>
>>>>>>
>>>>>> tsanOopMap.cpp:
>>>>>>
>>>>>> We fixed this, comment is wrong:
>>>>>>  448 // TODO: We never call __tsan_java_fini; we should.
>>>>>> And right above, maybe we could deallocate where we call
>>>>>> __tsan_java_fini?
>>>>>>  447 // TODO: We never deallocate the oopMap; we should.
>>>>>>
>>>>>> Should this instead be an assert and we do the check for
>>>>>> ThreadSanitizer in the caller?
>>>>>>  456   if (!ThreadSanitizer) return;
>>>>>>
>>>>>>
>>>>>> I also need more time to review rebuild_oops_map() :).
>>>>>>
>>>>>> On Thu, Jun 27, 2019 at 2:49 PM Jean Christophe Beyler <
>>>>>> jcbeyler at google.com> wrote:
>>>>>>
>>>>>>> Hi Man,
>>>>>>>
>>>>>>>
>>>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html
>>>>>>> // Note Well.
>>>>>>> ->
>>>>>>> // NOTE:
>>>>>>>
>>>>>>>
>>>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java.html
>>>>>>> -> I would put a note somewhere that if you change the 40 for
>>>>>>> HEAP_SIZE you
>>>>>>> should change the Xms/Xmx... (I imagined we could create the Xms/Xmx
>>>>>>> with
>>>>>>> string concatenation but that seems overkill)
>>>>>>>
>>>>>>>
>>>>>>> https://cr.openjdk.java.net/~manc/tsanOopMap/webrev.00/src/hotspot/share/tsan/tsanOopMap.cpp.html
>>>>>>>    - I find TsanOopBucket is not the right name, it might be TsanOop
>>>>>>>
>>>>>>>   -  120       do {
>>>>>>>  121         h = hash(h);
>>>>>>>  122         bucket = &buckets[h % _size];
>>>>>>>  123       } while (bucket->has_oop() && bucket->get_oop() != o);
>>>>>>>
>>>>>>> This could be put in a helper method because done twice, no?
>>>>>>>
>>>>>>> - print_map is not used, is it?
>>>>>>> - oopMap should be oop_map no?
>>>>>>>
>>>>>>> I admit I have not finished looking at rebuild_oops_map, it is a bit
>>>>>>> big
>>>>>>> ;-) could we break it up?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Jc
>>>>>>>
>>>>>>
>>
>> --
>>
>> Thanks,
>> Jc
>>
>

From manc at google.com  Mon Jul  1 18:32:31 2019
From: manc at google.com (manc at google.com)
Date: Mon, 01 Jul 2019 18:32:31 +0000
Subject: hg: tsan/dev: Add TSAN instrumentation for allocation and garbage
 collection.
Message-ID: <201907011832.x61IWWRj022448@aojmv0008.oracle.com>

Changeset: 1d3b2281db42
Author:    manc
Date:      2019-07-01 11:31 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/1d3b2281db42

Add TSAN instrumentation for allocation and garbage collection.
Reviewed-by: aeubanks, jcbeyler

! src/hotspot/cpu/x86/templateTable_x86.cpp
! src/hotspot/share/gc/shared/memAllocator.cpp
! src/hotspot/share/gc/shared/weakProcessorPhases.cpp
! src/hotspot/share/gc/shared/weakProcessorPhases.hpp
! src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp
! src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.cpp
! src/hotspot/share/runtime/mutexLocker.cpp
! src/hotspot/share/runtime/mutexLocker.hpp
! src/hotspot/share/runtime/sharedRuntime.cpp
! src/hotspot/share/runtime/sharedRuntime.hpp
! src/hotspot/share/tsan/tsan.cpp
! src/hotspot/share/tsan/tsanExternalDecls.hpp
+ src/hotspot/share/tsan/tsanOopMap.cpp
+ src/hotspot/share/tsan/tsanOopMap.hpp
+ test/hotspot/jtreg/tsan/NonRacyGarbageCollectionLoopTest.java
! test/hotspot/jtreg/tsan/TsanRunner.java


From aeubanks at google.com  Mon Jul  1 22:47:42 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 1 Jul 2019 15:47:42 -0700
Subject: [RFR]: Field suppression list
Message-ID: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>

webrev:
http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/index.html

Add a new flag -XX:ThreadSanitizerIgnoreFile=path/to/ignorefile. The file
is parsed for patterns like

# To whitelist field myBaz in a class named com.foo.Bar
com.foo.Bar myBaz
# Every field with primitive type starting with my in that class:
com.foo.Bar my*
# And every primitive field in package com.foo:
com.foo.* *

and those fields are ignored in regards to TSAN.

From manc at google.com  Tue Jul  2 23:31:05 2019
From: manc at google.com (Man Cao)
Date: Tue, 2 Jul 2019 16:31:05 -0700
Subject: [RFR]: Field suppression list
In-Reply-To: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
References: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
Message-ID: <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>

Looks good. Some nits below, no need for another webrev.

In classFileParser.cpp:
37 #include "classfile/tsanIgnoreList.hpp"
This can be guarded by TSAN_ONLY.

In
http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.cpp.html,
some whitespace can be improved:

  62  protected:
  63   const Symbol*        _class_name;
  64   const Symbol*        _field_name;
  65   Mode           _class_mode;
  66   Mode           _field_mode;
  67   FieldMatcher* _next;

No need to have more than 1 space.

 135     _exact_match = new FieldMatcher(class_symbol, field_symbol, class_mode,
 136                                    field_mode, _exact_match);

Need to align second line.

 156   char token[MAX_LINE_SIZE];
 157   int  pos = 0;
 158   int  c = fgetc(stream);

No need to have more than 1 space.

-Man


On Mon, Jul 1, 2019 at 3:59 PM Arthur Eubanks <aeubanks at google.com> wrote:

> webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/index.html
>
> Add a new flag -XX:ThreadSanitizerIgnoreFile=path/to/ignorefile. The file
> is parsed for patterns like
>
> # To whitelist field myBaz in a class named com.foo.Bar
> com.foo.Bar myBaz
> # Every field with primitive type starting with my in that class:
> com.foo.Bar my*
> # And every primitive field in package com.foo:
> com.foo.* *
>
> and those fields are ignored in regards to TSAN.
>

From aeubanks at google.com  Wed Jul  3 04:18:53 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Tue, 2 Jul 2019 21:18:53 -0700
Subject: [RFR]: Field suppression list
In-Reply-To: <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>
References: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
 <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>
Message-ID: <CAPW48spLUaCJRFXzk0KGsGK2XyabiTS5Wwf-5ABmG_p94VPi4g@mail.gmail.com>

On Tue, Jul 2, 2019 at 4:31 PM Man Cao <manc at google.com> wrote:

> Looks good. Some nits below, no need for another webrev.
>
> In classFileParser.cpp:
> 37 #include "classfile/tsanIgnoreList.hpp"
> This can be guarded by TSAN_ONLY.
>
We are using #if INCLUDE_TSAN rather than TSAN_ONLY for guarding #includes,
doing that instead.

>
> In
> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.cpp.html,
> some whitespace can be improved:
>
>   62  protected:
>   63   const Symbol*        _class_name;
>   64   const Symbol*        _field_name;
>   65   Mode           _class_mode;
>   66   Mode           _field_mode;
>   67   FieldMatcher* _next;
>
> No need to have more than 1 space.
>
>  135     _exact_match = new FieldMatcher(class_symbol, field_symbol, class_mode,
>  136                                    field_mode, _exact_match);
>
> Need to align second line.
>
>  156   char token[MAX_LINE_SIZE];
>  157   int  pos = 0;
>  158   int  c = fgetc(stream);
>
> No need to have more than 1 space.
>
> Done.

> -Man
>
>
> On Mon, Jul 1, 2019 at 3:59 PM Arthur Eubanks <aeubanks at google.com> wrote:
>
>> webrev:
>> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/index.html
>>
>> Add a new flag -XX:ThreadSanitizerIgnoreFile=path/to/ignorefile. The file
>> is parsed for patterns like
>>
>> # To whitelist field myBaz in a class named com.foo.Bar
>> com.foo.Bar myBaz
>> # Every field with primitive type starting with my in that class:
>> com.foo.Bar my*
>> # And every primitive field in package com.foo:
>> com.foo.* *
>>
>> and those fields are ignored in regards to TSAN.
>>
>

From aeubanks at google.com  Wed Jul  3 17:44:16 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 3 Jul 2019 10:44:16 -0700
Subject: Crash on finalizer in tsanOopMap
Message-ID: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>

Trying to implement finalizer support, I created a test (attached), and
tsanOopMap.cpp seems buggy.

# after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error
(/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
pid=16182, tid=16188
#  assert(heap->is_in(target_oop)) failed: Adjustment failed
#
# JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
13-internal+0-adhoc.aeubanks.tsan)
# Java VM: OpenJDK 64-Bit Server VM (fastdebug
13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered, compressed
oops, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x164e63b]
 TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*, char**,
char**)+0x24b
#
# No core dump will be written. Core dumps have been disabled. To enable
core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
#
/usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

From aeubanks at google.com  Mon Jul  8 20:22:47 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 8 Jul 2019 13:22:47 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
Message-ID: <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>

I have a simpler reproducer attached.

Some logging:
           // The object survived GC, add its updated oop to the new oops
map.
           oop target_oop = cast_to_oop((intptr_t)source_obj);
           pointer_adjuster->do_oop(&target_oop);
+          oopDesc *target_obj = target_oop;
+          if (!heap->is_in(target_oop)) {
+            fprintf(stderr, "%p\n", source_obj);
+            fprintf(stderr, "%p\n", target_obj);
+          }
           assert(heap->is_in(target_oop), "Adjustment failed");
-          oopDesc *target_obj = target_oop;
           new_map->put(target_obj, obj_size);

0xffea5570
0xf8238138

On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com> wrote:

> Trying to implement finalizer support, I created a test (attached), and
> tsanOopMap.cpp seems buggy.
>
> # after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  Internal Error
> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
> pid=16182, tid=16188
> #  assert(heap->is_in(target_oop)) failed: Adjustment failed
> #
> # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
> 13-internal+0-adhoc.aeubanks.tsan)
> # Java VM: OpenJDK 64-Bit Server VM (fastdebug
> 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered, compressed
> oops, g1 gc, linux-amd64)
> # Problematic frame:
> # V  [libjvm.so+0x164e63b]
>  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
> OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*, char**,
> char**)+0x24b
> #
> # No core dump will be written. Core dumps have been disabled. To enable
> core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> #
> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
>

From jcbeyler at google.com  Tue Jul  9 16:05:59 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Tue, 9 Jul 2019 09:05:59 -0700
Subject: [RFR]: Field suppression list
In-Reply-To: <CAPW48spLUaCJRFXzk0KGsGK2XyabiTS5Wwf-5ABmG_p94VPi4g@mail.gmail.com>
References: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
 <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>
 <CAPW48spLUaCJRFXzk0KGsGK2XyabiTS5Wwf-5ABmG_p94VPi4g@mail.gmail.com>
Message-ID: <CAF9BGBw1fEG=K5NpZ+ZcnHUYfxun69vsw_zfg6X4M1a1Mrc_yg@mail.gmail.com>

Hi Arthur,

Sorry for the late reply :)

http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.hpp.html

  33 // Lines that start with '#' are considered comment.

-> are considered comments.


  36 // Here are few examples.

-> Here are a few examples:


http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.cpp.html

  - line 156 could be removed (extra empty line)

  - whereas line 175 could use an extra line :)

  - Is there a reason that we are using fgetc? I would just use fgets
and be done with all this logic handling lines...

  - FWIW, I think you should extract the whole file parsing into a
parse_from_file method, which would make the init : open a file, parse
it via parse_from_file, close it

  - parse_from_line:

     - I find this error prone:

        - I would really find a '#' and replace it with '\0' if found

        - then I would do a sscanf and check its return

        - I would add a few extra lines to better understand this code btw :)


  -   35     Exact,

     -> If we are going to do = 1, = 2, ... ; might as well do = 0


  41   FieldMatcher(const Symbol* class_name, const Symbol* field_name,
  42                Mode class_mode, Mode field_mode, FieldMatcher* next)

     -> Somehow class_name, class_mode, field_name, field_mode, next
seems like a better order when looking at it (to me :-))


Thanks,

Jc



On Tue, Jul 2, 2019 at 9:19 PM Arthur Eubanks <aeubanks at google.com> wrote:

> On Tue, Jul 2, 2019 at 4:31 PM Man Cao <manc at google.com> wrote:
>
> > Looks good. Some nits below, no need for another webrev.
> >
> > In classFileParser.cpp:
> > 37 #include "classfile/tsanIgnoreList.hpp"
> > This can be guarded by TSAN_ONLY.
> >
> We are using #if INCLUDE_TSAN rather than TSAN_ONLY for guarding #includes,
> doing that instead.
>
> >
> > In
> >
> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.cpp.html
> ,
> > some whitespace can be improved:
> >
> >   62  protected:
> >   63   const Symbol*        _class_name;
> >   64   const Symbol*        _field_name;
> >   65   Mode           _class_mode;
> >   66   Mode           _field_mode;
> >   67   FieldMatcher* _next;
> >
> > No need to have more than 1 space.
> >
> >  135     _exact_match = new FieldMatcher(class_symbol, field_symbol,
> class_mode,
> >  136                                    field_mode, _exact_match);
> >
> > Need to align second line.
> >
> >  156   char token[MAX_LINE_SIZE];
> >  157   int  pos = 0;
> >  158   int  c = fgetc(stream);
> >
> > No need to have more than 1 space.
> >
> > Done.
>
> > -Man
> >
> >
> > On Mon, Jul 1, 2019 at 3:59 PM Arthur Eubanks <aeubanks at google.com>
> wrote:
> >
> >> webrev:
> >> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/index.html
> >>
> >> Add a new flag -XX:ThreadSanitizerIgnoreFile=path/to/ignorefile. The
> file
> >> is parsed for patterns like
> >>
> >> # To whitelist field myBaz in a class named com.foo.Bar
> >> com.foo.Bar myBaz
> >> # Every field with primitive type starting with my in that class:
> >> com.foo.Bar my*
> >> # And every primitive field in package com.foo:
> >> com.foo.* *
> >>
> >> and those fields are ignored in regards to TSAN.
> >>
> >
>


-- 

Thanks,
Jc

From aeubanks at google.com  Tue Jul  9 22:23:35 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Tue, 9 Jul 2019 15:23:35 -0700
Subject: [RFR]: Field suppression list
In-Reply-To: <CAF9BGBw1fEG=K5NpZ+ZcnHUYfxun69vsw_zfg6X4M1a1Mrc_yg@mail.gmail.com>
References: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
 <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>
 <CAPW48spLUaCJRFXzk0KGsGK2XyabiTS5Wwf-5ABmG_p94VPi4g@mail.gmail.com>
 <CAF9BGBw1fEG=K5NpZ+ZcnHUYfxun69vsw_zfg6X4M1a1Mrc_yg@mail.gmail.com>
Message-ID: <CAPW48soGto5QN3nQrz4N41vKTx4exhXdV=9YFFEMoJ4GW2it+A@mail.gmail.com>

On Tue, Jul 9, 2019 at 9:06 AM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Hi Arthur,
>
> Sorry for the late reply :)
>
>
> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.hpp.html
>
>   33 // Lines that start with '#' are considered comment.
>
> -> are considered comments.
>
> Done

>
>   36 // Here are few examples.
>
> -> Here are a few examples:
>
> Done

>
> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.cpp.html
>
>   - line 156 could be removed (extra empty line)
>
> Done

>   - whereas line 175 could use an extra line :)
>
> Done

>   - Is there a reason that we are using fgetc? I would just use fgets and be done with all this logic handling lines...
>
> Done

>   - FWIW, I think you should extract the whole file parsing into a parse_from_file method, which would make the init : open a file, parse it via parse_from_file, close it
>
> Done

>   - parse_from_line:
>
>      - I find this error prone:
>
>         - I would really find a '#' and replace it with '\0' if found
>
> If we have to check for '#' either way, I think bailing out when we see it
is fine.

>         - then I would do a sscanf and check its return
>
>  Done.

>         - I would add a few extra lines to better understand this code btw :)
>
> Added some comments

>
>   -   35     Exact,
>
>      -> If we are going to do = 1, = 2, ... ; might as well do = 0
>
> Done

>
>   41   FieldMatcher(const Symbol* class_name, const Symbol* field_name,
>   42                Mode class_mode, Mode field_mode, FieldMatcher* next)
>
>      -> Somehow class_name, class_mode, field_name, field_mode, next seems like a better order when looking at it (to me :-))
>
> Done

>
> Thanks,
>
> Jc
>
> Also added some tests for testing Any and Prefix matching
Updated webrev:
http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.01/

From jcbeyler at google.com  Tue Jul  9 22:41:34 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Tue, 9 Jul 2019 15:41:34 -0700
Subject: [RFR]: Field suppression list
In-Reply-To: <CAPW48soGto5QN3nQrz4N41vKTx4exhXdV=9YFFEMoJ4GW2it+A@mail.gmail.com>
References: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
 <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>
 <CAPW48spLUaCJRFXzk0KGsGK2XyabiTS5Wwf-5ABmG_p94VPi4g@mail.gmail.com>
 <CAF9BGBw1fEG=K5NpZ+ZcnHUYfxun69vsw_zfg6X4M1a1Mrc_yg@mail.gmail.com>
 <CAPW48soGto5QN3nQrz4N41vKTx4exhXdV=9YFFEMoJ4GW2it+A@mail.gmail.com>
Message-ID: <CAF9BGBydfiWx1yzyAy_3dux7kKEG=wkcuCQe_q4tmsLs1GEHZw@mail.gmail.com>

Hi Arthur,

FWIW, the comment here:

>         - I would really find a '#' and replace it with '\0' if found
>
> If we have to check for '#' either way, I think bailing out when we see it
is fine.

I don't entirely agree. User input is always error-prone and I could easily
do:

# Ignore this comment
 # But ignore this one
whatever #Incomplete line but now accepted by the system since it does a
"%s %s"
whatever really #But this also is accepted because sscanf does not look
afterwards...

In theory, we'd like the system to ignore the three first lines at least
since they are not correct, your code will accept the second and third...

If you don't want to fix it now that's ok but it should be something fixed
a bit to be a little bit more robust, no?
Jc


On Tue, Jul 9, 2019 at 3:23 PM Arthur Eubanks <aeubanks at google.com> wrote:

> On Tue, Jul 9, 2019 at 9:06 AM Jean Christophe Beyler <jcbeyler at google.com>
> wrote:
>
>> Hi Arthur,
>>
>> Sorry for the late reply :)
>>
>>
>> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.hpp.html
>>
>>   33 // Lines that start with '#' are considered comment.
>>
>> -> are considered comments.
>>
>> Done
>
>>
>>   36 // Here are few examples.
>>
>> -> Here are a few examples:
>>
>> Done
>
>>
>> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.00/src/hotspot/share/classfile/tsanIgnoreList.cpp.html
>>
>>   - line 156 could be removed (extra empty line)
>>
>> Done
>
>>   - whereas line 175 could use an extra line :)
>>
>> Done
>
>>   - Is there a reason that we are using fgetc? I would just use fgets and be done with all this logic handling lines...
>>
>> Done
>
>>   - FWIW, I think you should extract the whole file parsing into a parse_from_file method, which would make the init : open a file, parse it via parse_from_file, close it
>>
>> Done
>
>>   - parse_from_line:
>>
>>      - I find this error prone:
>>
>>         - I would really find a '#' and replace it with '\0' if found
>>
>> If we have to check for '#' either way, I think bailing out when we see
> it is fine.
>
>>         - then I would do a sscanf and check its return
>>
>>  Done.
>
>>         - I would add a few extra lines to better understand this code btw :)
>>
>> Added some comments
>
>>
>>   -   35     Exact,
>>
>>      -> If we are going to do = 1, = 2, ... ; might as well do = 0
>>
>> Done
>
>>
>>   41   FieldMatcher(const Symbol* class_name, const Symbol* field_name,
>>   42                Mode class_mode, Mode field_mode, FieldMatcher* next)
>>
>>      -> Somehow class_name, class_mode, field_name, field_mode, next seems like a better order when looking at it (to me :-))
>>
>> Done
>
>>
>> Thanks,
>>
>> Jc
>>
>> Also added some tests for testing Any and Prefix matching
> Updated webrev:
> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.01/
>
>


-- 

Thanks,
Jc

From aeubanks at google.com  Wed Jul 10 17:29:11 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 10 Jul 2019 10:29:11 -0700
Subject: [RFR]: Field suppression list
In-Reply-To: <CAF9BGBydfiWx1yzyAy_3dux7kKEG=wkcuCQe_q4tmsLs1GEHZw@mail.gmail.com>
References: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
 <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>
 <CAPW48spLUaCJRFXzk0KGsGK2XyabiTS5Wwf-5ABmG_p94VPi4g@mail.gmail.com>
 <CAF9BGBw1fEG=K5NpZ+ZcnHUYfxun69vsw_zfg6X4M1a1Mrc_yg@mail.gmail.com>
 <CAPW48soGto5QN3nQrz4N41vKTx4exhXdV=9YFFEMoJ4GW2it+A@mail.gmail.com>
 <CAF9BGBydfiWx1yzyAy_3dux7kKEG=wkcuCQe_q4tmsLs1GEHZw@mail.gmail.com>
Message-ID: <CAPW48srekUBUzmrcouL79ztHVp1StsXpLo7PA3eqovYyLuDy+g@mail.gmail.com>

On Tue, Jul 9, 2019 at 3:41 PM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Hi Arthur,
>
> FWIW, the comment here:
>
>>         - I would really find a '#' and replace it with '\0' if found
>>
>> If we have to check for '#' either way, I think bailing out when we see
> it is fine.
>
> I don't entirely agree. User input is always error-prone and I could
> easily do:
>
> # Ignore this comment
>  # But ignore this one
> whatever #Incomplete line but now accepted by the system since it does a
> "%s %s"
> whatever really #But this also is accepted because sscanf does not look
> afterwards...
>
> In theory, we'd like the system to ignore the three first lines at least
> since they are not correct, your code will accept the second and third...
>
> If you don't want to fix it now that's ok but it should be something fixed
> a bit to be a little bit more robust, no?
> Jc
>
You convinced me:
http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.02/

From jcbeyler at google.com  Wed Jul 10 18:24:58 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Wed, 10 Jul 2019 11:24:58 -0700
Subject: [RFR]: Field suppression list
In-Reply-To: <CAPW48srekUBUzmrcouL79ztHVp1StsXpLo7PA3eqovYyLuDy+g@mail.gmail.com>
References: <CAPW48sqW7hM9gpz-QkNX2voL4LdYv8mC9RO99oy-eehQ14KQUA@mail.gmail.com>
 <CA+w6HxbarpFb9WS9gWXSEw3UQRhaYAQ3PeFDBWZkRuXco=WDiA@mail.gmail.com>
 <CAPW48spLUaCJRFXzk0KGsGK2XyabiTS5Wwf-5ABmG_p94VPi4g@mail.gmail.com>
 <CAF9BGBw1fEG=K5NpZ+ZcnHUYfxun69vsw_zfg6X4M1a1Mrc_yg@mail.gmail.com>
 <CAPW48soGto5QN3nQrz4N41vKTx4exhXdV=9YFFEMoJ4GW2it+A@mail.gmail.com>
 <CAF9BGBydfiWx1yzyAy_3dux7kKEG=wkcuCQe_q4tmsLs1GEHZw@mail.gmail.com>
 <CAPW48srekUBUzmrcouL79ztHVp1StsXpLo7PA3eqovYyLuDy+g@mail.gmail.com>
Message-ID: <CAF9BGBzDcSb9qbxgrb7mQ57fO8=dEvtk+gb8FYXojzYuMam7eA@mail.gmail.com>

LGTM now, thanks!

My last nit would be that if we do do a "wrong" line, we get no mention of
it but just a silent fail due to sscanf's just doing a return. But it's
tricky now because a comment line goes to the sscanf to fail; so now you'd
have to distinguish between an empty line (which might have spaces,
tabs...) and just a one element line.

So for now, I'd skip that perhaps :)
Jc

On Wed, Jul 10, 2019 at 10:29 AM Arthur Eubanks <aeubanks at google.com> wrote:

>
>
> On Tue, Jul 9, 2019 at 3:41 PM Jean Christophe Beyler <jcbeyler at google.com>
> wrote:
>
>> Hi Arthur,
>>
>> FWIW, the comment here:
>>
>>>         - I would really find a '#' and replace it with '\0' if found
>>>
>>> If we have to check for '#' either way, I think bailing out when we see
>> it is fine.
>>
>> I don't entirely agree. User input is always error-prone and I could
>> easily do:
>>
>> # Ignore this comment
>>  # But ignore this one
>> whatever #Incomplete line but now accepted by the system since it does a
>> "%s %s"
>> whatever really #But this also is accepted because sscanf does not look
>> afterwards...
>>
>> In theory, we'd like the system to ignore the three first lines at least
>> since they are not correct, your code will accept the second and third...
>>
>> If you don't want to fix it now that's ok but it should be something
>> fixed a bit to be a little bit more robust, no?
>> Jc
>>
> You convinced me:
> http://cr.openjdk.java.net/~aeubanks/tsanfieldsupp/webrev.02/
>


-- 

Thanks,
Jc

From aeubanks at google.com  Wed Jul 10 18:33:50 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Wed, 10 Jul 2019 18:33:50 +0000
Subject: hg: tsan/dev: Implement field suppression via a suppression list
Message-ID: <201907101833.x6AIXo5E008943@aojmv0008.oracle.com>

Changeset: 3bae5ecee45f
Author:    aeubanks
Date:      2019-07-01 15:34 -0700
URL:       http://hg.openjdk.java.net/tsan/dev/rev/3bae5ecee45f

Implement field suppression via a suppression list

! src/hotspot/share/classfile/classFileParser.cpp
+ src/hotspot/share/classfile/tsanIgnoreList.cpp
+ src/hotspot/share/classfile/tsanIgnoreList.hpp
! src/hotspot/share/runtime/globals.hpp
! src/hotspot/share/tsan/tsan.cpp
+ test/hotspot/jtreg/tsan/FieldIgnoreAnyPrimitiveLoopTest.java
+ test/hotspot/jtreg/tsan/FieldIgnoreAnyReferenceLoopTest.java
+ test/hotspot/jtreg/tsan/FieldIgnoreLoopTest.java
+ test/hotspot/jtreg/tsan/FieldIgnorePrefixLoopTest.java
+ test/hotspot/jtreg/tsan/FieldIgnorePrefixNoMatchLoopTest.java
+ test/hotspot/jtreg/tsan/IgnoreFields


From manc at google.com  Wed Jul 10 23:59:23 2019
From: manc at google.com (Man Cao)
Date: Wed, 10 Jul 2019 16:59:23 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
 <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
Message-ID: <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>

I don't see the attached reproducer. I don't think you could add attachment
on the mailing list though.
Could you inline it in the email or put it on webrev?

-Man


On Mon, Jul 8, 2019 at 1:23 PM Arthur Eubanks <aeubanks at google.com> wrote:

> I have a simpler reproducer attached.
>
> Some logging:
>            // The object survived GC, add its updated oop to the new oops
> map.
>            oop target_oop = cast_to_oop((intptr_t)source_obj);
>            pointer_adjuster->do_oop(&target_oop);
> +          oopDesc *target_obj = target_oop;
> +          if (!heap->is_in(target_oop)) {
> +            fprintf(stderr, "%p\n", source_obj);
> +            fprintf(stderr, "%p\n", target_obj);
> +          }
>            assert(heap->is_in(target_oop), "Adjustment failed");
> -          oopDesc *target_obj = target_oop;
>            new_map->put(target_obj, obj_size);
>
> 0xffea5570
> 0xf8238138
>
> On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
> > Trying to implement finalizer support, I created a test (attached), and
> > tsanOopMap.cpp seems buggy.
> >
> > # after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
> > #
> > # A fatal error has been detected by the Java Runtime Environment:
> > #
> > #  Internal Error
> >
> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
> > pid=16182, tid=16188
> > #  assert(heap->is_in(target_oop)) failed: Adjustment failed
> > #
> > # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
> > 13-internal+0-adhoc.aeubanks.tsan)
> > # Java VM: OpenJDK 64-Bit Server VM (fastdebug
> > 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered, compressed
> > oops, g1 gc, linux-amd64)
> > # Problematic frame:
> > # V  [libjvm.so+0x164e63b]
> >  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
> > OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*, char**,
> > char**)+0x24b
> > #
> > # No core dump will be written. Core dumps have been disabled. To enable
> > core dumping, try "ulimit -c unlimited" before starting Java again
> > #
> > # An error report file with more information is saved as:
> > #
> >
> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
> > #
> > # If you would like to submit a bug report, please visit:
> > #   http://bugreport.java.com/bugreport/crash.jsp
> > #
> >
>

From aeubanks at google.com  Thu Jul 11 17:52:18 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Thu, 11 Jul 2019 10:52:18 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
 <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
 <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>
Message-ID: <CAPW48srXkdRuzYYBmN1AeipQSAnQgEPi5UnJamT1pgYrG2CjOw@mail.gmail.com>

Here it is:
http://cr.openjdk.java.net/~aeubanks/tsanfinalizercrash/NonRacyFinalizerLoopTest.java

On Wed, Jul 10, 2019 at 4:59 PM Man Cao <manc at google.com> wrote:

> I don't see the attached reproducer. I don't think you could add
> attachment on the mailing list though.
> Could you inline it in the email or put it on webrev?
>
> -Man
>
>
> On Mon, Jul 8, 2019 at 1:23 PM Arthur Eubanks <aeubanks at google.com> wrote:
>
>> I have a simpler reproducer attached.
>>
>> Some logging:
>>            // The object survived GC, add its updated oop to the new oops
>> map.
>>            oop target_oop = cast_to_oop((intptr_t)source_obj);
>>            pointer_adjuster->do_oop(&target_oop);
>> +          oopDesc *target_obj = target_oop;
>> +          if (!heap->is_in(target_oop)) {
>> +            fprintf(stderr, "%p\n", source_obj);
>> +            fprintf(stderr, "%p\n", target_obj);
>> +          }
>>            assert(heap->is_in(target_oop), "Adjustment failed");
>> -          oopDesc *target_obj = target_oop;
>>            new_map->put(target_obj, obj_size);
>>
>> 0xffea5570
>> 0xf8238138
>>
>> On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>> > Trying to implement finalizer support, I created a test (attached), and
>> > tsanOopMap.cpp seems buggy.
>> >
>> > # after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
>> > #
>> > # A fatal error has been detected by the Java Runtime Environment:
>> > #
>> > #  Internal Error
>> >
>> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
>> > pid=16182, tid=16188
>> > #  assert(heap->is_in(target_oop)) failed: Adjustment failed
>> > #
>> > # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
>> > 13-internal+0-adhoc.aeubanks.tsan)
>> > # Java VM: OpenJDK 64-Bit Server VM (fastdebug
>> > 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered, compressed
>> > oops, g1 gc, linux-amd64)
>> > # Problematic frame:
>> > # V  [libjvm.so+0x164e63b]
>> >  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
>> > OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*, char**,
>> > char**)+0x24b
>> > #
>> > # No core dump will be written. Core dumps have been disabled. To enable
>> > core dumping, try "ulimit -c unlimited" before starting Java again
>> > #
>> > # An error report file with more information is saved as:
>> > #
>> >
>> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
>> > #
>> > # If you would like to submit a bug report, please visit:
>> > #   http://bugreport.java.com/bugreport/crash.jsp
>> > #
>> >
>>
>

From manc at google.com  Fri Jul 12 01:08:48 2019
From: manc at google.com (Man Cao)
Date: Thu, 11 Jul 2019 18:08:48 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CAPW48srXkdRuzYYBmN1AeipQSAnQgEPi5UnJamT1pgYrG2CjOw@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
 <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
 <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>
 <CAPW48srXkdRuzYYBmN1AeipQSAnQgEPi5UnJamT1pgYrG2CjOw@mail.gmail.com>
Message-ID: <CA+w6HxZda6ULaESAh5Sott9mc9EXND+TJmRscNEoUSGL8a_5yw@mail.gmail.com>

Can I have reviews for this bug fix?
https://cr.openjdk.java.net/~manc/tsan20190711/webrev.00/
The assertion check is too strong for target_oop after calling
pointer_adjuster->do_oop().


-Man


On Thu, Jul 11, 2019 at 10:52 AM Arthur Eubanks <aeubanks at google.com> wrote:

> Here it is:
> http://cr.openjdk.java.net/~aeubanks/tsanfinalizercrash/NonRacyFinalizerLoopTest.java
>
> On Wed, Jul 10, 2019 at 4:59 PM Man Cao <manc at google.com> wrote:
>
>> I don't see the attached reproducer. I don't think you could add
>> attachment on the mailing list though.
>> Could you inline it in the email or put it on webrev?
>>
>> -Man
>>
>>
>> On Mon, Jul 8, 2019 at 1:23 PM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>>> I have a simpler reproducer attached.
>>>
>>> Some logging:
>>>            // The object survived GC, add its updated oop to the new oops
>>> map.
>>>            oop target_oop = cast_to_oop((intptr_t)source_obj);
>>>            pointer_adjuster->do_oop(&target_oop);
>>> +          oopDesc *target_obj = target_oop;
>>> +          if (!heap->is_in(target_oop)) {
>>> +            fprintf(stderr, "%p\n", source_obj);
>>> +            fprintf(stderr, "%p\n", target_obj);
>>> +          }
>>>            assert(heap->is_in(target_oop), "Adjustment failed");
>>> -          oopDesc *target_obj = target_oop;
>>>            new_map->put(target_obj, obj_size);
>>>
>>> 0xffea5570
>>> 0xf8238138
>>>
>>> On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com>
>>> wrote:
>>>
>>> > Trying to implement finalizer support, I created a test (attached), and
>>> > tsanOopMap.cpp seems buggy.
>>> >
>>> > # after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
>>> > #
>>> > # A fatal error has been detected by the Java Runtime Environment:
>>> > #
>>> > #  Internal Error
>>> >
>>> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
>>> > pid=16182, tid=16188
>>> > #  assert(heap->is_in(target_oop)) failed: Adjustment failed
>>> > #
>>> > # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
>>> > 13-internal+0-adhoc.aeubanks.tsan)
>>> > # Java VM: OpenJDK 64-Bit Server VM (fastdebug
>>> > 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered, compressed
>>> > oops, g1 gc, linux-amd64)
>>> > # Problematic frame:
>>> > # V  [libjvm.so+0x164e63b]
>>> >  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
>>> > OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*, char**,
>>> > char**)+0x24b
>>> > #
>>> > # No core dump will be written. Core dumps have been disabled. To
>>> enable
>>> > core dumping, try "ulimit -c unlimited" before starting Java again
>>> > #
>>> > # An error report file with more information is saved as:
>>> > #
>>> >
>>> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
>>> > #
>>> > # If you would like to submit a bug report, please visit:
>>> > #   http://bugreport.java.com/bugreport/crash.jsp
>>> > #
>>> >
>>>
>>

From jcbeyler at google.com  Fri Jul 12 02:22:09 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Thu, 11 Jul 2019 19:22:09 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CA+w6HxZda6ULaESAh5Sott9mc9EXND+TJmRscNEoUSGL8a_5yw@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
 <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
 <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>
 <CAPW48srXkdRuzYYBmN1AeipQSAnQgEPi5UnJamT1pgYrG2CjOw@mail.gmail.com>
 <CA+w6HxZda6ULaESAh5Sott9mc9EXND+TJmRscNEoUSGL8a_5yw@mail.gmail.com>
Message-ID: <CAF9BGBxWiJUuGe_cX_FfxZkEne3zNeRsm7C7AmfffsGHPmaLMA@mail.gmail.com>

Just by curiosity, isn't any memory allocated in the heap in the reserved
space? So essentially, this assert is no longer really checking for
anything?

Or does this actually do what we want? :-)
Jc

On Thu, Jul 11, 2019 at 6:09 PM Man Cao <manc at google.com> wrote:

> Can I have reviews for this bug fix?
> https://cr.openjdk.java.net/~manc/tsan20190711/webrev.00/
> The assertion check is too strong for target_oop after calling
> pointer_adjuster->do_oop().
>
>
> -Man
>
>
> On Thu, Jul 11, 2019 at 10:52 AM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
> > Here it is:
> >
> http://cr.openjdk.java.net/~aeubanks/tsanfinalizercrash/NonRacyFinalizerLoopTest.java
> >
> > On Wed, Jul 10, 2019 at 4:59 PM Man Cao <manc at google.com> wrote:
> >
> >> I don't see the attached reproducer. I don't think you could add
> >> attachment on the mailing list though.
> >> Could you inline it in the email or put it on webrev?
> >>
> >> -Man
> >>
> >>
> >> On Mon, Jul 8, 2019 at 1:23 PM Arthur Eubanks <aeubanks at google.com>
> >> wrote:
> >>
> >>> I have a simpler reproducer attached.
> >>>
> >>> Some logging:
> >>>            // The object survived GC, add its updated oop to the new
> oops
> >>> map.
> >>>            oop target_oop = cast_to_oop((intptr_t)source_obj);
> >>>            pointer_adjuster->do_oop(&target_oop);
> >>> +          oopDesc *target_obj = target_oop;
> >>> +          if (!heap->is_in(target_oop)) {
> >>> +            fprintf(stderr, "%p\n", source_obj);
> >>> +            fprintf(stderr, "%p\n", target_obj);
> >>> +          }
> >>>            assert(heap->is_in(target_oop), "Adjustment failed");
> >>> -          oopDesc *target_obj = target_oop;
> >>>            new_map->put(target_obj, obj_size);
> >>>
> >>> 0xffea5570
> >>> 0xf8238138
> >>>
> >>> On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com>
> >>> wrote:
> >>>
> >>> > Trying to implement finalizer support, I created a test (attached),
> and
> >>> > tsanOopMap.cpp seems buggy.
> >>> >
> >>> > # after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
> >>> > #
> >>> > # A fatal error has been detected by the Java Runtime Environment:
> >>> > #
> >>> > #  Internal Error
> >>> >
> >>>
> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
> >>> > pid=16182, tid=16188
> >>> > #  assert(heap->is_in(target_oop)) failed: Adjustment failed
> >>> > #
> >>> > # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
> >>> > 13-internal+0-adhoc.aeubanks.tsan)
> >>> > # Java VM: OpenJDK 64-Bit Server VM (fastdebug
> >>> > 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered,
> compressed
> >>> > oops, g1 gc, linux-amd64)
> >>> > # Problematic frame:
> >>> > # V  [libjvm.so+0x164e63b]
> >>> >  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
> >>> > OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*,
> char**,
> >>> > char**)+0x24b
> >>> > #
> >>> > # No core dump will be written. Core dumps have been disabled. To
> >>> enable
> >>> > core dumping, try "ulimit -c unlimited" before starting Java again
> >>> > #
> >>> > # An error report file with more information is saved as:
> >>> > #
> >>> >
> >>>
> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
> >>> > #
> >>> > # If you would like to submit a bug report, please visit:
> >>> > #   http://bugreport.java.com/bugreport/crash.jsp
> >>> > #
> >>> >
> >>>
> >>
>


-- 

Thanks,
Jc

From manc at google.com  Fri Jul 12 16:19:46 2019
From: manc at google.com (Man Cao)
Date: Fri, 12 Jul 2019 09:19:46 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CAF9BGBxWiJUuGe_cX_FfxZkEne3zNeRsm7C7AmfffsGHPmaLMA@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
 <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
 <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>
 <CAPW48srXkdRuzYYBmN1AeipQSAnQgEPi5UnJamT1pgYrG2CjOw@mail.gmail.com>
 <CA+w6HxZda6ULaESAh5Sott9mc9EXND+TJmRscNEoUSGL8a_5yw@mail.gmail.com>
 <CAF9BGBxWiJUuGe_cX_FfxZkEne3zNeRsm7C7AmfffsGHPmaLMA@mail.gmail.com>
Message-ID: <CA+w6HxYR9swhCcvjX9agrhb=h8CRP6zzHwHvZfUGoOiyZ2WQ+g@mail.gmail.com>

Yes, any valid oop pointer should be in the reserved space. Here the code
is not allocating anything on the heap, and pointer_adjuster->do_oop()
typically just loads from the header word and updates the holder (
target_oop).

So this assert just checks if target_oop looks like a valid pointer after
adjusting. It is a quite relaxed check, but better than nothing.

-Man

On Thu, Jul 11, 2019, 19:22 Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Just by curiosity, isn't any memory allocated in the heap in the reserved
> space? So essentially, this assert is no longer really checking for
> anything?
>
> Or does this actually do what we want? :-)
> Jc
>
> On Thu, Jul 11, 2019 at 6:09 PM Man Cao <manc at google.com> wrote:
>
>> Can I have reviews for this bug fix?
>> https://cr.openjdk.java.net/~manc/tsan20190711/webrev.00/
>> The assertion check is too strong for target_oop after calling
>> pointer_adjuster->do_oop().
>>
>>
>> -Man
>>
>>
>> On Thu, Jul 11, 2019 at 10:52 AM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>> > Here it is:
>> >
>> http://cr.openjdk.java.net/~aeubanks/tsanfinalizercrash/NonRacyFinalizerLoopTest.java
>> >
>> > On Wed, Jul 10, 2019 at 4:59 PM Man Cao <manc at google.com> wrote:
>> >
>> >> I don't see the attached reproducer. I don't think you could add
>> >> attachment on the mailing list though.
>> >> Could you inline it in the email or put it on webrev?
>> >>
>> >> -Man
>> >>
>> >>
>> >> On Mon, Jul 8, 2019 at 1:23 PM Arthur Eubanks <aeubanks at google.com>
>> >> wrote:
>> >>
>> >>> I have a simpler reproducer attached.
>> >>>
>> >>> Some logging:
>> >>>            // The object survived GC, add its updated oop to the new
>> oops
>> >>> map.
>> >>>            oop target_oop = cast_to_oop((intptr_t)source_obj);
>> >>>            pointer_adjuster->do_oop(&target_oop);
>> >>> +          oopDesc *target_obj = target_oop;
>> >>> +          if (!heap->is_in(target_oop)) {
>> >>> +            fprintf(stderr, "%p\n", source_obj);
>> >>> +            fprintf(stderr, "%p\n", target_obj);
>> >>> +          }
>> >>>            assert(heap->is_in(target_oop), "Adjustment failed");
>> >>> -          oopDesc *target_obj = target_oop;
>> >>>            new_map->put(target_obj, obj_size);
>> >>>
>> >>> 0xffea5570
>> >>> 0xf8238138
>> >>>
>> >>> On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com>
>> >>> wrote:
>> >>>
>> >>> > Trying to implement finalizer support, I created a test (attached),
>> and
>> >>> > tsanOopMap.cpp seems buggy.
>> >>> >
>> >>> > # after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
>> >>> > #
>> >>> > # A fatal error has been detected by the Java Runtime Environment:
>> >>> > #
>> >>> > #  Internal Error
>> >>> >
>> >>>
>> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
>> >>> > pid=16182, tid=16188
>> >>> > #  assert(heap->is_in(target_oop)) failed: Adjustment failed
>> >>> > #
>> >>> > # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
>> >>> > 13-internal+0-adhoc.aeubanks.tsan)
>> >>> > # Java VM: OpenJDK 64-Bit Server VM (fastdebug
>> >>> > 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered,
>> compressed
>> >>> > oops, g1 gc, linux-amd64)
>> >>> > # Problematic frame:
>> >>> > # V  [libjvm.so+0x164e63b]
>> >>> >  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
>> >>> > OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*,
>> char**,
>> >>> > char**)+0x24b
>> >>> > #
>> >>> > # No core dump will be written. Core dumps have been disabled. To
>> >>> enable
>> >>> > core dumping, try "ulimit -c unlimited" before starting Java again
>> >>> > #
>> >>> > # An error report file with more information is saved as:
>> >>> > #
>> >>> >
>> >>>
>> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
>> >>> > #
>> >>> > # If you would like to submit a bug report, please visit:
>> >>> > #   http://bugreport.java.com/bugreport/crash.jsp
>> >>> > #
>> >>> >
>> >>>
>> >>
>>
>
>
> --
>
> Thanks,
> Jc
>

From aeubanks at google.com  Fri Jul 12 17:40:37 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Fri, 12 Jul 2019 10:40:37 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CA+w6HxYR9swhCcvjX9agrhb=h8CRP6zzHwHvZfUGoOiyZ2WQ+g@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
 <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
 <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>
 <CAPW48srXkdRuzYYBmN1AeipQSAnQgEPi5UnJamT1pgYrG2CjOw@mail.gmail.com>
 <CA+w6HxZda6ULaESAh5Sott9mc9EXND+TJmRscNEoUSGL8a_5yw@mail.gmail.com>
 <CAF9BGBxWiJUuGe_cX_FfxZkEne3zNeRsm7C7AmfffsGHPmaLMA@mail.gmail.com>
 <CA+w6HxYR9swhCcvjX9agrhb=h8CRP6zzHwHvZfUGoOiyZ2WQ+g@mail.gmail.com>
Message-ID: <CAPW48somqpYsfSSRVkGAE1ZnbKSrijvWhphP4W9RwFBgEw-6wQ@mail.gmail.com>

LGTM

On Fri, Jul 12, 2019 at 9:19 AM Man Cao <manc at google.com> wrote:

> Yes, any valid oop pointer should be in the reserved space. Here the code
> is not allocating anything on the heap, and pointer_adjuster->do_oop()
> typically just loads from the header word and updates the holder (
> target_oop).
>
> So this assert just checks if target_oop looks like a valid pointer after
> adjusting. It is a quite relaxed check, but better than nothing.
>
> -Man
>
> On Thu, Jul 11, 2019, 19:22 Jean Christophe Beyler <jcbeyler at google.com>
> wrote:
>
>> Just by curiosity, isn't any memory allocated in the heap in the reserved
>> space? So essentially, this assert is no longer really checking for
>> anything?
>>
>> Or does this actually do what we want? :-)
>> Jc
>>
>> On Thu, Jul 11, 2019 at 6:09 PM Man Cao <manc at google.com> wrote:
>>
>>> Can I have reviews for this bug fix?
>>> https://cr.openjdk.java.net/~manc/tsan20190711/webrev.00/
>>> The assertion check is too strong for target_oop after calling
>>> pointer_adjuster->do_oop().
>>>
>>>
>>> -Man
>>>
>>>
>>> On Thu, Jul 11, 2019 at 10:52 AM Arthur Eubanks <aeubanks at google.com>
>>> wrote:
>>>
>>> > Here it is:
>>> >
>>> http://cr.openjdk.java.net/~aeubanks/tsanfinalizercrash/NonRacyFinalizerLoopTest.java
>>> >
>>> > On Wed, Jul 10, 2019 at 4:59 PM Man Cao <manc at google.com> wrote:
>>> >
>>> >> I don't see the attached reproducer. I don't think you could add
>>> >> attachment on the mailing list though.
>>> >> Could you inline it in the email or put it on webrev?
>>> >>
>>> >> -Man
>>> >>
>>> >>
>>> >> On Mon, Jul 8, 2019 at 1:23 PM Arthur Eubanks <aeubanks at google.com>
>>> >> wrote:
>>> >>
>>> >>> I have a simpler reproducer attached.
>>> >>>
>>> >>> Some logging:
>>> >>>            // The object survived GC, add its updated oop to the new
>>> oops
>>> >>> map.
>>> >>>            oop target_oop = cast_to_oop((intptr_t)source_obj);
>>> >>>            pointer_adjuster->do_oop(&target_oop);
>>> >>> +          oopDesc *target_obj = target_oop;
>>> >>> +          if (!heap->is_in(target_oop)) {
>>> >>> +            fprintf(stderr, "%p\n", source_obj);
>>> >>> +            fprintf(stderr, "%p\n", target_obj);
>>> >>> +          }
>>> >>>            assert(heap->is_in(target_oop), "Adjustment failed");
>>> >>> -          oopDesc *target_obj = target_oop;
>>> >>>            new_map->put(target_obj, obj_size);
>>> >>>
>>> >>> 0xffea5570
>>> >>> 0xf8238138
>>> >>>
>>> >>> On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com>
>>> >>> wrote:
>>> >>>
>>> >>> > Trying to implement finalizer support, I created a test
>>> (attached), and
>>> >>> > tsanOopMap.cpp seems buggy.
>>> >>> >
>>> >>> > # after -XX: or in .hotspotrc:  SuppressErrorAt=/tsanOopMap.cpp:294
>>> >>> > #
>>> >>> > # A fatal error has been detected by the Java Runtime Environment:
>>> >>> > #
>>> >>> > #  Internal Error
>>> >>> >
>>> >>>
>>> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
>>> >>> > pid=16182, tid=16188
>>> >>> > #  assert(heap->is_in(target_oop)) failed: Adjustment failed
>>> >>> > #
>>> >>> > # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
>>> >>> > 13-internal+0-adhoc.aeubanks.tsan)
>>> >>> > # Java VM: OpenJDK 64-Bit Server VM (fastdebug
>>> >>> > 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered,
>>> compressed
>>> >>> > oops, g1 gc, linux-amd64)
>>> >>> > # Problematic frame:
>>> >>> > # V  [libjvm.so+0x164e63b]
>>> >>> >  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
>>> >>> > OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*,
>>> char**,
>>> >>> > char**)+0x24b
>>> >>> > #
>>> >>> > # No core dump will be written. Core dumps have been disabled. To
>>> >>> enable
>>> >>> > core dumping, try "ulimit -c unlimited" before starting Java again
>>> >>> > #
>>> >>> > # An error report file with more information is saved as:
>>> >>> > #
>>> >>> >
>>> >>>
>>> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
>>> >>> > #
>>> >>> > # If you would like to submit a bug report, please visit:
>>> >>> > #   http://bugreport.java.com/bugreport/crash.jsp
>>> >>> > #
>>> >>> >
>>> >>>
>>> >>
>>>
>>
>>
>> --
>>
>> Thanks,
>> Jc
>>
>

From jcbeyler at google.com  Fri Jul 12 18:35:13 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Fri, 12 Jul 2019 11:35:13 -0700
Subject: Crash on finalizer in tsanOopMap
In-Reply-To: <CAPW48somqpYsfSSRVkGAE1ZnbKSrijvWhphP4W9RwFBgEw-6wQ@mail.gmail.com>
References: <CAPW48spizCi88aYM0ToDx0V+_9N1_0rszkmMORoPFbC4apjiAg@mail.gmail.com>
 <CAPW48sqr6r5ZZexWHGa_=QUdAd5KTEUOh87OCerXyPK=ojY5Mg@mail.gmail.com>
 <CA+w6Hxb0qV=ZxcGNkXq0TP+EVRpK9QQvarq8XXLyj_X5TmF-+w@mail.gmail.com>
 <CAPW48srXkdRuzYYBmN1AeipQSAnQgEPi5UnJamT1pgYrG2CjOw@mail.gmail.com>
 <CA+w6HxZda6ULaESAh5Sott9mc9EXND+TJmRscNEoUSGL8a_5yw@mail.gmail.com>
 <CAF9BGBxWiJUuGe_cX_FfxZkEne3zNeRsm7C7AmfffsGHPmaLMA@mail.gmail.com>
 <CA+w6HxYR9swhCcvjX9agrhb=h8CRP6zzHwHvZfUGoOiyZ2WQ+g@mail.gmail.com>
 <CAPW48somqpYsfSSRVkGAE1ZnbKSrijvWhphP4W9RwFBgEw-6wQ@mail.gmail.com>
Message-ID: <CAF9BGBw=S-eeQaj=4GYUeP3zCfEEZ8qPEh1=81p=1Ejhmn3MAw@mail.gmail.com>

Yeah, I was wondering if it was just a bit too relaxed but yes better than
nothing :)

LGTM,
Jc

On Fri, Jul 12, 2019 at 10:40 AM Arthur Eubanks <aeubanks at google.com> wrote:

> LGTM
>
> On Fri, Jul 12, 2019 at 9:19 AM Man Cao <manc at google.com> wrote:
>
>> Yes, any valid oop pointer should be in the reserved space. Here the code
>> is not allocating anything on the heap, and pointer_adjuster->do_oop()
>> typically just loads from the header word and updates the holder (
>> target_oop).
>>
>> So this assert just checks if target_oop looks like a valid pointer
>> after adjusting. It is a quite relaxed check, but better than nothing.
>>
>> -Man
>>
>> On Thu, Jul 11, 2019, 19:22 Jean Christophe Beyler <jcbeyler at google.com>
>> wrote:
>>
>>> Just by curiosity, isn't any memory allocated in the heap in the
>>> reserved space? So essentially, this assert is no longer really checking
>>> for anything?
>>>
>>> Or does this actually do what we want? :-)
>>> Jc
>>>
>>> On Thu, Jul 11, 2019 at 6:09 PM Man Cao <manc at google.com> wrote:
>>>
>>>> Can I have reviews for this bug fix?
>>>> https://cr.openjdk.java.net/~manc/tsan20190711/webrev.00/
>>>> The assertion check is too strong for target_oop after calling
>>>> pointer_adjuster->do_oop().
>>>>
>>>>
>>>> -Man
>>>>
>>>>
>>>> On Thu, Jul 11, 2019 at 10:52 AM Arthur Eubanks <aeubanks at google.com>
>>>> wrote:
>>>>
>>>> > Here it is:
>>>> >
>>>> http://cr.openjdk.java.net/~aeubanks/tsanfinalizercrash/NonRacyFinalizerLoopTest.java
>>>> >
>>>> > On Wed, Jul 10, 2019 at 4:59 PM Man Cao <manc at google.com> wrote:
>>>> >
>>>> >> I don't see the attached reproducer. I don't think you could add
>>>> >> attachment on the mailing list though.
>>>> >> Could you inline it in the email or put it on webrev?
>>>> >>
>>>> >> -Man
>>>> >>
>>>> >>
>>>> >> On Mon, Jul 8, 2019 at 1:23 PM Arthur Eubanks <aeubanks at google.com>
>>>> >> wrote:
>>>> >>
>>>> >>> I have a simpler reproducer attached.
>>>> >>>
>>>> >>> Some logging:
>>>> >>>            // The object survived GC, add its updated oop to the
>>>> new oops
>>>> >>> map.
>>>> >>>            oop target_oop = cast_to_oop((intptr_t)source_obj);
>>>> >>>            pointer_adjuster->do_oop(&target_oop);
>>>> >>> +          oopDesc *target_obj = target_oop;
>>>> >>> +          if (!heap->is_in(target_oop)) {
>>>> >>> +            fprintf(stderr, "%p\n", source_obj);
>>>> >>> +            fprintf(stderr, "%p\n", target_obj);
>>>> >>> +          }
>>>> >>>            assert(heap->is_in(target_oop), "Adjustment failed");
>>>> >>> -          oopDesc *target_obj = target_oop;
>>>> >>>            new_map->put(target_obj, obj_size);
>>>> >>>
>>>> >>> 0xffea5570
>>>> >>> 0xf8238138
>>>> >>>
>>>> >>> On Wed, Jul 3, 2019 at 10:44 AM Arthur Eubanks <aeubanks at google.com
>>>> >
>>>> >>> wrote:
>>>> >>>
>>>> >>> > Trying to implement finalizer support, I created a test
>>>> (attached), and
>>>> >>> > tsanOopMap.cpp seems buggy.
>>>> >>> >
>>>> >>> > # after -XX: or in .hotspotrc:
>>>> SuppressErrorAt=/tsanOopMap.cpp:294
>>>> >>> > #
>>>> >>> > # A fatal error has been detected by the Java Runtime Environment:
>>>> >>> > #
>>>> >>> > #  Internal Error
>>>> >>> >
>>>> >>>
>>>> (/usr/local/google/home/aeubanks/jdk/tsan/src/hotspot/share/tsan/tsanOopMap.cpp:294),
>>>> >>> > pid=16182, tid=16188
>>>> >>> > #  assert(heap->is_in(target_oop)) failed: Adjustment failed
>>>> >>> > #
>>>> >>> > # JRE version: OpenJDK Runtime Environment (13.0) (fastdebug build
>>>> >>> > 13-internal+0-adhoc.aeubanks.tsan)
>>>> >>> > # Java VM: OpenJDK 64-Bit Server VM (fastdebug
>>>> >>> > 13-internal+0-adhoc.aeubanks.tsan, interpreted mode, tiered,
>>>> compressed
>>>> >>> > oops, g1 gc, linux-amd64)
>>>> >>> > # Problematic frame:
>>>> >>> > # V  [libjvm.so+0x164e63b]
>>>> >>> >  TsanOopMapImpl::TsanOopSizeMap::collect_oops(BoolObjectClosure*,
>>>> >>> > OopClosure*, GrowableArray<TsanOopMapImpl::PendingMove>*, int*,
>>>> char**,
>>>> >>> > char**)+0x24b
>>>> >>> > #
>>>> >>> > # No core dump will be written. Core dumps have been disabled. To
>>>> >>> enable
>>>> >>> > core dumping, try "ulimit -c unlimited" before starting Java again
>>>> >>> > #
>>>> >>> > # An error report file with more information is saved as:
>>>> >>> > #
>>>> >>> >
>>>> >>>
>>>> /usr/local/google/home/aeubanks/jdk/tsan/build/test-support/jtreg_test_hotspot_jtreg_tsan_NonRacyFinalizerLoopTest_java/scratch/0/hs_err_pid16182.log
>>>> >>> > #
>>>> >>> > # If you would like to submit a bug report, please visit:
>>>> >>> > #   http://bugreport.java.com/bugreport/crash.jsp
>>>> >>> > #
>>>> >>> >
>>>> >>>
>>>> >>
>>>>
>>>
>>>
>>> --
>>>
>>> Thanks,
>>> Jc
>>>
>>

-- 

Thanks,
Jc

From manc at google.com  Fri Jul 12 19:12:50 2019
From: manc at google.com (manc at google.com)
Date: Fri, 12 Jul 2019 19:12:50 +0000
Subject: hg: tsan/dev: Fix assertion error in TsanOopSizeMap::collect_oops()
Message-ID: <201907121912.x6CJCotk029254@aojmv0008.oracle.com>

Changeset: 396485470187
Author:    manc
Date:      2019-07-11 18:05 -0700
URL:       https://hg.openjdk.java.net/tsan/dev/rev/396485470187

Fix assertion error in TsanOopSizeMap::collect_oops()

The assertion check for target_oop after calling pointer_adjuster->do_oop()
is too strong. The target_oop may not point to a valid oop yet, because
the collector may move the object to target_oop later.

Reviewed-by: aeubanks, jcbeyler

! src/hotspot/share/tsan/tsanOopMap.cpp


From manc at google.com  Tue Jul 16 19:08:27 2019
From: manc at google.com (Man Cao)
Date: Tue, 16 Jul 2019 12:08:27 -0700
Subject: Build breakage
Message-ID: <CA+w6HxZ__JMprjZbf5Ndkff_2sP3Tg=ECTwn19=Yd0sNx0Ltpg@mail.gmail.com>

I cannot build TSAN after
http://hg.openjdk.java.net/tsan/dev/rev/3bae5ecee45f. I'm seeing the
following errors:

In file included from
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/tsan/tsan.cpp:26:
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
error: expected class name
class TsanIgnoreList : AllStatic {
                       ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
error: unknown type name 'Symbol'
  static bool match(const Symbol* class_name, const Symbol* field_name,
                          ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
error: unknown type name 'Symbol'
  static bool match(const Symbol* class_name, const Symbol* field_name,
                                                    ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
error: unknown type name 'BasicType'
                    BasicType type);
                    ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
error: unknown type name 'FILE'
  static void parse_from_file(FILE* stream);
                              ^
In file included from
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.cpp:27:
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
error: expected class name
class TsanIgnoreList : AllStatic {
                       ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
error: unknown type name 'Symbol'
  static bool match(const Symbol* class_name, const Symbol* field_name,
                          ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
error: unknown type name 'Symbol'
  static bool match(const Symbol* class_name, const Symbol* field_name,
                                                    ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
error: unknown type name 'BasicType'
                    BasicType type);
                    ^
/usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
error: unknown type name 'FILE'
  static void parse_from_file(FILE* stream);
...

Also, can we move tsanIgnoreList.hpp/cpp to the hotspot/share/tsan
directory?

-Man

From aeubanks at google.com  Tue Jul 16 21:31:58 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Tue, 16 Jul 2019 14:31:58 -0700
Subject: Build breakage
In-Reply-To: <CA+w6HxZ__JMprjZbf5Ndkff_2sP3Tg=ECTwn19=Yd0sNx0Ltpg@mail.gmail.com>
References: <CA+w6HxZ__JMprjZbf5Ndkff_2sP3Tg=ECTwn19=Yd0sNx0Ltpg@mail.gmail.com>
Message-ID: <CAPW48soA8kcUYq8sRTF631Ksu2GPfkaT193+nJrMEmF2zsZpGg@mail.gmail.com>

RFR:
http://cr.openjdk.java.net/~aeubanks/tsanmissinginclude/webrev.00/index.html
should fix the missing includes. Will considering moving tsanIgnoreList in
another change.

On Tue, Jul 16, 2019 at 12:09 PM Man Cao <manc at google.com> wrote:

> I cannot build TSAN after
> http://hg.openjdk.java.net/tsan/dev/rev/3bae5ecee45f. I'm seeing the
> following errors:
>
> In file included from
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/tsan/tsan.cpp:26:
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
> error: expected class name
> class TsanIgnoreList : AllStatic {
>                        ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
> error: unknown type name 'Symbol'
>   static bool match(const Symbol* class_name, const Symbol* field_name,
>                           ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
> error: unknown type name 'Symbol'
>   static bool match(const Symbol* class_name, const Symbol* field_name,
>                                                     ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
> error: unknown type name 'BasicType'
>                     BasicType type);
>                     ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
> error: unknown type name 'FILE'
>   static void parse_from_file(FILE* stream);
>                               ^
> In file included from
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.cpp:27:
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
> error: expected class name
> class TsanIgnoreList : AllStatic {
>                        ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
> error: unknown type name 'Symbol'
>   static bool match(const Symbol* class_name, const Symbol* field_name,
>                           ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
> error: unknown type name 'Symbol'
>   static bool match(const Symbol* class_name, const Symbol* field_name,
>                                                     ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
> error: unknown type name 'BasicType'
>                     BasicType type);
>                     ^
>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
> error: unknown type name 'FILE'
>   static void parse_from_file(FILE* stream);
> ...
>
> Also, can we move tsanIgnoreList.hpp/cpp to the hotspot/share/tsan
> directory?
>
> -Man
>

From manc at google.com  Tue Jul 16 22:55:15 2019
From: manc at google.com (Man Cao)
Date: Tue, 16 Jul 2019 15:55:15 -0700
Subject: Build breakage
In-Reply-To: <CAPW48soA8kcUYq8sRTF631Ksu2GPfkaT193+nJrMEmF2zsZpGg@mail.gmail.com>
References: <CA+w6HxZ__JMprjZbf5Ndkff_2sP3Tg=ECTwn19=Yd0sNx0Ltpg@mail.gmail.com>
 <CAPW48soA8kcUYq8sRTF631Ksu2GPfkaT193+nJrMEmF2zsZpGg@mail.gmail.com>
Message-ID: <CA+w6HxbzSNgWMEyMhVb3iHU=hsgXy847KJQuw51V+H2Pus6Qqw@mail.gmail.com>

Looks good.

-Man


On Tue, Jul 16, 2019 at 2:32 PM Arthur Eubanks <aeubanks at google.com> wrote:

> RFR:
>
> http://cr.openjdk.java.net/~aeubanks/tsanmissinginclude/webrev.00/index.html
> should fix the missing includes. Will considering moving tsanIgnoreList in
> another change.
>
> On Tue, Jul 16, 2019 at 12:09 PM Man Cao <manc at google.com> wrote:
>
>> I cannot build TSAN after
>> http://hg.openjdk.java.net/tsan/dev/rev/3bae5ecee45f. I'm seeing the
>> following errors:
>>
>> In file included from
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/tsan/tsan.cpp:26:
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
>> error: expected class name
>> class TsanIgnoreList : AllStatic {
>>                        ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
>> error: unknown type name 'Symbol'
>>   static bool match(const Symbol* class_name, const Symbol* field_name,
>>                           ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
>> error: unknown type name 'Symbol'
>>   static bool match(const Symbol* class_name, const Symbol* field_name,
>>                                                     ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
>> error: unknown type name 'BasicType'
>>                     BasicType type);
>>                     ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
>> error: unknown type name 'FILE'
>>   static void parse_from_file(FILE* stream);
>>                               ^
>> In file included from
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.cpp:27:
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
>> error: expected class name
>> class TsanIgnoreList : AllStatic {
>>                        ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
>> error: unknown type name 'Symbol'
>>   static bool match(const Symbol* class_name, const Symbol* field_name,
>>                           ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
>> error: unknown type name 'Symbol'
>>   static bool match(const Symbol* class_name, const Symbol* field_name,
>>                                                     ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
>> error: unknown type name 'BasicType'
>>                     BasicType type);
>>                     ^
>>
>> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
>> error: unknown type name 'FILE'
>>   static void parse_from_file(FILE* stream);
>> ...
>>
>> Also, can we move tsanIgnoreList.hpp/cpp to the hotspot/share/tsan
>> directory?
>>
>> -Man
>>
>

From aeubanks at google.com  Wed Jul 17 15:33:22 2019
From: aeubanks at google.com (aeubanks at google.com)
Date: Wed, 17 Jul 2019 15:33:22 +0000
Subject: hg: tsan/dev: Add missing includes
Message-ID: <201907171533.x6HFXN12005448@aojmv0008.oracle.com>

Changeset: 2ab7d6616161
Author:    aeubanks
Date:      2019-07-16 14:30 -0700
URL:       https://hg.openjdk.java.net/tsan/dev/rev/2ab7d6616161

Add missing includes

! src/hotspot/share/classfile/tsanIgnoreList.cpp
! src/hotspot/share/classfile/tsanIgnoreList.hpp


From manc at google.com  Wed Jul 17 20:02:40 2019
From: manc at google.com (Man Cao)
Date: Wed, 17 Jul 2019 13:02:40 -0700
Subject: RFR (S): Add a configure flag --<enable|disable>-tsan-launcher
Message-ID: <CA+w6HxbXYGs7jjy_QhpVwio24m-VgQa1fGJepBCY1jOoF7dhHA@mail.gmail.com>

Hi,

Can I have reviews for this:
https://cr.openjdk.java.net/~manc/tsan20190717/webrev.00/

Add a configure flag --<enable|disable>-tsan-launcher. This flag allows
more control on whether to link TSAN runtime with launcher. Default is
consistent with whether the JVM feature "tsan" is enabled.

Tested with 4 combinations of "--with-jvm-features=<tsan|-tsan>
--<enable|disable>-tsan-launcher".

-Man

From aeubanks at google.com  Wed Jul 17 20:19:06 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Wed, 17 Jul 2019 13:19:06 -0700
Subject: RFR (S): Add a configure flag --<enable|disable>-tsan-launcher
In-Reply-To: <CA+w6HxbXYGs7jjy_QhpVwio24m-VgQa1fGJepBCY1jOoF7dhHA@mail.gmail.com>
References: <CA+w6HxbXYGs7jjy_QhpVwio24m-VgQa1fGJepBCY1jOoF7dhHA@mail.gmail.com>
Message-ID: <CAPW48sqphAj-_Jj5yPa=phrRKUhDe-uemD6oF+r4PkoBi-xL8w@mail.gmail.com>

LGTM

On Wed, Jul 17, 2019 at 1:13 PM Man Cao <manc at google.com> wrote:

> Hi,
>
> Can I have reviews for this:
> https://cr.openjdk.java.net/~manc/tsan20190717/webrev.00/
>
> Add a configure flag --<enable|disable>-tsan-launcher. This flag allows
> more control on whether to link TSAN runtime with launcher. Default is
> consistent with whether the JVM feature "tsan" is enabled.
>
> Tested with 4 combinations of "--with-jvm-features=<tsan|-tsan>
> --<enable|disable>-tsan-launcher".
>
> -Man
>

From jcbeyler at google.com  Thu Jul 18 00:19:05 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Wed, 17 Jul 2019 17:19:05 -0700
Subject: Build breakage
In-Reply-To: <CA+w6HxbzSNgWMEyMhVb3iHU=hsgXy847KJQuw51V+H2Pus6Qqw@mail.gmail.com>
References: <CA+w6HxZ__JMprjZbf5Ndkff_2sP3Tg=ECTwn19=Yd0sNx0Ltpg@mail.gmail.com>
 <CAPW48soA8kcUYq8sRTF631Ksu2GPfkaT193+nJrMEmF2zsZpGg@mail.gmail.com>
 <CA+w6HxbzSNgWMEyMhVb3iHU=hsgXy847KJQuw51V+H2Pus6Qqw@mail.gmail.com>
Message-ID: <CAF9BGBy=KgKmzj6_mfKu069ZvZB_5s7zqmNU6or+SA5P7EiAWg@mail.gmail.com>

LGTM too :)
Jc

On Tue, Jul 16, 2019 at 3:55 PM Man Cao <manc at google.com> wrote:

> Looks good.
>
> -Man
>
>
> On Tue, Jul 16, 2019 at 2:32 PM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
> > RFR:
> >
> >
> http://cr.openjdk.java.net/~aeubanks/tsanmissinginclude/webrev.00/index.html
> > should fix the missing includes. Will considering moving tsanIgnoreList
> in
> > another change.
> >
> > On Tue, Jul 16, 2019 at 12:09 PM Man Cao <manc at google.com> wrote:
> >
> >> I cannot build TSAN after
> >> http://hg.openjdk.java.net/tsan/dev/rev/3bae5ecee45f. I'm seeing the
> >> following errors:
> >>
> >> In file included from
> >> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/tsan/tsan.cpp:26:
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
> >> error: expected class name
> >> class TsanIgnoreList : AllStatic {
> >>                        ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
> >> error: unknown type name 'Symbol'
> >>   static bool match(const Symbol* class_name, const Symbol* field_name,
> >>                           ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
> >> error: unknown type name 'Symbol'
> >>   static bool match(const Symbol* class_name, const Symbol* field_name,
> >>                                                     ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
> >> error: unknown type name 'BasicType'
> >>                     BasicType type);
> >>                     ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
> >> error: unknown type name 'FILE'
> >>   static void parse_from_file(FILE* stream);
> >>                               ^
> >> In file included from
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.cpp:27:
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:45:24:
> >> error: expected class name
> >> class TsanIgnoreList : AllStatic {
> >>                        ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:27:
> >> error: unknown type name 'Symbol'
> >>   static bool match(const Symbol* class_name, const Symbol* field_name,
> >>                           ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:54:53:
> >> error: unknown type name 'Symbol'
> >>   static bool match(const Symbol* class_name, const Symbol* field_name,
> >>                                                     ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:55:21:
> >> error: unknown type name 'BasicType'
> >>                     BasicType type);
> >>                     ^
> >>
> >>
> /usr/local/google/home/manc/ws/tsan/src/hotspot/share/classfile/tsanIgnoreList.hpp:58:31:
> >> error: unknown type name 'FILE'
> >>   static void parse_from_file(FILE* stream);
> >> ...
> >>
> >> Also, can we move tsanIgnoreList.hpp/cpp to the hotspot/share/tsan
> >> directory?
> >>
> >> -Man
> >>
> >
>


-- 

Thanks,
Jc

From jcbeyler at google.com  Thu Jul 18 00:19:24 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Wed, 17 Jul 2019 17:19:24 -0700
Subject: RFR (S): Add a configure flag --<enable|disable>-tsan-launcher
In-Reply-To: <CAPW48sqphAj-_Jj5yPa=phrRKUhDe-uemD6oF+r4PkoBi-xL8w@mail.gmail.com>
References: <CA+w6HxbXYGs7jjy_QhpVwio24m-VgQa1fGJepBCY1jOoF7dhHA@mail.gmail.com>
 <CAPW48sqphAj-_Jj5yPa=phrRKUhDe-uemD6oF+r4PkoBi-xL8w@mail.gmail.com>
Message-ID: <CAF9BGBxEOsoHZp+5viF7Z-C7Xh4ePbVGm0HwrRBLuhXAMzg7Cg@mail.gmail.com>

LGTM as well,
Jc

On Wed, Jul 17, 2019 at 1:19 PM Arthur Eubanks <aeubanks at google.com> wrote:

> LGTM
>
> On Wed, Jul 17, 2019 at 1:13 PM Man Cao <manc at google.com> wrote:
>
> > Hi,
> >
> > Can I have reviews for this:
> > https://cr.openjdk.java.net/~manc/tsan20190717/webrev.00/
> >
> > Add a configure flag --<enable|disable>-tsan-launcher. This flag allows
> > more control on whether to link TSAN runtime with launcher. Default is
> > consistent with whether the JVM feature "tsan" is enabled.
> >
> > Tested with 4 combinations of "--with-jvm-features=<tsan|-tsan>
> > --<enable|disable>-tsan-launcher".
> >
> > -Man
> >
>


-- 

Thanks,
Jc

From manc at google.com  Thu Jul 18 02:25:16 2019
From: manc at google.com (manc at google.com)
Date: Thu, 18 Jul 2019 02:25:16 +0000
Subject: hg: tsan/dev: Add a configure flag --<enable|disable>-tsan-launcher.
Message-ID: <201907180225.x6I2PGU8019023@aojmv0008.oracle.com>

Changeset: dc19f181c3b9
Author:    manc
Date:      2019-07-17 12:58 -0700
URL:       https://hg.openjdk.java.net/tsan/dev/rev/dc19f181c3b9

Add a configure flag --<enable|disable>-tsan-launcher.

This flag allows more control on whether to link TSAN runtime with launcher.
Default is consistent with whether the JVM feature "tsan" is enabled.

! make/autoconf/hotspot.m4


From jdanek at redhat.com  Sun Jul 21 08:35:30 2019
From: jdanek at redhat.com (=?UTF-8?Q?Jiri_Dan=C4=9Bk?=)
Date: Sun, 21 Jul 2019 10:35:30 +0200
Subject: Adding System.out.printlns causes tsan to miss a race?
Message-ID: <CA+S-PU+t+qSqX1zX8sTq-YRUfr40yFWzLc3H_-mwX4vkQH+rBg@mail.gmail.com>

The following program contains unsynchronized access to a variable. When
the VERBOSE variable is false, jdk-tsan prints the appropriate warning.
When VERBOSE is set to true in the source code, no race is reported on my
machine.

Is this expected behavior at this stage of the project, or is there already
a relevant configuration option that would detect this race all the time?

I remember I saw this sort of problem discussed somewhere in relation to
tsan, or maybe DiViNe (https://divine.fi.muni.cz/): how much
synchronization does doing I/O imply.

I compiled jdk-tsan with gcc9, because I had issues compiling with clang_8
on nixOS. Does that make the test above invalid? I presume I could use
ubuntu in docker, if asked.

///Using jdk-tsan at commit///

changeset:   54170:dc19f181c3b9
branch:      tsan
tag:         tip
user:        manc
date:        Wed Jul 17 12:58:33 2019 -0700
summary:     Add a configure flag --<enable|disable>-tsan-launcher.

///command///

I am using just the -XX:+ThreadSanitizer flag with java

///Hello.java///

public class Main {
    private static boolean VERBOSE = false;

    public static void main(String[] args) {
        var ref = new Object() {
            int a = 5;
        };
        System.out.println("Hello World!");
        if (VERBOSE) System.out.println("pes");

        Thread t = new Thread(()-> {
            if (VERBOSE) System.out.println("pes");
            System.out.println(ref.a);
            if (VERBOSE) System.out.println("pes");
            ref.a = 6;
            if (VERBOSE) System.out.println("pes");
        });
        if (VERBOSE) System.out.println("pes");
        t.start();

        if (VERBOSE) System.out.println("pes");
        ref.a = 7;
        if (VERBOSE) System.out.println("pes");
        System.out.println(ref.a);
        if (VERBOSE) System.out.println("pes");
    }
}

///output of jdk-tsan with VERBOSE=false///

Hello World!
7
==================
WARNING: ThreadSanitizer: data race (pid=11124)
  Read of size 4 at 0x00066382ed8c by thread T15:
    #0 Main.lambda$main$0(LMain$1;)V Main.java:13
    #1 Main$$Lambda$38.run()V ??
    #2 java.lang.Thread.run()V Thread.java:835
    #3 (Generated Stub) <null>

  Previous write of size 4 at 0x00066382ed8c by thread T1:
    #0 Main.main([Ljava/lang/String;)V Main.java:22
    #1 (Generated Stub) <null>
    #2
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
NativeMethodAccessorImpl.java:62
    #3
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
DelegatingMethodAccessorImpl.java:43
    #4
java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
Method.java:567
    #5
com.intellij.rt.execution.application.AppMainV2.main([Ljava/lang/String;)V
AppMainV2.java:131
    #6 (Generated Stub) <null>

  Thread T15 (tid=11140, running) created by thread T1 at:
    #0 pthread_create <null> (libtsan.so.0+0x3055b)
    #1 os::create_thread(Thread*, os::ThreadType, unsigned long)
/home/jdanek/repos/jdk-tsan/src/hotspot/os/linux/os_linux.cpp:774
(libjvm.so+0xbd3345)
    #2 java.lang.Thread.start()V Thread.java:804
    #3 Main.main([Ljava/lang/String;)V Main.java:19
    #4 (Generated Stub) <null>
    #5
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
NativeMethodAccessorImpl.java:62
    #6
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
DelegatingMethodAccessorImpl.java:43
    #7
java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
Method.java:567
    #8
com.intellij.rt.execution.application.AppMainV2.main([Ljava/lang/String;)V
AppMainV2.java:131
    #9 (Generated Stub) <null>

  Thread T1 (tid=11126, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x3055b)
    #1 CallJavaMainInNewThread
/home/jdanek/repos/jdk-tsan/src/java.base/unix/native/libjli/java_md_solinux.c:769
(libjli.so+0xa3c8)

SUMMARY: ThreadSanitizer: data race Main.java:13 in
Main.lambda$main$0(LMain$1;)V
==================
7
ThreadSanitizer: reported 1 warnings
ThreadSanitizer: reported 1 warnings
-- 
Mit freundlichen Gr??en / Kind regards
Jiri Dan?k

From jdanek at redhat.com  Sun Jul 21 10:56:25 2019
From: jdanek at redhat.com (=?UTF-8?Q?Jiri_Dan=C4=9Bk?=)
Date: Sun, 21 Jul 2019 12:56:25 +0200
Subject: Adding System.out.printlns causes tsan to miss a race?
In-Reply-To: <CA+S-PU+t+qSqX1zX8sTq-YRUfr40yFWzLc3H_-mwX4vkQH+rBg@mail.gmail.com>
References: <CA+S-PU+t+qSqX1zX8sTq-YRUfr40yFWzLc3H_-mwX4vkQH+rBg@mail.gmail.com>
Message-ID: <CA+S-PUJtCUTQtpk6uauOMW-tiCJhRBZu6B9kZPgVm82_2A_u3g@mail.gmail.com>

On Sun, Jul 21, 2019 at 10:35 AM Jiri Dan?k <jdanek at redhat.com> wrote:

> I compiled jdk-tsan with gcc9, because I had issues compiling with clang_8
> on nixOS. Does that make the test above invalid? I presume I could use
> ubuntu in docker, if asked.
>

Managed to compile with clang_8. With the VERBOSE=true, I get the sanitizer
warning in what feels about 1 in 10 runs. I don't have the gcc9 build any
more to compare, there I never saw warning with VERBOSE=true, but I did not
run it many times and I did not try running in a loop.

One of the lucky runs looks like this

$ build/linux-x86_64-server-release/jdk/bin/java -XX:+ThreadSanitizer
-classpath /home/jdanek/repos/java-sanitizer/out/production/java-sanitizer/
Main
Hello World!
pes
pes
pes
pes
pes
7
pes
==================
WARNING: ThreadSanitizer: data race (pid=22615)
  Read of size 4 at 0x00066390da34 by thread T14:
    #0 Main.lambda$main$0(LMain$1;)V Main.java:13
    #1 Main$$Lambda$36.run()V ??
    #2 java.lang.Thread.run()V Thread.java:835
    #3 (Generated Stub) <null>

  Previous write of size 4 at 0x00066390da34 by thread T1:
    #0 Main.main([Ljava/lang/String;)V Main.java:22
    #1 (Generated Stub) <null>

  Thread T14 (tid=22631, running) created by thread T1 at:
    #0 pthread_create ??:? (java+0x42c5d6)
    #1 os::create_thread(Thread*, os::ThreadType, unsigned long)
/home/jdanek/repos/jdk-tsan/src/hotspot/os/linux/os_linux.cpp:774
(libjvm.so+0xb2683d)
    #2 java.lang.Thread.start()V Thread.java:804
    #3 Main.main([Ljava/lang/String;)V Main.java:19
    #4 (Generated Stub) <null>

  Thread T1 (tid=22617, running) created by main thread at:
    #0 pthread_create ??:? (java+0x42c5d6)
    #1 CallJavaMainInNewThread
/home/jdanek/repos/jdk-tsan/src/java.base/unix/native/libjli/java_md_solinux.c:769
(libjli.so+0x9cba)

SUMMARY: ThreadSanitizer: data race Main.java:13 in
Main.lambda$main$0(LMain$1;)V
==================
7
pes
pes
ThreadSanitizer: reported 1 warnings
ThreadSanitizer: reported 1 warnings
-- 
Mit freundlichen Gr??en / Kind regards
Jiri Dan?k

From jdanek at redhat.com  Sun Jul 21 11:06:24 2019
From: jdanek at redhat.com (=?UTF-8?Q?Jiri_Dan=C4=9Bk?=)
Date: Sun, 21 Jul 2019 13:06:24 +0200
Subject: Difficulties compiling: clang (not clang++) to precompile c++
 headers; various compiler warnings (treated as errors)
In-Reply-To: <CA+S-PU+t+qSqX1zX8sTq-YRUfr40yFWzLc3H_-mwX4vkQH+rBg@mail.gmail.com>
References: <CA+S-PU+t+qSqX1zX8sTq-YRUfr40yFWzLc3H_-mwX4vkQH+rBg@mail.gmail.com>
Message-ID: <CA+S-PU+GH=0a+e0MYVaj82ZkEhxviZ5uV2DmQxK4ezc+inQQLw@mail.gmail.com>

I attempted compilation of jdk-tsan with both gcc9 and clang_8. With either
compiler, I had to add additional options to configure, otherwise
compilation would fail. I thought I'll mention it here.

# Gcc 9

I had to do

bash configure --disable-warnings-as-errors

otherwise I'd have errors like

In file included from
> /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/stdio.h:862,
>                  from
> /home/jdanek/repos/jdk-tsan/src/hotspot/share/adlc/adlc.hpp:35,
>                  from
> /home/jdanek/repos/jdk-tsan/src/hotspot/share/adlc/adlparse.cpp:27:
> In function 'int fprintf(FILE*, const char*, ...)',
>     inlined from 'FormatRule* ADLParser::template_parse()' at
> /home/jdanek/repos/jdk-tsan/src/hotspot/share/adlc/adlparse.cpp:4080:34:
> /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/bits/stdio2.h:97:24:
> error: '%s' directive argument is null [-Werror=format-overflow=]
>    97 |   return __fprintf_chk (__stream, __USE_FORTIFY_LEVEL - 1, __fmt,
>       |          ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>    98 |    __va_arg_pack ());
>       |    ~~~~~~~~~~~~~~~~~
>

# Clang 8

I had to do

bash configure --with-toolchain-type=clang --disable-precompiled-headers
--disable-warnings-as-errors

without --disable-precompiled-headers, make would try to compile headers
with clang, instead of clang++, and my clang would not find c++ headers

$ LOG=trace make
> [...]
>
/nix/store/ki0cb2q9bib66zwjldfk6dkp9apkmgph-clang-wrapper-8.0.0/bin/clang
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
-D_GNU_SOURCE -pipe -fno-rtti -fno-exceptions -fvisibility=hidden
-fno-strict-aliasing -fno-omit-frame-pointer -flimit-debug-info
-mno-omit-leaf-frame-pointer -mstack-alignment=16
-DSUPPORTS_CLOCK_MONOTONIC -DLINUX -Wall -Wextra -Wformat=2 -Wpointer-arith
-Wsign-compare -Wreorder -Wunused-function -Wundef -Wunused-value
-Woverloaded-virtual -fPIC -DVM_LITTLE_ENDIAN -D_LP64=1 -m64 -DPRODUCT
-DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_linux -DINCLUDE_SUFFIX_CPU=_x86
-DINCLUDE_SUFFIX_COMPILER=_gcc -DTARGET_COMPILER_gcc -DAMD64
'-DHOTSPOT_LIB_ARCH="amd64"' -DCOMPILER1 -DCOMPILER2
-DSUPPORT_BARRIER_ON_PRIMITIVES -DSUPPORT_NOT_TO_SPACE_INVARIANT
-I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/gensrc/adfiles
-I/home/jdanek/repos/jdk-tsan/src/hotspot/share
-I/home/jdanek/repos/jdk-tsan/src/hotspot/os/linux
-I/home/jdanek/repos/jdk-tsan/src/hotspot/os/posix
-I/home/jdanek/repos/jdk-tsan/src/hotspot/cpu/x86
-I/home/jdanek/repos/jdk-tsan/src/hotspot/os_cpu/linux_x86
-I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/gensrc
-I/home/jdanek/repos/jdk-tsan/src/hotspot/share/precompiled
-I/home/jdanek/repos/jdk-tsan/src/hotspot/share/include
-I/home/jdanek/repos/jdk-tsan/src/hotspot/os/posix/include
-I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/support/modules_include/java.base
-I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/support/modules_include/java.base/linux
-I/home/jdanek/repos/jdk-tsan/src/java.base/share/native/libjimage -m64 -g
-Wno-unused-parameter -Wno-unused -Wno-tautological-compare
-Wno-undefined-var-template -Wno-sometimes-uninitialized
-Wno-unknown-pragmas -Wno-delete-non-virtual-dtor -Wno-missing-braces
-Wno-char-subscripts -Wno-ignored-qualifiers
-Wno-missing-field-initializers -Wno-mismatched-tags -Werror -O3 -x
c++-header -c -MMD -MF
/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch.d
/home/jdanek/repos/jdk-tsan/src/hotspot/share/precompiled/precompiled.hpp
-o
/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch
In file included from
/home/jdanek/repos/jdk-tsan/src/hotspot/share/precompiled/precompiled.hpp:34:
In file included from
/home/jdanek/repos/jdk-tsan/src/hotspot/share/classfile/classLoaderData.hpp:28:
/home/jdanek/repos/jdk-tsan/src/hotspot/share/memory/allocation.hpp:32:10:
fatal error: 'new' file not found
#include <new>
         ^~~~~
1 error generated.


when I rerun the command with clang++, it passed without error.

Next, --disable-warnings-as-errors was needed because of warnings like this

>
> /home/jdanek/repos/jdk-tsan/test/hotspot/gtest/classfile/test_symbolTable.cpp:62:6:
> error: explicitly assigning value of variable of type 'TempNewSymbol' to
> itself [-Werror,-Wself-assign-overloaded]
>   s1 = s1; // self assignment
>   ~~ ^ ~~
> 1 error generated.
>
> /home/jdanek/repos/jdk-tsan/src/java.smartcardio/share/native/libj2pcsc/pcsc.c:48:9:
> warning: 'dprintf' macro redefined [-Wmacro-redefined]
> #define dprintf(s)
>         ^
> /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/bits/stdio2.h:145:12:
> note: previous definition is here
> #   define dprintf(fd, ...) \
>            ^
> Creating support/modules_cmds/jdk.aot/jaotc from 1 file(s)
>

and later I saw also this

clang-8: warning: argument unused during compilation: '-D
> _FORTIFY_SOURCE=2' [-Wunused-command-line-argument]
> clang-8: warning: argument unused during compilation:
> '-fstack-protector-strong' [-Wunused-command-line-argument]
> clang-8: warning: argument unused during compilation: '--param
> ssp-buffer-size=4' [-Wunused-command-line-argument]
> clang-8: warning: argument unused during compilation:
> '-fno-strict-overflow' [-Wunused-command-line-argument]
> clang-8: warning: argument unused during compilation: '-idirafter
> /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include'
> [-Wunused-command-line-argument]
> clang-8: warning: argument unused during compilation: '-isystem
> /nix/store/1s2f1fhb63kv01s0mbjrs7sps0lvc5qi-libc++-8.0.0/include'
> [-Wunused-command-line-argument]
>

Thanks,
-- 
Mit freundlichen Gr??en / Kind regards
Jiri Dan?k

From dvyukov at google.com  Mon Jul 22 05:44:35 2019
From: dvyukov at google.com (Dmitry Vyukov)
Date: Mon, 22 Jul 2019 07:44:35 +0200
Subject: Adding System.out.printlns causes tsan to miss a race?
In-Reply-To: <CA+S-PUJtCUTQtpk6uauOMW-tiCJhRBZu6B9kZPgVm82_2A_u3g@mail.gmail.com>
References: <CA+S-PU+t+qSqX1zX8sTq-YRUfr40yFWzLc3H_-mwX4vkQH+rBg@mail.gmail.com>
 <CA+S-PUJtCUTQtpk6uauOMW-tiCJhRBZu6B9kZPgVm82_2A_u3g@mail.gmail.com>
Message-ID: <CACT4Y+ZeYpfE8YDnq=SOmK=5GE5yd_yZAp9K_EQ9goRueE7Rig@mail.gmail.com>

On Sun, Jul 21, 2019 at 12:57 PM Jiri Dan?k <jdanek at redhat.com> wrote:
>
> On Sun, Jul 21, 2019 at 10:35 AM Jiri Dan?k <jdanek at redhat.com> wrote:
>
> > I compiled jdk-tsan with gcc9, because I had issues compiling with clang_8
> > on nixOS. Does that make the test above invalid? I presume I could use
> > ubuntu in docker, if asked.
> >
>
> Managed to compile with clang_8. With the VERBOSE=true, I get the sanitizer
> warning in what feels about 1 in 10 runs. I don't have the gcc9 build any
> more to compare, there I never saw warning with VERBOSE=true, but I did not
> run it many times and I did not try running in a loop.
>
> One of the lucky runs looks like this
>
> $ build/linux-x86_64-server-release/jdk/bin/java -XX:+ThreadSanitizer
> -classpath /home/jdanek/repos/java-sanitizer/out/production/java-sanitizer/
> Main
> Hello World!
> pes
> pes
> pes
> pes
> pes
> 7
> pes
> ==================
> WARNING: ThreadSanitizer: data race (pid=22615)
>   Read of size 4 at 0x00066390da34 by thread T14:
>     #0 Main.lambda$main$0(LMain$1;)V Main.java:13
>     #1 Main$$Lambda$36.run()V ??
>     #2 java.lang.Thread.run()V Thread.java:835
>     #3 (Generated Stub) <null>
>
>   Previous write of size 4 at 0x00066390da34 by thread T1:
>     #0 Main.main([Ljava/lang/String;)V Main.java:22
>     #1 (Generated Stub) <null>
>
>   Thread T14 (tid=22631, running) created by thread T1 at:
>     #0 pthread_create ??:? (java+0x42c5d6)
>     #1 os::create_thread(Thread*, os::ThreadType, unsigned long)
> /home/jdanek/repos/jdk-tsan/src/hotspot/os/linux/os_linux.cpp:774
> (libjvm.so+0xb2683d)
>     #2 java.lang.Thread.start()V Thread.java:804
>     #3 Main.main([Ljava/lang/String;)V Main.java:19
>     #4 (Generated Stub) <null>
>
>   Thread T1 (tid=22617, running) created by main thread at:
>     #0 pthread_create ??:? (java+0x42c5d6)
>     #1 CallJavaMainInNewThread
> /home/jdanek/repos/jdk-tsan/src/java.base/unix/native/libjli/java_md_solinux.c:769
> (libjli.so+0x9cba)
>
> SUMMARY: ThreadSanitizer: data race Main.java:13 in
> Main.lambda$main$0(LMain$1;)V
> ==================
> 7
> pes
> pes
> ThreadSanitizer: reported 1 warnings
> ThreadSanitizer: reported 1 warnings


Hi Jiri,

Yes, this is an unfortunate aspect of happens-before-based race
detectors. But this is the cost for not having false positives.
Inserting something that contains synchronization (esp. global
synchronization) between operations makes detecting races on these
operations harder. We need to catch the racing operations happening
closer in time to catch races, close enough so that there is no
parasitic synchronization in between. Consider, if threads don't do
any synchronization, then tsan should be able to catch racing accesses
even if they happen an hour apart. But the more synchronization is
there, the closer we need to catch them.
The practical consequence: it's useful to run all tests multiple times
and continuously, then all races tend to pop up over time.
If we are talking about unit tests, then they may need to be specially
engineered to trigger races more consistently (in particular, no
printf's ;)).

From aeubanks at google.com  Mon Jul 22 15:28:40 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Mon, 22 Jul 2019 08:28:40 -0700
Subject: Difficulties compiling: clang (not clang++) to precompile c++
 headers; various compiler warnings (treated as errors)
In-Reply-To: <CA+S-PU+GH=0a+e0MYVaj82ZkEhxviZ5uV2DmQxK4ezc+inQQLw@mail.gmail.com>
References: <CA+S-PU+t+qSqX1zX8sTq-YRUfr40yFWzLc3H_-mwX4vkQH+rBg@mail.gmail.com>
 <CA+S-PU+GH=0a+e0MYVaj82ZkEhxviZ5uV2DmQxK4ezc+inQQLw@mail.gmail.com>
Message-ID: <CAPW48spTeM7k3MkAD50CakLkQtsxWT0tP0fPo=oi7hOWd9-G2g@mail.gmail.com>

I believe everybody has been passing --disable-warnings-as-errors, I added
that to the wiki page <https://wiki.openjdk.java.net/display/tsan/Main>.
I personally haven't been using --disable-precompiled-headers, so I'm not
sure why it's necessary for you. make shouldn't be invoking clang on cpp
files.

On Sun, Jul 21, 2019 at 4:07 AM Jiri Dan?k <jdanek at redhat.com> wrote:

> I attempted compilation of jdk-tsan with both gcc9 and clang_8. With either
> compiler, I had to add additional options to configure, otherwise
> compilation would fail. I thought I'll mention it here.
>
> # Gcc 9
>
> I had to do
>
> bash configure --disable-warnings-as-errors
>
> otherwise I'd have errors like
>
> In file included from
> >
> /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/stdio.h:862,
> >                  from
> > /home/jdanek/repos/jdk-tsan/src/hotspot/share/adlc/adlc.hpp:35,
> >                  from
> > /home/jdanek/repos/jdk-tsan/src/hotspot/share/adlc/adlparse.cpp:27:
> > In function 'int fprintf(FILE*, const char*, ...)',
> >     inlined from 'FormatRule* ADLParser::template_parse()' at
> > /home/jdanek/repos/jdk-tsan/src/hotspot/share/adlc/adlparse.cpp:4080:34:
> >
> /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/bits/stdio2.h:97:24:
> > error: '%s' directive argument is null [-Werror=format-overflow=]
> >    97 |   return __fprintf_chk (__stream, __USE_FORTIFY_LEVEL - 1, __fmt,
> >       |          ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >    98 |    __va_arg_pack ());
> >       |    ~~~~~~~~~~~~~~~~~
> >
>
> # Clang 8
>
> I had to do
>
> bash configure --with-toolchain-type=clang --disable-precompiled-headers
> --disable-warnings-as-errors
>
> without --disable-precompiled-headers, make would try to compile headers
> with clang, instead of clang++, and my clang would not find c++ headers
>
> $ LOG=trace make
> > [...]
> >
> /nix/store/ki0cb2q9bib66zwjldfk6dkp9apkmgph-clang-wrapper-8.0.0/bin/clang
> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS
> -D_GNU_SOURCE -pipe -fno-rtti -fno-exceptions -fvisibility=hidden
> -fno-strict-aliasing -fno-omit-frame-pointer -flimit-debug-info
> -mno-omit-leaf-frame-pointer -mstack-alignment=16
> -DSUPPORTS_CLOCK_MONOTONIC -DLINUX -Wall -Wextra -Wformat=2 -Wpointer-arith
> -Wsign-compare -Wreorder -Wunused-function -Wundef -Wunused-value
> -Woverloaded-virtual -fPIC -DVM_LITTLE_ENDIAN -D_LP64=1 -m64 -DPRODUCT
> -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_linux -DINCLUDE_SUFFIX_CPU=_x86
> -DINCLUDE_SUFFIX_COMPILER=_gcc -DTARGET_COMPILER_gcc -DAMD64
> '-DHOTSPOT_LIB_ARCH="amd64"' -DCOMPILER1 -DCOMPILER2
> -DSUPPORT_BARRIER_ON_PRIMITIVES -DSUPPORT_NOT_TO_SPACE_INVARIANT
>
> -I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/gensrc/adfiles
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/share
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/os/linux
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/os/posix
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/cpu/x86
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/os_cpu/linux_x86
>
> -I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/gensrc
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/share/precompiled
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/share/include
> -I/home/jdanek/repos/jdk-tsan/src/hotspot/os/posix/include
>
> -I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/support/modules_include/java.base
>
> -I/home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/support/modules_include/java.base/linux
> -I/home/jdanek/repos/jdk-tsan/src/java.base/share/native/libjimage -m64 -g
> -Wno-unused-parameter -Wno-unused -Wno-tautological-compare
> -Wno-undefined-var-template -Wno-sometimes-uninitialized
> -Wno-unknown-pragmas -Wno-delete-non-virtual-dtor -Wno-missing-braces
> -Wno-char-subscripts -Wno-ignored-qualifiers
> -Wno-missing-field-initializers -Wno-mismatched-tags -Werror -O3 -x
> c++-header -c -MMD -MF
>
> /home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch.d
> /home/jdanek/repos/jdk-tsan/src/hotspot/share/precompiled/precompiled.hpp
> -o
>
> /home/jdanek/repos/jdk-tsan/build/linux-x86_64-server-release/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch
> In file included from
>
> /home/jdanek/repos/jdk-tsan/src/hotspot/share/precompiled/precompiled.hpp:34:
> In file included from
>
> /home/jdanek/repos/jdk-tsan/src/hotspot/share/classfile/classLoaderData.hpp:28:
> /home/jdanek/repos/jdk-tsan/src/hotspot/share/memory/allocation.hpp:32:10:
> fatal error: 'new' file not found
> #include <new>
>          ^~~~~
> 1 error generated.
>
>
> when I rerun the command with clang++, it passed without error.
>
> Next, --disable-warnings-as-errors was needed because of warnings like this
>
> >
> >
> /home/jdanek/repos/jdk-tsan/test/hotspot/gtest/classfile/test_symbolTable.cpp:62:6:
> > error: explicitly assigning value of variable of type 'TempNewSymbol' to
> > itself [-Werror,-Wself-assign-overloaded]
> >   s1 = s1; // self assignment
> >   ~~ ^ ~~
> > 1 error generated.
> >
> >
> /home/jdanek/repos/jdk-tsan/src/java.smartcardio/share/native/libj2pcsc/pcsc.c:48:9:
> > warning: 'dprintf' macro redefined [-Wmacro-redefined]
> > #define dprintf(s)
> >         ^
> >
> /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/bits/stdio2.h:145:12:
> > note: previous definition is here
> > #   define dprintf(fd, ...) \
> >            ^
> > Creating support/modules_cmds/jdk.aot/jaotc from 1 file(s)
> >
>
> and later I saw also this
>
> clang-8: warning: argument unused during compilation: '-D
> > _FORTIFY_SOURCE=2' [-Wunused-command-line-argument]
> > clang-8: warning: argument unused during compilation:
> > '-fstack-protector-strong' [-Wunused-command-line-argument]
> > clang-8: warning: argument unused during compilation: '--param
> > ssp-buffer-size=4' [-Wunused-command-line-argument]
> > clang-8: warning: argument unused during compilation:
> > '-fno-strict-overflow' [-Wunused-command-line-argument]
> > clang-8: warning: argument unused during compilation: '-idirafter
> > /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include'
> > [-Wunused-command-line-argument]
> > clang-8: warning: argument unused during compilation: '-isystem
> > /nix/store/1s2f1fhb63kv01s0mbjrs7sps0lvc5qi-libc++-8.0.0/include'
> > [-Wunused-command-line-argument]
> >
>
> Thanks,
> --
> Mit freundlichen Gr??en / Kind regards
> Jiri Dan?k
>

From manc at google.com  Thu Jul 25 21:34:52 2019
From: manc at google.com (Man Cao)
Date: Thu, 25 Jul 2019 14:34:52 -0700
Subject: RFR (S): Make uncaught exception in child threads trigger test failure
Message-ID: <CA+w6Hxbh41g1+9ZhYE-3+8WBWTY+j6pjs18aZXFQq2iuAWxPXw@mail.gmail.com>

Hi all,

Can I have reviews for this:
https://cr.openjdk.java.net/~manc/tsan20190725/webrev.00/

Currently, if a child thread terminates due to an uncaught exception, such
as NullPointerException due to memory corruption, the TSAN test would still
pass. This change makes sure the test fail in such cases.

Tested with the following local modification:
--- a/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
+++ b/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
@@ -45,6 +45,9 @@ class NonRacyIntMemberLoopRunner extends
   @Override
   protected synchronized void run(int i) {
     x = x + 1;
+    if (i % 100 == 0) {
+      throw new NullPointerException();
+    }
   }

   public static void main(String[] args) throws InterruptedException

The test would pass without this change, and properly fails with this
change.

-Man

From jcbeyler at google.com  Thu Jul 25 21:42:27 2019
From: jcbeyler at google.com (Jean Christophe Beyler)
Date: Thu, 25 Jul 2019 14:42:27 -0700
Subject: RFR (S): Make uncaught exception in child threads trigger test
 failure
In-Reply-To: <CA+w6Hxbh41g1+9ZhYE-3+8WBWTY+j6pjs18aZXFQq2iuAWxPXw@mail.gmail.com>
References: <CA+w6Hxbh41g1+9ZhYE-3+8WBWTY+j6pjs18aZXFQq2iuAWxPXw@mail.gmail.com>
Message-ID: <CAF9BGByLGVt81HNKFG35DyeGJtiwYsCZhSFeJye_GLpar5pkXA@mail.gmail.com>

Hi Man,

Looks good to me!
Jc

On Thu, Jul 25, 2019 at 2:35 PM Man Cao <manc at google.com> wrote:

> Hi all,
>
> Can I have reviews for this:
> https://cr.openjdk.java.net/~manc/tsan20190725/webrev.00/
>
> Currently, if a child thread terminates due to an uncaught exception, such
> as NullPointerException due to memory corruption, the TSAN test would still
> pass. This change makes sure the test fail in such cases.
>
> Tested with the following local modification:
> --- a/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
> +++ b/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
> @@ -45,6 +45,9 @@ class NonRacyIntMemberLoopRunner extends
>    @Override
>    protected synchronized void run(int i) {
>      x = x + 1;
> +    if (i % 100 == 0) {
> +      throw new NullPointerException();
> +    }
>    }
>
>    public static void main(String[] args) throws InterruptedException
>
> The test would pass without this change, and properly fails with this
> change.
>
> -Man
>


-- 

Thanks,
Jc

From aeubanks at google.com  Thu Jul 25 22:05:38 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Thu, 25 Jul 2019 15:05:38 -0700
Subject: RFR (S): Make uncaught exception in child threads trigger test
 failure
In-Reply-To: <CAF9BGByLGVt81HNKFG35DyeGJtiwYsCZhSFeJye_GLpar5pkXA@mail.gmail.com>
References: <CA+w6Hxbh41g1+9ZhYE-3+8WBWTY+j6pjs18aZXFQq2iuAWxPXw@mail.gmail.com>
 <CAF9BGByLGVt81HNKFG35DyeGJtiwYsCZhSFeJye_GLpar5pkXA@mail.gmail.com>
Message-ID: <CAPW48sriqnP-eJwj1sDdSiz-BsdZ3XhXb1djOULbsXY=HHgjBw@mail.gmail.com>

LGTM

On Thu, Jul 25, 2019 at 2:43 PM Jean Christophe Beyler <jcbeyler at google.com>
wrote:

> Hi Man,
>
> Looks good to me!
> Jc
>
> On Thu, Jul 25, 2019 at 2:35 PM Man Cao <manc at google.com> wrote:
>
> > Hi all,
> >
> > Can I have reviews for this:
> > https://cr.openjdk.java.net/~manc/tsan20190725/webrev.00/
> >
> > Currently, if a child thread terminates due to an uncaught exception,
> such
> > as NullPointerException due to memory corruption, the TSAN test would
> still
> > pass. This change makes sure the test fail in such cases.
> >
> > Tested with the following local modification:
> > --- a/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
> > +++ b/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
> > @@ -45,6 +45,9 @@ class NonRacyIntMemberLoopRunner extends
> >    @Override
> >    protected synchronized void run(int i) {
> >      x = x + 1;
> > +    if (i % 100 == 0) {
> > +      throw new NullPointerException();
> > +    }
> >    }
> >
> >    public static void main(String[] args) throws InterruptedException
> >
> > The test would pass without this change, and properly fails with this
> > change.
> >
> > -Man
> >
>
>
> --
>
> Thanks,
> Jc
>

From aeubanks at google.com  Thu Jul 25 22:13:08 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Thu, 25 Jul 2019 15:13:08 -0700
Subject: RFR (S): Make uncaught exception in child threads trigger test
 failure
In-Reply-To: <CAPW48sriqnP-eJwj1sDdSiz-BsdZ3XhXb1djOULbsXY=HHgjBw@mail.gmail.com>
References: <CA+w6Hxbh41g1+9ZhYE-3+8WBWTY+j6pjs18aZXFQq2iuAWxPXw@mail.gmail.com>
 <CAF9BGByLGVt81HNKFG35DyeGJtiwYsCZhSFeJye_GLpar5pkXA@mail.gmail.com>
 <CAPW48sriqnP-eJwj1sDdSiz-BsdZ3XhXb1djOULbsXY=HHgjBw@mail.gmail.com>
Message-ID: <CAPW48somn3AUH1eaJawYMNXYe_k6cORicLuo=gBYKspGLhD-Xg@mail.gmail.com>

Actually, I think adding a test for this would be good.

On Thu, Jul 25, 2019 at 3:05 PM Arthur Eubanks <aeubanks at google.com> wrote:

> LGTM
>
> On Thu, Jul 25, 2019 at 2:43 PM Jean Christophe Beyler <
> jcbeyler at google.com> wrote:
>
>> Hi Man,
>>
>> Looks good to me!
>> Jc
>>
>> On Thu, Jul 25, 2019 at 2:35 PM Man Cao <manc at google.com> wrote:
>>
>> > Hi all,
>> >
>> > Can I have reviews for this:
>> > https://cr.openjdk.java.net/~manc/tsan20190725/webrev.00/
>> >
>> > Currently, if a child thread terminates due to an uncaught exception,
>> such
>> > as NullPointerException due to memory corruption, the TSAN test would
>> still
>> > pass. This change makes sure the test fail in such cases.
>> >
>> > Tested with the following local modification:
>> > --- a/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
>> > +++ b/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
>> > @@ -45,6 +45,9 @@ class NonRacyIntMemberLoopRunner extends
>> >    @Override
>> >    protected synchronized void run(int i) {
>> >      x = x + 1;
>> > +    if (i % 100 == 0) {
>> > +      throw new NullPointerException();
>> > +    }
>> >    }
>> >
>> >    public static void main(String[] args) throws InterruptedException
>> >
>> > The test would pass without this change, and properly fails with this
>> > change.
>> >
>> > -Man
>> >
>>
>>
>> --
>>
>> Thanks,
>> Jc
>>
>

From manc at google.com  Thu Jul 25 22:31:58 2019
From: manc at google.com (Man Cao)
Date: Thu, 25 Jul 2019 15:31:58 -0700
Subject: RFR (S): Make uncaught exception in child threads trigger test
 failure
In-Reply-To: <CAPW48somn3AUH1eaJawYMNXYe_k6cORicLuo=gBYKspGLhD-Xg@mail.gmail.com>
References: <CA+w6Hxbh41g1+9ZhYE-3+8WBWTY+j6pjs18aZXFQq2iuAWxPXw@mail.gmail.com>
 <CAF9BGByLGVt81HNKFG35DyeGJtiwYsCZhSFeJye_GLpar5pkXA@mail.gmail.com>
 <CAPW48sriqnP-eJwj1sDdSiz-BsdZ3XhXb1djOULbsXY=HHgjBw@mail.gmail.com>
 <CAPW48somn3AUH1eaJawYMNXYe_k6cORicLuo=gBYKspGLhD-Xg@mail.gmail.com>
Message-ID: <CA+w6HxYBgOiiKhJEm5=Ej09WaY9AXXJ90QWYHNnmg-8AeHApLg@mail.gmail.com>

I added a test:
https://cr.openjdk.java.net/~manc/tsan20190725/webrev.01/

I will merge the two webrevs into a single commit.

-Man


On Thu, Jul 25, 2019 at 3:13 PM Arthur Eubanks <aeubanks at google.com> wrote:

> Actually, I think adding a test for this would be good.
>
> On Thu, Jul 25, 2019 at 3:05 PM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
>> LGTM
>>
>> On Thu, Jul 25, 2019 at 2:43 PM Jean Christophe Beyler <
>> jcbeyler at google.com> wrote:
>>
>>> Hi Man,
>>>
>>> Looks good to me!
>>> Jc
>>>
>>> On Thu, Jul 25, 2019 at 2:35 PM Man Cao <manc at google.com> wrote:
>>>
>>> > Hi all,
>>> >
>>> > Can I have reviews for this:
>>> > https://cr.openjdk.java.net/~manc/tsan20190725/webrev.00/
>>> >
>>> > Currently, if a child thread terminates due to an uncaught exception,
>>> such
>>> > as NullPointerException due to memory corruption, the TSAN test would
>>> still
>>> > pass. This change makes sure the test fail in such cases.
>>> >
>>> > Tested with the following local modification:
>>> > --- a/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
>>> > +++ b/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
>>> > @@ -45,6 +45,9 @@ class NonRacyIntMemberLoopRunner extends
>>> >    @Override
>>> >    protected synchronized void run(int i) {
>>> >      x = x + 1;
>>> > +    if (i % 100 == 0) {
>>> > +      throw new NullPointerException();
>>> > +    }
>>> >    }
>>> >
>>> >    public static void main(String[] args) throws InterruptedException
>>> >
>>> > The test would pass without this change, and properly fails with this
>>> > change.
>>> >
>>> > -Man
>>> >
>>>
>>>
>>> --
>>>
>>> Thanks,
>>> Jc
>>>
>>

From aeubanks at google.com  Thu Jul 25 22:37:10 2019
From: aeubanks at google.com (Arthur Eubanks)
Date: Thu, 25 Jul 2019 15:37:10 -0700
Subject: RFR (S): Make uncaught exception in child threads trigger test
 failure
In-Reply-To: <CA+w6HxYBgOiiKhJEm5=Ej09WaY9AXXJ90QWYHNnmg-8AeHApLg@mail.gmail.com>
References: <CA+w6Hxbh41g1+9ZhYE-3+8WBWTY+j6pjs18aZXFQq2iuAWxPXw@mail.gmail.com>
 <CAF9BGByLGVt81HNKFG35DyeGJtiwYsCZhSFeJye_GLpar5pkXA@mail.gmail.com>
 <CAPW48sriqnP-eJwj1sDdSiz-BsdZ3XhXb1djOULbsXY=HHgjBw@mail.gmail.com>
 <CAPW48somn3AUH1eaJawYMNXYe_k6cORicLuo=gBYKspGLhD-Xg@mail.gmail.com>
 <CA+w6HxYBgOiiKhJEm5=Ej09WaY9AXXJ90QWYHNnmg-8AeHApLg@mail.gmail.com>
Message-ID: <CAPW48sp37DfxwKEGstQWN3TgdYaPO-SWU2fwSyAJrLW6DkR1fQ@mail.gmail.com>

LGTM

On Thu, Jul 25, 2019 at 3:32 PM Man Cao <manc at google.com> wrote:

> I added a test:
> https://cr.openjdk.java.net/~manc/tsan20190725/webrev.01/
>
> I will merge the two webrevs into a single commit.
>
> -Man
>
>
> On Thu, Jul 25, 2019 at 3:13 PM Arthur Eubanks <aeubanks at google.com>
> wrote:
>
>> Actually, I think adding a test for this would be good.
>>
>> On Thu, Jul 25, 2019 at 3:05 PM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>>> LGTM
>>>
>>> On Thu, Jul 25, 2019 at 2:43 PM Jean Christophe Beyler <
>>> jcbeyler at google.com> wrote:
>>>
>>>> Hi Man,
>>>>
>>>> Looks good to me!
>>>> Jc
>>>>
>>>> On Thu, Jul 25, 2019 at 2:35 PM Man Cao <manc at google.com> wrote:
>>>>
>>>> > Hi all,
>>>> >
>>>> > Can I have reviews for this:
>>>> > https://cr.openjdk.java.net/~manc/tsan20190725/webrev.00/
>>>> >
>>>> > Currently, if a child thread terminates due to an uncaught exception,
>>>> such
>>>> > as NullPointerException due to memory corruption, the TSAN test would
>>>> still
>>>> > pass. This change makes sure the test fail in such cases.
>>>> >
>>>> > Tested with the following local modification:
>>>> > --- a/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
>>>> > +++ b/test/hotspot/jtreg/tsan/NonRacyIntMemberLoopTest.java
>>>> > @@ -45,6 +45,9 @@ class NonRacyIntMemberLoopRunner extends
>>>> >    @Override
>>>> >    protected synchronized void run(int i) {
>>>> >      x = x + 1;
>>>> > +    if (i % 100 == 0) {
>>>> > +      throw new NullPointerException();
>>>> > +    }
>>>> >    }
>>>> >
>>>> >    public static void main(String[] args) throws InterruptedException
>>>> >
>>>> > The test would pass without this change, and properly fails with this
>>>> > change.
>>>> >
>>>> > -Man
>>>> >
>>>>
>>>>
>>>> --
>>>>
>>>> Thanks,
>>>> Jc
>>>>
>>>

From manc at google.com  Thu Jul 25 22:48:42 2019
From: manc at google.com (manc at google.com)
Date: Thu, 25 Jul 2019 22:48:42 +0000
Subject: hg: tsan/dev: Make uncaught exception in child threads trigger test
 failure.
Message-ID: <201907252248.x6PMmgxd027438@aojmv0008.oracle.com>

Changeset: 0a691930e701
Author:    manc
Date:      2019-07-25 14:27 -0700
URL:       https://hg.openjdk.java.net/tsan/dev/rev/0a691930e701

Make uncaught exception in child threads trigger test failure.

! test/hotspot/jtreg/tsan/AbstractLoop.java
+ test/hotspot/jtreg/tsan/ChildThreadExceptionTest.java
! test/hotspot/jtreg/tsan/TsanRunner.java