From alanb at openjdk.java.net Sun Nov 1 16:08:54 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 1 Nov 2020 16:08:54 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) In-Reply-To: References: Message-ID: On Thu, 29 Oct 2020 14:13:40 GMT, Maurizio Cimadamore wrote: >>> @mcimadamore, if you pull from current master, you would get the Linux x86_32 tier1 run "for free". >> >> Just did that - I also removed TestMismatch from the problem list in the latest iteration, and fixed the alignment for long/double layouts, after chatting with the team (https://bugs.openjdk.java.net/browse/JDK-8255350) > > I've just uploaded another iteration which addresses some comments from @AlanBateman. Basically, there are some operations on Channel and Socket which take ByteBuffer as arguments, and then, if such buffers are *direct*, they get the address and pass it down to some native function. This idiom is problematic because there's no way to guarantee that the buffer won't be closed (if obtained from a memory segment) after the address has been obtained. As a stop gap solution, I've introduced checks in `DirectBuffer::address` method, which is used in around 30 places in the JDK. This method will now throw if (a) the buffer has a shared scope, or (b) if the scope is confined, but already closed. With this extra check, I believe there's no way to misuse the buffer obtained from a segment. We have discussed plans to remove this limitations (which we think will be possible) - but for the time being, it's better to play the conservative card. I looked through the changes in this update. The shared memory segment support looks sound and the mechanism to close a shared memory segment is clever (albeit a bit surprising at first that it does global handshake to look for a frame in a scoped region. Also surprising that close can cause failure at both ends - it took me a while to see that this is pragmatic approach). The share method specifies NPE if thread == null but there is no thread parameter, is this a cut 'n paste error? Another one in registerCleaner where it should be NPE if the cleaner is null. I think the javadoc for the close method needs to be a bit clearer on the state of the memory segment when IllegalStateException is thrown. Will it be marked "not alive" when it fails? Does this mean there is a resource leak? I think an apiNote to explain the rational for why close is not idempotent is also needed, or maybe it should be re-visited so that close is a no-op when the memory segment is not alive. Now that MemorySegment is AutoCloseable then maybe the term "alive" should be replaced with "open" or "closed" and isAlive replaced with isOpen is isClosed. FileDescriptor can be attraction nuisance and forced reference counting everywhere that it is used. Is it needed? Could an isMapped method work instead? mapFromPath was in the second preview but I think the method name should be re-examined as it maps a file, the path just locates the file. Naming is subjectives but in this case using "map" or "mapFile" would fit beside the allocateNative methods. MappedMemorySegments. The force method specifies a write back guarantee but at the same time, the implNote in the class description suggests that the methods might be a no-op. You might want to adjust the wording to avoid any suggestion that force might be a no-op. The javadoc for copyFrom isn't changed in this update but I notice it specifies IndexOutOfBoundException when the source segment is larger than the receiver, have other exceptions been examined? I don't have any any comments on MemoryAccess except that it's not immediately clear why there are "Byte" methods that take a ByteOrder. Make sense for the multi-byte types of course. The updates the java/nio sources look okay but it would be helpful if the really long lines could be chopped down as it's just too hard to do side-by-side reviews when the lines are so long. A minor nit but the changes X-Buffer.java.template mess up the alignment of the parameters to copyMemory/copySwapMemory methods. ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From vlivanov at openjdk.java.net Sun Nov 1 19:10:58 2020 From: vlivanov at openjdk.java.net (Vladimir Ivanov) Date: Sun, 1 Nov 2020 19:10:58 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` Looks good. ------------- Marked as reviewed by vlivanov (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/960 From iveresov at openjdk.java.net Sun Nov 1 20:17:55 2020 From: iveresov at openjdk.java.net (Igor Veresov) Date: Sun, 1 Nov 2020 20:17:55 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` Marked as reviewed by iveresov (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/960 From kvn at openjdk.java.net Sun Nov 1 21:02:53 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Sun, 1 Nov 2020 21:02:53 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Sun, 1 Nov 2020 20:15:01 GMT, Igor Veresov wrote: >> We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. >> >> We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. >> >> We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. >> >> Tested changes in all tiers. >> >> I verified that with these changes I still able to build Graal in open repo and run graalunit testing: >> >> `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` >> `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` >> `open$ make jdk-image` >> `open$ make test-image` >> `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` > > Marked as reviewed by iveresov (Reviewer). Thank you for reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/960 From prr at openjdk.java.net Mon Nov 2 04:29:10 2020 From: prr at openjdk.java.net (Phil Race) Date: Mon, 2 Nov 2020 04:29:10 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: On Mon, 2 Nov 2020 04:19:57 GMT, Phil Race wrote: > This upgrades JDK to import the current 2.7.2 version of harfbuzz - an OpenType text shaping library > > https://bugs.openjdk.java.net/browse/JDK-8247872 > > This has passed building and headless and headful automated tests on all platforms. Aside from the code change there is a small build change is to remove some obsolete defines ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From prr at openjdk.java.net Mon Nov 2 04:29:10 2020 From: prr at openjdk.java.net (Phil Race) Date: Mon, 2 Nov 2020 04:29:10 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 Message-ID: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> This upgrades JDK to import the current 2.7.2 version of harfbuzz - an OpenType text shaping library https://bugs.openjdk.java.net/browse/JDK-8247872 This has passed building and headless and headful automated tests on all platforms. ------------- Commit messages: - 8247872: Upgrade HarfBuzz to the latest 2.7.2 Changes: https://git.openjdk.java.net/jdk/pull/993/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=993&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8247872 Stats: 44663 lines in 207 files changed: 25198 ins; 12729 del; 6736 mod Patch: https://git.openjdk.java.net/jdk/pull/993.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/993/head:pull/993 PR: https://git.openjdk.java.net/jdk/pull/993 From mcimadamore at openjdk.java.net Mon Nov 2 11:11:56 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 2 Nov 2020 11:11:56 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) In-Reply-To: References: Message-ID: On Sun, 1 Nov 2020 16:06:32 GMT, Alan Bateman wrote: > Now that MemorySegment is AutoCloseable then maybe the term "alive" should be replaced with "open" or "closed" and isAlive replaced with isOpen is isClosed. While the reason for the method being called "isAlive" are mostly historical (the old Panama pointer API had such a method), I think I still stand behind the current naming scheme. For temporal bounds, I think "isAlive" works better than "isOpened". > MappedMemorySegments. The force method specifies a write back guarantee but at the same time, the implNote in the class description suggests that the methods might be a no-op. You might want to adjust the wording to avoid any suggestion that force might be a no-op. The comment that this operation could be no-op was borrowed from the `MappedByteBuffer` API; looking at the impl, it seems that you are right that, under no circumstances (unless the segment has length zero) this should be a no-op. How do you suggest I proceed? ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From mcimadamore at openjdk.java.net Mon Nov 2 11:29:54 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 2 Nov 2020 11:29:54 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) In-Reply-To: References: Message-ID: On Sun, 1 Nov 2020 16:06:32 GMT, Alan Bateman wrote: > The javadoc for copyFrom isn't changed in this update but I notice it specifies IndexOutOfBoundException when the source segment is larger than the receiver, have other exceptions been examined? This exception is consistent with other uses of this exception throughout this API (e.g. when writing a segment out of bounds). ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From stefank at openjdk.java.net Mon Nov 2 11:40:04 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Mon, 2 Nov 2020 11:40:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 08:25:28 GMT, Stefan Karlsson wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > src/hotspot/share/prims/jvmtiTagMap.cpp line 126: > >> 124: // concurrent GCs. So fix it here once we have a lock or are >> 125: // at a safepoint. >> 126: // SetTag and GetTag should not post events! > > I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. Maybe even move this comment to the set_tag/get_tag code. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From stefank at openjdk.java.net Mon Nov 2 11:40:04 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Mon, 2 Nov 2020 11:40:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Commented on nits, and reviewed GC code and tag map code. Didn't look closely on the hashmap changes. src/hotspot/share/gc/shared/oopStorageSet.hpp line 41: > 39: // Must be updated when new OopStorages are introduced > 40: static const uint strong_count = 4 JVMTI_ONLY(+ 1); > 41: static const uint weak_count = 5 JVMTI_ONLY(+1) JFR_ONLY(+ 1); All other uses `+ 1` instead of `+1`. src/hotspot/share/gc/shared/weakProcessorPhaseTimes.hpp line 49: > 47: double _phase_times_sec[1]; > 48: size_t _phase_dead_items[1]; > 49: size_t _phase_total_items[1]; This should be removed and the associated reset_items src/hotspot/share/gc/z/zOopClosures.hpp line 64: > 62: }; > 63: > 64: class ZPhantomKeepAliveOopClosure : public ZRootsIteratorClosure { Seems like you flipped the location of these two. Maybe revert? src/hotspot/share/prims/jvmtiExport.hpp line 405: > 403: > 404: // Delete me and all my callers! > 405: static void weak_oops_do(BoolObjectClosure* b, OopClosure* f) {} Maybe delete? src/hotspot/share/prims/jvmtiTagMap.cpp line 126: > 124: // concurrent GCs. So fix it here once we have a lock or are > 125: // at a safepoint. > 126: // SetTag and GetTag should not post events! I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. src/hotspot/share/prims/jvmtiTagMap.cpp line 131: > 129: // Operating on the hashmap must always be locked, since concurrent GC threads may > 130: // notify while running through a safepoint. > 131: assert(is_locked(), "checking"); Maybe move this to the top of the function to make it very clear. src/hotspot/share/prims/jvmtiTagMap.cpp line 133: > 131: assert(is_locked(), "checking"); > 132: if (post_events && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { > 133: log_info(jvmti, table)("TagMap table needs posting before heap walk"); Not sure about the "before heap walk" since this is also done from GetObjectsWithTags, which does *not* do a heap walk but still requires posting. src/hotspot/share/prims/jvmtiTagMap.cpp line 140: > 138: hashmap()->rehash(); > 139: _needs_rehashing = false; > 140: } It's not clear to me that it's correct to rehash *after* posting. I think it is, because unlink_and_post will use load barriers to fixup old pointers. src/hotspot/share/prims/jvmtiTagMap.cpp line 146: > 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. > 145: void JvmtiTagMap::check_hashmaps_for_heapwalk() { > 146: Extra white space. (Also double whitespace after this function) src/hotspot/share/prims/jvmtiTagMap.cpp line 144: > 142: > 143: // This checks for posting and rehashing and is called from the heap walks. > 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. Should this comment be moved down to the lock taking? src/hotspot/share/prims/jvmtiTagMap.cpp line 377: > 375: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); > 376: > 377: // Check if we have to processing for concurrent GCs. Sentence seems to be missing a few words. src/hotspot/share/prims/jvmtiTagMap.cpp line 954: > 952: o->klass()->external_name()); > 953: return; > 954: } Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch? src/hotspot/share/prims/jvmtiTagMap.cpp line 1152: > 1150: void JvmtiTagMap::unlink_and_post_locked() { > 1151: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); > 1152: log_info(jvmti, table)("TagMap table needs posting before GetObjectTags"); There's no function called GetObjectTags. This log line needs to be adjusted. src/hotspot/share/prims/jvmtiTagMap.cpp line 1162: > 1160: VMOp_Type type() const { return VMOp_Cleanup; } > 1161: void doit() { > 1162: _tag_map->unlink_and_post_locked(); Either inline unlink_and_post_locked() or updated gc_notification to use it? src/hotspot/share/prims/jvmtiTagMap.cpp line 1279: > 1277: // Can't post ObjectFree events here from a JavaThread, so this > 1278: // will race with the gc_notification thread in the tiny > 1279: // window where the oop is not marked but hasn't been notified that Please don't use "oop" when referring to "objects". src/hotspot/share/prims/jvmtiTagMap.cpp line 2975: > 2973: } > 2974: > 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved oops => objects src/hotspot/share/prims/jvmtiTagMap.cpp line 2977: > 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved > 2976: // and have their new addresses, the table can be rehashed. > 2977: void JvmtiTagMap::set_needs_processing() { Maybe rename to set_needs_rehashing? src/hotspot/share/prims/jvmtiTagMap.cpp line 2985: > 2983: > 2984: JvmtiEnvIterator it; > 2985: for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { The iterator seems fast enough, so it seems unnecessary to have the environments_might_exist check. src/hotspot/share/prims/jvmtiTagMap.cpp line 2998: > 2996: // thread creation and before VMThread creation (1 thread); initial GC > 2997: // verification can happen in that window which gets to here. > 2998: if (!JvmtiEnv::environments_might_exist()) { return; } I don't know what this comment is saying, and why the code is needed. src/hotspot/share/prims/jvmtiTagMap.cpp line 3020: > 3018: JvmtiTagMap* tag_map = env->tag_map_acquire(); > 3019: if (tag_map != NULL && !tag_map->is_empty()) { > 3020: if (num_dead_entries > 0) { The other num_dead_entries check for != 0. Maybe use the same in the two branches? src/hotspot/share/prims/jvmtiTagMap.cpp line 3023: > 3021: tag_map->hashmap()->unlink_and_post(tag_map->env()); > 3022: } > 3023: tag_map->_needs_rehashing = true; Maybe add a small comment why this is deferred. src/hotspot/share/prims/jvmtiTagMap.hpp line 56: > 54: void entry_iterate(JvmtiTagMapEntryClosure* closure); > 55: void post_dead_object_on_vm_thread(); > 56: public: Looked nicer when there was a blank line before public. Now it looks like public "relates" more to the code before than after. src/hotspot/share/prims/jvmtiTagMap.hpp line 114: > 112: static void check_hashmaps_for_heapwalk(); > 113: static void set_needs_processing() NOT_JVMTI_RETURN; > 114: static void gc_notification(size_t num_dead_entries) NOT_JVMTI_RETURN; Have you verified that this builds without JVMTI? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 50: > 48: // A subsequent oop_load without AS_NO_KEEPALIVE (the object() accessor) > 49: // keeps the oop alive before doing so. > 50: return literal().peek(); I'm not sure we should be talking about the low-level Access names. Maybe reword in terms of WeakHandle operations? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 81: > 79: void JvmtiTagMapTable::free_entry(JvmtiTagMapEntry* entry) { > 80: unlink_entry(entry); > 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage release *to* OopStorage? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 82: > 80: unlink_entry(entry); > 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage > 82: FREE_C_HEAP_ARRAY(char, entry); // C_Heap free. Seems excessively redundant: // C_Heap free. src/hotspot/share/prims/jvmtiTagMapTable.cpp line 98: > 96: > 97: // The obj is in the table as a target already > 98: if (target != NULL && target == obj) { Wonder if we could assert that obj is not NULL at the entry of this function, and then change this to simply target == obj? src/hotspot/share/prims/jvmtiTagMapTable.cpp line 122: > 120: int index = hash_to_index(hash); > 121: // One was added while acquiring the lock > 122: JvmtiTagMapEntry* entry = find(index, hash, obj); Should this be done inside ASSERT? test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 64: > 62: static jclass klass = NULL; > 63: static jobject testedObject = NULL; > 64: const jlong TESTED_TAG_VALUE = (5555555L); Remove parenthesis? ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From mcimadamore at openjdk.java.net Mon Nov 2 11:59:09 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 2 Nov 2020 11:59:09 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v20] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Address comments from @AlanBateman ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/548/files - new: https://git.openjdk.java.net/jdk/pull/548/files/bd400615..8225bf2e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=19 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=18-19 Stats: 121 lines in 9 files changed: 14 ins; 53 del; 54 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From serb at openjdk.java.net Mon Nov 2 12:15:02 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Mon, 2 Nov 2020 12:15:02 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v20] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 11:59:09 GMT, Maurizio Cimadamore wrote: >> This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: >> >> * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads >> * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually >> * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. >> >> A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. >> >> This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). >> >> A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. >> >> A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. >> >> Thanks >> Maurizio >> >> Javadoc: >> >> http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html >> >> Specdiff: >> >> http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html >> >> CSR: >> >> https://bugs.openjdk.java.net/browse/JDK-8254163 >> >> >> >> ### API Changes >> >> * `MemorySegment` >> * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) >> * added a no-arg factory for a native restricted segment representing entire native heap >> * rename `withOwnerThread` to `handoff` >> * add new `share` method, to create shared segments >> * add new `registerCleaner` method, to register a segment against a cleaner >> * add more helpers to create arrays from a segment e.g. `toIntArray` >> * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) >> * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) >> * `MemoryAddress` >> * drop `segment` accessor >> * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment >> * `MemoryAccess` >> * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). >> * `MemoryHandles` >> * drop `withOffset` combinator >> * drop `withStride` combinator >> * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. >> * `Addressable` >> * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. >> * `MemoryLayouts` >> * A new layout, for machine addresses, has been added to the mix. >> >> >> >> ### Implementation changes >> >> There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. >> >> #### Shared segments >> >> The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. >> >> After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. >> >> Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). >> >> The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. >> >> As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. >> >> In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. >> >> To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). >> >> Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). >> >> `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. >> >> The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. >> >> #### Memory access var handles overhaul >> >> The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. >> >> This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. >> >> This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. >> >> #### Test changes >> >> Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. >> >> [1] - https://openjdk.java.net/jeps/393 >> [2] - https://openjdk.java.net/jeps/389 >> [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html >> [4] - https://openjdk.java.net/jeps/312 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address comments from @AlanBateman test/jdk/java/foreign/TestCleaner.java line 8: > 6: * under the terms of the GNU General Public License version 2 only, as > 7: * published by the Free Software Foundation. Oracle designates this > 8: * particular file as subject to the "Classpath" exception as provided "Classpath exception" could be dropped? ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From ihse at openjdk.java.net Mon Nov 2 12:30:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 12:30:56 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: <4yShPUjSyZSPGnbVh-jr58lGT9psXrg88HZIk5Wa-40=.c2f77dae-d582-4fb4-9f6e-bdcb5ced63b0@github.com> On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Build changes look good. Not reviewed hotspot changes. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From ihse at openjdk.java.net Mon Nov 2 12:31:59 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 12:31:59 GMT Subject: RFR: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` Build changes look good. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/960 From ihse at openjdk.java.net Mon Nov 2 12:35:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 12:35:56 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: On Mon, 2 Nov 2020 04:19:57 GMT, Phil Race wrote: > This upgrades JDK to import the current 2.7.2 version of harfbuzz - an OpenType text shaping library > > https://bugs.openjdk.java.net/browse/JDK-8247872 > > This has passed building and headless and headful automated tests on all platforms. Build changes look good. I'm just a bit curious about the added, empty, `src/java.desktop/share/native/libharfbuzz/abc.txt`... If it really is in upstream source, I'm not saying you should remove it. It just looks very odd. It's not a merge artifact? (I could not even add review comments to an empty file in github! ?) ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/993 From coleenp at openjdk.java.net Mon Nov 2 13:22:15 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:22:15 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 08:08:53 GMT, Stefan Karlsson wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > src/hotspot/share/gc/shared/oopStorageSet.hpp line 41: > >> 39: // Must be updated when new OopStorages are introduced >> 40: static const uint strong_count = 4 JVMTI_ONLY(+ 1); >> 41: static const uint weak_count = 5 JVMTI_ONLY(+1) JFR_ONLY(+ 1); > > All other uses `+ 1` instead of `+1`. Fixed, although I think the space looks strange there but I'll go along. > src/hotspot/share/gc/shared/weakProcessorPhaseTimes.hpp line 49: > >> 47: double _phase_times_sec[1]; >> 48: size_t _phase_dead_items[1]; >> 49: size_t _phase_total_items[1]; > > This should be removed and the associated reset_items Removed. > src/hotspot/share/gc/z/zOopClosures.hpp line 64: > >> 62: }; >> 63: >> 64: class ZPhantomKeepAliveOopClosure : public ZRootsIteratorClosure { > > Seems like you flipped the location of these two. Maybe revert? Reverted. There was a rebasing conflict here so this was unintentional. > src/hotspot/share/prims/jvmtiExport.hpp line 405: > >> 403: >> 404: // Delete me and all my callers! >> 405: static void weak_oops_do(BoolObjectClosure* b, OopClosure* f) {} > > Maybe delete? Yes, meant to do that. > src/hotspot/share/prims/jvmtiTagMap.cpp line 131: > >> 129: // Operating on the hashmap must always be locked, since concurrent GC threads may >> 130: // notify while running through a safepoint. >> 131: assert(is_locked(), "checking"); > > Maybe move this to the top of the function to make it very clear. ok. > src/hotspot/share/prims/jvmtiTagMap.cpp line 133: > >> 131: assert(is_locked(), "checking"); >> 132: if (post_events && env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { >> 133: log_info(jvmti, table)("TagMap table needs posting before heap walk"); > > Not sure about the "before heap walk" since this is also done from GetObjectsWithTags, which does *not* do a heap walk but still requires posting. I don't call check_hashmap for GetObjectsWithTags. > src/hotspot/share/prims/jvmtiTagMap.cpp line 140: > >> 138: hashmap()->rehash(); >> 139: _needs_rehashing = false; >> 140: } > > It's not clear to me that it's correct to rehash *after* posting. I think it is, because unlink_and_post will use load barriers to fixup old pointers. I think it's better that the rehashing doesn't encounter null entries and the WeakHandle.peek() operation is used for both so I hope it would get the same answer. If not, which seems bad, the last answer should be what we hash on. > src/hotspot/share/prims/jvmtiTagMap.cpp line 144: > >> 142: >> 143: // This checks for posting and rehashing and is called from the heap walks. >> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. > > Should this comment be moved down to the lock taking? ok, I also made it singular since Erik pointed out that we don't need the other lock. > src/hotspot/share/prims/jvmtiTagMap.cpp line 146: > >> 144: // The ZDriver may be walking the hashmaps concurrently so all these locks are needed. >> 145: void JvmtiTagMap::check_hashmaps_for_heapwalk() { >> 146: > > Extra white space. (Also double whitespace after this function) ? I removed the double whitespace after the function and put the whitespace here. It needs some whitespace. // This checks for posting and rehashing and is called from the heap walks. void JvmtiTagMap::check_hashmaps_for_heapwalk() { assert(SafepointSynchronize::is_at_safepoint(), "called from safepoints"); // Verify that the tag map tables are valid and unconditionally post events // that are expected to be posted before gc_notification. JvmtiEnvIterator it; > src/hotspot/share/prims/jvmtiTagMap.cpp line 377: > >> 375: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); >> 376: >> 377: // Check if we have to processing for concurrent GCs. > > Sentence seems to be missing a few words. removed the sentence, because non concurrent GCs also defer rehashing to next use. > src/hotspot/share/prims/jvmtiTagMap.cpp line 954: > >> 952: o->klass()->external_name()); >> 953: return; >> 954: } > > Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch? Because it crashed with my changes and didn't without. I cannot recollect why. > src/hotspot/share/prims/jvmtiTagMap.cpp line 1152: > >> 1150: void JvmtiTagMap::unlink_and_post_locked() { >> 1151: MutexLocker ml(lock(), Mutex::_no_safepoint_check_flag); >> 1152: log_info(jvmti, table)("TagMap table needs posting before GetObjectTags"); > > There's no function called GetObjectTags. This log line needs to be adjusted. GetObjectsWithTags fixed. > src/hotspot/share/prims/jvmtiTagMap.cpp line 1162: > >> 1160: VMOp_Type type() const { return VMOp_Cleanup; } >> 1161: void doit() { >> 1162: _tag_map->unlink_and_post_locked(); > > Either inline unlink_and_post_locked() or updated gc_notification to use it? I thought of trying to share it but one logs and the other doesn't and it only saves 1 lines of code. > src/hotspot/share/prims/jvmtiTagMap.cpp line 1279: > >> 1277: // Can't post ObjectFree events here from a JavaThread, so this >> 1278: // will race with the gc_notification thread in the tiny >> 1279: // window where the oop is not marked but hasn't been notified that > > Please don't use "oop" when referring to "objects". fixed. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2975: > >> 2973: } >> 2974: >> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved > > oops => objects fixed. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2977: > >> 2975: // Concurrent GC needs to call this in relocation pause, so after the oops are moved >> 2976: // and have their new addresses, the table can be rehashed. >> 2977: void JvmtiTagMap::set_needs_processing() { > > Maybe rename to set_needs_rehashing? Since I went back and forth about what this function did (it posted events at one time), I thought the generic _processing name would be better. GC callers shouldn't really have to know what processing we're doing here. Hopefully it won't change from rehashing. That's why I like processing. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2985: > >> 2983: >> 2984: JvmtiEnvIterator it; >> 2985: for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > > The iterator seems fast enough, so it seems unnecessary to have the environments_might_exist check. yes, it looks like it does the same thing. I'll remove it. > src/hotspot/share/prims/jvmtiTagMap.cpp line 2998: > >> 2996: // thread creation and before VMThread creation (1 thread); initial GC >> 2997: // verification can happen in that window which gets to here. >> 2998: if (!JvmtiEnv::environments_might_exist()) { return; } > > I don't know what this comment is saying, and why the code is needed. I've spent tons of time trying to understand this comment too. I think gc verification used to call oops do on the tagmap table. This comments obsolete now, and I'll remove it. > src/hotspot/share/prims/jvmtiTagMap.cpp line 3020: > >> 3018: JvmtiTagMap* tag_map = env->tag_map_acquire(); >> 3019: if (tag_map != NULL && !tag_map->is_empty()) { >> 3020: if (num_dead_entries > 0) { > > The other num_dead_entries check for != 0. Maybe use the same in the two branches? ok. > src/hotspot/share/prims/jvmtiTagMap.cpp line 3023: > >> 3021: tag_map->hashmap()->unlink_and_post(tag_map->env()); >> 3022: } >> 3023: tag_map->_needs_rehashing = true; > > Maybe add a small comment why this is deferred. // Later GC code will relocate the oops, so defer rehashing until then. ? > src/hotspot/share/prims/jvmtiTagMap.hpp line 56: > >> 54: void entry_iterate(JvmtiTagMapEntryClosure* closure); >> 55: void post_dead_object_on_vm_thread(); >> 56: public: > > Looked nicer when there was a blank line before public. Now it looks like public "relates" more to the code before than after. ok > src/hotspot/share/prims/jvmtiTagMap.hpp line 114: > >> 112: static void check_hashmaps_for_heapwalk(); >> 113: static void set_needs_processing() NOT_JVMTI_RETURN; >> 114: static void gc_notification(size_t num_dead_entries) NOT_JVMTI_RETURN; > > Have you verified that this builds without JVMTI? I will do (might have already done) that. Building non-oracle platforms builds minimal vm. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 50: > >> 48: // A subsequent oop_load without AS_NO_KEEPALIVE (the object() accessor) >> 49: // keeps the oop alive before doing so. >> 50: return literal().peek(); > > I'm not sure we should be talking about the low-level Access names. Maybe reword in terms of WeakHandle operations? I'm going to say: // Just peek at the object without keeping it alive. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 81: > >> 79: void JvmtiTagMapTable::free_entry(JvmtiTagMapEntry* entry) { >> 80: unlink_entry(entry); >> 81: entry->literal().release(JvmtiExport::weak_tag_storage()); // release OopStorage > > release *to* OopStorage? fixed > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 98: > >> 96: >> 97: // The obj is in the table as a target already >> 98: if (target != NULL && target == obj) { > > Wonder if we could assert that obj is not NULL at the entry of this function, and then change this to simply target == obj? makes sense. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 122: > >> 120: int index = hash_to_index(hash); >> 121: // One was added while acquiring the lock >> 122: JvmtiTagMapEntry* entry = find(index, hash, obj); > > Should this be done inside ASSERT? yes. > test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 64: > >> 62: static jclass klass = NULL; >> 63: static jobject testedObject = NULL; >> 64: const jlong TESTED_TAG_VALUE = (5555555L); > > Remove parenthesis? I copied this from some other place that had parenthesis. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 13:22:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:22:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. I think I addressed your comments, retesting now. Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 13:22:15 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:22:15 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 08:34:17 GMT, Stefan Karlsson wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 126: >> >>> 124: // concurrent GCs. So fix it here once we have a lock or are >>> 125: // at a safepoint. >>> 126: // SetTag and GetTag should not post events! >> >> I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. > > Maybe even move this comment to the set_tag/get_tag code. I was trying to explain why there's a boolean there but I can put this comment at both get_tag and set_tag. // Check if we have to processing for concurrent GCs. // GetTag should not post events because the JavaThread has to // transition to native for the callback and this cannot stop for // safepoints with the hashmap lock held. check_hashmap(false); ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 13:28:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 13:28:59 GMT Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:46:31 GMT, Erik Joelsson wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Build changes look ok. There should be a thank you emoji that doesn't send email except maybe to the person reviewing the code. Thank you @erikj79 and @magicus for reviewing the build changes. There should also be a 'fixed' emoji. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From burban at openjdk.java.net Mon Nov 2 13:45:57 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 2 Nov 2020 13:45:57 GMT Subject: Integrated: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build In-Reply-To: References: Message-ID: On Tue, 6 Oct 2020 18:09:05 GMT, Bernhard Urban-Forster wrote: > I organized this PR so that each commit contains the warning emitted by MSVC as commit message and its relevant fix. > > Verified on > * Linux+ARM64: `{hotspot,jdk,langtools}:tier1`, no failures. > * Windows+ARM64: `{hotspot,jdk,langtools}:tier1`, no (new) failures. > * internal macOS+ARM64 port: build without `--disable-warnings-as-errors` still works. Just mentioning this here, because it's yet another toolchain (Xcode / clang) that needs to be kept happy [going forward](https://openjdk.java.net/jeps/391). This pull request has now been integrated. Changeset: d2812f78 Author: Bernhard Urban-Forster Committer: Andrew Haley URL: https://git.openjdk.java.net/jdk/commit/d2812f78 Stats: 23 lines in 8 files changed: 2 ins; 0 del; 21 mod 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build Reviewed-by: ihse, aph ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From aph at openjdk.java.net Mon Nov 2 13:45:57 2020 From: aph at openjdk.java.net (Andrew Haley) Date: Mon, 2 Nov 2020 13:45:57 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: References: Message-ID: On Tue, 27 Oct 2020 14:04:04 GMT, Andrew Haley wrote: >> Bernhard Urban-Forster has updated the pull request incrementally with two additional commits since the last revision: >> >> - uppercase suffix >> - add assert > > Marked as reviewed by aph (Reviewer). > Would you mind to sponsor it @theRealAph or @magicus? Hmm, I think you have to integrate it first. https://wiki.openjdk.java.net/display/SKARA/Pull+Request+Commands#PullRequestCommands-/sponsor ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From burban at openjdk.java.net Mon Nov 2 14:02:56 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 2 Nov 2020 14:02:56 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 13:41:53 GMT, Andrew Haley wrote: >> Marked as reviewed by aph (Reviewer). > >> Would you mind to sponsor it @theRealAph or @magicus? > > Hmm, I think you have to integrate it first. > https://wiki.openjdk.java.net/display/SKARA/Pull+Request+Commands#PullRequestCommands-/sponsor Thank you Andrew. ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From mcimadamore at openjdk.java.net Mon Nov 2 14:18:14 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 2 Nov 2020 14:18:14 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v21] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Addess remaining feedback from @AlanBateman and @mrserb ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/548/files - new: https://git.openjdk.java.net/jdk/pull/548/files/8225bf2e..e2f69ec0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=20 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=19-20 Stats: 5 lines in 2 files changed: 1 ins; 2 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From ihse at openjdk.java.net Mon Nov 2 14:29:54 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 14:29:54 GMT Subject: RFR: JDK-8255673: Wrong version in docs bundles In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 17:19:56 GMT, Erik Joelsson wrote: > In JDK-8206311, when I introduced a new docs build profile, I forgot to add the version configure arguments to it. This causes the docs bundles for CI/promoted builds to be generated with the default adhoc version information instead of the specific version for that build. LGTM ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/959 From erikj at openjdk.java.net Mon Nov 2 14:48:02 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 2 Nov 2020 14:48:02 GMT Subject: RFR: JDK-8255732: OpenJDK fails to build if $A is set to a value with spaces Message-ID: We have at least one java file with a '$' in the name. As we have learned over the years, having $ in unexpected places quickly leads to unexpected behavior in a shell/make based build. In this case it's our override mechanism of java files that needs protection against expanding such occurrences of $. ------------- Commit messages: - Add DoubleDollar call to protect java class file names with dollar in them Changes: https://git.openjdk.java.net/jdk/pull/1007/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1007&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255732 Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1007.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1007/head:pull/1007 PR: https://git.openjdk.java.net/jdk/pull/1007 From erikj at openjdk.java.net Mon Nov 2 14:51:56 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 2 Nov 2020 14:51:56 GMT Subject: Integrated: JDK-8255673: Wrong version in docs bundles In-Reply-To: References: Message-ID: <5-FIcbFy5CJd5lPcB5hyZuDGsFZTuIfWpAoUufpa5_Y=.592e3f3b-9e13-4314-998e-ec055f90c300@github.com> On Fri, 30 Oct 2020 17:19:56 GMT, Erik Joelsson wrote: > In JDK-8206311, when I introduced a new docs build profile, I forgot to add the version configure arguments to it. This causes the docs bundles for CI/promoted builds to be generated with the default adhoc version information instead of the specific version for that build. This pull request has now been integrated. Changeset: b0280743 Author: Erik Joelsson URL: https://git.openjdk.java.net/jdk/commit/b0280743 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod 8255673: Wrong version in docs bundles Reviewed-by: tbell, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/959 From ihse at openjdk.java.net Mon Nov 2 15:03:02 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 15:03:02 GMT Subject: RFR: JDK-8255732: OpenJDK fails to build if $A is set to a value with spaces In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 14:40:35 GMT, Erik Joelsson wrote: > We have at least one java file with a '$' in the name. As we have learned over the years, having $ in unexpected places quickly leads to unexpected behavior in a shell/make based build. In this case it's our override mechanism of java files that needs protection against expanding such occurrences of $. Looks good to me. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1007 From ihse at openjdk.java.net Mon Nov 2 15:46:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 15:46:57 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 14:00:33 GMT, Bernhard Urban-Forster wrote: >>> Would you mind to sponsor it @theRealAph or @magicus? >> >> Hmm, I think you have to integrate it first. >> https://wiki.openjdk.java.net/display/SKARA/Pull+Request+Commands#PullRequestCommands-/sponsor > > Thank you Andrew. @lewurm This patch seems to break on linux-aarch64 with gcc: open/src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp:1501:52: error: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'int' [-Werror=sign-compare] 1501 | assert(StackOverflow::stack_shadow_zone_size() == (int)StackOverflow::stack_shadow_zone_size(), "must be same"); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Did you test building this on any aarch64 platforms besides Windows? ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From coleenp at openjdk.java.net Mon Nov 2 15:58:15 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 15:58:15 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Code review comments from StefanK. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/534326da..cb4c83e0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=00-01 Stats: 75 lines in 9 files changed: 12 ins; 48 del; 15 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From zgu at openjdk.java.net Mon Nov 2 16:07:02 2020 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Mon, 2 Nov 2020 16:07:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> Message-ID: On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from StefanK. Shenandoah part looks good. ------------- Marked as reviewed by zgu (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From burban at openjdk.java.net Mon Nov 2 16:08:58 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 2 Nov 2020 16:08:58 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: References: Message-ID: <8gg1Viyrq9xvlobYWDR2MTvXYFEBtVHXIfAkrptF240=.8936d808-6aac-43d7-9893-e8c095c6abb9@github.com> On Mon, 2 Nov 2020 15:41:06 GMT, Magnus Ihse Bursie wrote: >> Thank you Andrew. > > @lewurm > This patch seems to break on linux-aarch64 with gcc: > open/src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp:1501:52: error: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'int' [-Werror=sign-compare] > 1501 | assert(StackOverflow::stack_shadow_zone_size() == (int)StackOverflow::stack_shadow_zone_size(), "must be same"); > | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Did you test building this on any aarch64 platforms besides Windows? @magicus I did test the initial version of this PR on linux+arm64, but not the latest iteration. sorry about that ? What is the policy here? Submit a revert right away or investigate a fix? ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From kvn at openjdk.java.net Mon Nov 2 16:09:55 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 2 Nov 2020 16:09:55 GMT Subject: Integrated: 8255616: Disable AOT and Graal in Oracle OpenJDK In-Reply-To: References: Message-ID: <-4ETgOIXCz-lGU-CQQTpYtgUZ5gs6TYaeAZtBRqXmmg=.16b603f0-0af9-425f-aa62-b3ae29e41e56@github.com> On Fri, 30 Oct 2020 17:40:51 GMT, Vladimir Kozlov wrote: > We shipped Ahead-of-Time compilation (the jaotc tool) in JDK 9, as an experimental feature. We shipped Graal as an experimental JIT compiler in JDK 10. We haven't seen much use of these features, and the effort required to support and enhance them is significant. We therefore intend to disable these features in Oracle builds as of JDK 16. > > We'll leave the sources for these features in the repository, in case any one else is interested in building them. But we will not update or test them. > > We'll continue to build and ship JVMCI as an experimental feature in Oracle builds. > > Tested changes in all tiers. > > I verified that with these changes I still able to build Graal in open repo and run graalunit testing: > > `open$ bash test/hotspot/jtreg/compiler/graalunit/downloadLibs.sh /mydir/graalunit_lib/` > `open$ bash configure --with-debug-level=fastdebug --with-graalunit-lib=/mydir/graalunit_lib/ --with-jtreg=/mydir/jtreg` > `open$ make jdk-image` > `open$ make test-image` > `open$ make run-test TEST=compiler/graalunit/HotspotTest.java` This pull request has now been integrated. Changeset: 2f7d34f2 Author: Vladimir Kozlov URL: https://git.openjdk.java.net/jdk/commit/2f7d34f2 Stats: 36 lines in 4 files changed: 21 ins; 11 del; 4 mod 8255616: Disable AOT and Graal in Oracle OpenJDK Reviewed-by: iignatyev, vlivanov, iveresov, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/960 From eosterlund at openjdk.java.net Mon Nov 2 16:14:06 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 16:14:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> Message-ID: <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from StefanK. Looks great in general. Great work Coleen, and thanks again for fixing this. I like all the red lines in the GC code. I added a few nits/questions. test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 656: > 654: result = NSK_FALSE; > 655: > 656: printf("Object free events %d\n", ObjectFreeEventsCount); Is this old debug info you forgot to remove? Other code seems to use NSK_DISPLAY macros instead. src/hotspot/share/prims/jvmtiTagMap.cpp line 345: > 343: > 344: // Check if we have to process for concurrent GCs. > 345: check_hashmap(false); Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap. src/hotspot/share/prims/jvmtiTagMap.cpp line 3009: > 3007: // Lock each hashmap from concurrent posting and cleaning > 3008: MutexLocker ml(tag_map->lock(), Mutex::_no_safepoint_check_flag); > 3009: tag_map->hashmap()->unlink_and_post(tag_map->env()); This could call unlink_and_post_locked instead of manually locking. ------------- Changes requested by eosterlund (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Mon Nov 2 16:14:06 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Mon, 2 Nov 2020 16:14:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 12:50:23 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 954: >> >>> 952: o->klass()->external_name()); >>> 953: return; >>> 954: } >> >> Why is this done as a part of this RFE? Is this a bug fix that should be done as a separate patch? > > Because it crashed with my changes and didn't without. I cannot recollect why. I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From ihse at openjdk.java.net Mon Nov 2 16:19:07 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 2 Nov 2020 16:19:07 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: <8gg1Viyrq9xvlobYWDR2MTvXYFEBtVHXIfAkrptF240=.8936d808-6aac-43d7-9893-e8c095c6abb9@github.com> References: <8gg1Viyrq9xvlobYWDR2MTvXYFEBtVHXIfAkrptF240=.8936d808-6aac-43d7-9893-e8c095c6abb9@github.com> Message-ID: <-mnJRsB2bDYTyIboORS0S7t2rVJC_YolZrqmr2lqloM=.da15438d-463a-4462-8292-f6595b4bcb06@github.com> On Mon, 2 Nov 2020 16:06:15 GMT, Bernhard Urban-Forster wrote: >> @lewurm >> This patch seems to break on linux-aarch64 with gcc: >> open/src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp:1501:52: error: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'int' [-Werror=sign-compare] >> 1501 | assert(StackOverflow::stack_shadow_zone_size() == (int)StackOverflow::stack_shadow_zone_size(), "must be same"); >> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> Did you test building this on any aarch64 platforms besides Windows? > > @magicus I did test the initial version of this PR on linux+arm64, but not the latest iteration. sorry about that ? > > What is the policy here? Submit a revert right away or investigate a fix? @lewurm Open a new JBS issue with the bug. If you can find a fix in a short amount of time (which I would believe should be possible; probably just need a proper cast) it's acceptable to fix it directly. What amounts to a "short amount of time" is left to reasonable judgement; minutes to hours are okay, days are not. Otherwise, create an anti-delta (revert changeset) to back out your changes, and open yet another JBS issue for re-implementing them correctly. In this case, an alternative short-term fix could also be to remove the assert instead of backing out the entire patch. ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From coleenp at openjdk.java.net Mon Nov 2 16:27:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 16:27:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> Message-ID: On Mon, 2 Nov 2020 15:18:43 GMT, Erik ?sterlund wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from StefanK. > > test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/capability/CM02/cm02t001/cm02t001.cpp line 656: > >> 654: result = NSK_FALSE; >> 655: >> 656: printf("Object free events %d\n", ObjectFreeEventsCount); > > Is this old debug info you forgot to remove? Other code seems to use NSK_DISPLAY macros instead. yes, removed it. > src/hotspot/share/prims/jvmtiTagMap.cpp line 345: > >> 343: >> 344: // Check if we have to process for concurrent GCs. >> 345: check_hashmap(false); > > Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap. Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before? > src/hotspot/share/prims/jvmtiTagMap.cpp line 3009: > >> 3007: // Lock each hashmap from concurrent posting and cleaning >> 3008: MutexLocker ml(tag_map->lock(), Mutex::_no_safepoint_check_flag); >> 3009: tag_map->hashmap()->unlink_and_post(tag_map->env()); > > This could call unlink_and_post_locked instead of manually locking. Ok, 2 requests. I can call that then and move the logging. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From prr at openjdk.java.net Mon Nov 2 16:28:22 2020 From: prr at openjdk.java.net (Phil Race) Date: Mon, 2 Nov 2020 16:28:22 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v2] In-Reply-To: References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: <2um9af6839x1BDDn3P10xB6WcxXVc_DGAjCDueyfDcI=.5feeee81-b628-465a-8123-3750ea4db3d2@github.com> On Mon, 2 Nov 2020 12:33:17 GMT, Magnus Ihse Bursie wrote: > I'm just a bit curious about the added, empty, `src/java.desktop/share/native/libharfbuzz/abc.txt`... > > If it really is in upstream source, I'm not saying you should remove it. It just looks very odd. It's not a merge artifact? > > (I could not even add review comments to an empty file in github! ?) oops. that should not have been in that folder. It was me practicing a different white space removal script. I thought I checked for any extraneous files. I'll get rid of it now. ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From prr at openjdk.java.net Mon Nov 2 16:28:22 2020 From: prr at openjdk.java.net (Phil Race) Date: Mon, 2 Nov 2020 16:28:22 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v2] In-Reply-To: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: > This upgrades JDK to import the current 2.7.2 version of harfbuzz - an OpenType text shaping library > > https://bugs.openjdk.java.net/browse/JDK-8247872 > > This has passed building and headless and headful automated tests on all platforms. Phil Race has updated the pull request incrementally with one additional commit since the last revision: 8247872: Upgrade HarfBuzz to the latest 2.7.2 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/993/files - new: https://git.openjdk.java.net/jdk/pull/993/files/8d1ea467..1f47957b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=993&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=993&range=00-01 Stats: 0 lines in 1 file changed: 0 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/993.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/993/head:pull/993 PR: https://git.openjdk.java.net/jdk/pull/993 From kravikumar at openjdk.java.net Mon Nov 2 16:38:00 2020 From: kravikumar at openjdk.java.net (Kiran Sidhartha Ravikumar) Date: Mon, 2 Nov 2020 16:38:00 GMT Subject: RFR: 8255226: (tz) Upgrade time-zone data to tzdata2020d Message-ID: Hi Guys, Please review the integration of tzdata2020d to JDK. Details regarding the change can be viewed at - https://mm.icann.org/pipermail/tz-announce/2020-October/000062.html Bug: https://bugs.openjdk.java.net/browse/JDK-8255226 TestZoneInfo310.java test failure is addressed along with it. The last rule affects "Asia/Gaza" and "Asia/Hebron" and therefore excluded from the test. Regression Tests pass along with JCK. Please let me know if the changes are good to push. Thanks, Kiran ------------- Commit messages: - 8255226: (tz) Upgrade time-zone data to tzdata2020d Changes: https://git.openjdk.java.net/jdk/pull/1012/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1012&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255226 Stats: 54 lines in 4 files changed: 34 ins; 2 del; 18 mod Patch: https://git.openjdk.java.net/jdk/pull/1012.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1012/head:pull/1012 PR: https://git.openjdk.java.net/jdk/pull/1012 From coleenp at openjdk.java.net Mon Nov 2 16:57:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 16:57:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 16:52:02 GMT, Coleen Phillimore wrote: >> I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference. > > Now I remember. I added an assert in JvmtiTagMapTable::find() for oop != NULL which didn't exist in the current hashmap code. The current hashmap code just didn't find a null oop. I tracked it down to the fact that we're finding dormant objects whose class hasn't been loaded yet. So I think we do load the archived heap from CDS. The heap walker capabilities can be added dynamically. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 2 16:57:13 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 2 Nov 2020 16:57:13 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 15:45:18 GMT, Erik ?sterlund wrote: >> Because it crashed with my changes and didn't without. I cannot recollect why. > > I thought that we didn't load the archived heap from CDS, if JVMTI heap walker capabilities are in place, as we didn't want this kind of interactions. But maybe I'm missing something, since you said having this if statement here made a difference. Now I remember. I added an assert in JvmtiTagMapTable::find() for oop != NULL which didn't exist in the current hashmap code. The current hashmap code just didn't find a null oop. I tracked it down to the fact that we're finding dormant objects whose class hasn't been loaded yet. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From burban at openjdk.java.net Mon Nov 2 17:07:54 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 2 Nov 2020 17:07:54 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: <-mnJRsB2bDYTyIboORS0S7t2rVJC_YolZrqmr2lqloM=.da15438d-463a-4462-8292-f6595b4bcb06@github.com> References: <8gg1Viyrq9xvlobYWDR2MTvXYFEBtVHXIfAkrptF240=.8936d808-6aac-43d7-9893-e8c095c6abb9@github.com> <-mnJRsB2bDYTyIboORS0S7t2rVJC_YolZrqmr2lqloM=.da15438d-463a-4462-8292-f6595b4bcb06@github.com> Message-ID: On Mon, 2 Nov 2020 16:16:25 GMT, Magnus Ihse Bursie wrote: >> @magicus I did test the initial version of this PR on linux+arm64, but not the latest iteration. sorry about that ? >> >> What is the policy here? Submit a revert right away or investigate a fix? > > @lewurm Open a new JBS issue with the bug. If you can find a fix in a short amount of time (which I would believe should be possible; probably just need a proper cast) it's acceptable to fix it directly. What amounts to a "short amount of time" is left to reasonable judgement; minutes to hours are okay, days are not. > > Otherwise, create an anti-delta (revert changeset) to back out your changes, and open yet another JBS issue for re-implementing them correctly. > > In this case, an alternative short-term fix could also be to remove the assert instead of backing out the entire patch. https://github.com/openjdk/jdk/pull/1013 ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From naoto at openjdk.java.net Mon Nov 2 17:18:02 2020 From: naoto at openjdk.java.net (Naoto Sato) Date: Mon, 2 Nov 2020 17:18:02 GMT Subject: RFR: 8255226: (tz) Upgrade time-zone data to tzdata2020d In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 16:29:07 GMT, Kiran Sidhartha Ravikumar wrote: > Hi Guys, > > Please review the integration of tzdata2020d to JDK. > > Details regarding the change can be viewed at - https://mm.icann.org/pipermail/tz-announce/2020-October/000062.html > Bug: https://bugs.openjdk.java.net/browse/JDK-8255226 > > TestZoneInfo310.java test failure is addressed along with it. The last rule affects "Asia/Gaza" and "Asia/Hebron" and therefore excluded from the test. > > Regression Tests pass along with JCK. > > Please let me know if the changes are good to push. > > Thanks, > Kiran The test case needs copyright year change to 2020. test/jdk/sun/util/calendar/zi/TestZoneInfo310.java line 201: > 199: zid.equals("Iran") || // last rule mismatch > 200: zid.equals("Asia/Gaza") || // last rule mismatch > 201: zid.equals("Asia/Hebron")) { // last rule mismatch Can you explain why these zones are failing? The criteria for excluding failing tests here is that the zone has negative dst and last rules beyond 2037, and I don't think those newly excluded zones suffice those. ------------- Changes requested by naoto (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1012 From aph at openjdk.java.net Mon Nov 2 17:45:57 2020 From: aph at openjdk.java.net (Andrew Haley) Date: Mon, 2 Nov 2020 17:45:57 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: References: <8gg1Viyrq9xvlobYWDR2MTvXYFEBtVHXIfAkrptF240=.8936d808-6aac-43d7-9893-e8c095c6abb9@github.com> <-mnJRsB2bDYTyIboORS0S7t2rVJC_YolZrqmr2lqloM=.da15438d-463a-4462-8292-f6595b4bcb06@github.com> Message-ID: On Mon, 2 Nov 2020 17:05:19 GMT, Bernhard Urban-Forster wrote: >> @lewurm Open a new JBS issue with the bug. If you can find a fix in a short amount of time (which I would believe should be possible; probably just need a proper cast) it's acceptable to fix it directly. What amounts to a "short amount of time" is left to reasonable judgement; minutes to hours are okay, days are not. >> >> Otherwise, create an anti-delta (revert changeset) to back out your changes, and open yet another JBS issue for re-implementing them correctly. >> >> In this case, an alternative short-term fix could also be to remove the assert instead of backing out the entire patch. > > https://github.com/openjdk/jdk/pull/1013 > @lewurm > This patch seems to break on linux-aarch64 with gcc: Builds cleanly on Linux/GCC or me. ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From jjg at openjdk.java.net Mon Nov 2 17:52:56 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Mon, 2 Nov 2020 17:52:56 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v4] In-Reply-To: References: <281-fNSKFfd-qngLAcLkMFdmKe-XHD-0qKCEh4OGvog=.82b534a0-e57a-41bf-ba94-e487601b2773@github.com> Message-ID: <0nw1ywk_bFqAUlU9DuC_79QXVgx63FEsf3Jwt_Y187M=.1164173f-b6d5-4f20-bc66-f722ebb0be08@github.com> On Thu, 29 Oct 2020 09:43:56 GMT, Jan Lahoda wrote: >> I don't think there should be much interaction with -source . We don't support preview features from previous version or preview class files from previous versions, so I think it should be enough to handle the currently present preview features. > > We don't support preview features from previous releases, AFAIK. So javadoc for JDK 16 should not accept e.g. record class when running with -source 15. Yeah, my recollection is that I was wondering whether preview-related code needs to be "guarded" to only work in the current release. But, I guess we may get the right effect (of forbidding preview features in older code) from the javac front end, so that in javadoc we can be assured that there are no instances of what may still be preview features in older code (i.e with older --source/--rlease options) ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From jjg at openjdk.java.net Mon Nov 2 17:52:56 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Mon, 2 Nov 2020 17:52:56 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v4] In-Reply-To: References: Message-ID: On Thu, 29 Oct 2020 09:26:05 GMT, Jan Lahoda wrote: >> src/jdk.javadoc/share/classes/jdk/javadoc/internal/tool/ElementsTable.java line 1288: >> >>> 1286: case FIELD: case INSTANCE_INIT: case LOCAL_VARIABLE: case PARAMETER: >>> 1287: case RESOURCE_VARIABLE: case STATIC_INIT: case TYPE_PARAMETER: >>> 1288: case RECORD_COMPONENT: >> >> I'm not saying this is wrong, but I'd like to understand why it is necessary. > > HtmlDocletWriter.getPreviewNotes analyzes classes and their members, to see if some are using aspects that are under preview. When looking at the members, it uses utils.isIncluded on the member, and that eventually gets here. And if the member is a RECORD_COMPONENT, it would fail here. But we can avoid asking for RECORD_COMPONENTS, if needed. ok ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From kravikumar at openjdk.java.net Mon Nov 2 18:09:03 2020 From: kravikumar at openjdk.java.net (Kiran Sidhartha Ravikumar) Date: Mon, 2 Nov 2020 18:09:03 GMT Subject: RFR: 8255226: (tz) Upgrade time-zone data to tzdata2020d In-Reply-To: References: Message-ID: <_-CfMNeat8oHCidIqA-s6I6ZQPcHAa-z8NRg1h9mm4c=.c4846ff2-a653-4a50-b66a-8fd455e2d729@github.com> On Mon, 2 Nov 2020 17:10:34 GMT, Naoto Sato wrote: >> Hi Guys, >> >> Please review the integration of tzdata2020d to JDK. >> >> Details regarding the change can be viewed at - https://mm.icann.org/pipermail/tz-announce/2020-October/000062.html >> Bug: https://bugs.openjdk.java.net/browse/JDK-8255226 >> >> TestZoneInfo310.java test failure is addressed along with it. The last rule affects "Asia/Gaza" and "Asia/Hebron" and therefore excluded from the test. >> >> Regression Tests pass along with JCK. >> >> Please let me know if the changes are good to push. >> >> Thanks, >> Kiran > > test/jdk/sun/util/calendar/zi/TestZoneInfo310.java line 201: > >> 199: zid.equals("Iran") || // last rule mismatch >> 200: zid.equals("Asia/Gaza") || // last rule mismatch >> 201: zid.equals("Asia/Hebron")) { // last rule mismatch > > Can you explain why these zones are failing? The criteria for excluding failing tests here is that the zone has negative dst and last rules beyond 2037, and I don't think those newly excluded zones suffice those. It's probably these last rule what is causing the issue Rule Palestine 2020 max - Mar Sat>=24 0:00 1:00 S Rule Palestine 2020 max - Oct Sat>=24 1:00 0 - The failure seen is ****************************** Asia/Gaza : Asia/Gaza ------------- SimpleTimeZone (NG) stz=java.util.SimpleTimeZone[id=Asia/Gaza,offset=7200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=7,startTime=0,startTimeMode=0,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=7,endTime=3600000,endTimeMode=0] stz0=java.util.SimpleTimeZone[id=Asia/Gaza,offset=7200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=24,startDayOfWeek=7,startTime=0,startTimeMode=0,endMode=3,endMonth=9,endDay=24,endDayOfWeek=7,endTime=3600000,endTimeMode=0] ------------- PR: https://git.openjdk.java.net/jdk/pull/1012 From jlahoda at openjdk.java.net Mon Nov 2 18:15:09 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Mon, 2 Nov 2020 18:15:09 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: References: Message-ID: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> > This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. > > The notable changes are: > > * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: > * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. > * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings > * improving error/warning messages. Please see [1] for a list of cases/examples. > * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. > * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). > * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. > * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. > > Please also see the CSR [6] for more information. > > [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html > [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html > [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html > [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html > [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ > [6] https://bugs.openjdk.java.net/browse/JDK-8250769 Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: - Removing trailing whitespace. - Merging master into JDK-8250768. - Updating tests after records are a final feature. - Fixing tests. - Finalizing removal of record preview hooks. - Merging master into JDK-8250768 - Reflecting review comments. - Merge branch 'master' into JDK-8250768 - Removing unnecessary cast. - Using a more correct way to get URLs. - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 ------------- Changes: https://git.openjdk.java.net/jdk/pull/703/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=05 Stats: 3012 lines in 142 files changed: 2521 ins; 260 del; 231 mod Patch: https://git.openjdk.java.net/jdk/pull/703.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/703/head:pull/703 PR: https://git.openjdk.java.net/jdk/pull/703 From naoto at openjdk.java.net Mon Nov 2 18:18:00 2020 From: naoto at openjdk.java.net (Naoto Sato) Date: Mon, 2 Nov 2020 18:18:00 GMT Subject: RFR: 8255226: (tz) Upgrade time-zone data to tzdata2020d In-Reply-To: <_-CfMNeat8oHCidIqA-s6I6ZQPcHAa-z8NRg1h9mm4c=.c4846ff2-a653-4a50-b66a-8fd455e2d729@github.com> References: <_-CfMNeat8oHCidIqA-s6I6ZQPcHAa-z8NRg1h9mm4c=.c4846ff2-a653-4a50-b66a-8fd455e2d729@github.com> Message-ID: <3BuHmUaG3MrBs7zQdpb8uiAxEQGUGO8YhdADRKBGaqw=.7276ab51-8823-4f20-be80-6c003212f317@github.com> On Mon, 2 Nov 2020 18:06:28 GMT, Kiran Sidhartha Ravikumar wrote: >> test/jdk/sun/util/calendar/zi/TestZoneInfo310.java line 201: >> >>> 199: zid.equals("Iran") || // last rule mismatch >>> 200: zid.equals("Asia/Gaza") || // last rule mismatch >>> 201: zid.equals("Asia/Hebron")) { // last rule mismatch >> >> Can you explain why these zones are failing? The criteria for excluding failing tests here is that the zone has negative dst and last rules beyond 2037, and I don't think those newly excluded zones suffice those. > > It's probably these last rule what is causing the issue > > Rule Palestine 2020 max - Mar Sat>=24 0:00 1:00 S > Rule Palestine 2020 max - Oct Sat>=24 1:00 0 - > > The failure seen is > > ****************************** > Asia/Gaza : Asia/Gaza > ------------- > SimpleTimeZone (NG) > stz=java.util.SimpleTimeZone[id=Asia/Gaza,offset=7200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=7,startTime=0,startTimeMode=0,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=7,endTime=3600000,endTimeMode=0] > stz0=java.util.SimpleTimeZone[id=Asia/Gaza,offset=7200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=24,startDayOfWeek=7,startTime=0,startTimeMode=0,endMode=3,endMonth=9,endDay=24,endDayOfWeek=7,endTime=3600000,endTimeMode=0] My question is why it is failing. Have you figured it? The existing exceptions are either negative DST or last rule beyond 2037, which javazic cannot handle. The changes introduced with 2020d does not meet either of them. Unless we know why it is failing, we cannot be sure we can exclude it. ------------- PR: https://git.openjdk.java.net/jdk/pull/1012 From burban at openjdk.java.net Mon Nov 2 18:37:56 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 2 Nov 2020 18:37:56 GMT Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: References: <8gg1Viyrq9xvlobYWDR2MTvXYFEBtVHXIfAkrptF240=.8936d808-6aac-43d7-9893-e8c095c6abb9@github.com> <-mnJRsB2bDYTyIboORS0S7t2rVJC_YolZrqmr2lqloM=.da15438d-463a-4462-8292-f6595b4bcb06@github.com> Message-ID: On Mon, 2 Nov 2020 17:43:31 GMT, Andrew Haley wrote: >> https://github.com/openjdk/jdk/pull/1013 > >> @lewurm >> This patch seems to break on linux-aarch64 with gcc: > > Builds cleanly on Linux/GCC or me. @theRealAph what gcc version? I can reproduce with $ gcc --version gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008 which ships in Ubuntu 19.10 as default ------------- PR: https://git.openjdk.java.net/jdk/pull/530 From never at openjdk.java.net Mon Nov 2 19:25:03 2020 From: never at openjdk.java.net (Tom Rodriguez) Date: Mon, 2 Nov 2020 19:25:03 GMT Subject: RFR: 8254827: JVMCI: Enable it for Windows+AArch64 [v3] In-Reply-To: References: Message-ID: <-eax17fyDoxXQFT0QSzGlFaJC1z8nLtkn_EbJKApNPQ=.93cceeb4-2046-4761-81a7-87eb89e78660@github.com> On Tue, 20 Oct 2020 15:46:36 GMT, Bernhard Urban-Forster wrote: >> Use r18 as allocatable register on Linux only. >> >> A bootstrap works now (it has been crashing before due to r18 being allocated): >> $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version >> Bootstrapping JVMCI................................. in 17990 ms >> (compiled 3330 methods) >> openjdk version "16-internal" 2021-03-16 >> OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) >> OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) >> >> Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. > > Bernhard Urban-Forster has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - add missing precompiled.hpp include > - Merge remote-tracking branch 'upstream/master' into 8254827-enable-jvmci-win-aarch64 > - rename argument to canUsePlatformRegister > - comment for platformRegister > - 8254827: JVMCI: Enable it for Windows+AArch64 > > Use r18 as allocatable register on Linux only. > > A bootstrap works now (it has been crashing before due to r18 being allocated): > ```console > $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version > Bootstrapping JVMCI................................. in 17990 ms > (compiled 3330 methods) > openjdk version "16-internal" 2021-03-16 > OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) > OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) > ``` > > Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. Marked as reviewed by never (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/685 From kvn at openjdk.java.net Mon Nov 2 19:36:05 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 2 Nov 2020 19:36:05 GMT Subject: RFR: 8254827: JVMCI: Enable it for Windows+AArch64 [v3] In-Reply-To: References: Message-ID: On Tue, 20 Oct 2020 15:46:36 GMT, Bernhard Urban-Forster wrote: >> Use r18 as allocatable register on Linux only. >> >> A bootstrap works now (it has been crashing before due to r18 being allocated): >> $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version >> Bootstrapping JVMCI................................. in 17990 ms >> (compiled 3330 methods) >> openjdk version "16-internal" 2021-03-16 >> OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) >> OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) >> >> Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. > > Bernhard Urban-Forster has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - add missing precompiled.hpp include > - Merge remote-tracking branch 'upstream/master' into 8254827-enable-jvmci-win-aarch64 > - rename argument to canUsePlatformRegister > - comment for platformRegister > - 8254827: JVMCI: Enable it for Windows+AArch64 > > Use r18 as allocatable register on Linux only. > > A bootstrap works now (it has been crashing before due to r18 being allocated): > ```console > $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version > Bootstrapping JVMCI................................. in 17990 ms > (compiled 3330 methods) > openjdk version "16-internal" 2021-03-16 > OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) > OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) > ``` > > Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. Changes requested by kvn (Reviewer). make/autoconf/jvm-features.m4 line 309: > 307: if test "x$OPENJDK_TARGET_CPU" = "xx86_64"; then > 308: AC_MSG_RESULT([yes]) > 309: elif test "x$OPENJDK_TARGET_CPU" = "xaarch64"; then You are missing the same change for JVM_FEATURES_CHECK_JVMCI. Unless it is done intentionally. ------------- PR: https://git.openjdk.java.net/jdk/pull/685 From burban at openjdk.java.net Mon Nov 2 20:08:03 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 2 Nov 2020 20:08:03 GMT Subject: RFR: 8254827: JVMCI: Enable it for Windows+AArch64 [v3] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 19:33:39 GMT, Vladimir Kozlov wrote: >> Bernhard Urban-Forster has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: >> >> - add missing precompiled.hpp include >> - Merge remote-tracking branch 'upstream/master' into 8254827-enable-jvmci-win-aarch64 >> - rename argument to canUsePlatformRegister >> - comment for platformRegister >> - 8254827: JVMCI: Enable it for Windows+AArch64 >> >> Use r18 as allocatable register on Linux only. >> >> A bootstrap works now (it has been crashing before due to r18 being allocated): >> ```console >> $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version >> Bootstrapping JVMCI................................. in 17990 ms >> (compiled 3330 methods) >> openjdk version "16-internal" 2021-03-16 >> OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) >> OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) >> ``` >> >> Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. > > make/autoconf/jvm-features.m4 line 309: > >> 307: if test "x$OPENJDK_TARGET_CPU" = "xx86_64"; then >> 308: AC_MSG_RESULT([yes]) >> 309: elif test "x$OPENJDK_TARGET_CPU" = "xaarch64"; then > > You are missing the same change for JVM_FEATURES_CHECK_JVMCI. > Unless it is done intentionally. It's done a couple lines below: https://github.com/openjdk/jdk/pull/685/files#diff-a09b08bcd422d0a8fb32a95ccf85051ac1e69bef2bd420d579f74d8efa286d2fL343 Or do you mean something else? ------------- PR: https://git.openjdk.java.net/jdk/pull/685 From kvn at openjdk.java.net Mon Nov 2 20:25:01 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 2 Nov 2020 20:25:01 GMT Subject: RFR: 8254827: JVMCI: Enable it for Windows+AArch64 [v3] In-Reply-To: References: Message-ID: On Tue, 20 Oct 2020 15:46:36 GMT, Bernhard Urban-Forster wrote: >> Use r18 as allocatable register on Linux only. >> >> A bootstrap works now (it has been crashing before due to r18 being allocated): >> $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version >> Bootstrapping JVMCI................................. in 17990 ms >> (compiled 3330 methods) >> openjdk version "16-internal" 2021-03-16 >> OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) >> OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) >> >> Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. > > Bernhard Urban-Forster has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - add missing precompiled.hpp include > - Merge remote-tracking branch 'upstream/master' into 8254827-enable-jvmci-win-aarch64 > - rename argument to canUsePlatformRegister > - comment for platformRegister > - 8254827: JVMCI: Enable it for Windows+AArch64 > > Use r18 as allocatable register on Linux only. > > A bootstrap works now (it has been crashing before due to r18 being allocated): > ```console > $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version > Bootstrapping JVMCI................................. in 17990 ms > (compiled 3330 methods) > openjdk version "16-internal" 2021-03-16 > OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) > OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) > ``` > > Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. Marked as reviewed by kvn (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/685 From kvn at openjdk.java.net Mon Nov 2 20:25:02 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 2 Nov 2020 20:25:02 GMT Subject: RFR: 8254827: JVMCI: Enable it for Windows+AArch64 [v3] In-Reply-To: References: Message-ID: <-w7BvxPLjNvak1PLDvan5QXC8435O2Pq4kGf-5X8LTU=.34aae93f-69a0-456b-a389-ebc1e1dfe218@github.com> On Mon, 2 Nov 2020 20:05:10 GMT, Bernhard Urban-Forster wrote: >> make/autoconf/jvm-features.m4 line 309: >> >>> 307: if test "x$OPENJDK_TARGET_CPU" = "xx86_64"; then >>> 308: AC_MSG_RESULT([yes]) >>> 309: elif test "x$OPENJDK_TARGET_CPU" = "xaarch64"; then >> >> You are missing the same change for JVM_FEATURES_CHECK_JVMCI. >> Unless it is done intentionally. > > It's done a couple lines below: https://github.com/openjdk/jdk/pull/685/files#diff-a09b08bcd422d0a8fb32a95ccf85051ac1e69bef2bd420d579f74d8efa286d2fL343 > > Or do you mean something else? Uhh. Sorry, I thought JVMCI definition will be first, before Graal and did not look below. ------------- PR: https://git.openjdk.java.net/jdk/pull/685 From jjg at openjdk.java.net Mon Nov 2 20:25:10 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Mon, 2 Nov 2020 20:25:10 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> Message-ID: <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> On Mon, 2 Nov 2020 18:15:09 GMT, Jan Lahoda wrote: >> This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. >> >> The notable changes are: >> >> * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: >> * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. >> * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings >> * improving error/warning messages. Please see [1] for a list of cases/examples. >> * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. >> * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). >> * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. >> * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. >> >> Please also see the CSR [6] for more information. >> >> [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html >> [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html >> [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html >> [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html >> [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ >> [6] https://bugs.openjdk.java.net/browse/JDK-8250769 > > Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: > > - Removing trailing whitespace. > - Merging master into JDK-8250768. > - Updating tests after records are a final feature. > - Fixing tests. > - Finalizing removal of record preview hooks. > - Merging master into JDK-8250768 > - Reflecting review comments. > - Merge branch 'master' into JDK-8250768 > - Removing unnecessary cast. > - Using a more correct way to get URLs. > - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 I have read all the files. I have added a n umber of various minor non-blocking comments (no need for re-review( to fix these. But I have a couple of comments/questions before finally giving approval. There's a comment in `PreviewListWriter` about annotation members that needs too be addressed, and I wonder is RECORD and RECORD_COMPONENT need to be added into PreviewElementKind. src/java.base/share/classes/jdk/internal/javac/PreviewFeature.java line 75: > 73: * A key for testing. > 74: */ > 75: TEST, Slightly weird src/jdk.compiler/share/classes/com/sun/tools/javac/api/JavacTaskPool.java line 257: > 255: //when patching modules (esp. java.base), it may be impossible to > 256: //clear the symbols read from the patch path: > 257: polluted |= get(JavaFileManager.class).hasLocation(StandardLocation.PATCH_MODULE_PATH); OK, but looks unrelated to primary work src/jdk.compiler/share/classes/com/sun/tools/javac/code/Preview.java line 218: > 216: return Errors.PreviewFeatureDisabledClassfile(classfile, majorVersionToSource.get(majorVersion).name); > 217: } > 218: Up above in isPreview, lines 185-188, I'm supervised it's not a `switch` statement. (Can't annotate those lines directly) src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractExecutableMemberWriter.java line 89: > 87: @Override > 88: protected Content getDeprecatedOrPreviewLink(Element member) { > 89: Content content = new ContentBuilder(); Yeah the shorter name is good here and more in keeping with the code style src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractExecutableMemberWriter.java line 93: > 91: if (!utils.isConstructor(member)) { > 92: content.add("."); > 93: content.add(member.getSimpleName()); this is OK, but generally FYI, `Content` is now set up to allow chained method calls. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ClassUseWriter.java line 436: > 434: configuration, LinkInfoImpl.Kind.CLASS_USE_HEADER, typeElement) > 435: .label(resources.getText("doclet.Class")) > 436: .skipPreview(true)); @hns General comment, not directly related to this review: is it worth a cleanup to fold the sequence `getLink(new LinkInfoImpl(...))` into a hypothetical new `LinkBuilder` ? src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ClassWriterImpl.java line 221: > 219: if (modifiers.endsWith(" ")) { > 220: pre.add(" "); > 221: } Obligatory ugh that `modifiers` is a string and that it might end with a space. This is a possibility for future cleanup. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ClassWriterImpl.java line 283: > 281: if (isFirst) { > 282: pre.add(DocletConstants.NL); > 283: pre.add("permits"); Note for javadoc-dev folk: we should have a better more consistent way of handling keywords src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HelpWriter.java line 254: > 252: contentTree.add(section); > 253: } > 254: ? src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlConfiguration.java line 145: > 143: // which performs a somewhat similar role > 144: public enum ConditionalPage { > 145: CONSTANT_VALUES, DEPRECATED, PREVIEW, SERIALIZED_FORM, SYSTEM_PROPERTIES ? src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/LinkFactoryImpl.java line 120: > 118: title, > 119: classLinkInfo.target)); > 120: if (flags.contains(ElementFlag.PREVIEW)) { I see a lot of `new HtmlTree(TagName.SUP).add(...)` ... enough to warrant a new factory method ``HtmlTree.SUP(Content)` or similar. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModuleIndexWriter.java line 39: > 37: import jdk.javadoc.internal.doclets.formats.html.markup.HtmlStyle; > 38: import jdk.javadoc.internal.doclets.formats.html.markup.HtmlTree; > 39: import jdk.javadoc.internal.doclets.formats.html.markup.RawHtml; Are these imports still required? src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PreviewListWriter.java line 40: > 38: > 39: import com.sun.source.doctree.DocTree; > 40: import javax.lang.model.element.ElementKind; unordered imports src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PreviewListWriter.java line 223: > 221: case CONSTRUCTOR -> new ConstructorWriterImpl(this); > 222: case ENUM_CONSTANT -> new EnumConstantWriterImpl(this); > 223: case ANNOTATION_TYPE_MEMBER -> new AnnotationTypeOptionalMemberWriterImpl(this, null); Hmmm. Not sure about this one. For better or worse, javadoc current handles optional and required members differently (i.e. with different classes). That's arguably something that should be cleaned up at some point, but in the meantime, I'm surprised to not see the distinction here. I guess if we're not likely to see an interaction between preview-ness and annotation-members, this will not likely show up in practice. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PreviewListWriter.java line 67: > 65: public class PreviewListWriter extends SubWriterHolderWriter { > 66: > 67: private String getAnchorName(PreviewElementKind kind) { 1. I'm mildly surprised to not see anything record-related here. 2. I'm mildly surprised you haven't used the new switch-expression syntax. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/PreviewAPIListBuilder.java line 67: > 65: ENUM_CONSTANT, > 66: ANNOTATION_TYPE_MEMBER // no ElementKind mapping > 67: }; Should there be RECORD and RECORD_COMPONENT in this list? src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlTree.java line 932: > 930: switch (tagName) { > 931: case A: case BUTTON: case BR: case CODE: case EM: case I: case IMG: > 932: case LABEL: case SMALL: case SPAN: case STRONG: case SUB: case SUP: See comment elsewhere about possibly adding a factory method for `SUP` trees. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/standard.properties line 290: > 288: doclet.Declared_Using_Preview.SEALED_PERMITS=Sealed Classes > 289: doclet.PreviewPlatformLeadingNote={0} is a preview API of the Java platform. > 290: doclet.ReflectivePreviewPlatformLeadingNote={0} is a reflective preview API of the Java platform. Question: is the following "inconsistency" deliberate: 1. Java platform 2. Java language 3. Java SE (in javac diagnostics) src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/FieldWriter.java line 75: > 73: > 74: /** > 75: * Add the preview output for the given member. (minor) Should be "Adds" src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/MethodWriter.java line 75: > 73: > 74: /** > 75: * Add the preview output for the given member. (minor) should be "Adds" src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/builders/AnnotationTypeRequiredMemberBuilder.java line 156: > 154: buildSignature(annotationDocTree); > 155: buildDeprecationInfo(annotationDocTree); > 156: buildPreviewInfo(annotationDocTree); (Just checking) I presume this behavior is inherited into `AnnotationTypeOptionalMemberBuilder` so no changes required there. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/builders/EnumConstantBuilder.java line 166: > 164: writer.addPreview(currentElement, enumConstantsTree); > 165: } > 166: Note to self: I regret that this duplication of adding this method everywhere seems to be necessary. This would be good to clean up at some point. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets.properties line 213: > 211: doclet.Preview=Preview. > 212: doclet.Properties=Properties > 213: doclet.constructors=constructors Is the period after `Preview` intentional? It seems inconsistent. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Comparators.java line 131: > 129: /** > 130: * Returns a Comparator for deprecated items listed on deprecated list page, by comparing the > 131: * fully qualified names, and if those are equal the names of the enclosing modules. Comment does not match method name src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Utils.java line 132: > 130: import static com.sun.source.doctree.DocTree.Kind.*; > 131: import javax.lang.model.AnnotatedConstruct; > 132: import javax.lang.model.util.SimpleAnnotationValueVisitor14; unordered imports test/langtools/jdk/javadoc/doclet/testPreview/TestPreview.java line 75: > 73: """; > 74: String expected = MessageFormat.format(expectedTemplate, zero, one, two, three); > 75: checkOutput("m/pkg/TestPreviewDeclaration.html", true, expected); Interesting technique... Future: it might be interesting to consider folding the bundle access on line 59 into a utility call in JavadocTester test/langtools/jdk/javadoc/doclet/testPreview/api/preview/Core.java line 28: > 26: import jdk.internal.javac.PreviewFeature.Feature; > 27: > 28: @PreviewFeature(feature=Feature.TEST) Yeah, I remember `Feature.TEST` from earlier. I guess it's OK for now, as a workaround for a testing a feature which is inherently, by design, a moving target across releases. These days, javadoc tests are trending towards generating small sample test programs, instead of having small static side-files dominated by a legal header. I wonder if there is a possibility of having a "generator class" in the `javadoc.tester` package that can generate sample code using one or more of the current set of preview features, as a way of reducing the need for the TEST feature. test/langtools/tools/javac/patterns/BindingsTest2.out line 54: > 52: - compiler.note.preview.filename: BindingsTest2.java, DEFAULT > 53: - compiler.note.preview.recompile > 54: 51 errors (issue unrelated to review) it seems wrong that we encourage/allow the use of files that do not end in newline. test/langtools/tools/javac/preview/PreviewAutoSuppress.java line 25: > 23: > 24: /* > 25: * @test no `@bug`? test/langtools/tools/javac/preview/PreviewErrors.java line 38: > 36: * @build toolbox.ToolBox toolbox.JavacTask > 37: * @build combo.ComboTestHelper > 38: * @compile PreviewErrors.java do you need `@compile` here: is `@build` not good enough? ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From jjg at openjdk.java.net Mon Nov 2 20:25:12 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Mon, 2 Nov 2020 20:25:12 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <5xE8fziHH9n04bSHhODnOB9On3t_zGz-JY0gJhVhP74=.0c79430f-fac0-44f9-82be-40ef2f94a0d9@github.com> References: <5xE8fziHH9n04bSHhODnOB9On3t_zGz-JY0gJhVhP74=.0c79430f-fac0-44f9-82be-40ef2f94a0d9@github.com> Message-ID: <8FF_CARupWr7LcKZm5Th7wpIZ-QhNVOM_KytRUOPppE=.bf38fc0c-5c70-4bff-aa50-fe23c772fe2b@github.com> On Fri, 16 Oct 2020 16:07:41 GMT, Maurizio Cimadamore wrote: >> Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: >> >> - Removing trailing whitespace. >> - Merging master into JDK-8250768. >> - Updating tests after records are a final feature. >> - Fixing tests. >> - Finalizing removal of record preview hooks. >> - Merging master into JDK-8250768 >> - Reflecting review comments. >> - Merge branch 'master' into JDK-8250768 >> - Removing unnecessary cast. >> - Using a more correct way to get URLs. >> - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 > > src/jdk.compiler/share/classes/com/sun/tools/javac/util/RawDiagnosticFormatter.java line 166: > >> 164: s = "compiler.misc.tree.tag." + StringUtils.toLowerCase(((Tag) arg).name()); >> 165: } else if (arg instanceof Source && arg == Source.DEFAULT && >> 166: CODES_NEEDING_SOURCE_NORMALIZATION.contains(diag.getCode())) { > > Nice trick to keep raw output constant across versions :-) ? ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From jjg at openjdk.java.net Mon Nov 2 20:25:12 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Mon, 2 Nov 2020 20:25:12 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v4] In-Reply-To: <281-fNSKFfd-qngLAcLkMFdmKe-XHD-0qKCEh4OGvog=.82b534a0-e57a-41bf-ba94-e487601b2773@github.com> References: <281-fNSKFfd-qngLAcLkMFdmKe-XHD-0qKCEh4OGvog=.82b534a0-e57a-41bf-ba94-e487601b2773@github.com> Message-ID: On Tue, 27 Oct 2020 16:09:45 GMT, Jan Lahoda wrote: >> src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractExecutableMemberWriter.java line 88: >> >>> 86: >>> 87: @Override >>> 88: protected Content getDeprecatedOrPreviewLink(Element member) { >> >> This name is a side-effect of the `ElementFlag` question. We should either use explicit field and method names, or we should use `ElementFlag` more consistently. This method name works OK for now, but if if ever gets to have another `orFoo` component in the name, the signature should be parameterized with something like `ElementFlag` or its equivalent. > > Note this method returns the same link for deprecate or preview - it just was named "getDeprecatedLink", and when using it work preview, I renamed it "getDeprecatedOrPreviewLink". We may need to think of a better name. This name is OK for now. Maybe we'll have m ore insight into a better name if/when we add a third variant ;-) ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From burban at openjdk.java.net Mon Nov 2 20:37:56 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 2 Nov 2020 20:37:56 GMT Subject: RFR: 8254827: JVMCI: Enable it for Windows+AArch64 [v3] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 20:22:21 GMT, Vladimir Kozlov wrote: >> Bernhard Urban-Forster has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: >> >> - add missing precompiled.hpp include >> - Merge remote-tracking branch 'upstream/master' into 8254827-enable-jvmci-win-aarch64 >> - rename argument to canUsePlatformRegister >> - comment for platformRegister >> - 8254827: JVMCI: Enable it for Windows+AArch64 >> >> Use r18 as allocatable register on Linux only. >> >> A bootstrap works now (it has been crashing before due to r18 being allocated): >> ```console >> $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version >> Bootstrapping JVMCI................................. in 17990 ms >> (compiled 3330 methods) >> openjdk version "16-internal" 2021-03-16 >> OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) >> OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) >> ``` >> >> Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. > > Marked as reviewed by kvn (Reviewer). Thanks for the review Tom and Magnus, and for the comments to Vladimir and Doug ? ------------- PR: https://git.openjdk.java.net/jdk/pull/685 From erikj at openjdk.java.net Mon Nov 2 21:11:57 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 2 Nov 2020 21:11:57 GMT Subject: Integrated: JDK-8255732: OpenJDK fails to build if $A is set to a value with spaces In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 14:40:35 GMT, Erik Joelsson wrote: > We have at least one java file with a '$' in the name. As we have learned over the years, having $ in unexpected places quickly leads to unexpected behavior in a shell/make based build. In this case it's our override mechanism of java files that needs protection against expanding such occurrences of $. This pull request has now been integrated. Changeset: 184db64d Author: Erik Joelsson URL: https://git.openjdk.java.net/jdk/commit/184db64d Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod 8255732: OpenJDK fails to build if $A is set to a value with spaces Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1007 From kravikumar at openjdk.java.net Tue Nov 3 00:02:57 2020 From: kravikumar at openjdk.java.net (Kiran Sidhartha Ravikumar) Date: Tue, 3 Nov 2020 00:02:57 GMT Subject: RFR: 8255226: (tz) Upgrade time-zone data to tzdata2020d In-Reply-To: <3BuHmUaG3MrBs7zQdpb8uiAxEQGUGO8YhdADRKBGaqw=.7276ab51-8823-4f20-be80-6c003212f317@github.com> References: <_-CfMNeat8oHCidIqA-s6I6ZQPcHAa-z8NRg1h9mm4c=.c4846ff2-a653-4a50-b66a-8fd455e2d729@github.com> <3BuHmUaG3MrBs7zQdpb8uiAxEQGUGO8YhdADRKBGaqw=.7276ab51-8823-4f20-be80-6c003212f317@github.com> Message-ID: On Mon, 2 Nov 2020 18:14:47 GMT, Naoto Sato wrote: >> It's probably these last rule what is causing the issue >> >> Rule Palestine 2020 max - Mar Sat>=24 0:00 1:00 S >> Rule Palestine 2020 max - Oct Sat>=24 1:00 0 - >> >> The failure seen is >> >> ****************************** >> Asia/Gaza : Asia/Gaza >> ------------- >> SimpleTimeZone (NG) >> stz=java.util.SimpleTimeZone[id=Asia/Gaza,offset=7200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=7,startTime=0,startTimeMode=0,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=7,endTime=3600000,endTimeMode=0] >> stz0=java.util.SimpleTimeZone[id=Asia/Gaza,offset=7200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=24,startDayOfWeek=7,startTime=0,startTimeMode=0,endMode=3,endMonth=9,endDay=24,endDayOfWeek=7,endTime=3600000,endTimeMode=0] > > My question is why it is failing. Have you figured it? The existing exceptions are either negative DST or last rule beyond 2037, which javazic cannot handle. The changes introduced with 2020d does not meet either of them. Unless we know why it is failing, we cannot be sure we can exclude it. Thanks for getting back Naoto, I believe this is a long-standing issue - https://bugs.openjdk.java.net/browse/JDK-8227293. Looking back at the integration of tzdata2019a/tzdata2019b, the same issue was addressed by updating the source code - https://hg.openjdk.java.net/jdk/jdk/rev/79036e5e744b#l13.1. Here is some history to the issue - https://mail.openjdk.java.net/pipermail/i18n-dev/2019-July/002887.html Please let me know your thoughts ------------- PR: https://git.openjdk.java.net/jdk/pull/1012 From naoto at openjdk.java.net Tue Nov 3 00:24:00 2020 From: naoto at openjdk.java.net (Naoto Sato) Date: Tue, 3 Nov 2020 00:24:00 GMT Subject: RFR: 8255226: (tz) Upgrade time-zone data to tzdata2020d In-Reply-To: References: <_-CfMNeat8oHCidIqA-s6I6ZQPcHAa-z8NRg1h9mm4c=.c4846ff2-a653-4a50-b66a-8fd455e2d729@github.com> <3BuHmUaG3MrBs7zQdpb8uiAxEQGUGO8YhdADRKBGaqw=.7276ab51-8823-4f20-be80-6c003212f317@github.com> Message-ID: On Tue, 3 Nov 2020 00:00:26 GMT, Kiran Sidhartha Ravikumar wrote: >> My question is why it is failing. Have you figured it? The existing exceptions are either negative DST or last rule beyond 2037, which javazic cannot handle. The changes introduced with 2020d does not meet either of them. Unless we know why it is failing, we cannot be sure we can exclude it. > > Thanks for getting back Naoto, I believe this is a long-standing issue - https://bugs.openjdk.java.net/browse/JDK-8227293. > > Looking back at the integration of tzdata2019a/tzdata2019b, the same issue was addressed by updating the source code - https://hg.openjdk.java.net/jdk/jdk/rev/79036e5e744b#l13.1. > > Here is some history to the issue - https://mail.openjdk.java.net/pipermail/i18n-dev/2019-July/002887.html > > Please let me know your thoughts Should we then remove the hack in the ZoneInfoFile.java that excludes Gaza/Hebron instead? ------------- PR: https://git.openjdk.java.net/jdk/pull/1012 From ihse at openjdk.java.net Tue Nov 3 00:41:03 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 00:41:03 GMT Subject: RFR: 8255785: X11 libraries should not be required by configure for headless only Message-ID: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> If we build a headless only JDK, configure will require X11 libraries and headers to be present. This used to be necessary, but thanks to massive cleanups in the AWT headless code, this is no longer the case. We should fix configure so that headless can be built without X11 libraries and headers. ------------- Commit messages: - 8255785: X11 libraries should not be required by configure for headless only Changes: https://git.openjdk.java.net/jdk/pull/1025/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1025&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255785 Stats: 5 lines in 1 file changed: 3 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1025.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1025/head:pull/1025 PR: https://git.openjdk.java.net/jdk/pull/1025 From mikael at openjdk.java.net Tue Nov 3 00:47:54 2020 From: mikael at openjdk.java.net (Mikael Vidstedt) Date: Tue, 3 Nov 2020 00:47:54 GMT Subject: RFR: 8255785: X11 libraries should not be required by configure for headless only In-Reply-To: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> References: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> Message-ID: On Tue, 3 Nov 2020 00:33:09 GMT, Magnus Ihse Bursie wrote: > If we build a headless only JDK, configure will require X11 libraries and headers to be present. This used to be necessary, but thanks to massive cleanups in the AWT headless code, this is no longer the case. > > We should fix configure so that headless can be built without X11 libraries and headers. Marked as reviewed by mikael (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1025 From prr at openjdk.java.net Tue Nov 3 01:10:54 2020 From: prr at openjdk.java.net (Phil Race) Date: Tue, 3 Nov 2020 01:10:54 GMT Subject: RFR: 8255785: X11 libraries should not be required by configure for headless only In-Reply-To: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> References: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> Message-ID: On Tue, 3 Nov 2020 00:33:09 GMT, Magnus Ihse Bursie wrote: > If we build a headless only JDK, configure will require X11 libraries and headers to be present. This used to be necessary, but thanks to massive cleanups in the AWT headless code, this is no longer the case. > > We should fix configure so that headless can be built without X11 libraries and headers. Marked as reviewed by prr (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1025 From ihse at openjdk.java.net Tue Nov 3 01:20:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 01:20:57 GMT Subject: Integrated: 8255785: X11 libraries should not be required by configure for headless only In-Reply-To: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> References: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> Message-ID: On Tue, 3 Nov 2020 00:33:09 GMT, Magnus Ihse Bursie wrote: > If we build a headless only JDK, configure will require X11 libraries and headers to be present. This used to be necessary, but thanks to massive cleanups in the AWT headless code, this is no longer the case. > > We should fix configure so that headless can be built without X11 libraries and headers. This pull request has now been integrated. Changeset: f97ec359 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/f97ec359 Stats: 5 lines in 1 file changed: 3 ins; 1 del; 1 mod 8255785: X11 libraries should not be required by configure for headless only Reviewed-by: mikael, prr ------------- PR: https://git.openjdk.java.net/jdk/pull/1025 From david.holmes at oracle.com Tue Nov 3 01:37:05 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Nov 2020 11:37:05 +1000 Subject: =?UTF-8?Q?Re=3a_BUG=ef=bc=9aj2re-image_lost_two_dll_files=2c_j2sdk-?= =?UTF-8?Q?image/jre_have_them?= In-Reply-To: References: Message-ID: <7adbc640-68b9-1392-06e2-4a1eb85f3886@oracle.com> Hi, On 29/10/2020 1:21 pm, ??? wrote: > > Hello! > I have built OpenJDK8-272 on windows 10. I found there is a bug: > Compre files between j2re-image and j2sdk-image, j2re-image lost two files: > sawindbg.dll > attach.dll A JRE and a JDK have different contents - this is one of those differences. A JRE is just for running Java applications and doesn't have the development tools that are part of the JDK. Those two libraries are development tools. Cheers, David > I build severial times, it always ocurred. > > Thanks. > > Quantum6, Tai Shan IT. > From serb at openjdk.java.net Tue Nov 3 04:27:54 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Tue, 3 Nov 2020 04:27:54 GMT Subject: RFR: 8255785: X11 libraries should not be required by configure for headless only In-Reply-To: References: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> Message-ID: <4hq9M2uhojAm-MoLY7s9BlONTbwCJJm_nfmpNYo2H_s=.c6b9bc4e-a8d0-4e5b-bc0c-e1d0066b75da@github.com> On Tue, 3 Nov 2020 01:07:51 GMT, Phil Race wrote: >> If we build a headless only JDK, configure will require X11 libraries and headers to be present. This used to be necessary, but thanks to massive cleanups in the AWT headless code, this is no longer the case. >> >> We should fix configure so that headless can be built without X11 libraries and headers. > > Marked as reviewed by prr (Reviewer). Do we have a tier 1-build task or something that will check that we did not break building headless w/o x11? ------------- PR: https://git.openjdk.java.net/jdk/pull/1025 From prr at openjdk.java.net Tue Nov 3 05:02:57 2020 From: prr at openjdk.java.net (Phil Race) Date: Tue, 3 Nov 2020 05:02:57 GMT Subject: RFR: 8255785: X11 libraries should not be required by configure for headless only In-Reply-To: <4hq9M2uhojAm-MoLY7s9BlONTbwCJJm_nfmpNYo2H_s=.c6b9bc4e-a8d0-4e5b-bc0c-e1d0066b75da@github.com> References: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> <4hq9M2uhojAm-MoLY7s9BlONTbwCJJm_nfmpNYo2H_s=.c6b9bc4e-a8d0-4e5b-bc0c-e1d0066b75da@github.com> Message-ID: On Tue, 3 Nov 2020 04:25:37 GMT, Sergey Bylokhov wrote: >> Marked as reviewed by prr (Reviewer). > > Do we have a tier 1-build task or something that will check that we did not break building headless w/o x11? I think you are confusing the modest goal of making it possible, with ensure .. Would you want a headless build to actually fail if the build environment had X11 libraries ? If your real question is "will anyone bite my head off if I break this by accident" ? No. ------------- PR: https://git.openjdk.java.net/jdk/pull/1025 From magnus.ihse.bursie at oracle.com Tue Nov 3 07:51:10 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 08:51:10 +0100 Subject: RFR: 8255785: X11 libraries should not be required by configure for headless only In-Reply-To: <4hq9M2uhojAm-MoLY7s9BlONTbwCJJm_nfmpNYo2H_s=.c6b9bc4e-a8d0-4e5b-bc0c-e1d0066b75da@github.com> References: <8rnCZ5QGpnxuHCuEA7VvzLbFNOgqLfSzL7gViqBqEWE=.4fd5556b-7a1b-4a0c-8080-6daeaeff6baa@github.com> <4hq9M2uhojAm-MoLY7s9BlONTbwCJJm_nfmpNYo2H_s=.c6b9bc4e-a8d0-4e5b-bc0c-e1d0066b75da@github.com> Message-ID: On 2020-11-03 05:27, Sergey Bylokhov wrote: > On Tue, 3 Nov 2020 01:07:51 GMT, Phil Race wrote: > >>> If we build a headless only JDK, configure will require X11 libraries and headers to be present. This used to be necessary, but thanks to massive cleanups in the AWT headless code, this is no longer the case. >>> >>> We should fix configure so that headless can be built without X11 libraries and headers. >> Marked as reviewed by prr (Reviewer). > Do we have a tier 1-build task or something that will check that we did not break building headless w/o x11? There is no regular testing of headless-only. However, the only test needed for this fix was a single confirmation that the headless libraries indeed compile even without X_CFLAGS and X_LIBS. (This was apparent from the code since the make rules did not use X_CFLAGS and X_LIBS, but just to be 100% sure, I also tried building it.) /Magnus From ihse at openjdk.java.net Tue Nov 3 08:29:01 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 08:29:01 GMT Subject: RFR: 8255798: Remove dead headless code in CompileJavaModules.gmk Message-ID: <5Gi8FSVPAw7OmK6bq4YjYDWyCGr23IgedicITWsO0CE=.bfa58da7-69c2-472c-9a5f-1b6fa9dd2d36@github.com> The variable BUILD_HEADLESS_ONLY is no longer set. And sun/applet does not exist anymore. ------------- Commit messages: - 8255798: Remove dead headless code in CompileJavaModules.gmk Changes: https://git.openjdk.java.net/jdk/pull/1031/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1031&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255798 Stats: 4 lines in 1 file changed: 0 ins; 4 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1031.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1031/head:pull/1031 PR: https://git.openjdk.java.net/jdk/pull/1031 From shade at openjdk.java.net Tue Nov 3 09:18:52 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 3 Nov 2020 09:18:52 GMT Subject: RFR: 8255798: Remove dead headless code in CompileJavaModules.gmk In-Reply-To: <5Gi8FSVPAw7OmK6bq4YjYDWyCGr23IgedicITWsO0CE=.bfa58da7-69c2-472c-9a5f-1b6fa9dd2d36@github.com> References: <5Gi8FSVPAw7OmK6bq4YjYDWyCGr23IgedicITWsO0CE=.bfa58da7-69c2-472c-9a5f-1b6fa9dd2d36@github.com> Message-ID: <2PuJi3AuTvUKfC8pHGD7-wx0D4H34VknAKmoTfC9ipA=.7ff085eb-f25d-4b62-bb27-aff05d389eda@github.com> On Tue, 3 Nov 2020 08:21:22 GMT, Magnus Ihse Bursie wrote: > The variable BUILD_HEADLESS_ONLY is no longer set. And sun/applet does not exist anymore. Looks fine and trivial. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1031 From aph at redhat.com Tue Nov 3 10:05:40 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 3 Nov 2020 10:05:40 +0000 Subject: RFR: 8254072: AArch64: Get rid of --disable-warnings-as-errors on Windows+ARM64 build [v4] In-Reply-To: References: <8gg1Viyrq9xvlobYWDR2MTvXYFEBtVHXIfAkrptF240=.8936d808-6aac-43d7-9893-e8c095c6abb9@github.com> <-mnJRsB2bDYTyIboORS0S7t2rVJC_YolZrqmr2lqloM=.da15438d-463a-4462-8292-f6595b4bcb06@github.com> Message-ID: <34b4b08e-9114-5121-9b6f-542d4714a634@redhat.com> On 02/11/2020 18:37, Bernhard Urban-Forster wrote: > @theRealAph what gcc version? > > I can reproduce with > $ gcc --version > gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008 > which ships in Ubuntu 19.10 as default My mistake: I think it only triggers with a debug build, because assert is a macro. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From eosterlund at openjdk.java.net Tue Nov 3 10:25:00 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 3 Nov 2020 10:25:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> Message-ID: On Mon, 2 Nov 2020 16:22:57 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 345: >> >>> 343: >>> 344: // Check if we have to process for concurrent GCs. >>> 345: check_hashmap(false); >> >> Maybe add a comment stating the parameter name, as was done in other callsites for check_hashmap. > > Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before? ZGC people passionately place the comment after the argument value. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From stefank at openjdk.java.net Tue Nov 3 10:36:57 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Tue, 3 Nov 2020 10:36:57 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 12:58:49 GMT, Coleen Phillimore wrote: > GC callers shouldn't really have to know what processing we're doing here. I completely disagree with this. It's extremely important that the GC and Runtime code agrees on what this code does and where the GC *must* call it. Knowing the details allows us to skip calling this after mark end, but forces us to call it in relocate start, when objects should start to move. Though, I don't want to block this review because of this point, so if you still thinks that a non-descriptive name is better then we can argue that separately. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From stefank at openjdk.java.net Tue Nov 3 10:44:05 2020 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Tue, 3 Nov 2020 10:44:05 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> Message-ID: <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com> On Mon, 2 Nov 2020 15:58:15 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from StefanK. Some more nit-picking to make the code more consistent. src/hotspot/share/prims/jvmtiTagMapTable.cpp line 52: > 50: : Hashtable(_table_size, sizeof(JvmtiTagMapEntry)) {} > 51: > 52: Double whitespace src/hotspot/share/prims/jvmtiTagMapTable.cpp line 185: > 183: // Serially remove unused oops from the table, and notify jvmti. > 184: void JvmtiTagMapTable::unlink_and_post(JvmtiEnv* env) { > 185: Stray newline src/hotspot/share/prims/jvmtiTagMapTable.cpp line 224: > 222: // Rehash oops in the table > 223: void JvmtiTagMapTable::rehash() { > 224: Stray newline src/hotspot/share/prims/jvmtiTagMapTable.hpp line 75: > 73: > 74: void resize_if_needed(); > 75: public: Newline between src/hotspot/share/prims/jvmtiTagMapTable.hpp line 100: > 98: }; > 99: > 100: Double newline src/hotspot/share/prims/jvmtiTagMapTable.cpp line 258: > 256: int rehash_len = moved_entries.length(); > 257: // Now add back in the entries that were removed. > 258: for (int i = 0; i < moved_entries.length(); i++) { rehash_len is read, but not used in for loop condition. src/hotspot/share/prims/jvmtiTagMapTable.cpp line 165: > 163: } > 164: } > 165: const int _resize_load_trigger = 5; // load factor that will trigger the resize Newline between ------------- Marked as reviewed by stefank (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From ihse at openjdk.java.net Tue Nov 3 11:43:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 11:43:56 GMT Subject: RFR: 8251549: Update docs on building for Git In-Reply-To: References: Message-ID: On Mon, 26 Oct 2020 12:50:59 GMT, Erik Joelsson wrote: >> @jddarcy Recommended new wording: >> Suggestion: >> >> * Clone the JDK repository using the Cygwin command line `git` client >> as instructed in this document. >> >> Using the Cygwin `git` client is the recommended way since it >> guarantees that there will be no line ending issues and it works well >> with other Cygwin tools. It is however also possible to use [Git for >> Windows](https://gitforwindows.org). If you chose this path, make sure >> to set `core.autocrlf` to `false` (this is asked during installation). > > I would recommend putting in a link to the instructions on the Skara wiki page. They actually recommend Git for Windows and I have switched too. The main reason is to be able to use the token manager. To just clone and build, using Cygwin Git is definitely easier though. @erikj79 Ok, here is a completely rewritten portion, which highlights the differences between Cygwin git and Git for Windows. Suggestion: * You need to install a git client. You have two choices, Cygwin git or Git for Windows. Unfortunately there are pros and cons with each choice. * The Cygwin `git` client has no line ending issues and understands Cygwin paths (which are used throughout the JDK build system). However, it does not currently work well with the Skara CLI tooling. Please see the [Skara wiki on Git clients]( https://wiki.openjdk.java.net/display/SKARA/Skara#Skara-Git) for up-to-date information about the Skara git client support. * The [Git for Windows](https://gitforwindows.org) client has issues with line endings, and do not understand Cygwin paths. It does work well with the Skara CLI tooling, however. To alleviate the line ending problems, make sure you set `core.autocrlf` to `false` (this is asked during installation). @jddarcy Do you find this acceptable? This PR has been open for almost a month, and it's starting getting embarrassing that our build instruction does not match the new Github reality. :( ------------- PR: https://git.openjdk.java.net/jdk/pull/21 From coleenp at openjdk.java.net Tue Nov 3 12:22:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:22:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0heSKANZ4kJqlQOKY6MCs6cSZvT8-KRFRbbbKTlzNzA=.7224a164-a76d-47ce-8016-9db052b09773@github.com> Message-ID: On Tue, 3 Nov 2020 10:22:09 GMT, Erik ?sterlund wrote: >> Ok, will I run afoul of the ZGC people putting the parameter name after the parameter and the rest of the code, it is before? > > ZGC people passionately place the comment after the argument value. I see that but in the non-zgc code it's the opposite and this is non-zgc code. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 12:36:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:36:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com> References: <3rZagetDi1APoH1Gg2q4Z5mVPYjk0vBHwDnnhuh7d6M=.da07fe0e-91d6-43cb-b7c7-f3f9daedb931@github.com> <0DUaLOt_27wPkI2SwP4BwykioL4Hn2c-j7hMz3AbHYI=.2270cb60-bd2f-4d72-abc8-cd8ea44d30b5@github.com> Message-ID: On Tue, 3 Nov 2020 10:36:21 GMT, Stefan Karlsson wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from StefanK. > > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 185: > >> 183: // Serially remove unused oops from the table, and notify jvmti. >> 184: void JvmtiTagMapTable::unlink_and_post(JvmtiEnv* env) { >> 185: > > Stray newline that's a useful newline. > src/hotspot/share/prims/jvmtiTagMapTable.cpp line 258: > >> 256: int rehash_len = moved_entries.length(); >> 257: // Now add back in the entries that were removed. >> 258: for (int i = 0; i < moved_entries.length(); i++) { > > rehash_len is read, but not used in for loop condition. It's in the logging message below. I'll use it in the loop too. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 12:40:06 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:40:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v2] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 10:33:16 GMT, Stefan Karlsson wrote: >> Since I went back and forth about what this function did (it posted events at one time), I thought the generic _processing name would be better. GC callers shouldn't really have to know what processing we're doing here. Hopefully it won't change from rehashing. That's why I like processing. > >> GC callers shouldn't really have to know what processing we're doing here. > > I completely disagree with this. It's extremely important that the GC and Runtime code agrees on what this code does and where the GC *must* call it. Knowing the details allows us to skip calling this after mark end, but forces us to call it in relocate start, when objects should start to move. Though, I don't want to block this review because of this point, so if you still thinks that a non-descriptive name is better then we can argue that separately. Ok, I'll rename it to needs_hashing. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 12:58:22 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 12:58:22 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: Message-ID: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: More review comments from Stefan and ErikO ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/cb4c83e0..eeaf9aed Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=01-02 Stats: 18 lines in 7 files changed: 3 ins; 6 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Tue Nov 3 13:12:02 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 3 Nov 2020 13:12:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: <4rQSjnt6O05gF9AFgc-nOxXIPehn_TKL0jJj85fvbr4=.c80aafac-a743-47d0-9d1a-31caa35d54ec@github.com> On Tue, 3 Nov 2020 12:58:22 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > More review comments from Stefan and ErikO Marked as reviewed by eosterlund (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From github.com+1593194+croydon at openjdk.java.net Tue Nov 3 13:29:54 2020 From: github.com+1593194+croydon at openjdk.java.net (Michael Keck) Date: Tue, 3 Nov 2020 13:29:54 GMT Subject: RFR: 8251549: Update docs on building for Git In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 11:41:20 GMT, Magnus Ihse Bursie wrote: >> I would recommend putting in a link to the instructions on the Skara wiki page. They actually recommend Git for Windows and I have switched too. The main reason is to be able to use the token manager. To just clone and build, using Cygwin Git is definitely easier though. > > @erikj79 > > Ok, here is a completely rewritten portion, which highlights the differences between Cygwin git and Git for Windows. > > Suggestion: > > * You need to install a git client. You have two choices, Cygwin git or > Git for Windows. Unfortunately there are pros and cons with each choice. > > * The Cygwin `git` client has no line ending issues and understands > Cygwin paths (which are used throughout the JDK build system). > However, it does not currently work well with the Skara CLI tooling. > Please see the [Skara wiki on Git clients]( > https://wiki.openjdk.java.net/display/SKARA/Skara#Skara-Git) for > up-to-date information about the Skara git client support. > > * The [Git for Windows](https://gitforwindows.org) client has issues > with line endings, and do not understand Cygwin paths. It does work > well with the Skara CLI tooling, however. To alleviate the line ending > problems, make sure you set `core.autocrlf` to `false` (this is asked > during installation). > > @jddarcy Do you find this acceptable? This PR has been open for almost a month, and it's starting getting embarrassing that our build instruction does not match the new Github reality. :( You should add a `.gitattributes` file which defines the desired line endings for each file type or for all files. Recommending a specific git client just because of that (?) is unnecessary. Here is a arbitrary example: https://github.com/alexkaratarakis/gitattributes/blob/f017637f08305dd1402fd025f6b648c45f04f499/Common.gitattributes#L49 ------------- PR: https://git.openjdk.java.net/jdk/pull/21 From ihse at openjdk.java.net Tue Nov 3 13:49:06 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 13:49:06 GMT Subject: RFR: 8255801: Race when building ct.sym build tools Message-ID: There is a race when compiling build.tools.symbolgenerator.CreateSymbols: [2020-11-03T07:21:12,118Z] Error: LinkageError occurred while loading main class build.tools.symbolgenerator.CreateSymbols [2020-11-03T07:21:12,119Z] java.lang.ClassFormatError: Truncated class file [2020-11-03T07:21:12,125Z] Gendata.gmk:69: recipe for target '/.../build/macosx-x64-open/support/javadoc-symbols/symbols' failed [2020-11-03T07:21:12,125Z] make[3]: *** [/.../build/macosx-x64-open/support/javadoc-symbols/symbols] Error 1 [2020-11-03T07:21:12,127Z] make/Main.gmk:148: recipe for target 'jdk.javadoc-gendata' failed [2020-11-03T07:21:12,127Z] make[2]: *** [jdk.javadoc-gendata] Error 2 [2020-11-03T07:21:12,127Z] make[2]: *** Waiting for unfinished jobs.... This was caused by https://bugs.openjdk.java.net/browse/JDK-8216497, which started compiling the same buildtool twic, in different, independent makefiles, to the same output directory. The long term correct solution is to fix https://bugs.openjdk.java.net/browse/JDK-8241463. For now, I opted to just make sure the output is placed in separate directories to avoid the race. We will still compile the build tool twice; it's annoying but not a major performance problem. ------------- Commit messages: - 8255801: Race when building ct.sym build tools Changes: https://git.openjdk.java.net/jdk/pull/1037/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1037&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255801 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1037.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1037/head:pull/1037 PR: https://git.openjdk.java.net/jdk/pull/1037 From erikj at openjdk.java.net Tue Nov 3 13:54:00 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 3 Nov 2020 13:54:00 GMT Subject: RFR: 8255798: Remove dead headless code in CompileJavaModules.gmk In-Reply-To: <5Gi8FSVPAw7OmK6bq4YjYDWyCGr23IgedicITWsO0CE=.bfa58da7-69c2-472c-9a5f-1b6fa9dd2d36@github.com> References: <5Gi8FSVPAw7OmK6bq4YjYDWyCGr23IgedicITWsO0CE=.bfa58da7-69c2-472c-9a5f-1b6fa9dd2d36@github.com> Message-ID: <_dNefpiWfKR-zi8TELcmEiO8Sx_C9dnqDF_nhHu2N0I=.6ba7cf21-0a69-4049-af5c-d73e1d5decde@github.com> On Tue, 3 Nov 2020 08:21:22 GMT, Magnus Ihse Bursie wrote: > The variable BUILD_HEADLESS_ONLY is no longer set. And sun/applet does not exist anymore. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1031 From erikj at openjdk.java.net Tue Nov 3 13:58:57 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 3 Nov 2020 13:58:57 GMT Subject: RFR: 8251549: Update docs on building for Git In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 13:21:24 GMT, Michael Keck wrote: >> @erikj79 >> >> Ok, here is a completely rewritten portion, which highlights the differences between Cygwin git and Git for Windows. >> >> Suggestion: >> >> * You need to install a git client. You have two choices, Cygwin git or >> Git for Windows. Unfortunately there are pros and cons with each choice. >> >> * The Cygwin `git` client has no line ending issues and understands >> Cygwin paths (which are used throughout the JDK build system). >> However, it does not currently work well with the Skara CLI tooling. >> Please see the [Skara wiki on Git clients]( >> https://wiki.openjdk.java.net/display/SKARA/Skara#Skara-Git) for >> up-to-date information about the Skara git client support. >> >> * The [Git for Windows](https://gitforwindows.org) client has issues >> with line endings, and do not understand Cygwin paths. It does work >> well with the Skara CLI tooling, however. To alleviate the line ending >> problems, make sure you set `core.autocrlf` to `false` (this is asked >> during installation). >> >> @jddarcy Do you find this acceptable? This PR has been open for almost a month, and it's starting getting embarrassing that our build instruction does not match the new Github reality. :( > > You should add a `.gitattributes` file which defines the desired line endings for each file type or for all files. Recommending a specific git client just because of that (?) is unnecessary. > > Here is a arbitrary example: > https://github.com/alexkaratarakis/gitattributes/blob/f017637f08305dd1402fd025f6b648c45f04f499/Common.gitattributes#L49 I like your proposed text, Magnus. ------------- PR: https://git.openjdk.java.net/jdk/pull/21 From erikj at openjdk.java.net Tue Nov 3 13:59:55 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 3 Nov 2020 13:59:55 GMT Subject: RFR: 8255801: Race when building ct.sym build tools In-Reply-To: References: Message-ID: <2d8Wt1Pdy3E8o7nyvPqGI9RGCH8qC2f5mheVJTjjXOY=.8e8ddac7-ef1b-4822-a930-c805f87567f0@github.com> On Tue, 3 Nov 2020 13:39:19 GMT, Magnus Ihse Bursie wrote: > There is a race when compiling build.tools.symbolgenerator.CreateSymbols: > > [2020-11-03T07:21:12,118Z] Error: LinkageError occurred while loading main class build.tools.symbolgenerator.CreateSymbols > [2020-11-03T07:21:12,119Z] java.lang.ClassFormatError: Truncated class file > [2020-11-03T07:21:12,125Z] Gendata.gmk:69: recipe for target '/.../build/macosx-x64-open/support/javadoc-symbols/symbols' failed > [2020-11-03T07:21:12,125Z] make[3]: *** [/.../build/macosx-x64-open/support/javadoc-symbols/symbols] Error 1 > [2020-11-03T07:21:12,127Z] make/Main.gmk:148: recipe for target 'jdk.javadoc-gendata' failed > [2020-11-03T07:21:12,127Z] make[2]: *** [jdk.javadoc-gendata] Error 2 > [2020-11-03T07:21:12,127Z] make[2]: *** Waiting for unfinished jobs.... > > This was caused by https://bugs.openjdk.java.net/browse/JDK-8216497, which started compiling the same buildtool twic, in different, independent makefiles, to the same output directory. > > The long term correct solution is to fix https://bugs.openjdk.java.net/browse/JDK-8241463. > > For now, I opted to just make sure the output is placed in separate directories to avoid the race. We will still compile the build tool twice; it's annoying but not a major performance problem. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1037 From coleenp at openjdk.java.net Tue Nov 3 14:53:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 14:53:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Merge branch 'master' into jvmti-table - Merge branch 'master' into jvmti-table - More review comments from Stefan and ErikO - Code review comments from StefanK. - 8212879: Make JVMTI TagMap table not hash on oop address ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=03 Stats: 1749 lines in 41 files changed: 627 ins; 1020 del; 102 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From ihse at openjdk.java.net Tue Nov 3 14:58:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 14:58:57 GMT Subject: Integrated: 8255798: Remove dead headless code in CompileJavaModules.gmk In-Reply-To: <5Gi8FSVPAw7OmK6bq4YjYDWyCGr23IgedicITWsO0CE=.bfa58da7-69c2-472c-9a5f-1b6fa9dd2d36@github.com> References: <5Gi8FSVPAw7OmK6bq4YjYDWyCGr23IgedicITWsO0CE=.bfa58da7-69c2-472c-9a5f-1b6fa9dd2d36@github.com> Message-ID: On Tue, 3 Nov 2020 08:21:22 GMT, Magnus Ihse Bursie wrote: > The variable BUILD_HEADLESS_ONLY is no longer set. And sun/applet does not exist anymore. This pull request has now been integrated. Changeset: 64a98112 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/64a98112 Stats: 4 lines in 1 file changed: 0 ins; 4 del; 0 mod 8255798: Remove dead headless code in CompileJavaModules.gmk Reviewed-by: shade, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1031 From ihse at openjdk.java.net Tue Nov 3 14:58:59 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 14:58:59 GMT Subject: Integrated: 8255801: Race when building ct.sym build tools In-Reply-To: References: Message-ID: <0aSaUdOY0N29ENF8UPRx7h0FN-mKoZUMUiHqAwxbGFg=.dd333233-f854-43e3-8c11-c87adf7736e1@github.com> On Tue, 3 Nov 2020 13:39:19 GMT, Magnus Ihse Bursie wrote: > There is a race when compiling build.tools.symbolgenerator.CreateSymbols: > > [2020-11-03T07:21:12,118Z] Error: LinkageError occurred while loading main class build.tools.symbolgenerator.CreateSymbols > [2020-11-03T07:21:12,119Z] java.lang.ClassFormatError: Truncated class file > [2020-11-03T07:21:12,125Z] Gendata.gmk:69: recipe for target '/.../build/macosx-x64-open/support/javadoc-symbols/symbols' failed > [2020-11-03T07:21:12,125Z] make[3]: *** [/.../build/macosx-x64-open/support/javadoc-symbols/symbols] Error 1 > [2020-11-03T07:21:12,127Z] make/Main.gmk:148: recipe for target 'jdk.javadoc-gendata' failed > [2020-11-03T07:21:12,127Z] make[2]: *** [jdk.javadoc-gendata] Error 2 > [2020-11-03T07:21:12,127Z] make[2]: *** Waiting for unfinished jobs.... > > This was caused by https://bugs.openjdk.java.net/browse/JDK-8216497, which started compiling the same buildtool twic, in different, independent makefiles, to the same output directory. > > The long term correct solution is to fix https://bugs.openjdk.java.net/browse/JDK-8241463. > > For now, I opted to just make sure the output is placed in separate directories to avoid the race. We will still compile the build tool twice; it's annoying but not a major performance problem. This pull request has now been integrated. Changeset: 364b0fe8 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/364b0fe8 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod 8255801: Race when building ct.sym build tools Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1037 From stuefe at openjdk.java.net Tue Nov 3 15:57:01 2020 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Tue, 3 Nov 2020 15:57:01 GMT Subject: RFR: JDK-8255711: Fix and unify hotspot signal handlers Message-ID: Hi all, may I please have opinions and reviews on this cleanup-fix-patch for the hotspot signal handlers. Its main intent is to simplify coding and to commonize some of it across all Posix platforms where possible. Also to fix a number of smaller issues. This will have a number of benefits, mainly easing maintenance pain for porters and reducing bitrot for platform dependent code. This all builds upon the work @gerard-ziemski did with https://bugs.openjdk.java.net/browse/JDK-8252324. ---- This cleanup was made more complicated by the fact that there exists a non-obvious and undocumented way for a third party app to chain signal handlers (beside the documented one of using libjsig). It seems that the JVM_handle_xxx_functions() are in fact interfaces for third party coding to invoke hotspot signal handling. This only makes sense in combination with `-XX:+AllowUserSignalHandlers`. A cursory github search revealed that this flag is used quite a bit. See a more in-depth discussion here: [4]. Thanks to @dholmes-ora for untangling this bit of history. Unfortunately there is no official documentation from Sun or Oracle, and zero regression tests. So I try to preserve this interface as best as I can. I plan to add a proper regression test with a later change, but for now I don't have the time for that. --- The fixed issues: 1) `PosixSignals::_are_signal_handlers_installed` is used inside the platform handlers to guard a part of the platform handlers against execution in case the signal handlers are not yet installed. Initially this confused me since when this handler is called it would of course be installed. So that boolean would always be true. The only explanation I found was that since these handlers can be invoked directly from outside, this is some (ineffective) form of guard against calling this handler too early. But that guard can be left out and that boolean removed. Our signal handlers are safe to call before VM initialization is completed. 2) The return code of JVM_handle_xxx_signal() was inconsistently set (some as bool, some as int) as well as unused in normal code paths (excluding outside calls). 3) JVM_handle_xxx_signal are supposed to be exported, but on AIX there is a day-zero bug which caused it to not be exported. 4) Every platform handler has this section: JavaThread* thread = NULL; VMThread* vmthread = NULL; if (PosixSignals::are_signal_handlers_installed()) { if (t != NULL ){ if(t->is_Java_thread()) { thread = t->as_Java_thread(); } else if(t->is_VM_thread()){ vmthread = (VMThread *)t; } } } `vmthread` is unused on all platforms and can be removed. 5) Every platform handler has some variant of this section, to ignore SIGPIPE, SIGXFSZ (whose default actions are to terminate the VM): if (sig == SIGPIPE || sig == SIGXFSZ) { // allow chained handler to go first if (PosixSignals::chained_handler(sig, info, ucVoid)) { return true; } else { // Ignoring SIGPIPE/SIGXFSZ - see bugs 4229104 or 6499219 return true; } } - On s390 and ppc, we miss SIGXFSZ handling _Update: Fixed separately for easier backport, see [https://bugs.openjdk.java.net/browse/JDK-8255734](JDK-8255734)_. - both paths return true - section can be shortened Side note: having handlers for those signals may be unnecessary. We could just set the signal handler to `SIG_IGN`. We would have to tiptoe around any third party handlers for those signals, but it still may be simpler. 6) At the end of every platform header, before calling into fatal error handling, we unblock the signal: > // unmask current signal > sigset_t newset; > sigemptyset(&newset); > sigaddset(&newset, sig); > sigprocmask(SIG_UNBLOCK, &newset, NULL); > - Use of `sigprocmask()` is UB in a multithreaded program. - but then, this section is unnecessary anyway, since [https://bugs.openjdk.java.net/browse/JDK-8252533](JDK-8252533) we unmask error signals at the start of the signal handler. 7) the JFR crash protection is not consistently checked in all platform handlers. 8) On Zero, when entering fatal error handling, we do so via fatal() instead of VMError::report_and_die(), thereby discarding the real crash context and obfuscating the register content in the hs-err file (we still see registers, but those stem from the assertion-poison-page mechanism). 9) on Linux ppc64 and AIX, we have this section: > if (sig == SIGILL && (pc < (address) 0x200)) { > goto report_and_die; > } which is related to the fact that the zero page on AIX is readable, filled with 0, and reading instructions from it will yield us a SIGILL, not a SIGSEGV (0 is not a noop on PPC, so we don't nop-slide). This coding is irrelevant on Linux. On AIX, it can also be removed, since this SIGILL would be unrecognized by the hotspot and later count as fatal error anyway. 10) When invoking the fatal error handler, we extract the pc from the context and hand it over as "faulting pc". For SIGILL and SIGFPE, this is not totally correct. According to POSIX [3], for those signals the address of the faulting instruction is handed over in `si_info.si_addr`. On most platforms this does not matter, they are the same. But on some architectures the pc in the signal context actually points somewhere else, e.g. beyond the faulting instruction. Therefore `si_info.si_addr` is the better choice. ---- The changes in this patch: a) hotspot signal handling is now done by the following functions: | | v v javaSignalHandler JVM_handle_linux_signal() | / v v javaSignalHandler_inner | v PosixSignals::pd_hotspot_signal_handler() The right branch only exists to support the `AllowUserSignalHandlers` case, see [4]. `javaSignalHandler` is registered as handler, as it was before; JVM_handle_linux_signal() is exported as before. `javaSignalHandler_inner` contains the shared portion of the signal handler; `PosixSignals::pd_hotspot_signal_handler()` contains the remaining platform dependent portions. b) I commonized prologue- and epilogue coding. - I simplified (4) to a single line in the shared handler - I moved the JFR thread crash protection (7) up to the shared handler - I moved the complete epilogue up to the shared handler. That includes calling the chained handlers, should they exist, as well as invoking the fatal error handler. That fixes (8) and (6) - Zero has this tradition of showing a robot telling a cat about the error signal, which I like, and kept. - I simplified (5) and commonized it, and removed (9) completely - In PosixSignals::install_signal_handlers(), I removed the `signal_handlers_are_installed` guard and replaced it with an assert. Unfortunately this causes lots of indentation changes. @gerard-ziemski: if this clashes too much with your patch for JDK-8253742, I'll leave that part out. Thanks for reviewing. Testing: this patch ran through our nightlies, in an earlier form. They will be re-ran some more times. I'd be happy if aarch64 porters could take a look at the aarch64 portion of this change. Please note that I had to draw a line somewhere - this is an open ended issue and a lot more could be cleaned. See also Gerard's work on [5], which is under review too. ---- [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-October/043145.html [2] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-October/043191.html [3] https://pubs.opengroup.org/onlinepubs/009695399/basedefs/signal.h.html [4] https://mail.openjdk.java.net/pipermail/jdk-dev/2020-November/004887.html [5] https://bugs.openjdk.java.net/browse/JDK-8253742 ------------- Commit messages: - Initial patch Changes: https://git.openjdk.java.net/jdk/pull/1034/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1034&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255711 Stats: 916 lines in 13 files changed: 116 ins; 686 del; 114 mod Patch: https://git.openjdk.java.net/jdk/pull/1034.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1034/head:pull/1034 PR: https://git.openjdk.java.net/jdk/pull/1034 From jvernee at openjdk.java.net Tue Nov 3 17:09:00 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 17:09:00 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c Message-ID: Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c This removes the reported warning. Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. Testing: tier1-3 [1] https://bugs.openjdk.java.net/browse/JDK-8232022 ------------- Commit messages: - Add 32-bit safe version of jlong <-> casts to libJNIPoint.c Changes: https://git.openjdk.java.net/jdk/pull/1017/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1017&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255128 Stats: 17 lines in 2 files changed: 8 ins; 2 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/1017.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1017/head:pull/1017 PR: https://git.openjdk.java.net/jdk/pull/1017 From coleenp at openjdk.java.net Tue Nov 3 17:47:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 17:47:59 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 18:36:32 GMT, Jorn Vernee wrote: > Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c > > This removes the reported warning. > > Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). > > CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. > > Testing: tier1-3 > > [1] https://bugs.openjdk.java.net/browse/JDK-8232022 Marked as reviewed by coleenp (Reviewer). test/micro/org/openjdk/bench/jdk/incubator/foreign/points/support/libJNIPoint.c line 32: > 30: #define PTR_TO_JLONG(value) ((jlong) (value)) > 31: #else > 32: #define JLONG_TO_PTR(value, type) ((type*) (jint) (value)) Maybe the jlong thisPoint argument comes from a pointer so it's ok. Not nice, but if you say so, I'll go along. ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From coleenp at openjdk.java.net Tue Nov 3 17:48:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 17:48:00 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: <86GJGpWCz1RDjZwbPBay-WvvZMMTPidoJT22ppTqgRo=.23c9d7de-fa18-4b9a-aab8-99ccedb54b3c@github.com> On Tue, 3 Nov 2020 17:40:38 GMT, Coleen Phillimore wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > Marked as reviewed by coleenp (Reviewer). I was anxious for this to compile so I reviewed it. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From jvernee at openjdk.java.net Tue Nov 3 17:48:00 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 17:48:00 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 17:40:32 GMT, Coleen Phillimore wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > test/micro/org/openjdk/bench/jdk/incubator/foreign/points/support/libJNIPoint.c line 32: > >> 30: #define PTR_TO_JLONG(value) ((jlong) (value)) >> 31: #else >> 32: #define JLONG_TO_PTR(value, type) ((type*) (jint) (value)) > > Maybe the jlong thisPoint argument comes from a pointer so it's ok. Not nice, but if you say so, I'll go along. Yes, it's the same pointer that is returned from allocate. It's just stored in a `jlong` on the Java side (this would be a requirement on x64), but it's not expected that the high-order bits are used. Do you have a suggestion for handling this otherwise? ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From jvernee at openjdk.java.net Tue Nov 3 17:48:01 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 17:48:01 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: <6DXVMQgh9e1fzgQvDT2uEy3ttbSLzTj4hKJrH7gJ2y4=.42173fe7-2f20-4796-8500-426a7b872cf3@github.com> On Tue, 3 Nov 2020 17:42:08 GMT, Jorn Vernee wrote: >> test/micro/org/openjdk/bench/jdk/incubator/foreign/points/support/libJNIPoint.c line 32: >> >>> 30: #define PTR_TO_JLONG(value) ((jlong) (value)) >>> 31: #else >>> 32: #define JLONG_TO_PTR(value, type) ((type*) (jint) (value)) >> >> Maybe the jlong thisPoint argument comes from a pointer so it's ok. Not nice, but if you say so, I'll go along. > > Yes, it's the same pointer that is returned from allocate. It's just stored in a `jlong` on the Java side (this would be a requirement on x64), but it's not expected that the high-order bits are used. > > Do you have a suggestion for handling this otherwise? Hmm, now that I think about it, we could add overloads that work with `int` for 32-bit platforms. But, for now 32-bit usage of these benchmarks seems unlikely. Since the build passes, and the benchmarks run on both platforms already, I'm reluctant to put much more effort into this at the moment. ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From kbarrett at openjdk.java.net Tue Nov 3 18:12:10 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 3 Nov 2020 18:12:10 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:53:12 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Merge branch 'master' into jvmti-table > - Merge branch 'master' into jvmti-table > - More review comments from Stefan and ErikO > - Code review comments from StefanK. > - 8212879: Make JVMTI TagMap table not hash on oop address src/hotspot/share/prims/jvmtiTagMap.cpp line 3018: > 3016: } > 3017: // Later GC code will relocate the oops, so defer rehashing until then. > 3018: tag_map->_needs_rehashing = true; This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.) For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing. (I think CMS oldgen remark may have been similar, for what that's worth.) src/hotspot/share/prims/jvmtiTagMap.cpp line 3015: > 3013: if (tag_map != NULL && !tag_map->is_empty()) { > 3014: if (num_dead_entries != 0) { > 3015: tag_map->hashmap()->unlink_and_post(tag_map->env()); Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup. src/hotspot/share/prims/jvmtiTagMap.cpp line 2979: > 2977: > 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved > 2979: // and have their new addresses, the table can be rehashed. I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that). src/hotspot/share/prims/jvmtiTagMap.cpp line 127: > 125: // The table cleaning, posting and rehashing can race for > 126: // concurrent GCs. So fix it here once we have a lock or are > 127: // at a safepoint. I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time). ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Tue Nov 3 18:12:07 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 3 Nov 2020 18:12:07 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: On Tue, 3 Nov 2020 12:58:22 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > More review comments from Stefan and ErikO src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 41: > 39: class Iterator; > 40: > 41: typedef void (*Processor)(BoolObjectClosure*, OopClosure*); I think this typedef is to support serial phases and that it is probably no longer used. src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50: > 48: }; > 49: > 50: typedef uint WeakProcessorPhase; This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes. But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here: src/hotspot/share/gc/shared/weakProcessor.inline.hpp 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase); 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index); => 103 StorageState* cur_state = _storage_states.par_state(phase); I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet. enum class : uint WeakProcessorPhase {}; ENUMERATOR_RANGE(WeakProcessorPhase, static_cast(0), static_cast(OopStorageSet::weak_count)); and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator (which involves more than a type alias). Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From shade at openjdk.java.net Tue Nov 3 18:33:55 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 3 Nov 2020 18:33:55 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 18:36:32 GMT, Jorn Vernee wrote: > Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c > > This removes the reported warning. > > Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). > > CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. > > Testing: tier1-3 > > [1] https://bugs.openjdk.java.net/browse/JDK-8232022 I knew this looks familiar. Look at [existing macros in jlong_md.h](https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/jlong_md.h#L67-L81) and use/match them? There is a little difference in casting through `jint` in your code, while `jlong_md.h` does it via `int`. ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From ihse at openjdk.java.net Tue Nov 3 18:48:58 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 18:48:58 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 18:36:32 GMT, Jorn Vernee wrote: > Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c > > This removes the reported warning. > > Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). > > CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. > > Testing: tier1-3 > > [1] https://bugs.openjdk.java.net/browse/JDK-8232022 Changes requested by ihse (Reviewer). make/autoconf/flags-cflags.m4 line 667: > 665: > 666: if test "x$FLAGS_CPU_BITS" = x64; then > 667: if test "x$FLAGS_OS" = xlinux || test "x$FLAGS_OS" = xmacosx || test "x$FLAGS_OS" = xwindows; then At this point, you're almost testing for all supported OSes. :) I can only think of AIX that does not match this if clause. I think it would be better to just remove the if and always define _LP64=1 on 64-bit platforms. @simonis Would that be OK for AIX? ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From jvernee at openjdk.java.net Tue Nov 3 18:56:00 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 18:56:00 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: <5RiTHvRKDKf0_Zr2ZQICloKttbvZc_osDtJ_ylpxfrw=.7fe27b37-78fa-47fc-b37b-4ecde950818a@github.com> On Tue, 3 Nov 2020 18:46:29 GMT, Magnus Ihse Bursie wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > make/autoconf/flags-cflags.m4 line 667: > >> 665: >> 666: if test "x$FLAGS_CPU_BITS" = x64; then >> 667: if test "x$FLAGS_OS" = xlinux || test "x$FLAGS_OS" = xmacosx || test "x$FLAGS_OS" = xwindows; then > > At this point, you're almost testing for all supported OSes. :) I can only think of AIX that does not match this if clause. I think it would be better to just remove the if and always define _LP64=1 on 64-bit platforms. > > @simonis Would that be OK for AIX? Thanks for the suggestion. Note the code that adds _LP64 for the JVM (below): if test "x$FLAGS_OS" != xaix; then # xlc on AIX defines _LP64=1 by default and issues a warning if we redefine it. $1_DEFINES_CPU_JVM="${$1_DEFINES_CPU_JVM} -D_LP64=1" fi So, it seems xlc/aix explicitly does _not_ want this macro defined? I think we could reuse that if-block to add `-D_LP64=` to both `1_DEFINES_CPU_JDK`, and `1_DEFINES_CPU_JVM` though, and remove the first one that checks for linux/mac/windows. ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From burban at openjdk.java.net Tue Nov 3 19:07:58 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Tue, 3 Nov 2020 19:07:58 GMT Subject: Integrated: 8254827: JVMCI: Enable it for Windows+AArch64 In-Reply-To: References: Message-ID: On Thu, 15 Oct 2020 15:00:47 GMT, Bernhard Urban-Forster wrote: > Use r18 as allocatable register on Linux only. > > A bootstrap works now (it has been crashing before due to r18 being allocated): > $ ./windows-aarch64-server-fastdebug/bin/java.exe -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:+BootstrapJVMCI -version > Bootstrapping JVMCI................................. in 17990 ms > (compiled 3330 methods) > openjdk version "16-internal" 2021-03-16 > OpenJDK Runtime Environment (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk) > OpenJDK 64-Bit Server VM (fastdebug build 16-internal+0-adhoc.NORTHAMERICAbeurba.openjdk-jdk, mixed mode) > > Jtreg tests `test/hotspot/jtreg/compiler/jvmci` are passing as well. This pull request has now been integrated. Changeset: 88ee9733 Author: Bernhard Urban-Forster Committer: Tom Rodriguez URL: https://git.openjdk.java.net/jdk/commit/88ee9733 Stats: 24 lines in 4 files changed: 15 ins; 0 del; 9 mod 8254827: JVMCI: Enable it for Windows+AArch64 Reviewed-by: ihse, never, kvn ------------- PR: https://git.openjdk.java.net/jdk/pull/685 From jvernee at openjdk.java.net Tue Nov 3 19:10:55 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 19:10:55 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 18:30:46 GMT, Aleksey Shipilev wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > I knew this looks familiar. Look at [existing macros in jlong_md.h](https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/jlong_md.h#L67-L81) and use/match them? There is a little difference in casting through `jint` in your code, while `jlong_md.h` does it via `int`. @shipilev Now that I look at the file you linked, it does look familiar to me as well. Must have copy-pasted it from my subconscious ;) Looks like this file is usable from the benchmark lib code as well, will try to switch. ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From ihse at openjdk.java.net Tue Nov 3 19:18:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 19:18:56 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: <5RiTHvRKDKf0_Zr2ZQICloKttbvZc_osDtJ_ylpxfrw=.7fe27b37-78fa-47fc-b37b-4ecde950818a@github.com> References: <5RiTHvRKDKf0_Zr2ZQICloKttbvZc_osDtJ_ylpxfrw=.7fe27b37-78fa-47fc-b37b-4ecde950818a@github.com> Message-ID: On Tue, 3 Nov 2020 18:52:52 GMT, Jorn Vernee wrote: >> make/autoconf/flags-cflags.m4 line 667: >> >>> 665: >>> 666: if test "x$FLAGS_CPU_BITS" = x64; then >>> 667: if test "x$FLAGS_OS" = xlinux || test "x$FLAGS_OS" = xmacosx || test "x$FLAGS_OS" = xwindows; then >> >> At this point, you're almost testing for all supported OSes. :) I can only think of AIX that does not match this if clause. I think it would be better to just remove the if and always define _LP64=1 on 64-bit platforms. >> >> @simonis Would that be OK for AIX? > > Thanks for the suggestion. > > Note the code that adds _LP64 for the JVM (below): > > if test "x$FLAGS_OS" != xaix; then > # xlc on AIX defines _LP64=1 by default and issues a warning if we redefine it. > $1_DEFINES_CPU_JVM="${$1_DEFINES_CPU_JVM} -D_LP64=1" > fi > > So, it seems xlc/aix explicitly does _not_ want this macro defined? > > I think we could reuse that if-block to add `-D_LP64=` to both `1_DEFINES_CPU_JDK`, and `1_DEFINES_CPU_JVM` though, and remove the first one that checks for linux/mac/windows. Yes, that sounds good. I did not notice this (still not used to github reviews, which I think has too little context by default). ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From jvernee at openjdk.java.net Tue Nov 3 19:44:54 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 19:44:54 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: <5RiTHvRKDKf0_Zr2ZQICloKttbvZc_osDtJ_ylpxfrw=.7fe27b37-78fa-47fc-b37b-4ecde950818a@github.com> Message-ID: On Tue, 3 Nov 2020 19:16:02 GMT, Magnus Ihse Bursie wrote: >> Thanks for the suggestion. >> >> Note the code that adds _LP64 for the JVM (below): >> >> if test "x$FLAGS_OS" != xaix; then >> # xlc on AIX defines _LP64=1 by default and issues a warning if we redefine it. >> $1_DEFINES_CPU_JVM="${$1_DEFINES_CPU_JVM} -D_LP64=1" >> fi >> >> So, it seems xlc/aix explicitly does _not_ want this macro defined? >> >> I think we could reuse that if-block to add `-D_LP64=` to both `1_DEFINES_CPU_JDK`, and `1_DEFINES_CPU_JVM` though, and remove the first one that checks for linux/mac/windows. > > Yes, that sounds good. I did not notice this (still not used to github reviews, which I think has too little context by default). Ok, will do. (FWIW, you can expand the context of the diff with the arrow buttons on the left side of the view. Above or below the line numbers) ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From erikj at openjdk.java.net Tue Nov 3 20:15:04 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 3 Nov 2020 20:15:04 GMT Subject: RFR: JDK-8255850: Hotspot recompiled on first incremental build Message-ID: After building the JDK from clean, the first incremental build of hotspot will recompile all of it. This is caused by a difference in the CFLAGS generated on the second go. The difference is generated in JdkNativeCompilation.gmk, where the module specific java header dir is always added to the list of include dirs. When compiling hotspot the first time, there is no such dir, and so nothing is added, but the second go, later compilation steps have created the base headers dir ($(SUPPORT_OUTPUTDIR)/headers), which is found and picked up in CFLAGS. This difference is then detected by the DependOnVariable construct for libjvm. The fix here is to make sure SetupJdkLibrary is able to work in a context without a MODULE defined, since that is how libjvm is built. In that case, no java header dir should be added. While fixing this I decided to go through the whole file and make sure all uses of MODULE were protected when needed. ------------- Commit messages: - Handle a MODULE free context better in SetupJdk* macros Changes: https://git.openjdk.java.net/jdk/pull/1041/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1041&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255850 Stats: 30 lines in 1 file changed: 23 ins; 0 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/1041.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1041/head:pull/1041 PR: https://git.openjdk.java.net/jdk/pull/1041 From ihse at openjdk.java.net Tue Nov 3 20:14:54 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 20:14:54 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: <5RiTHvRKDKf0_Zr2ZQICloKttbvZc_osDtJ_ylpxfrw=.7fe27b37-78fa-47fc-b37b-4ecde950818a@github.com> Message-ID: On Tue, 3 Nov 2020 19:41:49 GMT, Jorn Vernee wrote: >> Yes, that sounds good. I did not notice this (still not used to github reviews, which I think has too little context by default). > > Ok, will do. (FWIW, you can expand the context of the diff with the arrow buttons on the left side of the view. Above or below the line numbers) (Yes, I know. I just didn't think that doing so would reveal anything about AIX. I just wish I had gotten a bit more context automatically.) ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From ihse at openjdk.java.net Tue Nov 3 20:29:53 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 20:29:53 GMT Subject: RFR: JDK-8255850: Hotspot recompiled on first incremental build In-Reply-To: References: Message-ID: <4eHbDTNW4hfEVF8csdrCBYG9KOm-4UTQA5s7DcfT5BM=.c4858d27-1d59-47d9-b5ed-c03f05fef043@github.com> On Tue, 3 Nov 2020 20:06:35 GMT, Erik Joelsson wrote: > After building the JDK from clean, the first incremental build of hotspot will recompile all of it. This is caused by a difference in the CFLAGS generated on the second go. The difference is generated in JdkNativeCompilation.gmk, where the module specific java header dir is always added to the list of include dirs. When compiling hotspot the first time, there is no such dir, and so nothing is added, but the second go, later compilation steps have created the base headers dir ($(SUPPORT_OUTPUTDIR)/headers), which is found and picked up in CFLAGS. This difference is then detected by the DependOnVariable construct for libjvm. > > The fix here is to make sure SetupJdkLibrary is able to work in a context without a MODULE defined, since that is how libjvm is built. In that case, no java header dir should be added. > > While fixing this I decided to go through the whole file and make sure all uses of MODULE were protected when needed. Looks good to me. Thanks for fixing this, it has been highly annoying! ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1041 From jvernee at openjdk.java.net Tue Nov 3 20:37:07 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 20:37:07 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c [v2] In-Reply-To: References: Message-ID: > Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c > > This removes the reported warning. > > Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). > > CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. > > Testing: tier1-3 > > [1] https://bugs.openjdk.java.net/browse/JDK-8232022 Jorn Vernee has updated the pull request incrementally with two additional commits since the last revision: - Collapse both _LP64 if blocks in flags-cflags.m4 - Use jlong.h macros instead of spinning new ones ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1017/files - new: https://git.openjdk.java.net/jdk/pull/1017/files/18e2507d..6309e2ae Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1017&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1017&range=00-01 Stats: 23 lines in 2 files changed: 1 ins; 12 del; 10 mod Patch: https://git.openjdk.java.net/jdk/pull/1017.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1017/head:pull/1017 PR: https://git.openjdk.java.net/jdk/pull/1017 From jvernee at openjdk.java.net Tue Nov 3 20:39:59 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Tue, 3 Nov 2020 20:39:59 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c [v2] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 19:08:05 GMT, Jorn Vernee wrote: >> I knew this looks familiar. Look at [existing macros in jlong_md.h](https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/jlong_md.h#L67-L81) and use/match them? There is a little difference in casting through `jint` in your code, while `jlong_md.h` does it via `int`. > > @shipilev Now that I look at the file you linked, it does look familiar to me as well. Must have copy-pasted it from my subconscious ;) > > Looks like this file is usable from the benchmark lib code as well, will try to switch. I've updated the PR with the following 2 changes: - Use the pre-exsiting macros from jlong.h to convert between jlong and pointer - Collapse the 2 if-blocks in flags-cflags.m4 for setting _LP64 into one ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From coleenp at openjdk.java.net Tue Nov 3 20:42:58 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 20:42:58 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c [v2] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 20:37:07 GMT, Jorn Vernee wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > Jorn Vernee has updated the pull request incrementally with two additional commits since the last revision: > > - Collapse both _LP64 if blocks in flags-cflags.m4 > - Use jlong.h macros instead of spinning new ones This looks better. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1017 From ihse at openjdk.java.net Tue Nov 3 20:50:07 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 20:50:07 GMT Subject: RFR: 8255853: Update all nroff manpages for JDK 16 release Message-ID: The man pages in src/$module/man/*.1 is generated from the (unfortunately still) closed markdown sources. These needs to be updated for JDK 16. During this updating, a typo in the GPL header is also fixed (which caused Oracle's internal header verification tool to complain) -- a double space had been collapsed into a single space. There is a certain possibility that man page content can still change for JDK 16. If that is the case, the export needs to be re-done. (In contrast to prior exports, we now have a standard way of doing this, so it's trivial to update.) ------------- Commit messages: - 8255853: Update all nroff manpages for JDK 16 release Changes: https://git.openjdk.java.net/jdk/pull/1043/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1043&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255853 Stats: 334 lines in 28 files changed: 202 ins; 25 del; 107 mod Patch: https://git.openjdk.java.net/jdk/pull/1043.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1043/head:pull/1043 PR: https://git.openjdk.java.net/jdk/pull/1043 From coleenp at openjdk.java.net Tue Nov 3 21:17:05 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:17:05 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:47:35 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 3018: > >> 3016: } >> 3017: // Later GC code will relocate the oops, so defer rehashing until then. >> 3018: tag_map->_needs_rehashing = true; > > This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.) > > For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing. > > (I think CMS oldgen remark may have been similar, for what that's worth.) Ok, so I'm going to need help to know where in all the different GCs to make this call. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From ayang at openjdk.java.net Tue Nov 3 21:17:03 2020 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 3 Nov 2020 21:17:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:53:12 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Merge branch 'master' into jvmti-table > - Merge branch 'master' into jvmti-table > - More review comments from Stefan and ErikO > - Code review comments from StefanK. > - 8212879: Make JVMTI TagMap table not hash on oop address src/hotspot/share/utilities/hashtable.cpp line 164: > 162: } > 163: } > 164: return newsize; It is existing code, but could be made clearer as part of this PR: int newsize; for (int i=0; i= requested) break; } return newsize; Additionally, this method could be made `const`, right? PS: not a review, just a comment in passing ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From daniel.daugherty at oracle.com Tue Nov 3 21:18:23 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 3 Nov 2020 16:18:23 -0500 Subject: RFR: 8212879: Make JVMTI TagMap table not hash on oop address In-Reply-To: References: Message-ID: <3fb77c6e-a99b-c90b-9170-1ce5fca6df1b@oracle.com> On 11/2/20 8:22 AM, Coleen Phillimore wrote: > On Mon, 2 Nov 2020 08:34:17 GMT, Stefan Karlsson wrote: > >>> src/hotspot/share/prims/jvmtiTagMap.cpp line 126: >>> >>>> 124: // concurrent GCs. So fix it here once we have a lock or are >>>> 125: // at a safepoint. >>>> 126: // SetTag and GetTag should not post events! >>> I think it would be good to explain why. Otherwise, this just leaves the readers wondering why this is the case. >> Maybe even move this comment to the set_tag/get_tag code. > I was trying to explain why there's a boolean there but I can put this comment at both get_tag and set_tag. > > // Check if we have to processing for concurrent GCs. Typo: s/we have to processing/we have to do processing/ > // GetTag should not post events because the JavaThread has to > // transition to native for the callback and this cannot stop for > // safepoints with the hashmap lock held. > check_hashmap(false); > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:28:08 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:28:08 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 14:50:36 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 3015: > >> 3013: if (tag_map != NULL && !tag_map->is_empty()) { >> 3014: if (num_dead_entries != 0) { >> 3015: tag_map->hashmap()->unlink_and_post(tag_map->env()); > > Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup. The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From dcubed at openjdk.java.net Tue Nov 3 21:28:06 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 3 Nov 2020 21:28:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: <6uV_hLuxf7fu90EgzZckv9LwT-jAVboukH35MKaTtcU=.40efbbd6-fd6e-427a-a940-784ec86ddb15@github.com> On Mon, 2 Nov 2020 13:19:08 GMT, Coleen Phillimore wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > I think I addressed your comments, retesting now. Thank you! @coleenp - please make sure you hear from someone on the Serviceability team for this PR... ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:33:59 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:33:59 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 16:12:21 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 2979: > >> 2977: >> 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved >> 2979: // and have their new addresses, the table can be rehashed. > > I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that). So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case? ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:46:04 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:46:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 16:17:58 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/prims/jvmtiTagMap.cpp line 127: > >> 125: // The table cleaning, posting and rehashing can race for >> 126: // concurrent GCs. So fix it here once we have a lock or are >> 127: // at a safepoint. > > I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time). The comment is trying to describe the situation like: 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked) 2. safepoint for heap walk 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A. 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable. Both operations require the table to be locked. The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:46:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:46:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 13:19:08 GMT, Coleen Phillimore wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > I think I addressed your comments, retesting now. Thank you! > @coleenp - please make sure you hear from someone on the Serviceability team > for this PR... I've asked @sspitsyn to review this. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:46:05 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:46:05 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: On Tue, 3 Nov 2020 13:43:32 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> More review comments from Stefan and ErikO > > src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 41: > >> 39: class Iterator; >> 40: >> 41: typedef void (*Processor)(BoolObjectClosure*, OopClosure*); > > I think this typedef is to support serial phases and that it is probably no longer used. ok, removed. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 21:51:03 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 21:51:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> Message-ID: <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> On Tue, 3 Nov 2020 13:45:57 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> More review comments from Stefan and ErikO > > src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50: > >> 48: }; >> 49: >> 50: typedef uint WeakProcessorPhase; > > This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes. > > But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here: > > src/hotspot/share/gc/shared/weakProcessor.inline.hpp > 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase); > 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index); > => > 103 StorageState* cur_state = _storage_states.par_state(phase); > > I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet. > enum class : uint WeakProcessorPhase {}; > > ENUMERATOR_RANGE(WeakProcessorPhase, > static_cast(0), > static_cast(OopStorageSet::weak_count)); > and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator (which involves more than a type alias). > > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. Ok, so I'm not sure what to do with this: enum Phase { // Serial phase. JVMTI_ONLY(jvmti) // Additional implicit phase values follow for oopstorages. `};` I've removed the only thing in this enum. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From erikj at openjdk.java.net Tue Nov 3 22:07:55 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 3 Nov 2020 22:07:55 GMT Subject: RFR: 8255853: Update all nroff manpages for JDK 16 release In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 20:43:01 GMT, Magnus Ihse Bursie wrote: > The man pages in src/$module/man/*.1 is generated from the (unfortunately still) closed markdown sources. > > These needs to be updated for JDK 16. During this updating, a typo in the GPL header is also fixed (which caused Oracle's internal header verification tool to complain) -- a double space had been collapsed into a single space. > > There is a certain possibility that man page content can still change for JDK 16. If that is the case, the export needs to be re-done. (In contrast to prior exports, we now have a standard way of doing this, so it's trivial to update.) Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1043 From ihse at openjdk.java.net Tue Nov 3 22:14:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 22:14:57 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c [v2] In-Reply-To: References: Message-ID: <_DhMz8Zb2ZoH9S-_MSX5VClHyGfdLhcj4aSl5L2mIS8=.9f7e6452-4171-48c2-8340-ccba15cf0407@github.com> On Tue, 3 Nov 2020 20:37:07 GMT, Jorn Vernee wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > Jorn Vernee has updated the pull request incrementally with two additional commits since the last revision: > > - Collapse both _LP64 if blocks in flags-cflags.m4 > - Use jlong.h macros instead of spinning new ones Looks good to me now. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1017 From ihse at openjdk.java.net Tue Nov 3 22:19:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 22:19:56 GMT Subject: Integrated: 8255853: Update all nroff manpages for JDK 16 release In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 20:43:01 GMT, Magnus Ihse Bursie wrote: > The man pages in src/$module/man/*.1 is generated from the (unfortunately still) closed markdown sources. > > These needs to be updated for JDK 16. During this updating, a typo in the GPL header is also fixed (which caused Oracle's internal header verification tool to complain) -- a double space had been collapsed into a single space. > > There is a certain possibility that man page content can still change for JDK 16. If that is the case, the export needs to be re-done. (In contrast to prior exports, we now have a standard way of doing this, so it's trivial to update.) This pull request has now been integrated. Changeset: 622f72bc Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/622f72bc Stats: 334 lines in 28 files changed: 202 ins; 25 del; 107 mod 8255853: Update all nroff manpages for JDK 16 release Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1043 From sspitsyn at openjdk.java.net Tue Nov 3 22:23:00 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Tue, 3 Nov 2020 22:23:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:41:55 GMT, Coleen Phillimore wrote: > > @coleenp - please make sure you hear from someone on the Serviceability team > > for this PR... > > I've asked @sspitsyn to review this. Yes, I'm reviewing this. Still need another pass. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From ihse at openjdk.java.net Tue Nov 3 22:34:03 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 22:34:03 GMT Subject: RFR: 8253892: Disable misleading-indentation on clang as well as gcc Message-ID: With clang 10.0, the compiler now detects a new class of warnings. The `misleading-indentation` warning has previously been disabled on gcc for hotspot and libfdlibm. Now we need to disable it for clang as well. ------------- Commit messages: - 8253892: Disable misleading-indentation on clang as well as gcc Changes: https://git.openjdk.java.net/jdk/pull/1044/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1044&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8253892 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1044.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1044/head:pull/1044 PR: https://git.openjdk.java.net/jdk/pull/1044 From glaubitz at openjdk.java.net Tue Nov 3 22:38:56 2020 From: glaubitz at openjdk.java.net (John Paul Adrian Glaubitz) Date: Tue, 3 Nov 2020 22:38:56 GMT Subject: RFR: 8253892: Disable misleading-indentation on clang as well as gcc In-Reply-To: References: Message-ID: <4ndvIuL0hY08UQkoaye7QsI2UxVSTJs6yD66R7Re9dY=.1b18ff9f-108b-4132-9d8c-a9716efb8be8@github.com> On Tue, 3 Nov 2020 22:25:08 GMT, Magnus Ihse Bursie wrote: > With clang 10.0, the compiler now detects a new class of warnings. The `misleading-indentation` warning has previously been disabled on gcc for hotspot and libfdlibm. Now we need to disable it for clang as well. Why do we disable the warning instead of fixing the incorrect indentations? ------------- PR: https://git.openjdk.java.net/jdk/pull/1044 From ihse at openjdk.java.net Tue Nov 3 22:57:54 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 3 Nov 2020 22:57:54 GMT Subject: RFR: 8253892: Disable misleading-indentation on clang as well as gcc In-Reply-To: <4ndvIuL0hY08UQkoaye7QsI2UxVSTJs6yD66R7Re9dY=.1b18ff9f-108b-4132-9d8c-a9716efb8be8@github.com> References: <4ndvIuL0hY08UQkoaye7QsI2UxVSTJs6yD66R7Re9dY=.1b18ff9f-108b-4132-9d8c-a9716efb8be8@github.com> Message-ID: On Tue, 3 Nov 2020 22:36:34 GMT, John Paul Adrian Glaubitz wrote: >> With clang 10.0, the compiler now detects a new class of warnings. The `misleading-indentation` warning has previously been disabled on gcc for hotspot and libfdlibm. Now we need to disable it for clang as well. > > Why do we disable the warning instead of fixing the incorrect indentations? @glaubitz Good question. :) If you want to start fixing code to get rid of disabled warnings, I will not stand in your way! For example, in hotspot and gcc, we have `parentheses comment unknown-pragmas address delete-non-virtual-dtor char-subscripts array-bounds int-in-bool-context ignored-qualifiers missing-field-initializers implicit-fallthrough empty-body strict-overflow sequence-point maybe-uninitialized misleading-indentation cast-function-type shift-negative-value`. I believe many of these should be fixed and removed from the list of disabled warnings. But this bug is about disabling a warning in one compiler that we have already decided to disable in another. ------------- PR: https://git.openjdk.java.net/jdk/pull/1044 From erikj at openjdk.java.net Tue Nov 3 23:16:53 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 3 Nov 2020 23:16:53 GMT Subject: Integrated: JDK-8255850: Hotspot recompiled on first incremental build In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 20:06:35 GMT, Erik Joelsson wrote: > After building the JDK from clean, the first incremental build of hotspot will recompile all of it. This is caused by a difference in the CFLAGS generated on the second go. The difference is generated in JdkNativeCompilation.gmk, where the module specific java header dir is always added to the list of include dirs. When compiling hotspot the first time, there is no such dir, and so nothing is added, but the second go, later compilation steps have created the base headers dir ($(SUPPORT_OUTPUTDIR)/headers), which is found and picked up in CFLAGS. This difference is then detected by the DependOnVariable construct for libjvm. > > The fix here is to make sure SetupJdkLibrary is able to work in a context without a MODULE defined, since that is how libjvm is built. In that case, no java header dir should be added. > > While fixing this I decided to go through the whole file and make sure all uses of MODULE were protected when needed. This pull request has now been integrated. Changeset: 76fa974c Author: Erik Joelsson URL: https://git.openjdk.java.net/jdk/commit/76fa974c Stats: 30 lines in 1 file changed: 23 ins; 0 del; 7 mod 8255850: Hotspot recompiled on first incremental build Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1041 From coleenp at openjdk.java.net Tue Nov 3 23:41:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 23:41:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> Message-ID: <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> On Tue, 3 Nov 2020 21:47:24 GMT, Coleen Phillimore wrote: >> src/hotspot/share/gc/shared/weakProcessorPhases.hpp line 50: >> >>> 48: }; >>> 49: >>> 50: typedef uint WeakProcessorPhase; >> >> This was originally written with the idea that WeakProcessorPhases::Phase (and WeakProcessorPhase) should be a scoped enum (but we didn't have that feature yet). It's possible there are places that don't cope with a scoped enum, since that feature wasn't available when the code was written, so there might have be mistakes. >> >> But because of that, I'd prefer to keep the WeakProcessorPhases::Phase type and the existing definition of WeakProcessorPhase. Except this proposed change is breaking that at least here: >> >> src/hotspot/share/gc/shared/weakProcessor.inline.hpp >> 116 uint oopstorage_index = WeakProcessorPhases::oopstorage_index(phase); >> 117 StorageState* cur_state = _storage_states.par_state(oopstorage_index); >> => >> 103 StorageState* cur_state = _storage_states.par_state(phase); >> >> I think eventually (as in some future RFE) this could all be collapsed to something provided by OopStorageSet. >> enum class : uint WeakProcessorPhase {}; >> >> ENUMERATOR_RANGE(WeakProcessorPhase, >> static_cast(0), >> static_cast(OopStorageSet::weak_count)); >> and replacing all uses of WeakProcessorPhases::Iterator with EnumIterator (which involves more than a type alias). >> >> Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. > > Ok, so I'm not sure what to do with this: > > enum Phase { > // Serial phase. > JVMTI_ONLY(jvmti) > // Additional implicit phase values follow for oopstorages. > `};` > > I've removed the only thing in this enum. >Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Tue Nov 3 23:41:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 3 Nov 2020 23:41:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:12:49 GMT, Albert Mingkun Yang wrote: >> Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: >> >> - Merge branch 'master' into jvmti-table >> - Merge branch 'master' into jvmti-table >> - More review comments from Stefan and ErikO >> - Code review comments from StefanK. >> - 8212879: Make JVMTI TagMap table not hash on oop address > > src/hotspot/share/utilities/hashtable.cpp line 164: > >> 162: } >> 163: } >> 164: return newsize; > > It is existing code, but could be made clearer as part of this PR: > int newsize; > for (int i=0; i newsize = primelist[i]; > if (newsize >= requested) > break; > } > return newsize; > Additionally, this method could be made `const`, right? > > PS: not a review, just a comment in passing Yes, that is a lot simpler and better. I'd copied that code from another file without changing it that much. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Wed Nov 4 00:08:10 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 4 Nov 2020 00:08:10 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Code review comments from Kim and Albert. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/1c3f2e1e..f66ea839 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=03-04 Stats: 10 lines in 3 files changed: 0 ins; 4 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From david.holmes at oracle.com Wed Nov 4 00:47:13 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Nov 2020 10:47:13 +1000 Subject: RFR: 8255853: Update all nroff manpages for JDK 16 release In-Reply-To: References: Message-ID: <0ea0e410-4589-adcf-3dfe-1553cb85dadc@oracle.com> Hi Magnus, On 4/11/2020 6:50 am, Magnus Ihse Bursie wrote: > The man pages in src/$module/man/*.1 is generated from the (unfortunately still) closed markdown sources. > > These needs to be updated for JDK 16. During this updating, a typo in the GPL header is also fixed (which caused Oracle's internal header verification tool to complain) -- a double space had been collapsed into a single space. > > There is a certain possibility that man page content can still change for JDK 16. If that is the case, the export needs to be re-done. (In contrast to prior exports, we now have a standard way of doing this, so it's trivial to update.) I'd say there is a certainty that some man page content will still be updated before FC. Last time we at least waited until after RDP1 before doing the bulk update. And I'm not sure this is really something that should be classified as a build task that only folk on build-dev will see. Cheers, David > ------------- > > Commit messages: > - 8255853: Update all nroff manpages for JDK 16 release > > Changes: https://git.openjdk.java.net/jdk/pull/1043/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1043&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8255853 > Stats: 334 lines in 28 files changed: 202 ins; 25 del; 107 mod > Patch: https://git.openjdk.java.net/jdk/pull/1043.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/1043/head:pull/1043 > > PR: https://git.openjdk.java.net/jdk/pull/1043 > From sspitsyn at openjdk.java.net Wed Nov 4 02:19:00 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 02:19:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 00:08:10 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from Kim and Albert. Hi Coleen, Wow, there are a lot of simplifications and code removal with this fix! It looks great in general, just some nits below. I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :) src/hotspot/share/prims/jvmtiTagMap.hpp: Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? : `+ void post_dead_object_on_vm_thread();` src/hotspot/share/prims/jvmtiTagMap.cpp: Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk: 122 void JvmtiTagMap::check_hashmap(bool post_events) { . . . 143 // This checks for posting and rehashing and is called from the heap walks. 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() { I'm just curious how this fragment was added. Did you get any failures in testing? : 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) { 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj), 1041 obj->klass()->external_name()); 1042 return; 1043 } Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? : `1186 bool _some_dead;` Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts. 2996 if (!is_vm_thread) { 2997 if (num_dead_entries != 0) { 2998 JvmtiEnvIterator it; 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { 3000 JvmtiTagMap* tag_map = env->tag_map_acquire(); 3001 if (tag_map != NULL) { 3002 // Lock each hashmap from concurrent posting and cleaning 3003 tag_map->unlink_and_post_locked(); 3004 } 3005 } 3006 // there's another callback for needs_rehashing 3007 } 3008 } else { 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be"); 3010 JvmtiEnvIterator it; 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { 3012 JvmtiTagMap* tag_map = env->tag_map_acquire(); 3013 if (tag_map != NULL && !tag_map->is_empty()) { 3014 if (num_dead_entries != 0) { 3015 tag_map->hashmap()->unlink_and_post(tag_map->env()); 3016 } 3017 // Later GC code will relocate the oops, so defer rehashing until then. 3018 tag_map->_needs_rehashing = true; 3019 } It feels like it can be refactored/simplified, at least, a little bit. Is it possible to check and just return if (num_dead_entries == 0)? If not, then, at least, it can be done the same way (except of locking). Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts. src/hotspot/share/utilities/hashtable.cpp: Nit: Need space after the '{' : `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;` src/hotspot/share/prims/jvmtiTagMapTable.cpp: Nit: Extra space after assert: `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");` Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From sspitsyn at openjdk.java.net Wed Nov 4 05:40:03 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 05:40:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 02:15:52 GMT, Serguei Spitsyn wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from Kim and Albert. > > Hi Coleen, > > Wow, there are a lot of simplifications and code removal with this fix! > It looks great in general, just some nits below. > I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :) > > src/hotspot/share/prims/jvmtiTagMap.hpp: > > Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? : > `+ void post_dead_object_on_vm_thread();` > > src/hotspot/share/prims/jvmtiTagMap.cpp: > > Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk: > 122 void JvmtiTagMap::check_hashmap(bool post_events) { > . . . > 143 // This checks for posting and rehashing and is called from the heap walks. > 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() { > > I'm just curious how this fragment was added. Did you get any failures in testing? : > 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded > 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) { > 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj), > 1041 obj->klass()->external_name()); > 1042 return; > 1043 } > > Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? : > `1186 bool _some_dead;` > > Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts. > 2996 if (!is_vm_thread) { > 2997 if (num_dead_entries != 0) { > 2998 JvmtiEnvIterator it; > 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > 3000 JvmtiTagMap* tag_map = env->tag_map_acquire(); > 3001 if (tag_map != NULL) { > 3002 // Lock each hashmap from concurrent posting and cleaning > 3003 tag_map->unlink_and_post_locked(); > 3004 } > 3005 } > 3006 // there's another callback for needs_rehashing > 3007 } > 3008 } else { > 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be"); > 3010 JvmtiEnvIterator it; > 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > 3012 JvmtiTagMap* tag_map = env->tag_map_acquire(); > 3013 if (tag_map != NULL && !tag_map->is_empty()) { > 3014 if (num_dead_entries != 0) { > 3015 tag_map->hashmap()->unlink_and_post(tag_map->env()); > 3016 } > 3017 // Later GC code will relocate the oops, so defer rehashing until then. > 3018 tag_map->_needs_rehashing = true; > 3019 } > It feels like it can be refactored/simplified, at least, a little bit. > Is it possible to check and just return if (num_dead_entries == 0)? > If not, then, at least, it can be done the same way (except of locking). > Q: Should the _needs_rehashing be set in both contexts? > > Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts. > > src/hotspot/share/utilities/hashtable.cpp: > > Nit: Need space after the '{' : > `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;` > > src/hotspot/share/prims/jvmtiTagMapTable.cpp: > > Nit: Extra space after assert: > `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");` > > Thanks, > Serguei More about possible refactoring of the JvmtiTagMap::gc_notification(). I'm thinking about something like below: void JvmtiTagMap::unlink_and_post_for_all_envs() { if (num_dead_entries == 0) { return; // nothing to unlink and post } JvmtiEnvIterator it; for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { JvmtiTagMap* tag_map = env->tag_map_acquire(); if (tag_map != NULL && !tag_map->is_empty()) { tag_map->unlink_and_post(); } } } void JvmtiTagMap::gc_notification(size_t num_dead_entries) { if (Thread::current()->is_VM_thread()) { assert(SafepointSynchronize::is_at_safepoint(), "must be"); unlink_and_post_for_all_envs(); set_needs_rehashing(); } else { MutexLocker ml(JvmtiTagMap_lock(), Mutex::_no_safepoint_check_flag); unlink_and_post_for_all_envs(); // there's another callback for needs_rehashing } } If we still need a lock per each JvmtiTagMap then it is possible to add this fragment to the unlink_and_post_for_all_envs: bool is_vm_thread = Thread::current()->is_VM_thread() MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); Then the code above could look like below: void JvmtiTagMap::unlink_and_post_for_all_envs() { if (num_dead_entries == 0) { return; // nothing to unlink and post } bool is_vm_thread = Thread::current()->is_VM_thread() JvmtiEnvIterator it; for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { JvmtiTagMap* tag_map = env->tag_map_acquire(); if (tag_map != NULL && !tag_map->is_empty()) { MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); tag_map->unlink_and_post(); } } } void JvmtiTagMap::gc_notification(size_t num_dead_entries) { if (Thread::current()->is_VM_thread()) { assert(SafepointSynchronize::is_at_safepoint(), "must be"); set_needs_rehashing(); } unlink_and_post_for_all_envs(); } ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From david.holmes at oracle.com Wed Nov 4 05:56:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Nov 2020 15:56:34 +1000 Subject: RFR: JDK-8255711: Fix and unify hotspot signal handlers In-Reply-To: References: Message-ID: <7888a9d2-a3b9-9981-665f-1cb3181d199d@oracle.com> Hi Thomas, Some initial comments as this is quite big - but thanks for all the detailed explanations. On 4/11/2020 1:57 am, Thomas Stuefe wrote: > Hi all, > > may I please have opinions and reviews on this cleanup-fix-patch for the hotspot signal handlers. > > Its main intent is to simplify coding and to commonize some of it across all Posix platforms where possible. Also to fix a number of smaller issues. > > This will have a number of benefits, mainly easing maintenance pain for porters and reducing bitrot for platform dependent code. > > This all builds upon the work @gerard-ziemski did with https://bugs.openjdk.java.net/browse/JDK-8252324. > > ---- > > This cleanup was made more complicated by the fact that there exists a non-obvious and undocumented way for a third party app to chain signal handlers (beside the documented one of using libjsig). It seems that the JVM_handle_xxx_functions() are in fact interfaces for third party coding to invoke hotspot signal handling. This only makes sense in combination with `-XX:+AllowUserSignalHandlers`. A cursory github search revealed that this flag is used quite a bit. Thanks for doing that search. > See a more in-depth discussion here: [4]. Thanks to @dholmes-ora for untangling this bit of history. > > Unfortunately there is no official documentation from Sun or Oracle, and zero regression tests. So I try to preserve this interface as best as I can. I plan to add a proper regression test with a later change, but for now I don't have the time for that. On the documentation front we could at least explain what the flag really does. Rather than saying: product(bool, AllowUserSignalHandlers, false, \ "Do not complain if the application installs signal handlers " we could say something like: product(bool, AllowUserSignalHandlers, false, \ "Allow the application to install the primary signal handlers instead of the JVM." \ and we (I?) could update the java manpage. > > --- > > The fixed issues: > > 1) `PosixSignals::_are_signal_handlers_installed` is used inside the platform handlers to guard a part of the platform handlers against execution in case the signal handlers are not yet installed. > > Initially this confused me since when this handler is called it would of course be installed. So that boolean would always be true. The only explanation I found was that since these handlers can be invoked directly from outside, this is some (ineffective) form of guard against calling this handler too early. > But that guard can be left out and that boolean removed. Our signal handlers are safe to call before VM initialization is completed. The code that is guarded by this check is implicitly safe as it is trivial and early in VM init the current thread will be null anyway and so we won't execute the guarded code. > 2) The return code of JVM_handle_xxx_signal() was inconsistently set (some as bool, some as int) as well as unused in normal code paths (excluding outside calls). Ok. > 3) JVM_handle_xxx_signal are supposed to be exported, but on AIX there is a day-zero bug which caused it to not be exported. > > 4) Every platform handler has this section: > > JavaThread* thread = NULL; > VMThread* vmthread = NULL; > if (PosixSignals::are_signal_handlers_installed()) { > if (t != NULL ){ > if(t->is_Java_thread()) { > thread = t->as_Java_thread(); > } > else if(t->is_VM_thread()){ > vmthread = (VMThread *)t; > } > } > } > > `vmthread` is unused on all platforms and can be removed. Ok. Once upon a time we probably did something different if in the vmThread versus some other non-JavaThread. > 5) Every platform handler has some variant of this section, to ignore SIGPIPE, SIGXFSZ (whose default actions are to terminate the VM): > > if (sig == SIGPIPE || sig == SIGXFSZ) { > // allow chained handler to go first > if (PosixSignals::chained_handler(sig, info, ucVoid)) { > return true; > } else { > // Ignoring SIGPIPE/SIGXFSZ - see bugs 4229104 or 6499219 > return true; > } > } > > - On s390 and ppc, we miss SIGXFSZ handling > _Update: Fixed separately for easier backport, see [https://bugs.openjdk.java.net/browse/JDK-8255734](JDK-8255734)_. > - both paths return true - section can be shortened > > Side note: having handlers for those signals may be unnecessary. We could just set the signal handler to `SIG_IGN`. We would have to tiptoe around any third party handlers for those signals, but it still may be simpler. > > 6) At the end of every platform header, before calling into fatal error handling, we unblock the signal: > >> // unmask current signal >> sigset_t newset; >> sigemptyset(&newset); >> sigaddset(&newset, sig); >> sigprocmask(SIG_UNBLOCK, &newset, NULL); >> > > - Use of `sigprocmask()` is UB in a multithreaded program. > - but then, this section is unnecessary anyway, since [https://bugs.openjdk.java.net/browse/JDK-8252533](JDK-8252533) we unmask error signals at the start of the signal handler. But is this guaranteed to be one of the error signals? What if the application calls our handler for some other signal? I guess that is their problem. > 7) the JFR crash protection is not consistently checked in all platform handlers. > > 8) On Zero, when entering fatal error handling, we do so via fatal() instead of VMError::report_and_die(), thereby discarding the real crash context and obfuscating the register content in the hs-err file (we still see registers, but those stem from the assertion-poison-page mechanism). > > 9) on Linux ppc64 and AIX, we have this section: > >> if (sig == SIGILL && (pc < (address) 0x200)) { >> goto report_and_die; >> } > > which is related to the fact that the zero page on AIX is readable, filled with 0, and reading instructions from it will yield us a SIGILL, not a SIGSEGV (0 is not a noop on PPC, so we don't nop-slide). > > This coding is irrelevant on Linux. On AIX, it can also be removed, since this SIGILL would be unrecognized by the hotspot and later count as fatal error anyway. > > 10) When invoking the fatal error handler, we extract the pc from the context and hand it over as "faulting pc". For SIGILL and SIGFPE, this is not totally correct. According to POSIX [3], for those signals the address of the faulting instruction is handed over in `si_info.si_addr`. > > On most platforms this does not matter, they are the same. But on some architectures the pc in the signal context actually points somewhere else, e.g. beyond the faulting instruction. Therefore `si_info.si_addr` is the better choice. The Posix spec also states "For some implementations, the value of si_addr may be inaccurate." - so I'm not at all sure which "pc" we should be trusting here? I thought the ucontext was the detailed platform specific "context" object that we should extract information from. Which architectures give different values in the two and is there some documentation stating what happens for any given os/cpu? > ---- > > The changes in this patch: > > a) hotspot signal handling is now done by the following functions: > > > | | > v v > javaSignalHandler JVM_handle_linux_signal() > | / > v v > javaSignalHandler_inner Not clear why we need the _inner version. Why can't we just have javaSignalHandler which is installed as the handler and which is called by JVM_handle_XXX_signal? > | > v > PosixSignals::pd_hotspot_signal_handler() > > > The right branch only exists to support the `AllowUserSignalHandlers` case, see [4]. > > `javaSignalHandler` is registered as handler, as it was before; JVM_handle_linux_signal() is exported as before. > `javaSignalHandler_inner` contains the shared portion of the signal handler; `PosixSignals::pd_hotspot_signal_handler()` contains the remaining platform dependent portions. > > > b) I commonized prologue- and epilogue coding. > - I simplified (4) to a single line in the shared handler > - I moved the JFR thread crash protection (7) up to the shared handler > - I moved the complete epilogue up to the shared handler. That includes calling the chained handlers, should they exist, as well as invoking the fatal error handler. That fixes (8) and (6) > - Zero has this tradition of showing a robot telling a cat about the error signal, which I like, and kept. > - I simplified (5) and commonized it, and removed (9) completely > - In PosixSignals::install_signal_handlers(), I removed the `signal_handlers_are_installed` guard and replaced it with an assert. Unfortunately this causes lots of indentation changes. @gerard-ziemski: if this clashes too much with your patch for JDK-8253742, I'll leave that part out. > > Thanks for reviewing. > > Testing: this patch ran through our nightlies, in an earlier form. They will be re-ran some more times. > > I'd be happy if aarch64 porters could take a look at the aarch64 portion of this change. > > Please note that I had to draw a line somewhere - this is an open ended issue and a lot more could be cleaned. See also Gerard's work on [5], which is under review too. :) I've actually gone through this in far more detail as I've been composing this email and overall it is looking very good. I've made a couple of comments directly in the PR, in addition to the above. Thanks, David ----- > > ---- > > [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-October/043145.html > [2] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-October/043191.html > [3] https://pubs.opengroup.org/onlinepubs/009695399/basedefs/signal.h.html > [4] https://mail.openjdk.java.net/pipermail/jdk-dev/2020-November/004887.html > [5] https://bugs.openjdk.java.net/browse/JDK-8253742 > > ------------- > > Commit messages: > - Initial patch > > Changes: https://git.openjdk.java.net/jdk/pull/1034/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1034&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8255711 > Stats: 916 lines in 13 files changed: 116 ins; 686 del; 114 mod > Patch: https://git.openjdk.java.net/jdk/pull/1034.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/1034/head:pull/1034 > > PR: https://git.openjdk.java.net/jdk/pull/1034 > From alanb at openjdk.java.net Wed Nov 4 07:47:55 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Wed, 4 Nov 2020 07:47:55 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) In-Reply-To: References: Message-ID: <-u4eSM47o_e_KlfTRYBNGyLNhjqAeG-84u_uEd3ppH0=.c49b4269-1001-49d1-96fd-ecacbf2417e9@github.com> On Mon, 2 Nov 2020 11:26:51 GMT, Maurizio Cimadamore wrote: >> I looked through the changes in this update. >> >> The shared memory segment support looks sound and the mechanism to close a shared memory segment is clever (albeit a bit surprising at first that it does global handshake to look for a frame in a scoped region. Also surprising that close can cause failure at both ends - it took me a while to see that this is pragmatic approach). >> >> The share method specifies NPE if thread == null but there is no thread parameter, is this a cut 'n paste error? Another one in registerCleaner where it should be NPE if the cleaner is null. >> >> I think the javadoc for the close method needs to be a bit clearer on the state of the memory segment when IllegalStateException is thrown. Will it be marked "not alive" when it fails? Does this mean there is a resource leak? I think an apiNote to explain the rational for why close is not idempotent is also needed, or maybe it should be re-visited so that close is a no-op when the memory segment is not alive. >> >> Now that MemorySegment is AutoCloseable then maybe the term "alive" should be replaced with "open" or "closed" and isAlive replaced with isOpen is isClosed. >> >> FileDescriptor can be attraction nuisance and forced reference counting everywhere that it is used. Is it needed? Could an isMapped method work instead? >> >> mapFromPath was in the second preview but I think the method name should be re-examined as it maps a file, the path just locates the file. Naming is subjectives but in this case using "map" or "mapFile" would fit beside the allocateNative methods. >> >> MappedMemorySegments. The force method specifies a write back guarantee but at the same time, the implNote in the class description suggests that the methods might be a no-op. You might want to adjust the wording to avoid any suggestion that force might be a no-op. >> >> The javadoc for copyFrom isn't changed in this update but I notice it specifies IndexOutOfBoundException when the source segment is larger than the receiver, have other exceptions been examined? >> >> I don't have any any comments on MemoryAccess except that it's not immediately clear why there are "Byte" methods that take a ByteOrder. Make sense for the multi-byte types of course. >> >> The updates the java/nio sources look okay but it would be helpful if the really long lines could be chopped down as it's just too hard to do side-by-side reviews when the lines are so long. A minor nit but the changes X-Buffer.java.template mess up the alignment of the parameters to copyMemory/copySwapMemory methods. > >> The javadoc for copyFrom isn't changed in this update but I notice it specifies IndexOutOfBoundException when the source segment is larger than the receiver, have other exceptions been examined? > > This exception is consistent with other uses of this exception throughout this API (e.g. when writing a segment out of bounds). I assume the CSR needs to be updated so that it's in sync with the API changes in the latest round. ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From kbarrett at openjdk.java.net Wed Nov 4 09:10:27 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:10:27 GMT Subject: RFR: 8188055: (ref) Add Reference::refersTo predicate [v7] In-Reply-To: References: Message-ID: > Finally returning to this review that was started in April 2020. I've > recast it as a github PR. I think the security concern raised by Gil > has been adequately answered. > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-April/029203.html > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-July/030401.html > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-August/030677.html > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-September/030793.html > > Please review a new function: java.lang.ref.Reference.refersTo. > > This function is needed to test the referent of a Reference object without > artificially extending the lifetime of the referent object, as may happen > when calling Reference.get. Some garbage collectors require extending the > lifetime of a weak referent when accessed, in order to maintain collector > invariants. Lifetime extension may occur with any collector when the > Reference is a SoftReference, as calling get indicates recent access. This > new function also allows testing the referent of a PhantomReference, which > can't be accessed by calling get. > > The new function uses native methods whose implementations are in the VM so > they can use the Access API. It is the intent that these methods will be > intrinsified by optimizing compilers like C2 or graal, but that hasn't been > implemented yet. Bear that in mind before rushing off to change existing > uses of Reference.get. > > There are two native methods involved, one in Reference and an override in > PhantomReference, both package private in java.lang.ref. The reason for this > split is to simplify the intrinsification. This is a change from the version > from April 2020; that version had a single native method in Reference, > implemented using the ON_UNKNOWN_OOP_REF Access reference strength category. > However, adding support for that category in the compilers adds significant > implementation effort and complexity. Splitting avoids that complexity. > > Testing: > mach5 tier1 > Locally (linux-x64) verified the new test passes with various garbage collectors. Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 13 additional commits since the last revision: - Merge branch 'master' into refersto - improve wording in refersTo javadoc - Merge branch 'master' into refersto - More explicit refersTo0 comment. - simplify test - cleanup nits from Mandy - use Object instead of TestObject - improve refersTo0 descriptions - basic functional test - change referent access - ... and 3 more: https://git.openjdk.java.net/jdk/compare/f06d7348...79277ff3 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/498/files - new: https://git.openjdk.java.net/jdk/pull/498/files/3a15b6a9..79277ff3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=498&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=498&range=05-06 Stats: 90837 lines in 1555 files changed: 63919 ins; 19502 del; 7416 mod Patch: https://git.openjdk.java.net/jdk/pull/498.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/498/head:pull/498 PR: https://git.openjdk.java.net/jdk/pull/498 From kbarrett at openjdk.java.net Wed Nov 4 09:23:00 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:23:00 GMT Subject: Integrated: 8188055: (ref) Add Reference::refersTo predicate In-Reply-To: References: Message-ID: On Sun, 4 Oct 2020 03:59:59 GMT, Kim Barrett wrote: > Finally returning to this review that was started in April 2020. I've > recast it as a github PR. I think the security concern raised by Gil > has been adequately answered. > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-April/029203.html > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-July/030401.html > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-August/030677.html > https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-September/030793.html > > Please review a new function: java.lang.ref.Reference.refersTo. > > This function is needed to test the referent of a Reference object without > artificially extending the lifetime of the referent object, as may happen > when calling Reference.get. Some garbage collectors require extending the > lifetime of a weak referent when accessed, in order to maintain collector > invariants. Lifetime extension may occur with any collector when the > Reference is a SoftReference, as calling get indicates recent access. This > new function also allows testing the referent of a PhantomReference, which > can't be accessed by calling get. > > The new function uses native methods whose implementations are in the VM so > they can use the Access API. It is the intent that these methods will be > intrinsified by optimizing compilers like C2 or graal, but that hasn't been > implemented yet. Bear that in mind before rushing off to change existing > uses of Reference.get. > > There are two native methods involved, one in Reference and an override in > PhantomReference, both package private in java.lang.ref. The reason for this > split is to simplify the intrinsification. This is a change from the version > from April 2020; that version had a single native method in Reference, > implemented using the ON_UNKNOWN_OOP_REF Access reference strength category. > However, adding support for that category in the compilers adds significant > implementation effort and complexity. Splitting avoids that complexity. > > Testing: > mach5 tier1 > Locally (linux-x64) verified the new test passes with various garbage collectors. This pull request has now been integrated. Changeset: 6023f6b1 Author: Kim Barrett URL: https://git.openjdk.java.net/jdk/commit/6023f6b1 Stats: 501 lines in 13 files changed: 488 ins; 0 del; 13 mod 8188055: (ref) Add Reference::refersTo predicate Reviewed-by: mchung, pliden, rriggs, dholmes, ihse, smarks, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/498 From shade at openjdk.java.net Wed Nov 4 09:23:59 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 4 Nov 2020 09:23:59 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c [v2] In-Reply-To: References: Message-ID: <0kIzU7Og4dKHYW1mYDOFf83yXJ4eUwnNax-ljOF2ap0=.ad40cac6-689d-4d27-98a7-77aa24d291d2@github.com> On Tue, 3 Nov 2020 20:37:07 GMT, Jorn Vernee wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > Jorn Vernee has updated the pull request incrementally with two additional commits since the last revision: > > - Collapse both _LP64 if blocks in flags-cflags.m4 > - Use jlong.h macros instead of spinning new ones Yup, using `jlong.h` looks much better. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1017 From tvaleev at openjdk.java.net Wed Nov 4 09:34:11 2020 From: tvaleev at openjdk.java.net (Tagir F.Valeev) Date: Wed, 4 Nov 2020 09:34:11 GMT Subject: RFR: 8188055: (ref) Add Reference::refersTo predicate [v6] In-Reply-To: References: <0dhF_xxcp1VoUowwdZenB2qWa9ILcZjTMe3lsaRrg7k=.3c633db8-f745-4353-ad34-a64fbc96d4e0@github.com> Message-ID: On Wed, 28 Oct 2020 15:56:48 GMT, Alan Bateman wrote: >> Kim Barrett has updated the pull request incrementally with one additional commit since the last revision: >> >> improve wording in refersTo javadoc > > The API looks good, thanks for getting this in. Hello! As an IDE developer, I'm thinking about IDE inspection that may suggest the new method. My idea is to suggest replacing every `ref.get() == obj` with `ref.refersTo(obj)`. Is this a good idea or there are cases when `ref.get() == obj` could be preferred? What do you think? ------------- PR: https://git.openjdk.java.net/jdk/pull/498 From simonis at openjdk.java.net Wed Nov 4 09:34:05 2020 From: simonis at openjdk.java.net (Volker Simonis) Date: Wed, 4 Nov 2020 09:34:05 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c [v2] In-Reply-To: References: <5RiTHvRKDKf0_Zr2ZQICloKttbvZc_osDtJ_ylpxfrw=.7fe27b37-78fa-47fc-b37b-4ecde950818a@github.com> Message-ID: On Tue, 3 Nov 2020 20:12:35 GMT, Magnus Ihse Bursie wrote: >> Ok, will do. (FWIW, you can expand the context of the diff with the arrow buttons on the left side of the view. Above or below the line numbers) > > (Yes, I know. I just didn't think that doing so would reveal anything about AIX. I just wish I had gotten a bit more context automatically.) Sorry, but I'm neither workin on AIX any more nor do I have access to an AIX machine :) Pulling in @TheRealMDoerr or @tstuefe ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From kbarrett at openjdk.java.net Wed Nov 4 09:36:02 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> Message-ID: On Wed, 4 Nov 2020 07:41:39 GMT, Kim Barrett wrote: >>>Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. >> >> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. > >> Ok, so I'm not sure what to do with this: >> >> enum Phase { >> // Serial phase. >> JVMTI_ONLY(jvmti) >> // Additional implicit phase values follow for oopstorages. >> `};` >> >> I've removed the only thing in this enum. > > Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums. > > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. > > This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. I think the deletion stopped at the wrong place; it either went too far, or not far enough. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:02 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> Message-ID: On Tue, 3 Nov 2020 23:38:08 GMT, Coleen Phillimore wrote: >> Ok, so I'm not sure what to do with this: >> >> enum Phase { >> // Serial phase. >> JVMTI_ONLY(jvmti) >> // Additional implicit phase values follow for oopstorages. >> `};` >> >> I've removed the only thing in this enum. > >>Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. > > This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. > Ok, so I'm not sure what to do with this: > > enum Phase { > // Serial phase. > JVMTI_ONLY(jvmti) > // Additional implicit phase values follow for oopstorages. > `};` > > I've removed the only thing in this enum. Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:03 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:03 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:31:35 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 2979: >> >>> 2977: >>> 2978: // Concurrent GC needs to call this in relocation pause, so after the objects are moved >>> 2979: // and have their new addresses, the table can be rehashed. >> >> I think the comment is confusing and wrong. The requirement is that the collector must call this before exposing moved objects to the mutator, and must provide the to-space invariant. (This whole design would not work with the old Shenandoah barriers without additional work. I don't know if tagmaps ever worked at all for them? Maybe they added calls to Access<>::resolve (since happily deceased) to deal with that?) I also think there are a bunch of missing calls; piggybacking on the num-dead callback isn't correct (see later comment about that). > > So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case? I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.) >> src/hotspot/share/prims/jvmtiTagMap.cpp line 3015: >> >>> 3013: if (tag_map != NULL && !tag_map->is_empty()) { >>> 3014: if (num_dead_entries != 0) { >>> 3015: tag_map->hashmap()->unlink_and_post(tag_map->env()); >> >> Why are we doing this in the callback, rather than just setting a flag? I thought part of the point of this change was to get tagmap processing out of GC pauses. The same question applies to the non-safepoint side. The idea was to be lazy about updating the tagmap, waiting until someone actually needed to use it. Or if more prompt ObjectFree notifications are needed then signal some thread (maybe the service thread?) for followup. > > The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table. I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:04 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 00:08:10 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Code review comments from Kim and Albert. src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36: > 34: class JvmtiTagMapEntryClosure; > 35: > 36: class JvmtiTagMapEntry : public HashtableEntry { By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%). It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:36:04 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:36:04 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 07:52:12 GMT, Kim Barrett wrote: >> So the design is that when the oops have new addresses, we set a flag in the table to rehash it. Not sure why this is wrong and why wouldn't it work for shenandoah? @zhengyu123 ? When we call WeakHandle.peek()/resolve() after the call, the new/moved oop address should be returned. Why wouldn't this be the case? > > I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.) It used to be that jvmti tagmap processing was all-in-one (in GC weak reference processing, with the weak clearing, dead table entry removal, and rehashing all done in one pass. This change has split that up, with the weak clearing happening in a different place (still as part of the GC's weak reference processing) than the others (which I claim can now be part of the mutator, whether further separated or not). "Concurrent GC" has nothing to do with whether tagmaps need rehashing. Any copying collector needs to do so. A non-copying collector (whether concurrent or not) would not. (We don't have any of those in HotSpot today.) And weak reference clearing (whether concurrent or not) has nothing to do with whether objects have been moved and so the hashing has been invalidated. There's also a "well known" issue with address-based hashing and generational or similar collectors, where a simple boolean "objects have moved" flag can be problematic, and which tagmaps seem likely to be prone to. The old do_weak_oops tries to mitigate it by recognizing when the object didn't move. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Wed Nov 4 09:38:00 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 09:38:00 GMT Subject: RFR: 8188055: (ref) Add Reference::refersTo predicate [v6] In-Reply-To: References: <0dhF_xxcp1VoUowwdZenB2qWa9ILcZjTMe3lsaRrg7k=.3c633db8-f745-4353-ad34-a64fbc96d4e0@github.com> Message-ID: On Wed, 4 Nov 2020 09:31:13 GMT, Tagir F. Valeev wrote: >> The API looks good, thanks for getting this in. > > Hello! > > As an IDE developer, I'm thinking about IDE inspection that may suggest the new method. My idea is to suggest replacing every `ref.get() == obj` with `ref.refersTo(obj)`. Is this a good idea or there are cases when `ref.get() == obj` could be preferred? What do you think? Thanks to a whole host of folks for reviews and comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/498 From kbarrett at openjdk.java.net Wed Nov 4 10:08:00 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 10:08:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> On Tue, 3 Nov 2020 21:40:39 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 127: >> >>> 125: // The table cleaning, posting and rehashing can race for >>> 126: // concurrent GCs. So fix it here once we have a lock or are >>> 127: // at a safepoint. >> >> I think this comment and the one below about locking are confused, at least about rehashing. I _think_ this is referring to concurrent num-dead notification? I've already commented there about it being a problem to do the unlink &etc in the GC pause (see later comment). It also seems like a bad idea to be doing this here and block progress by a concurrent GC because we're holding the tagmap lock for a long time, which is another reason to not have the num-dead notification do very much (and not require a lock that might be held here for a long time). > > The comment is trying to describe the situation like: > 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked) > 2. safepoint for heap walk > 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A. > 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened > > The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable. > > Both operations require the table to be locked. > > The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints. For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing, an d so still effectively be a serial subtask after all the parallel subtasks are done or mostly done. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From mdoerr at openjdk.java.net Wed Nov 4 10:12:57 2020 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Wed, 4 Nov 2020 10:12:57 GMT Subject: RFR: 8255128: linux x86 build failure with libJNIPoint.c [v2] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 20:37:07 GMT, Jorn Vernee wrote: >> Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c >> >> This removes the reported warning. >> >> Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). >> >> CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. >> >> Testing: tier1-3 >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8232022 > > Jorn Vernee has updated the pull request incrementally with two additional commits since the last revision: > > - Collapse both _LP64 if blocks in flags-cflags.m4 > - Use jlong.h macros instead of spinning new ones Builds on AIX. ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From kbarrett at openjdk.java.net Wed Nov 4 10:16:59 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 4 Nov 2020 10:16:59 GMT Subject: RFR: 8188055: (ref) Add Reference::refersTo predicate [v6] In-Reply-To: References: <0dhF_xxcp1VoUowwdZenB2qWa9ILcZjTMe3lsaRrg7k=.3c633db8-f745-4353-ad34-a64fbc96d4e0@github.com> Message-ID: On Wed, 4 Nov 2020 09:31:13 GMT, Tagir F. Valeev wrote: > Hello! > > As an IDE developer, I'm thinking about IDE inspection that may suggest the new method. My idea is to suggest replacing every `ref.get() == obj` with `ref.refersTo(obj)`. Is this a good idea or there are cases when `ref.get() == obj` could be preferred? What do you think? Those have different behaviors when ref's class overrides get. Sometimes that might be intentional (PhantomReference, where get blocks access to the referent, and SoftReference, where get may update heuristics for recent accesses delaying GC clearing). But if some further subclass overrides get for some reason, such a change might not be appropriate. ------------- PR: https://git.openjdk.java.net/jdk/pull/498 From jlahoda at openjdk.java.net Wed Nov 4 10:35:02 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Wed, 4 Nov 2020 10:35:02 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> Message-ID: On Mon, 2 Nov 2020 18:39:59 GMT, Jonathan Gibbons wrote: >> Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: >> >> - Removing trailing whitespace. >> - Merging master into JDK-8250768. >> - Updating tests after records are a final feature. >> - Fixing tests. >> - Finalizing removal of record preview hooks. >> - Merging master into JDK-8250768 >> - Reflecting review comments. >> - Merge branch 'master' into JDK-8250768 >> - Removing unnecessary cast. >> - Using a more correct way to get URLs. >> - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 > > src/jdk.compiler/share/classes/com/sun/tools/javac/api/JavacTaskPool.java line 257: > >> 255: //when patching modules (esp. java.base), it may be impossible to >> 256: //clear the symbols read from the patch path: >> 257: polluted |= get(JavaFileManager.class).hasLocation(StandardLocation.PATCH_MODULE_PATH); > > OK, but looks unrelated to primary work The tests need to inject preview APIs into compilation, and without this tweak, the incorrect values are re-used for tests that use the javac pool. So it is related in the sense it is needed for the tests to pass. ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Wed Nov 4 10:42:59 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Wed, 4 Nov 2020 10:42:59 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> Message-ID: On Mon, 2 Nov 2020 19:37:39 GMT, Jonathan Gibbons wrote: >> Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: >> >> - Removing trailing whitespace. >> - Merging master into JDK-8250768. >> - Updating tests after records are a final feature. >> - Fixing tests. >> - Finalizing removal of record preview hooks. >> - Merging master into JDK-8250768 >> - Reflecting review comments. >> - Merge branch 'master' into JDK-8250768 >> - Removing unnecessary cast. >> - Using a more correct way to get URLs. >> - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 > > src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/builders/AnnotationTypeRequiredMemberBuilder.java line 156: > >> 154: buildSignature(annotationDocTree); >> 155: buildDeprecationInfo(annotationDocTree); >> 156: buildPreviewInfo(annotationDocTree); > > (Just checking) I presume this behavior is inherited into `AnnotationTypeOptionalMemberBuilder` so no changes required there. Right, I think so. ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Wed Nov 4 10:59:02 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Wed, 4 Nov 2020 10:59:02 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> Message-ID: On Mon, 2 Nov 2020 19:41:08 GMT, Jonathan Gibbons wrote: >> Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: >> >> - Removing trailing whitespace. >> - Merging master into JDK-8250768. >> - Updating tests after records are a final feature. >> - Fixing tests. >> - Finalizing removal of record preview hooks. >> - Merging master into JDK-8250768 >> - Reflecting review comments. >> - Merge branch 'master' into JDK-8250768 >> - Removing unnecessary cast. >> - Using a more correct way to get URLs. >> - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 > > src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets.properties line 213: > >> 211: doclet.Preview=Preview. >> 212: doclet.Properties=Properties >> 213: doclet.constructors=constructors > > Is the period after `Preview` intentional? It seems inconsistent. This is consistent with doclet.Deprecated key a few lines up. In the javadoc it then says something along these lines: Preview. A binding pattern tree ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Wed Nov 4 11:04:04 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Wed, 4 Nov 2020 11:04:04 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> Message-ID: On Mon, 2 Nov 2020 19:59:10 GMT, Jonathan Gibbons wrote: >> Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: >> >> - Removing trailing whitespace. >> - Merging master into JDK-8250768. >> - Updating tests after records are a final feature. >> - Fixing tests. >> - Finalizing removal of record preview hooks. >> - Merging master into JDK-8250768 >> - Reflecting review comments. >> - Merge branch 'master' into JDK-8250768 >> - Removing unnecessary cast. >> - Using a more correct way to get URLs. >> - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 > > test/langtools/jdk/javadoc/doclet/testPreview/api/preview/Core.java line 28: > >> 26: import jdk.internal.javac.PreviewFeature.Feature; >> 27: >> 28: @PreviewFeature(feature=Feature.TEST) > > Yeah, I remember `Feature.TEST` from earlier. I guess it's OK for now, as a workaround for a testing a feature which is inherently, by design, a moving target across releases. > > These days, javadoc tests are trending towards generating small sample test programs, instead of having small static side-files dominated by a legal header. I wonder if there is a possibility of having a "generator class" in the `javadoc.tester` package that can generate sample code using one or more of the current set of preview features, as a way of reducing the need for the TEST feature. I have intentionally added Feature.TEST to improve testability. Before, tests were using one of the constants (typically whatever was the first constant), but that seems somewhat problematic - what if (at some point, transiently) we have no preview features? ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From coleenp at openjdk.java.net Wed Nov 4 12:21:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 4 Nov 2020 12:21:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 05:37:00 GMT, Serguei Spitsyn wrote: >> Hi Coleen, >> >> Wow, there are a lot of simplifications and code removal with this fix! >> It looks great in general, just some nits below. >> I also wanted to suggest renaming the 'set_needs_processing' to 'set_needs_rehashing'. :) >> >> src/hotspot/share/prims/jvmtiTagMap.hpp: >> >> Nit: Would it better to use a plural form 'post_dead_objects_on_vm_thread'? : >> `+ void post_dead_object_on_vm_thread();` >> >> src/hotspot/share/prims/jvmtiTagMap.cpp: >> >> Nit: It'd be nice to add a short comment before the check_hashmap similar to L143 also explaining a difference (does check and post just for one env) with the check_hashmaps_for_heapwalk: >> 122 void JvmtiTagMap::check_hashmap(bool post_events) { >> . . . >> 143 // This checks for posting and rehashing and is called from the heap walks. >> 144 void JvmtiTagMap::check_hashmaps_for_heapwalk() { >> >> I'm just curious how this fragment was added. Did you get any failures in testing? : >> 1038 // skip if object is a dormant shared object whose mirror hasn't been loaded >> 1039 if (obj != NULL && obj->klass()->java_mirror() == NULL) { >> 1040 log_debug(cds, heap)("skipped dormant archived object " INTPTR_FORMAT " (%s)", p2i(obj), >> 1041 obj->klass()->external_name()); >> 1042 return; >> 1043 } >> >> Nit: Can we rename this field to something like '_some_dead_found' or '_dead_found'? : >> `1186 bool _some_dead;` >> >> Nit: The lines 2997-3007 and 3009-3019 do the same but in different contexts. >> 2996 if (!is_vm_thread) { >> 2997 if (num_dead_entries != 0) { >> 2998 JvmtiEnvIterator it; >> 2999 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { >> 3000 JvmtiTagMap* tag_map = env->tag_map_acquire(); >> 3001 if (tag_map != NULL) { >> 3002 // Lock each hashmap from concurrent posting and cleaning >> 3003 tag_map->unlink_and_post_locked(); >> 3004 } >> 3005 } >> 3006 // there's another callback for needs_rehashing >> 3007 } >> 3008 } else { >> 3009 assert(SafepointSynchronize::is_at_safepoint(), "must be"); >> 3010 JvmtiEnvIterator it; >> 3011 for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { >> 3012 JvmtiTagMap* tag_map = env->tag_map_acquire(); >> 3013 if (tag_map != NULL && !tag_map->is_empty()) { >> 3014 if (num_dead_entries != 0) { >> 3015 tag_map->hashmap()->unlink_and_post(tag_map->env()); >> 3016 } >> 3017 // Later GC code will relocate the oops, so defer rehashing until then. >> 3018 tag_map->_needs_rehashing = true; >> 3019 } >> It feels like it can be refactored/simplified, at least, a little bit. >> Is it possible to check and just return if (num_dead_entries == 0)? >> If not, then, at least, it can be done the same way (except of locking). >> Q: Should the _needs_rehashing be set in both contexts? >> >> Also, can we have just one (static?) lock for the whole gc_notification (not per JVMTI env/JvmtiTagMap)? How much do we win by locking per each env/JvmtiTagMap? Note, that in normal case there is just one agent. It is very rare to have multiple agents requesting object tagging and ObjectFree events. It seems, this can be refactored to more simple code with one function doing work in both contexts. >> >> src/hotspot/share/utilities/hashtable.cpp: >> >> Nit: Need space after the '{' : >> `+const int _small_table_sizes[] = {107, 1009, 2017, 4049, 5051, 10103, 20201, 40423 } ;` >> >> src/hotspot/share/prims/jvmtiTagMapTable.cpp: >> >> Nit: Extra space after assert: >> `119 assert (find(index, hash, obj) == NULL, "shouldn't already be present");` >> >> Thanks, >> Serguei > > More about possible refactoring of the JvmtiTagMap::gc_notification(). > I'm thinking about something like below: > > void JvmtiTagMap::unlink_and_post_for_all_envs() { > if (num_dead_entries == 0) { > return; // nothing to unlink and post > } > JvmtiEnvIterator it; > for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > JvmtiTagMap* tag_map = env->tag_map_acquire(); > if (tag_map != NULL && !tag_map->is_empty()) { > tag_map->unlink_and_post(); > } > } > } > > void JvmtiTagMap::gc_notification(size_t num_dead_entries) { > if (Thread::current()->is_VM_thread()) { > assert(SafepointSynchronize::is_at_safepoint(), "must be"); > unlink_and_post_for_all_envs(); > set_needs_rehashing(); > } else { > MutexLocker ml(JvmtiTagMap_lock(), Mutex::_no_safepoint_check_flag); > unlink_and_post_for_all_envs(); > // there's another callback for needs_rehashing > } > } > > If we still need a lock per each JvmtiTagMap then it is possible to add this fragment to the unlink_and_post_for_all_envs: > bool is_vm_thread = Thread::current()->is_VM_thread() > MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); > > Then the code above could look like below: > > void JvmtiTagMap::unlink_and_post_for_all_envs() { > if (num_dead_entries == 0) { > return; // nothing to unlink and post > } > bool is_vm_thread = Thread::current()->is_VM_thread() > JvmtiEnvIterator it; > for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) { > JvmtiTagMap* tag_map = env->tag_map_acquire(); > if (tag_map != NULL && !tag_map->is_empty()) { > MutexLocker ml(is_vm_thread ? NULL : lock(), Mutex::_no_safepoint_check_flag); > tag_map->unlink_and_post(); > } > } > } > > void JvmtiTagMap::gc_notification(size_t num_dead_entries) { > if (Thread::current()->is_VM_thread()) { > assert(SafepointSynchronize::is_at_safepoint(), "must be"); > set_needs_rehashing(); > } > unlink_and_post_for_all_envs(); > } @sspitsyn Thank you for reviewing the code. The gc_notification refactoring is awkward because each table needs a lock and not a global lock. If we find another place in the GCs to call set_needs_rehashing() it might be possible to make gc_notification call two functions with a boolean to decide whether to take the lock. We're still working on @kimbarrett comments so maybe the notification will change to some new thread and be refactored that way if necessary. I fixed the code for your other comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Wed Nov 4 12:21:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 4 Nov 2020 12:21:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision: - Add back WeakProcessorPhases::Phase enum. - Serguei 1. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/f66ea839..7d3fdf68 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=04-05 Stats: 18 lines in 5 files changed: 7 ins; 0 del; 11 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From shade at openjdk.java.net Wed Nov 4 13:06:05 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 4 Nov 2020 13:06:05 GMT Subject: RFR: 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch Message-ID: Current submit workflow omits `hs_err*.log` and `replay*.log` files, because zip matches the paths, and so only the hs_errs/replays in the current dir are picked up (and there are none). It should be preceded with `*` to match files in subdirs. Incidentally, this is why `*.jtr` match works. Observe: $ mkdir 1/2 -p $ touch 1/2/hs_err_213123.log $ zip -r9 test.zip . -i hs_err* zip warning: zip file empty $ zip -r9 test.zip . -i */hs_err* adding: 1/2/hs_err_213123.log (stored 0%) This was introduced in the initial implementation in JDK-8255352. Additional testing: - [x] Observing current Windows failures package hs_err into archive ------------- Commit messages: - 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch Changes: https://git.openjdk.java.net/jdk/pull/1055/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1055&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255895 Stats: 8 lines in 1 file changed: 0 ins; 0 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/1055.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1055/head:pull/1055 PR: https://git.openjdk.java.net/jdk/pull/1055 From erikj at openjdk.java.net Wed Nov 4 13:32:58 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 4 Nov 2020 13:32:58 GMT Subject: RFR: 8253892: Disable misleading-indentation on clang as well as gcc In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 22:25:08 GMT, Magnus Ihse Bursie wrote: > With clang 10.0, the compiler now detects a new class of warnings. The `misleading-indentation` warning has previously been disabled on gcc for hotspot and libfdlibm. Now we need to disable it for clang as well. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1044 From erikj at openjdk.java.net Wed Nov 4 13:43:55 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 4 Nov 2020 13:43:55 GMT Subject: RFR: 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 12:59:24 GMT, Aleksey Shipilev wrote: > Current submit workflow omits `hs_err*.log` and `replay*.log` files, because zip matches the paths, and so only the hs_errs/replays in the current dir are picked up (and there are none). It should be preceded with `*` to match files in subdirs. Incidentally, this is why `*.jtr` match works. > > Observe: > > $ mkdir 1/2 -p > $ touch 1/2/hs_err_213123.log > $ zip -r9 test.zip . -i hs_err* > zip warning: zip file empty > $ zip -r9 test.zip . -i */hs_err* > adding: 1/2/hs_err_213123.log (stored 0%) > > This was introduced in the initial implementation in JDK-8255352. > > Additional testing: > - [x] Observing current Windows failures package hs_err into archive Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1055 From shade at openjdk.java.net Wed Nov 4 13:46:55 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 4 Nov 2020 13:46:55 GMT Subject: RFR: 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 13:40:49 GMT, Erik Joelsson wrote: >> Current submit workflow omits `hs_err*.log` and `replay*.log` files, because zip matches the paths, and so only the hs_errs/replays in the current dir are picked up (and there are none). It should be preceded with `*` to match files in subdirs. Incidentally, this is why `*.jtr` match works. >> >> Observe: >> >> $ mkdir 1/2 -p >> $ touch 1/2/hs_err_213123.log >> $ zip -r9 test.zip . -i hs_err* >> zip warning: zip file empty >> $ zip -r9 test.zip . -i */hs_err* >> adding: 1/2/hs_err_213123.log (stored 0%) >> >> This was introduced in the initial implementation in JDK-8255352. >> >> Additional testing: >> - [x] Observing current Windows failures package hs_err into archive > > Marked as reviewed by erikj (Reviewer). Thank you, @erikj79. Trivial? (I would like to get is soon, in order to capture more hs_errs for current failures) ------------- PR: https://git.openjdk.java.net/jdk/pull/1055 From erikj at openjdk.java.net Wed Nov 4 14:41:55 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 4 Nov 2020 14:41:55 GMT Subject: RFR: 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 13:43:58 GMT, Aleksey Shipilev wrote: >> Marked as reviewed by erikj (Reviewer). > > Thank you, @erikj79. Trivial? (I would like to get is soon, in order to capture more hs_errs for current failures) I'm fine with a quick integration of this. ------------- PR: https://git.openjdk.java.net/jdk/pull/1055 From shade at openjdk.java.net Wed Nov 4 14:47:57 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 4 Nov 2020 14:47:57 GMT Subject: RFR: 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch In-Reply-To: References: Message-ID: <1iLblTSMB2gGojuPo0ECMsD3XUdwdlmGNEClJ0lqqp4=.40134e48-c304-4d47-81fd-f08c7781ef65@github.com> On Wed, 4 Nov 2020 14:38:53 GMT, Erik Joelsson wrote: >> Thank you, @erikj79. Trivial? (I would like to get is soon, in order to capture more hs_errs for current failures) > > I'm fine with a quick integration of this. Thank you. ------------- PR: https://git.openjdk.java.net/jdk/pull/1055 From shade at openjdk.java.net Wed Nov 4 14:48:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 4 Nov 2020 14:48:00 GMT Subject: Integrated: 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 12:59:24 GMT, Aleksey Shipilev wrote: > Current submit workflow omits `hs_err*.log` and `replay*.log` files, because zip matches the paths, and so only the hs_errs/replays in the current dir are picked up (and there are none). It should be preceded with `*` to match files in subdirs. Incidentally, this is why `*.jtr` match works. > > Observe: > > $ mkdir 1/2 -p > $ touch 1/2/hs_err_213123.log > $ zip -r9 test.zip . -i hs_err* > zip warning: zip file empty > $ zip -r9 test.zip . -i */hs_err* > adding: 1/2/hs_err_213123.log (stored 0%) > > This was introduced in the initial implementation in JDK-8255352. > > Additional testing: > - [x] Observing current Windows failures package hs_err into archive This pull request has now been integrated. Changeset: 61c92470 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/61c92470 Stats: 8 lines in 1 file changed: 0 ins; 0 del; 8 mod 8255895: Submit workflow artifacts miss hs_errs/replays due to ZIP include mismatch Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1055 From hannesw at openjdk.java.net Wed Nov 4 15:53:59 2020 From: hannesw at openjdk.java.net (Hannes =?UTF-8?B?V2FsbG7DtmZlcg==?=) Date: Wed, 4 Nov 2020 15:53:59 GMT Subject: RFR: JDK-6251738: Want a top-level summary page that itemizes all spec documents referenced from javadocs (OEM spec) In-Reply-To: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> References: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> Message-ID: On Thu, 22 Oct 2020 02:40:44 GMT, Jonathan Gibbons wrote: > This introduces support for a new `@spec` tag that can be used as either an inline tag or as a block tag. It is used to identify references to external specifications, in such a way that the references can be collected together on a new summary page, called "Other Specifications", available from either the static INDEX page or the interactive search box. > > As an inline tag, the format is `{@spec url label}`, which is roughly translated to `label` in the generated docs. > > As a block tag, the format is simply > > @spec url label > > which is handled in a manner analogous to > > @see label > > The tag is notable for being the first standard/supported tag that can be used as either an inline or block tag. (We have had support for bimodal non-standard/custom tags for a while, such as `{@jls}` and `{@jvms}`.) To support bimodal standard tags, some changes to `DocCommentParser` are incorporated here. > > This change is only the _support_ for the new tag; it does _not_ include any changes to API docs to _use_ the new tag. It's very instructive to see a new tag introduced from scratch with all the accompanying infrastructure and tests. The changes looks good to me, but I need to stress than I'm not an expert on some of the affected areas such as the jdk.compiler parts. ------------- Marked as reviewed by hannesw (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/790 From ihse at openjdk.java.net Wed Nov 4 16:20:54 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 4 Nov 2020 16:20:54 GMT Subject: Integrated: 8253892: Disable misleading-indentation on clang as well as gcc In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 22:25:08 GMT, Magnus Ihse Bursie wrote: > With clang 10.0, the compiler now detects a new class of warnings. The `misleading-indentation` warning has previously been disabled on gcc for hotspot and libfdlibm. Now we need to disable it for clang as well. This pull request has now been integrated. Changeset: 7dcaba63 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/7dcaba63 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod 8253892: Disable misleading-indentation on clang as well as gcc Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1044 From shade at openjdk.java.net Wed Nov 4 16:41:01 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 4 Nov 2020 16:41:01 GMT Subject: RFR: 8255822: Zero: improve build-time JVMTI handling Message-ID: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> Current Zero interpreter has the optimization for JVMTI support. It recognizes that JVMTI is disabled most of the time, and that JVMTI checks in the interpreter code slows it down considerably. (I measured it myself for this patch, it gives about 20% hit in build times). Current optimization works as follows. At build time, an XSLT transform is performed on `bytecodeInterpreter.cpp`, yielding `bytecodeInterpreterWithChecks.cpp`. In that new compilation unit, `VM_JVMTI` macro is defined, and a new entry point -- `BytecodeInterpreter::withChecks` -- is defined. Then, both compilation units are compiled. In one of them, `JVMTI` hooks are stripped out. In another, they persist. Then, callers have to choose which entry point to use. I believe this can be rewritten to use C++ templates instead of XLST and defines dance. This also allows to clean up JVMTI checks a bit. Additional testing: - [x] Linux x86_64 Zero fastdebug build with `-jvmti` - [x] Linux x86_64 Zero fastdebug/release build times are not regressing ------------- Commit messages: - Revert one dubious change - 8255822: Zero: improve build-time JVMTI handling Changes: https://git.openjdk.java.net/jdk/pull/1061/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1061&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255822 Stats: 155 lines in 6 files changed: 3 ins; 118 del; 34 mod Patch: https://git.openjdk.java.net/jdk/pull/1061.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1061/head:pull/1061 PR: https://git.openjdk.java.net/jdk/pull/1061 From mchung at openjdk.java.net Wed Nov 4 17:40:00 2020 From: mchung at openjdk.java.net (Mandy Chung) Date: Wed, 4 Nov 2020 17:40:00 GMT Subject: RFR: 8188055: (ref) Add Reference::refersTo predicate [v6] In-Reply-To: References: <0dhF_xxcp1VoUowwdZenB2qWa9ILcZjTMe3lsaRrg7k=.3c633db8-f745-4353-ad34-a64fbc96d4e0@github.com> Message-ID: <95NVU6qFbtLLeI2DfNZBHdpbkrXlc0jWTbHpebLpHj0=.31a36bc2-e770-453c-940f-ca21ff79e4c2@github.com> On Wed, 4 Nov 2020 10:13:59 GMT, Kim Barrett wrote: >> Hello! >> >> As an IDE developer, I'm thinking about IDE inspection that may suggest the new method. My idea is to suggest replacing every `ref.get() == obj` with `ref.refersTo(obj)`. Is this a good idea or there are cases when `ref.get() == obj` could be preferred? What do you think? > >> Hello! >> >> As an IDE developer, I'm thinking about IDE inspection that may suggest the new method. My idea is to suggest replacing every `ref.get() == obj` with `ref.refersTo(obj)`. Is this a good idea or there are cases when `ref.get() == obj` could be preferred? What do you think? > > Those have different behaviors when ref's class overrides get. Sometimes that might be intentional (PhantomReference, where get blocks access to the referent, and SoftReference, where get may update heuristics for recent accesses delaying GC clearing). But if some further subclass overrides get for some reason, such a change might not be appropriate. Checking if a reference has been cleared i.e. `ref.get() == null` or `ref.get() != null` may benefit with IDE giving a hint. ------------- PR: https://git.openjdk.java.net/jdk/pull/498 From jvernee at openjdk.java.net Wed Nov 4 18:13:59 2020 From: jvernee at openjdk.java.net (Jorn Vernee) Date: Wed, 4 Nov 2020 18:13:59 GMT Subject: Integrated: 8255128: linux x86 build failure with libJNIPoint.c In-Reply-To: References: Message-ID: On Mon, 2 Nov 2020 18:36:32 GMT, Jorn Vernee wrote: > Add 32-bit-safe version of jlong <-> casts to libJNIPoint.c > > This removes the reported warning. > > Note that the _LP64 macro was not being propagated to the benchmark native libraries on Windows. The comment says that this is due to pack200, but since this has been removed [1], it seemed safe to propagate the macro now (backed up by testing). > > CC'ing hotspot-runtime since I know some people there were looking forward to having this fixed. > > Testing: tier1-3 > > [1] https://bugs.openjdk.java.net/browse/JDK-8232022 This pull request has now been integrated. Changeset: 804bd725 Author: Jorn Vernee URL: https://git.openjdk.java.net/jdk/commit/804bd725 Stats: 17 lines in 2 files changed: 1 ins; 6 del; 10 mod 8255128: linux x86 build failure with libJNIPoint.c Reviewed-by: coleenp, shade, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1017 From herrick at openjdk.java.net Wed Nov 4 19:17:11 2020 From: herrick at openjdk.java.net (Andy Herrick) Date: Wed, 4 Nov 2020 19:17:11 GMT Subject: RFR: JDK-8252870: Finalize (remove "incubator" from) jpackage [v6] In-Reply-To: References: Message-ID: > JDK-8252870: Finalize (remove "incubator" from) jpackage Andy Herrick has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: - Merge branch 'master' into JDK-8252870 - Merge branch 'master' into JDK-8252870 - Merge branch master into JDK-8252870 - Merge branch 'master' into JDK-8252870 - JDK-8252870: Finalize (remove "incubator" from) jpackage - reverting two auto-generated files, and changing module-info to "@since 16" - JDK-8252870: Finalize (remove "incubator" from) jpackage Merge branch 'finalize' into JDK-8252870 - 8252869 Finalize (remove incubator from) jpackage (implementation) ------------- Changes: https://git.openjdk.java.net/jdk/pull/633/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=633&range=05 Stats: 1733 lines in 259 files changed: 692 ins; 701 del; 340 mod Patch: https://git.openjdk.java.net/jdk/pull/633.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/633/head:pull/633 PR: https://git.openjdk.java.net/jdk/pull/633 From jlahoda at openjdk.java.net Wed Nov 4 19:36:11 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Wed, 4 Nov 2020 19:36:11 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v7] In-Reply-To: References: Message-ID: > This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. > > The notable changes are: > > * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: > * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. > * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings > * improving error/warning messages. Please see [1] for a list of cases/examples. > * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. > * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). > * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. > * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. > > Please also see the CSR [6] for more information. > > [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html > [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html > [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html > [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html > [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ > [6] https://bugs.openjdk.java.net/browse/JDK-8250769 Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 49 commits: - Cleanup - removing unnecessary code. - Merging master into JDK-8250768-dev4 - Reflecting review comments. - Removing trailing whitespace. - Merging master into JDK-8250768. - Updating tests after records are a final feature. - Fixing tests. - Finalizing removal of record preview hooks. - Merging master into JDK-8250768 - Reflecting review comments. - ... and 39 more: https://git.openjdk.java.net/jdk/compare/7f4d873d...e4b02827 ------------- Changes: https://git.openjdk.java.net/jdk/pull/703/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=06 Stats: 3719 lines in 148 files changed: 2701 ins; 730 del; 288 mod Patch: https://git.openjdk.java.net/jdk/pull/703.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/703/head:pull/703 PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Wed Nov 4 19:43:06 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Wed, 4 Nov 2020 19:43:06 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> Message-ID: On Mon, 2 Nov 2020 20:21:44 GMT, Jonathan Gibbons wrote: >> Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: >> >> - Removing trailing whitespace. >> - Merging master into JDK-8250768. >> - Updating tests after records are a final feature. >> - Fixing tests. >> - Finalizing removal of record preview hooks. >> - Merging master into JDK-8250768 >> - Reflecting review comments. >> - Merge branch 'master' into JDK-8250768 >> - Removing unnecessary cast. >> - Using a more correct way to get URLs. >> - ... and 36 more: https://git.openjdk.java.net/jdk/compare/d93e3a7d...2e403900 > > I have read all the files. > > I have added a n umber of various minor non-blocking comments (no need for re-review( to fix these. But I have a couple of comments/questions before finally giving approval. > There's a comment in `PreviewListWriter` about annotation members that needs too be addressed, and I wonder is RECORD and RECORD_COMPONENT need to be added into PreviewElementKind. Thanks @jonathan-gibbons for your comments! I've tried to update the code based on them, mostly in https://github.com/lahodaj/jdk/commit/743f516c660b577035cdda4510a0bb97937fd9b2 and https://github.com/lahodaj/jdk/commit/e4b02827998fc2e8f19f983aabfb3d720b03d111 A big chunk of the update is generalization of the deprecated and preview list builders and writers into a "summary" list builder and writer. These should also now handle records. For record components, those are a little tricky, as (AFAIK) can't currently have deprecation/preview-ness for them (and hence there is no good way to test any support for record components in these). But the summary build and writer are looking for record components and will fail in case a record component is sent into them. ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From herrick at openjdk.java.net Wed Nov 4 20:30:58 2020 From: herrick at openjdk.java.net (Andy Herrick) Date: Wed, 4 Nov 2020 20:30:58 GMT Subject: Integrated: JDK-8252870: Finalize (remove "incubator" from) jpackage In-Reply-To: References: Message-ID: <7UWYWJYiZj1QKeFBjKC1fPxOdQENTEeBp0QwFxhOcuY=.b202c984-622a-4651-98e0-907a671e7ccc@github.com> On Tue, 13 Oct 2020 12:51:54 GMT, Andy Herrick wrote: > JDK-8252870: Finalize (remove "incubator" from) jpackage This pull request has now been integrated. Changeset: 26e7ef78 Author: Andy Herrick URL: https://git.openjdk.java.net/jdk/commit/26e7ef78 Stats: 1733 lines in 259 files changed: 692 ins; 701 del; 340 mod 8252870: Finalize (remove "incubator" from) jpackage Reviewed-by: kcr, erikj, almatvee, asemenyuk, prr, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/633 From jjg at openjdk.java.net Wed Nov 4 20:57:06 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Wed, 4 Nov 2020 20:57:06 GMT Subject: RFR: JDK-6251738: Want a top-level summary page that itemizes all spec documents referenced from javadocs (OEM spec) [v2] In-Reply-To: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> References: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> Message-ID: > This introduces support for a new `@spec` tag that can be used as either an inline tag or as a block tag. It is used to identify references to external specifications, in such a way that the references can be collected together on a new summary page, called "Other Specifications", available from either the static INDEX page or the interactive search box. > > As an inline tag, the format is `{@spec url label}`, which is roughly translated to `label` in the generated docs. > > As a block tag, the format is simply > > @spec url label > > which is handled in a manner analogous to > > @see label > > The tag is notable for being the first standard/supported tag that can be used as either an inline or block tag. (We have had support for bimodal non-standard/custom tags for a while, such as `{@jls}` and `{@jvms}`.) To support bimodal standard tags, some changes to `DocCommentParser` are incorporated here. > > This change is only the _support_ for the new tag; it does _not_ include any changes to API docs to _use_ the new tag. Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: - Fix merge issues; review feedback - Merge with master - allow rich content in createAnchorAndSearchIndex - update Docs.gmk to stop disabling spec tag - fix TestSpecTag.testEncodedURI - fix tests - remove support to workaround legacy @spec tag - Merge remote-tracking branch 'upstream/master' into new-spec-tag - fix trailing whitespace in test - temporarily allow existing legacy usage `@spec JPMS` `@spec jsr-51` - ... and 1 more: https://git.openjdk.java.net/jdk/compare/804bd725...ed5512d9 ------------- Changes: https://git.openjdk.java.net/jdk/pull/790/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=790&range=01 Stats: 1374 lines in 42 files changed: 1337 ins; 14 del; 23 mod Patch: https://git.openjdk.java.net/jdk/pull/790.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/790/head:pull/790 PR: https://git.openjdk.java.net/jdk/pull/790 From tim.bell at oracle.com Wed Nov 4 21:24:56 2020 From: tim.bell at oracle.com (Tim Bell) Date: Wed, 4 Nov 2020 13:24:56 -0800 Subject: New lead for the JDK 7u Project: Andrew Brygin Message-ID: <361bbdf7-f61a-fd28-dc51-7b6f6b7eae1b@oracle.com> Hello Andrew Haley resigned as JDK 7u project lead [1]. Andrew Brygin offered to take over leadership of JDK 7u. [2] I have no other nominations. Under the bylaws for Project Roles [3], a new Project Lead may be nominated by the Group Leads of the Project's sponsoring groups. The Build Infrastructure Group is sponsor of the project. As Group Leader [4], I would like to nominate Andrew Brygin [5] to be the new Project Lead. The Build Infrastructure Group is the only sponsoring group for this project. The nomination is automatically approved. Tim Bell [1] https://mail.openjdk.java.net/pipermail/jdk7u-dev/2020-August/011040.html [2] https://mail.openjdk.java.net/pipermail/jdk7u-dev/2020-September/011041.html [3] https://openjdk.java.net/bylaws#project-lead [4] https://openjdk.java.net/census#build [5] http://openjdk.java.net/census#bae From sspitsyn at openjdk.java.net Wed Nov 4 22:12:06 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 4 Nov 2020 22:12:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 12:21:12 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision: > > - Add back WeakProcessorPhases::Phase enum. > - Serguei 1. Thank you for the update, Coleen! I leave it for you to decide to refactor the gc_notification or not. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From shade at openjdk.java.net Thu Nov 5 06:39:02 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 5 Nov 2020 06:39:02 GMT Subject: RFR: 8255822: Zero: improve build-time JVMTI handling [v2] In-Reply-To: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> References: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> Message-ID: > Current Zero interpreter has the optimization for JVMTI support. It recognizes that JVMTI is disabled most of the time, and that JVMTI checks in the interpreter code slows it down considerably. (I measured it myself for this patch, it gives about 20% hit in build times). > > Current optimization works as follows. At build time, an XSLT transform is performed on `bytecodeInterpreter.cpp`, yielding `bytecodeInterpreterWithChecks.cpp`. In that new compilation unit, `VM_JVMTI` macro is defined, and a new entry point -- `BytecodeInterpreter::withChecks` -- is defined. Then, both compilation units are compiled. In one of them, `JVMTI` hooks are stripped out. In another, they persist. Then, callers have to choose which entry point to use. > > I believe this can be rewritten to use C++ templates instead of XLST and defines dance. This also allows to clean up JVMTI checks a bit. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug build with `-jvmti` > - [x] Linux x86_64 Zero fastdebug/release build times are not regressing Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - Fix build error - Merge branch 'master' into JDK-8255822-zero-jvmti-rework - Revert one dubious change - 8255822: Zero: improve build-time JVMTI handling Summary: use C++ templates instead of XSLT transforms ------------- Changes: https://git.openjdk.java.net/jdk/pull/1061/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1061&range=01 Stats: 156 lines in 6 files changed: 3 ins; 119 del; 34 mod Patch: https://git.openjdk.java.net/jdk/pull/1061.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1061/head:pull/1061 PR: https://git.openjdk.java.net/jdk/pull/1061 From dholmes at openjdk.java.net Thu Nov 5 06:50:56 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 5 Nov 2020 06:50:56 GMT Subject: RFR: 8255822: Zero: improve build-time JVMTI handling [v2] In-Reply-To: References: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> Message-ID: <5iCgi65PZMWTwIIaBn4AdC7aM_Uvl2gqX3hUNRpK1Ok=.2352c564-b061-48ba-a40c-4a2d329676bf@github.com> On Thu, 5 Nov 2020 06:39:02 GMT, Aleksey Shipilev wrote: >> Current Zero interpreter has the optimization for JVMTI support. It recognizes that JVMTI is disabled most of the time, and that JVMTI checks in the interpreter code slows it down considerably. (I measured it myself for this patch, it gives about 20% hit in build times). >> >> Current optimization works as follows. At build time, an XSLT transform is performed on `bytecodeInterpreter.cpp`, yielding `bytecodeInterpreterWithChecks.cpp`. In that new compilation unit, `VM_JVMTI` macro is defined, and a new entry point -- `BytecodeInterpreter::withChecks` -- is defined. Then, both compilation units are compiled. In one of them, `JVMTI` hooks are stripped out. In another, they persist. Then, callers have to choose which entry point to use. >> >> I believe this can be rewritten to use C++ templates instead of XLST and defines dance. This also allows to clean up JVMTI checks a bit. >> >> Additional testing: >> - [x] Linux x86_64 Zero fastdebug build with `-jvmti` >> - [x] Linux x86_64 Zero fastdebug/release build times are not regressing > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: > > - Fix build error > - Merge branch 'master' into JDK-8255822-zero-jvmti-rework > - Revert one dubious change > - 8255822: Zero: improve build-time JVMTI handling > Summary: use C++ templates instead of XSLT transforms This looks like a good cleanup to me - far simpler! Cheers, David ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1061 From ihse at openjdk.java.net Thu Nov 5 07:58:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 5 Nov 2020 07:58:57 GMT Subject: RFR: 8255822: Zero: improve build-time JVMTI handling [v2] In-Reply-To: References: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> Message-ID: On Thu, 5 Nov 2020 06:39:02 GMT, Aleksey Shipilev wrote: >> Current Zero interpreter has the optimization for JVMTI support. It recognizes that JVMTI is disabled most of the time, and that JVMTI checks in the interpreter code slows it down considerably. (I measured it myself when working on this patch: removing this optimization yields about 20% hit in build times). >> >> Current optimization works as follows. At build time, an XSLT transform is performed on `bytecodeInterpreter.cpp`, yielding `bytecodeInterpreterWithChecks.cpp`. In that new compilation unit, `VM_JVMTI` macro is defined, and a new entry point -- `BytecodeInterpreter::withChecks` -- is defined. Then, both compilation units are compiled. In one of them, `JVMTI` hooks are stripped out. In another, they persist. Then, callers have to choose which entry point to use. >> >> I believe this can be rewritten to use C++ templates instead of XLST and defines dance. This also allows to clean up JVMTI checks a bit. >> >> Additional testing: >> - [x] Linux x86_64 Zero fastdebug build with `-jvmti` >> - [x] Linux x86_64 Zero fastdebug/release build times are not regressing > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: > > - Fix build error > - Merge branch 'master' into JDK-8255822-zero-jvmti-rework > - Revert one dubious change > - 8255822: Zero: improve build-time JVMTI handling > Summary: use C++ templates instead of XSLT transforms Looks good to me. Thanks for the cleanup! ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1061 From jlahoda at openjdk.java.net Thu Nov 5 09:31:20 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Thu, 5 Nov 2020 09:31:20 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v8] In-Reply-To: References: Message-ID: > This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. > > The notable changes are: > > * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: > * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. > * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings > * improving error/warning messages. Please see [1] for a list of cases/examples. > * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. > * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). > * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. > * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. > > Please also see the CSR [6] for more information. > > [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html > [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html > [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html > [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html > [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ > [6] https://bugs.openjdk.java.net/browse/JDK-8250769 Jan Lahoda has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 51 commits: - Merging master into JDK-8250768 - Removing unnecessary property keys. - Cleanup - removing unnecessary code. - Merging master into JDK-8250768-dev4 - Reflecting review comments. - Removing trailing whitespace. - Merging master into JDK-8250768. - Updating tests after records are a final feature. - Fixing tests. - Finalizing removal of record preview hooks. - ... and 41 more: https://git.openjdk.java.net/jdk/compare/700447f7...3d55f909 ------------- Changes: https://git.openjdk.java.net/jdk/pull/703/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=07 Stats: 3665 lines in 136 files changed: 2701 ins; 692 del; 272 mod Patch: https://git.openjdk.java.net/jdk/pull/703.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/703/head:pull/703 PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Thu Nov 5 09:51:13 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Thu, 5 Nov 2020 09:51:13 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v9] In-Reply-To: References: Message-ID: > This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. > > The notable changes are: > > * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: > * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. > * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings > * improving error/warning messages. Please see [1] for a list of cases/examples. > * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. > * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). > * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. > * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. > > Please also see the CSR [6] for more information. > > [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html > [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html > [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html > [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html > [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ > [6] https://bugs.openjdk.java.net/browse/JDK-8250769 Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision: Removing obsolette @PreviewFeature. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/703/files - new: https://git.openjdk.java.net/jdk/pull/703/files/3d55f909..370031f0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=08 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=07-08 Stats: 10 lines in 5 files changed: 0 ins; 10 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/703.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/703/head:pull/703 PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Thu Nov 5 11:08:18 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Thu, 5 Nov 2020 11:08:18 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v10] In-Reply-To: References: Message-ID: <0RHAqKvLjL4Y2yvmClOLoPrcEfdYyWid4BR-ntTWQ78=.e91baacb-444b-4cd1-89d1-8cb26f3de56f@github.com> > This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. > > The notable changes are: > > * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: > * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. > * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings > * improving error/warning messages. Please see [1] for a list of cases/examples. > * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. > * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). > * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. > * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. > > Please also see the CSR [6] for more information. > > [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html > [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html > [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html > [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html > [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ > [6] https://bugs.openjdk.java.net/browse/JDK-8250769 Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision: Fixing typo. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/703/files - new: https://git.openjdk.java.net/jdk/pull/703/files/370031f0..097ae3c1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=09 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=08-09 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/703.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/703/head:pull/703 PR: https://git.openjdk.java.net/jdk/pull/703 From serb at openjdk.java.net Thu Nov 5 11:26:57 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Thu, 5 Nov 2020 11:26:57 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v2] In-Reply-To: <2um9af6839x1BDDn3P10xB6WcxXVc_DGAjCDueyfDcI=.5feeee81-b628-465a-8123-3750ea4db3d2@github.com> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <2um9af6839x1BDDn3P10xB6WcxXVc_DGAjCDueyfDcI=.5feeee81-b628-465a-8123-3750ea4db3d2@github.com> Message-ID: On Mon, 2 Nov 2020 16:23:19 GMT, Phil Race wrote: >> I'm just a bit curious about the added, empty, `src/java.desktop/share/native/libharfbuzz/abc.txt`... >> >> If it really is in upstream source, I'm not saying you should remove it. It just looks very odd. It's not a merge artifact? >> >> (I could not even add review comments to an empty file in github! ?) > >> I'm just a bit curious about the added, empty, `src/java.desktop/share/native/libharfbuzz/abc.txt`... >> >> If it really is in upstream source, I'm not saying you should remove it. It just looks very odd. It's not a merge artifact? >> >> (I could not even add review comments to an empty file in github! ?) > > oops. that should not have been in that folder. It was me practicing a different white space removal script. > I thought I checked for any extraneous files. I'll get rid of it now. Looks like the build on macOS via GitHub will be broken by this fix? ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From coleenp at openjdk.java.net Thu Nov 5 12:30:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Wed, 4 Nov 2020 10:05:29 GMT, Kim Barrett wrote: >> The comment is trying to describe the situation like: >> 1. mark-end pause (WeakHandle.peek() returns NULL because object A is unmarked) >> 2. safepoint for heap walk >> 2a. Need to post ObjectFree event for object A before the heap walk doesn't find object A. >> 3. gc_notification - would have posted an ObjectFree event for object A if the heapwalk hadn't intervened >> >> The check_hashmap() function also checks whether the hash table needs to be rehashed before the next operation that uses the hashtable. >> >> Both operations require the table to be locked. >> >> The unlink and post needs to be in a GC pause for reasons that I stated above. The unlink and post were done in a GC pause so this isn't worse for any GCs. The lock can be held for concurrent GC while the number of entries are processed and this would be a delay for some applications that have requested a lot of tags, but these applications have asked for this and it's not worse than what we had with GC walking this table in safepoints. > > For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing, and so still effectively be a serial subtask after all the parallel subtasks are done or mostly done. Yes, you are right that the processing will be done serially and not by a parallel worker thread. This is could spawn a new GC worker thread to process the posts, as you suggest. We could do that if we find a customer that has a complaint about the pause time of this processing. >> The JVMTI code expects the posting to be done quite eagerly presumably during GC, before it has a chance to disable the event or some other such operation. So the posting is done during the notification because it's as soon as possible. Deferring to the ServiceThread had two problems. 1. the event came later than the caller is expecting it, and in at least one test the event was disabled before posting, and 2. there's a comment in the code why we can't post events with a JavaThread. We'd have to transition into native while holding a no safepoint lock (or else deadlock). The point of making this change was so that the JVMTI table does not need GC code to serially process the table. > > I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event. Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:30:00 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:00 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 08:56:54 GMT, Kim Barrett wrote: >> Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: >> >> Code review comments from Kim and Albert. > > src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36: > >> 34: class JvmtiTagMapEntryClosure; >> 35: >> 36: class JvmtiTagMapEntry : public HashtableEntry { > > By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%). > > It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost. Yes, there is 64/32 bits extra per hashtable entry with the standard hashtable implementation. It wouldn't have been hard to replace the oop object, but using shared code was a goal of this change. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:30:02 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v3] In-Reply-To: References: <5UvpVT3pWDNSJ1Vh_WIy-E3ZjOS8O-8sZi-9ZRyYYQI=.d5d244cf-5e1b-4790-9370-66ae3a2ea76c@github.com> <0dSlRGfxnjP_6IlH5CEYdF48d3ymvjCKE_s4UZb_WNc=.789216de-6d78-4c00-bdd4-e09f1dec3924@github.com> <1qM8Skbob0uL_KwdoJNDTyavFxOH_VHJc5o6yF881zI=.604bc76e-0536-48a0-91d5-4ba85e32bc11@github.com> Message-ID: On Wed, 4 Nov 2020 07:42:36 GMT, Kim Barrett wrote: >>> Ok, so I'm not sure what to do with this: >>> >>> enum Phase { >>> // Serial phase. >>> JVMTI_ONLY(jvmti) >>> // Additional implicit phase values follow for oopstorages. >>> `};` >>> >>> I've removed the only thing in this enum. >> >> Enums without any named enumerators are still meaningful types. More so with scoped enums, but still with unscoped enums. > >> > Though it might be possible to go even further and eliminate WeakProcessorPhases as a thing separate from OopStorageSet. >> >> This makes sense. Can we file another RFE for this? I was sort of surprised by how much code was involved so I tried to find a place to stop deleting. > > I think the deletion stopped at the wrong place; it either went too far, or not far enough. I restored the empty enum for Phase and can open a new RFE if there is more code to remove. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:30:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:30:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 09:27:39 GMT, Kim Barrett wrote: >> I didn't say it "doesn't work for shenandoah", I said it wouldn't have worked with the old shenandoah barriers without additional work, like adding calls to resolve. I understand the design intent of notifying the table management that its hash codes are out of date. And the num-dead callback isn't the right place, since there are num-dead callback invocations that aren't associated with hash code invalidation. (It's not a correctness wrong, it's a "these things are unrelated and this causes unnecessary work" wrong.) > > It used to be that jvmti tagmap processing was all-in-one (in GC weak reference processing, with the weak clearing, dead table entry removal, and rehashing all done in one pass. This change has split that up, with the weak clearing happening in a different place (still as part of the GC's weak reference processing) than the others (which I claim can now be part of the mutator, whether further separated or not). > > "Concurrent GC" has nothing to do with whether tagmaps need rehashing. Any copying collector needs to do so. A non-copying collector (whether concurrent or not) would not. (We don't have any purely non-copying collectors, but G1 concurrent oldgen collection is non-copying.) And weak reference clearing (whether concurrent or not) has nothing to do with whether objects have been moved and so the hashing has been invalidated. > > There's also a "well known" issue with address-based hashing and generational or similar collectors, where a simple boolean "objects have moved" flag can be problematic, and which tagmaps seem likely to be prone to. The old do_weak_oops tries to mitigate it by recognizing when the object didn't move. The new rehash function also doesn't move the objects either. It essentially does the same as the old weak_oops_do function. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 12:45:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 12:45:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Wed, 4 Nov 2020 22:09:21 GMT, Serguei Spitsyn wrote: >> Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision: >> >> - Add back WeakProcessorPhases::Phase enum. >> - Serguei 1. > > Thank you for the update, Coleen! > I leave it for you to decide to refactor the gc_notification or not. > Thanks, > Serguei Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From jlahoda at openjdk.java.net Thu Nov 5 12:45:56 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Thu, 5 Nov 2020 12:45:56 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> Message-ID: <9oQ_AyidieKnLIkJrx6koIR3TUJIqmxVmlzp82TX_64=.c44f863b-7873-498c-ae87-a922e0086fb9@github.com> On Wed, 4 Nov 2020 19:40:33 GMT, Jan Lahoda wrote: >> I have read all the files. >> >> I have added a n umber of various minor non-blocking comments (no need for re-review( to fix these. But I have a couple of comments/questions before finally giving approval. >> There's a comment in `PreviewListWriter` about annotation members that needs too be addressed, and I wonder is RECORD and RECORD_COMPONENT need to be added into PreviewElementKind. > > Thanks @jonathan-gibbons for your comments! I've tried to update the code based on them, mostly in https://github.com/lahodaj/jdk/commit/743f516c660b577035cdda4510a0bb97937fd9b2 and https://github.com/lahodaj/jdk/commit/e4b02827998fc2e8f19f983aabfb3d720b03d111 > > A big chunk of the update is generalization of the deprecated and preview list builders and writers into a "summary" list builder and writer. These should also now handle records. For record components, those are a little tricky, as (AFAIK) can't currently have deprecation/preview-ness for them (and hence there is no good way to test any support for record components in these). But the summary build and writer are looking for record components and will fail in case a record component is sent into them. FWIW, a javadoc generated with the current version of the patch: http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/index.html And a specdiff comparing it to the javadoc built from the corresponding master: http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.specdiff.01/overview-summary.html ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From prappo at openjdk.java.net Thu Nov 5 13:15:59 2020 From: prappo at openjdk.java.net (Pavel Rappo) Date: Thu, 5 Nov 2020 13:15:59 GMT Subject: RFR: JDK-6251738: Want a top-level summary page that itemizes all spec documents referenced from javadocs (OEM spec) [v2] In-Reply-To: References: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> Message-ID: On Wed, 4 Nov 2020 20:57:06 GMT, Jonathan Gibbons wrote: >> This introduces support for a new `@spec` tag that can be used as either an inline tag or as a block tag. It is used to identify references to external specifications, in such a way that the references can be collected together on a new summary page, called "Other Specifications", available from either the static INDEX page or the interactive search box. >> >> As an inline tag, the format is `{@spec url label}`, which is roughly translated to `label` in the generated docs. >> >> As a block tag, the format is simply >> >> @spec url label >> >> which is handled in a manner analogous to >> >> @see label >> >> The tag is notable for being the first standard/supported tag that can be used as either an inline or block tag. (We have had support for bimodal non-standard/custom tags for a while, such as `{@jls}` and `{@jvms}`.) To support bimodal standard tags, some changes to `DocCommentParser` are incorporated here. >> >> This change is only the _support_ for the new tag; it does _not_ include any changes to API docs to _use_ the new tag. > > Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: > > - Fix merge issues; review feedback > - Merge with master > - allow rich content in createAnchorAndSearchIndex > - update Docs.gmk to stop disabling spec tag > - fix TestSpecTag.testEncodedURI > - fix tests > - remove support to workaround legacy @spec tag > - Merge remote-tracking branch 'upstream/master' into new-spec-tag > - fix trailing whitespace in test > - temporarily allow existing legacy usage `@spec JPMS` `@spec jsr-51` > - ... and 1 more: https://git.openjdk.java.net/jdk/compare/804bd725...ed5512d9 src/jdk.compiler/share/classes/com/sun/source/util/DocTreeFactory.java line 309: > 307: * Creates a new {@code SpecTree} object, to represent a {@code @spec} tag. > 308: * > 309: * @param inline whether this is instance is an inline tag Should be: `* @param inline whether this instance is an inline tag` ------------- PR: https://git.openjdk.java.net/jdk/pull/790 From ihse at openjdk.java.net Thu Nov 5 13:38:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 5 Nov 2020 13:38:57 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v2] In-Reply-To: References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <2um9af6839x1BDDn3P10xB6WcxXVc_DGAjCDueyfDcI=.5feeee81-b628-465a-8123-3750ea4db3d2@github.com> Message-ID: On Thu, 5 Nov 2020 11:24:28 GMT, Sergey Bylokhov wrote: >>> I'm just a bit curious about the added, empty, `src/java.desktop/share/native/libharfbuzz/abc.txt`... >>> >>> If it really is in upstream source, I'm not saying you should remove it. It just looks very odd. It's not a merge artifact? >>> >>> (I could not even add review comments to an empty file in github! ?) >> >> oops. that should not have been in that folder. It was me practicing a different white space removal script. >> I thought I checked for any extraneous files. I'll get rid of it now. > > Looks like the build on macOS via GitHub will be broken by this fix? @mrserb Good catch! Yes indeed, it looks like `range-loop-analysis` needs to be disabled for clang for Xcode 12 to not complain. ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From eosterlund at openjdk.java.net Thu Nov 5 14:22:01 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Thu, 5 Nov 2020 14:22:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: Message-ID: On Tue, 3 Nov 2020 21:14:04 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMap.cpp line 3018: >> >>> 3016: } >>> 3017: // Later GC code will relocate the oops, so defer rehashing until then. >>> 3018: tag_map->_needs_rehashing = true; >> >> This is wrong for some collectors. I think all collectors ought to be calling set_needs_rehashing in appropriate places, and it can't be be correctly piggybacked on the num-dead callback. (See discussion above for that function.) >> >> For example, G1 remark pause does weak processing (including weak oopstorage) and will call the num-dead callback, but does not move objects, so does not require tagmap rehashing. >> >> (I think CMS oldgen remark may have been similar, for what that's worth.) > > Ok, so I'm going to need help to know where in all the different GCs to make this call. This seemed simpler at the expense of maybe causing a rehash at some points when it might not be necessary. For what GC is this wrong? I can see that it might yield more work than required, when performing a full GC, but not that it would do too little work. In other words, I can't see how it is wrong, as opposed to inaccurate. Littering GCs with JVMTI hooks so that we can optimize away an operation we do every young GC, from a full GC, does not really seem worth it IMO. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Thu Nov 5 14:40:02 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Thu, 5 Nov 2020 14:40:02 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Wed, 4 Nov 2020 13:32:07 GMT, Coleen Phillimore wrote: >> I know of nothing that leads to "presumably during GC" being a requirement. Having all pending events of some type occur before that type of event is disabled seems like a reasonable requirement, but just means that event disabling also requires the table to be "up to date", in the sense that any GC-cleared entries need to be dealt with. That can be handled just like other operations that use the table contents, rather than during the GC. That is, use post_dead_object_on_vm_thread if there are or might be any pending dead objects, before disabling the event. > > Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times. The point of this change was not necessarily to be lazy about updating the tagmap, until someone uses it. The point was to get rid of the last annoying serial GC phase. Doing it all lazily would certainly also achieve that. But it would also lead to situations where no event is ever posted from GC to GC. So you would get the event 20 GCs later, which might come as a surprise. It did come as a surprise to some tests, so it is reasonable to assume it would come as a surprise to users too. And I don't think we want such surprises unless we couldn't deal with them. And we can. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From eosterlund at openjdk.java.net Thu Nov 5 14:53:01 2020 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Thu, 5 Nov 2020 14:53:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Wed, 4 Nov 2020 13:22:57 GMT, Coleen Phillimore wrote: >> For the GCs that call the num_dead notification in a pause it is much worse than what we had. As I pointed out elsewhere, it used to be that tagmap processing was all-in-one, as a single serial subtask taken by the first thread that reached it in WeakProcessor processing. Other threads would find that subtask taken and move on to processing oopstores in parallel with the tagmap processing. Now everything except the oopstorage-based clearing of dead entries is a single threaded serial task done by the VMThread, after all the parallel WeakProcessor work is done, because that's where the num-dead callbacks are invoked. WeakProcessor's parallel oopstorage processing doesn't have a way to do the num-dead callbacks by the last thread out of each parallel oopstorage processing. Instead it's left to the end, on the assumption that the callbacks are relatively cheap. But that could still be much worse than the old code, since the tagmap oopstorage could be late in the order of processing, and so still effectively be a serial subtask after all the parallel subtasks are done or mostly done. > > Yes, you are right that the processing will be done serially and not by a parallel worker thread. This is could spawn a new GC worker thread to process the posts, as you suggest. We could do that if we find a customer that has a complaint about the pause time of this processing. So both before and now, this task is a single threaded task. The difference is that before that single threaded task could be performed in parallel to other tasks. So if the table is small, you probably won't be able to notice any difference as small table implies not much to do. And if the table is large, you still probably won't be able to notice any difference as a large table implies it will dominate the pause with both the old and new approach. Any difference at all is bounded at 2x processing time, as it was serial both before and after. But now if we have a perfectly medium balanced table, we can at the very worst observe a theoretical 2x worse processing of this JVMTI table. I think that if we truly did care about this difference, and that it is important to keep this code as well performed as possible, then we would not have a serial phase for this at all. The fact that this has been serial suggests to me that it is not a path that is critical, and therefore I don't think op timizing the theoretical max 2x worse processing times for perfectly medium sized JVMTI tag map tables, is worth the hassle. At least I can't see why this would be of any importance. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 5 15:07:01 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 5 Nov 2020 15:07:01 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v5] In-Reply-To: References: Message-ID: <_w3Wb6lCIkBeH6UxIGCZ-HoXgwx-qSYF6KpLY1txy6Y=.eeeb53b9-6b1f-41cb-b59b-109b714e8eed@github.com> On Wed, 4 Nov 2020 13:19:21 GMT, Coleen Phillimore wrote: >> src/hotspot/share/prims/jvmtiTagMapTable.hpp line 36: >> >>> 34: class JvmtiTagMapEntryClosure; >>> 35: >>> 36: class JvmtiTagMapEntry : public HashtableEntry { >> >> By using utilities/hashtable this buys into having to use HashtableEntry, which includes the _hash member, even though that value is trivially computed from the key (since we're using address-based hashing here). This costs an additional 8 bytes (_LP64) per entry (a 25% increase) compared to the old JvmtiTagHashmapEntry. (I think it doesn't currently make a difference on !_LP64 because of poorly chosen layout in the old code, but fixing that would make the difference 33%). >> >> It seems like it should not have been hard to replace the oop _object member in the old code with a WeakHandle while otherwise maintaining the Entry interface, allowing much of the rest of the code to remain the same or similar and not incurring this additional space cost. > > Yes, there is 64/32 bits extra per hashtable entry with the standard hashtable implementation. It wouldn't have been hard to replace the oop object, but using shared code was a goal of this change. So looking at the concurrent hashtable, the entries can be created without saving the hashcode. I was going to use that at first but didn't want to cut/paste the boilerplate to do so and the jvmti tag map hashtable is always accessed with a lock. This could be a future RFE if necessary and would also serve to eliminate another ad-hoc hashtable. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From prappo at openjdk.java.net Thu Nov 5 16:19:59 2020 From: prappo at openjdk.java.net (Pavel Rappo) Date: Thu, 5 Nov 2020 16:19:59 GMT Subject: RFR: JDK-6251738: Want a top-level summary page that itemizes all spec documents referenced from javadocs (OEM spec) [v2] In-Reply-To: References: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> Message-ID: On Wed, 4 Nov 2020 20:57:06 GMT, Jonathan Gibbons wrote: >> This introduces support for a new `@spec` tag that can be used as either an inline tag or as a block tag. It is used to identify references to external specifications, in such a way that the references can be collected together on a new summary page, called "Other Specifications", available from either the static INDEX page or the interactive search box. >> >> As an inline tag, the format is `{@spec url label}`, which is roughly translated to `label` in the generated docs. >> >> As a block tag, the format is simply >> >> @spec url label >> >> which is handled in a manner analogous to >> >> @see label >> >> The tag is notable for being the first standard/supported tag that can be used as either an inline or block tag. (We have had support for bimodal non-standard/custom tags for a while, such as `{@jls}` and `{@jvms}`.) To support bimodal standard tags, some changes to `DocCommentParser` are incorporated here. >> >> This change is only the _support_ for the new tag; it does _not_ include any changes to API docs to _use_ the new tag. > > Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: > > - Fix merge issues; review feedback > - Merge with master > - allow rich content in createAnchorAndSearchIndex > - update Docs.gmk to stop disabling spec tag > - fix TestSpecTag.testEncodedURI > - fix tests > - remove support to workaround legacy @spec tag > - Merge remote-tracking branch 'upstream/master' into new-spec-tag > - fix trailing whitespace in test > - temporarily allow existing legacy usage `@spec JPMS` `@spec jsr-51` > - ... and 1 more: https://git.openjdk.java.net/jdk/compare/804bd725...ed5512d9 1. Thanks for incorporating my previous offline feedback. 2. Since Hannes and Erik seem to have looked at everything else, I looked mostly at changes in `src/jdk.compiler/share/classes/com/sun/source/**`, which are good! 3. There should be a corresponding but separate change to "Documentation Comment Specification for the Standard Doclet". 4. Can we use this new `@since` tag to refer to the spec at `com/sun/tools/javac/parser/DocCommentParser.java:1116`? 5. Should we specify in `com.sun.source.doctree.SpecTree` that both `url` and `label` parts are mandatory? 6. `DCSpec extends DCEndPosTree`, sigh... Although that is not a public API, this design suggests we could improve that abstraction sometime later. src/jdk.compiler/share/classes/com/sun/tools/javac/parser/DocCommentParser.java line 1104: > 1102: > 1103: DCTree parse(int pos, Kind kind) throws ParseException { > 1104: if (kind != this.kind && this.kind != Kind.EITHER) { This condition looks right, but shouldn't we add another one to guard against accidental passing of `kind == EITHER`? src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlTree.java line 306: > 304: * The {@code ref} argument will be URL-encoded for use as the attribute value. > 305: * > 306: * @param ref the value for the {@code href} attribute} Since you are here, could you please remove that trailing `}`? src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlTree.java line 322: > 320: * {@link URI#toASCIIString() converted} to ASCII for use as the attribute value. > 321: * > 322: * @param ref the value for the {@code href} attribute} One more, perhaps copy-pasted, trailing `}`. test/langtools/tools/javac/diags/examples/NoLabel.java line 2: > 1: /* > 2: * Copyright (c) 2012, 2020 Oracle and/or its affiliates. All rights reserved. Should be: `Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved.` src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/TagletWriterImpl.java line 266: > 264: } > 265: > 266: String textOf(List trees) { I wonder if this method should work **recursively** rather than shallowly. Consider the following example: @spec http://example.com Some code: {@code text} I reckon we'll end up with only `Some code: `. src/jdk.compiler/share/classes/com/sun/tools/javac/tree/DCTree.java line 81: > 79: return list.stream().allMatch(DCTree::isBlank); > 80: } > 81: Adding these two methods **here** might be overkill. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/taglets/TagletManager.java line 737: > 735: > 736: for (Taglet t : taglets) { > 737: String name = t.isBlockTag() ? "@" + t.getName() : "{@" + t.getName() + "}"; Out of curiosity, why did you flip that? ------------- Changes requested by prappo (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/790 From mcimadamore at openjdk.java.net Thu Nov 5 17:14:16 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 5 Nov 2020 17:14:16 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v22] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 29 commits: - Fix post-merge issues caused by 8219014 - Merge branch 'master' into 8254162 - Addess remaining feedback from @AlanBateman and @mrserb - Address comments from @AlanBateman - Merge branch 'master' into 8254162 - Fix issues with derived buffers and IO operations - More 32-bit fixes for TestLayouts - * Add final to MappedByteBuffer::SCOPED_MEMORY_ACCESS field * Tweak TestLayouts to make it 32-bit friendly after recent MemoryLayouts tweaks - Remove TestMismatch from 32-bit problem list - Merge branch 'master' into 8254162 - ... and 19 more: https://git.openjdk.java.net/jdk/compare/a50fdd54...02f9e251 ------------- Changes: https://git.openjdk.java.net/jdk/pull/548/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=21 Stats: 7608 lines in 80 files changed: 4859 ins; 1545 del; 1204 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From neliasso at openjdk.java.net Thu Nov 5 17:28:06 2020 From: neliasso at openjdk.java.net (Nils Eliasson) Date: Thu, 5 Nov 2020 17:28:06 GMT Subject: RFR: 8255964: Add jcmd Thread.print to jtreg timeout handler Message-ID: This patch adds jcmd Thread.print to the jtreg timeout handler. Please review. ------------- Commit messages: - Add jcmd thread.print to timeout handler Changes: https://git.openjdk.java.net/jdk/pull/1080/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1080&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255964 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1080.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1080/head:pull/1080 PR: https://git.openjdk.java.net/jdk/pull/1080 From prr at openjdk.java.net Thu Nov 5 17:52:19 2020 From: prr at openjdk.java.net (Phil Race) Date: Thu, 5 Nov 2020 17:52:19 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v3] In-Reply-To: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: > This upgrades JDK to import the current 2.7.2 version of harfbuzz - an OpenType text shaping library > > https://bugs.openjdk.java.net/browse/JDK-8247872 > > This has passed building and headless and headful automated tests on all platforms. Phil Race has updated the pull request incrementally with one additional commit since the last revision: 8247872: Upgrade HarfBuzz to the latest 2.7.2 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/993/files - new: https://git.openjdk.java.net/jdk/pull/993/files/1f47957b..3d172716 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=993&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=993&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/993.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/993/head:pull/993 PR: https://git.openjdk.java.net/jdk/pull/993 From prr at openjdk.java.net Thu Nov 5 17:55:59 2020 From: prr at openjdk.java.net (Phil Race) Date: Thu, 5 Nov 2020 17:55:59 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v3] In-Reply-To: References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <2um9af6839x1BDDn3P10xB6WcxXVc_DGAjCDueyfDcI=.5feeee81-b628-465a-8123-3750ea4db3d2@github.com> Message-ID: On Thu, 5 Nov 2020 13:36:18 GMT, Magnus Ihse Bursie wrote: >> Looks like the build on macOS via GitHub will be broken by this fix? > > @mrserb Good catch! Yes indeed, it looks like `range-loop-analysis` needs to be disabled for clang for Xcode 12 to not complain. I added that to the disabled warnings .. now I suppose the actions will re-run but I don't see any sign of it yet. Just that the results that were there are now gone. A few thoughts on this - github sends so many messages that it is easy to miss the important ones - using the default for github we are at the whim of when ever they switch and if this change had been pushed 6 weeks ago likely it would have passed until the day github upgraded. - it also means passing a github build does not guarantee it will pass on the "official" toolchains. ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From alex.buckley at oracle.com Thu Nov 5 18:11:00 2020 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 5 Nov 2020 10:11:00 -0800 Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <9oQ_AyidieKnLIkJrx6koIR3TUJIqmxVmlzp82TX_64=.c44f863b-7873-498c-ae87-a922e0086fb9@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> <9oQ_AyidieKnLIkJrx6koIR3TUJIqmxVmlzp82TX_64=.c44f863b-7873-498c-ae87-a922e0086fb9@github.com> Message-ID: On 11/5/2020 4:45 AM, Jan Lahoda wrote: > FWIW, a javadoc generated with the current version of the patch: > http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/index.html Allow me to draw people's attention to the PREVIEW link in the banner of the generated javadoc. It shows all the preview APIs in the release on one page. This is very helpful for understanding the surface area of a preview feature. For example, with Sealed Classes being the only preview feature likely to target JDK 16, the PREVIEW page shows that the feature's API is solely about reflection. It's clear that Sealed Classes do not introduce, say, a java.lang.Sealed class analogous to the java.lang.Record class introduced by Records in JDK 14/15 (and which would have appeared on the PREVIEW page had it existed then). Alex From ecaspole at openjdk.java.net Thu Nov 5 22:19:03 2020 From: ecaspole at openjdk.java.net (Eric Caspole) Date: Thu, 5 Nov 2020 22:19:03 GMT Subject: RFR: 8255965: LogCompilation: add sort by nmethod code size Message-ID: While profiling an issue I added this sort by code size to LogCompilation, using -z $ java -ea -jar target/LogCompilation-1.0-SNAPSHOT.jar -z 2000-2.log | head 879 4 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 57344) 853 make_not_entrant 853 3 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 55968) 895 4 com.fee.fi.fo.Fum::baz (2238 bytes)(code size: 46112) 888 4 com.fee.fi.fo.Fum::quux (2165 bytes)(code size: 43200) The code size = stub_offset - insts_offset from what is in the log. This makes it easier to see, for example, if changing compiler XX options make huge differences in inlining. ------------- Commit messages: - 8255965: LogCompilation: add sort by nmethod code size Changes: https://git.openjdk.java.net/jdk/pull/1085/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1085&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8255965 Stats: 96 lines in 6 files changed: 91 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/1085.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1085/head:pull/1085 PR: https://git.openjdk.java.net/jdk/pull/1085 From kvn at openjdk.java.net Thu Nov 5 22:33:58 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 5 Nov 2020 22:33:58 GMT Subject: RFR: 8255965: LogCompilation: add sort by nmethod code size In-Reply-To: References: Message-ID: <1pzy5RPGK1v9O3utvsOwnPkKbXXYC29BEo58XTMjcbg=.427e9ca9-bbda-473e-a1af-89348f244fc5@github.com> On Thu, 5 Nov 2020 22:13:54 GMT, Eric Caspole wrote: > While profiling an issue I added this sort by code size to LogCompilation, using -z > > $ java -ea -jar target/LogCompilation-1.0-SNAPSHOT.jar -z 2000-2.log | head > > 879 4 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 57344) > 853 make_not_entrant > 853 3 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 55968) > 895 4 com.fee.fi.fo.Fum::baz (2238 bytes)(code size: 46112) > 888 4 com.fee.fi.fo.Fum::quux (2165 bytes)(code size: 43200) > > The code size = stub_offset - insts_offset from what is in the log. > This makes it easier to see, for example, if changing compiler XX options make huge differences in inlining. Good. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1085 From redestad at openjdk.java.net Thu Nov 5 22:40:58 2020 From: redestad at openjdk.java.net (Claes Redestad) Date: Thu, 5 Nov 2020 22:40:58 GMT Subject: RFR: 8255965: LogCompilation: add sort by nmethod code size In-Reply-To: References: Message-ID: On Thu, 5 Nov 2020 22:13:54 GMT, Eric Caspole wrote: > While profiling an issue I added this sort by code size to LogCompilation, using -z > > $ java -ea -jar target/LogCompilation-1.0-SNAPSHOT.jar -z 2000-2.log | head > > 879 4 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 57344) > 853 make_not_entrant > 853 3 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 55968) > 895 4 com.fee.fi.fo.Fum::baz (2238 bytes)(code size: 46112) > 888 4 com.fee.fi.fo.Fum::quux (2165 bytes)(code size: 43200) > > The code size = stub_offset - insts_offset from what is in the log. > This makes it easier to see, for example, if changing compiler XX options make huge differences in inlining. Marked as reviewed by redestad (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1085 From ihse at openjdk.java.net Thu Nov 5 23:55:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 5 Nov 2020 23:55:56 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v3] In-Reply-To: References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <2um9af6839x1BDDn3P10xB6WcxXVc_DGAjCDueyfDcI=.5feeee81-b628-465a-8123-3750ea4db3d2@github.com> Message-ID: On Thu, 5 Nov 2020 17:52:53 GMT, Phil Race wrote: >> @mrserb Good catch! Yes indeed, it looks like `range-loop-analysis` needs to be disabled for clang for Xcode 12 to not complain. > > I added that to the disabled warnings .. now I suppose the actions will re-run but I don't see any sign of it yet. > Just that the results that were there are now gone. > A few thoughts on this > - github sends so many messages that it is easy to miss the important ones > - using the default for github we are at the whim of when ever they switch and if this change > had been pushed 6 weeks ago likely it would have passed until the day github upgraded. > - it also means passing a github build does not guarantee it will pass on the "official" toolchains. @prrace I agree, the github actions are a double-edged sword -- at best. At worst, it's a clear regression in terms of how we work. :-( We might need to take a step back and rethink this. Anyway, your changes looks good from a build PoV now. ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From ecaspole at openjdk.java.net Thu Nov 5 23:56:59 2020 From: ecaspole at openjdk.java.net (Eric Caspole) Date: Thu, 5 Nov 2020 23:56:59 GMT Subject: Integrated: 8255965: LogCompilation: add sort by nmethod code size In-Reply-To: References: Message-ID: On Thu, 5 Nov 2020 22:13:54 GMT, Eric Caspole wrote: > While profiling an issue I added this sort by code size to LogCompilation, using -z > > $ java -ea -jar target/LogCompilation-1.0-SNAPSHOT.jar -z 2000-2.log | head > > 879 4 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 57344) > 853 make_not_entrant > 853 3 com.fee.fi.fo.Fum::foobar (3076 bytes)(code size: 55968) > 895 4 com.fee.fi.fo.Fum::baz (2238 bytes)(code size: 46112) > 888 4 com.fee.fi.fo.Fum::quux (2165 bytes)(code size: 43200) > > The code size = stub_offset - insts_offset from what is in the log. > This makes it easier to see, for example, if changing compiler XX options make huge differences in inlining. This pull request has now been integrated. Changeset: 57b98fa5 Author: Eric Caspole URL: https://git.openjdk.java.net/jdk/commit/57b98fa5 Stats: 96 lines in 6 files changed: 91 ins; 0 del; 5 mod 8255965: LogCompilation: add sort by nmethod code size Reviewed-by: kvn, redestad ------------- PR: https://git.openjdk.java.net/jdk/pull/1085 From coleenp at openjdk.java.net Fri Nov 6 12:16:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 6 Nov 2020 12:16:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v7] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: - Merge branch 'master' into jvmti-table - Add back WeakProcessorPhases::Phase enum. - Serguei 1. - Code review comments from Kim and Albert. - Merge branch 'master' into jvmti-table - Merge branch 'master' into jvmti-table - More review comments from Stefan and ErikO - Code review comments from StefanK. - 8212879: Make JVMTI TagMap table not hash on oop address ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=06 Stats: 1748 lines in 41 files changed: 628 ins; 1018 del; 102 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From chris.hegarty at oracle.com Fri Nov 6 15:12:45 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 6 Nov 2020 15:12:45 +0000 Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> <9oQ_AyidieKnLIkJrx6koIR3TUJIqmxVmlzp82TX_64=.c44f863b-7873-498c-ae87-a922e0086fb9@github.com> Message-ID: > On 5 Nov 2020, at 18:11, Alex Buckley wrote: > > On 11/5/2020 4:45 AM, Jan Lahoda wrote: >> FWIW, a javadoc generated with the current version of the patch: >> http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/index.html > > Allow me to draw people's attention to the PREVIEW link in the banner of the generated javadoc. It shows all the preview APIs in the release on one page. This is very helpful for understanding the surface area of a preview feature. > > For example, with Sealed Classes being the only preview feature likely to target JDK 16, the PREVIEW page shows that the feature's API is solely about reflection. It's clear that Sealed Classes do not introduce, say, a java.lang.Sealed class analogous to the java.lang.Record class introduced by Records in JDK 14/15 (and which would have appeared on the PREVIEW page had it existed then). Very cool! ( I didn?t notice this until you pointed it out ;-) ) http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/preview-list.html There appears to be a very minor bug; when I click on the PREVIEW link in the banner, the page that lists the preview API points loads, but the banner does not ?highlight? that PREVIEW is ?selected?, but rather it ?highlights? DEPRECATED. -Chris. From jjg at openjdk.java.net Fri Nov 6 15:35:00 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Fri, 6 Nov 2020 15:35:00 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: <9oQ_AyidieKnLIkJrx6koIR3TUJIqmxVmlzp82TX_64=.c44f863b-7873-498c-ae87-a922e0086fb9@github.com> References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> <9oQ_AyidieKnLIkJrx6koIR3TUJIqmxVmlzp82TX_64=.c44f863b-7873-498c-ae87-a922e0086fb9@github.com> Message-ID: On Thu, 5 Nov 2020 12:43:03 GMT, Jan Lahoda wrote: >> Thanks @jonathan-gibbons for your comments! I've tried to update the code based on them, mostly in https://github.com/lahodaj/jdk/commit/743f516c660b577035cdda4510a0bb97937fd9b2 and https://github.com/lahodaj/jdk/commit/e4b02827998fc2e8f19f983aabfb3d720b03d111 >> >> A big chunk of the update is generalization of the deprecated and preview list builders and writers into a "summary" list builder and writer. These should also now handle records. For record components, those are a little tricky, as (AFAIK) can't currently have deprecation/preview-ness for them (and hence there is no good way to test any support for record components in these). But the summary build and writer are looking for record components and will fail in case a record component is sent into them. > > FWIW, a javadoc generated with the current version of the patch: > http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/index.html > > And a specdiff comparing it to the javadoc built from the corresponding master: > http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.specdiff.01/overview-summary.html The page and comments are generally good, but there is a bug that needs to be fixed. After clicking on the PREVIEW link in the top bar to go to the Preview page, the word DEPRECATED is highlighted in the top navbar instead of PREVIEW. -- Jon On 11/5/20 10:13 AM, mlbridge[bot] wrote: > > /Mailing list message from Alex Buckley > on kulla-dev > :/ > > On 11/5/2020 4:45 AM, Jan Lahoda wrote: > > FWIW, a javadoc generated with the current version of the patch: > http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/index.html > > Allow me to draw people's attention to the PREVIEW link in the banner of > the generated javadoc. It shows all the preview APIs in the release on > one page. This is very helpful for understanding the surface area of a > preview feature. > > For example, with Sealed Classes being the only preview feature likely > to target JDK 16, the PREVIEW page shows that the feature's API is > solely about reflection. It's clear that Sealed Classes do not > introduce, say, a java.lang.Sealed class analogous to the > java.lang.Record class introduced by Records in JDK 14/15 (and which > would have appeared on the PREVIEW page had it existed then). > > Alex > > ? > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , > or unsubscribe > . > ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Fri Nov 6 18:41:15 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Fri, 6 Nov 2020 18:41:15 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v11] In-Reply-To: References: Message-ID: > This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. > > The notable changes are: > > * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: > * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. > * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings > * improving error/warning messages. Please see [1] for a list of cases/examples. > * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. > * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). > * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. > * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. > > Please also see the CSR [6] for more information. > > [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html > [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html > [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html > [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html > [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ > [6] https://bugs.openjdk.java.net/browse/JDK-8250769 Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision: Fixing navigator for the PREVIEW page. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/703/files - new: https://git.openjdk.java.net/jdk/pull/703/files/097ae3c1..61264fe4 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=10 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=703&range=09-10 Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/703.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/703/head:pull/703 PR: https://git.openjdk.java.net/jdk/pull/703 From jlahoda at openjdk.java.net Fri Nov 6 18:41:15 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Fri, 6 Nov 2020 18:41:15 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v6] In-Reply-To: References: <0_WQkFp21--3oBLlFpJ-CN3ACeLwKEB21ByfD_LUl7Q=.831ed155-e36f-4bb9-8289-98e30042b65b@github.com> <3jLIRMTBm23tqzjxY8zwTvMCL7zaZkLgjEKAUiCFc78=.279fb6fb-2f7f-4602-8ed5-5df354276a3e@github.com> <9oQ_AyidieKnLIkJrx6koIR3TUJIqmxVmlzp82TX_64=.c44f863b-7873-498c-ae87-a922e0086fb9@github.com> Message-ID: On Fri, 6 Nov 2020 15:32:16 GMT, Jonathan Gibbons wrote: >> FWIW, a javadoc generated with the current version of the patch: >> http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/index.html >> >> And a specdiff comparing it to the javadoc built from the corresponding master: >> http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.specdiff.01/overview-summary.html > > The page and comments are generally good, but there is a bug that needs > to be fixed. > > After clicking on the PREVIEW link in the top bar to go to the Preview > page, the word DEPRECATED is highlighted in the top navbar instead of > PREVIEW. > > -- Jon > > > On 11/5/20 10:13 AM, mlbridge[bot] wrote: >> >> /Mailing list message from Alex Buckley >> on kulla-dev >> :/ >> >> On 11/5/2020 4:45 AM, Jan Lahoda wrote: >> >> FWIW, a javadoc generated with the current version of the patch: >> http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.01/api/index.html >> >> Allow me to draw people's attention to the PREVIEW link in the banner of >> the generated javadoc. It shows all the preview APIs in the release on >> one page. This is very helpful for understanding the surface area of a >> preview feature. >> >> For example, with Sealed Classes being the only preview feature likely >> to target JDK 16, the PREVIEW page shows that the feature's API is >> solely about reflection. It's clear that Sealed Classes do not >> introduce, say, a java.lang.Sealed class analogous to the >> java.lang.Record class introduced by Records in JDK 14/15 (and which >> would have appeared on the PREVIEW page had it existed then). >> >> Alex >> >> ? >> You are receiving this because you were mentioned. >> Reply to this email directly, view it on GitHub >> , >> or unsubscribe >> . >> Thanks for noticing Chris and @jonathan-gibbons. Fixed in 61264fe, updated javadoc: http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.02/api/preview-list.html ------------- PR: https://git.openjdk.java.net/jdk/pull/703 From alanb at openjdk.java.net Sun Nov 8 08:12:56 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 8 Nov 2020 08:12:56 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) In-Reply-To: <-u4eSM47o_e_KlfTRYBNGyLNhjqAeG-84u_uEd3ppH0=.c49b4269-1001-49d1-96fd-ecacbf2417e9@github.com> References: <-u4eSM47o_e_KlfTRYBNGyLNhjqAeG-84u_uEd3ppH0=.c49b4269-1001-49d1-96fd-ecacbf2417e9@github.com> Message-ID: On Wed, 4 Nov 2020 07:45:09 GMT, Alan Bateman wrote: >>> The javadoc for copyFrom isn't changed in this update but I notice it specifies IndexOutOfBoundException when the source segment is larger than the receiver, have other exceptions been examined? >> >> This exception is consistent with other uses of this exception throughout this API (e.g. when writing a segment out of bounds). > > I assume the CSR needs to be updated so that it's in sync with the API changes in the latest round. I see the xxxByteAtIndex methods that took a ByteOrder have been removed from MemoryAccess. Should the xxxByte and xxxByteAtOffset that take a ByteOrder be removed too? ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From serb at openjdk.java.net Sun Nov 8 10:23:54 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Sun, 8 Nov 2020 10:23:54 GMT Subject: RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 [v3] In-Reply-To: References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: On Thu, 5 Nov 2020 17:52:19 GMT, Phil Race wrote: >> This upgrades JDK to import the current 2.7.2 version of harfbuzz - an OpenType text shaping library >> >> https://bugs.openjdk.java.net/browse/JDK-8247872 >> >> This has passed building and headless and headful automated tests on all platforms. > > Phil Race has updated the pull request incrementally with one additional commit since the last revision: > > 8247872: Upgrade HarfBuzz to the latest 2.7.2 Marked as reviewed by serb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From alanb at openjdk.java.net Sun Nov 8 16:31:58 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Sun, 8 Nov 2020 16:31:58 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v22] In-Reply-To: References: Message-ID: On Thu, 5 Nov 2020 17:14:16 GMT, Maurizio Cimadamore wrote: >> This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: >> >> * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads >> * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually >> * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. >> >> A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. >> >> This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). >> >> A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. >> >> A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. >> >> Thanks >> Maurizio >> >> Javadoc: >> >> http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html >> >> Specdiff: >> >> http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html >> >> CSR: >> >> https://bugs.openjdk.java.net/browse/JDK-8254163 >> >> >> >> ### API Changes >> >> * `MemorySegment` >> * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) >> * added a no-arg factory for a native restricted segment representing entire native heap >> * rename `withOwnerThread` to `handoff` >> * add new `share` method, to create shared segments >> * add new `registerCleaner` method, to register a segment against a cleaner >> * add more helpers to create arrays from a segment e.g. `toIntArray` >> * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) >> * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) >> * `MemoryAddress` >> * drop `segment` accessor >> * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment >> * `MemoryAccess` >> * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). >> * `MemoryHandles` >> * drop `withOffset` combinator >> * drop `withStride` combinator >> * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. >> * `Addressable` >> * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. >> * `MemoryLayouts` >> * A new layout, for machine addresses, has been added to the mix. >> >> >> >> ### Implementation changes >> >> There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. >> >> #### Shared segments >> >> The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. >> >> After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. >> >> Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). >> >> The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. >> >> As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. >> >> In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. >> >> To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). >> >> Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). >> >> `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. >> >> The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. >> >> #### Memory access var handles overhaul >> >> The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. >> >> This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. >> >> This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. >> >> #### Test changes >> >> Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. >> >> [1] - https://openjdk.java.net/jeps/393 >> [2] - https://openjdk.java.net/jeps/389 >> [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html >> [4] - https://openjdk.java.net/jeps/312 > > Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 29 commits: > > - Fix post-merge issues caused by 8219014 > - Merge branch 'master' into 8254162 > - Addess remaining feedback from @AlanBateman and @mrserb > - Address comments from @AlanBateman > - Merge branch 'master' into 8254162 > - Fix issues with derived buffers and IO operations > - More 32-bit fixes for TestLayouts > - * Add final to MappedByteBuffer::SCOPED_MEMORY_ACCESS field > * Tweak TestLayouts to make it 32-bit friendly after recent MemoryLayouts tweaks > - Remove TestMismatch from 32-bit problem list > - Merge branch 'master' into 8254162 > - ... and 19 more: https://git.openjdk.java.net/jdk/compare/a50fdd54...02f9e251 Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From prr at openjdk.java.net Sun Nov 8 17:08:56 2020 From: prr at openjdk.java.net (Phil Race) Date: Sun, 8 Nov 2020 17:08:56 GMT Subject: Integrated: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: On Mon, 2 Nov 2020 04:19:57 GMT, Phil Race wrote: > This upgrades JDK to import the current 2.7.2 version of harfbuzz - an OpenType text shaping library > > https://bugs.openjdk.java.net/browse/JDK-8247872 > > This has passed building and headless and headful automated tests on all platforms. This pull request has now been integrated. Changeset: ed7526a6 Author: Phil Race URL: https://git.openjdk.java.net/jdk/commit/ed7526a6 Stats: 44664 lines in 206 files changed: 25198 ins; 12729 del; 6737 mod 8247872: Upgrade HarfBuzz to the latest 2.7.2 Reviewed-by: serb ------------- PR: https://git.openjdk.java.net/jdk/pull/993 From jiefu at openjdk.java.net Mon Nov 9 12:14:03 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 9 Nov 2020 12:14:03 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project Message-ID: Hi all, The gitignore setting for netbeans project seems to be incomplete. Only nbproject/private/ [1] is ignored. But there are also other files under nbproject/, which should be ignored too. ---------------------------- $ tree -L 1 nbproject/ nbproject/ ??? configurations.xml ??? private ??? project.xml ---------------------------- It would be better to fix it. Thanks. Best regards, Jie [1] https://github.com/openjdk/jdk/blob/master/.gitignore#L5 ------------- Commit messages: - 8256048: Incomplete gitignore setting for netbeans project Changes: https://git.openjdk.java.net/jdk/pull/1122/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1122&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256048 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1122.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1122/head:pull/1122 PR: https://git.openjdk.java.net/jdk/pull/1122 From mcimadamore at openjdk.java.net Mon Nov 9 12:28:13 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 9 Nov 2020 12:28:13 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v23] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Remove endianness-aware byte getter/setter in MemoryAccess Remove index-based version of byte getter/setter in MemoryAccess ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/548/files - new: https://git.openjdk.java.net/jdk/pull/548/files/02f9e251..6940f0ac Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=22 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=21-22 Stats: 110 lines in 3 files changed: 0 ins; 98 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From mcimadamore at openjdk.java.net Mon Nov 9 12:28:16 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 9 Nov 2020 12:28:16 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v22] In-Reply-To: References: Message-ID: On Sun, 8 Nov 2020 16:28:41 GMT, Alan Bateman wrote: >> Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 29 commits: >> >> - Fix post-merge issues caused by 8219014 >> - Merge branch 'master' into 8254162 >> - Addess remaining feedback from @AlanBateman and @mrserb >> - Address comments from @AlanBateman >> - Merge branch 'master' into 8254162 >> - Fix issues with derived buffers and IO operations >> - More 32-bit fixes for TestLayouts >> - * Add final to MappedByteBuffer::SCOPED_MEMORY_ACCESS field >> * Tweak TestLayouts to make it 32-bit friendly after recent MemoryLayouts tweaks >> - Remove TestMismatch from 32-bit problem list >> - Merge branch 'master' into 8254162 >> - ... and 19 more: https://git.openjdk.java.net/jdk/compare/a50fdd54...02f9e251 > > Marked as reviewed by alanb (Reviewer). > I see the xxxByteAtIndex methods that took a ByteOrder have been removed from MemoryAccess. Should the xxxByte and xxxByteAtOffset that take a ByteOrder be removed too? I've addresses this in the latest iteration. Since I was there I also removed `getByteAtIndex` and `getByteAtIndex`, since their behavior is identical to that of `getByteAtOffset` and `setByteAtOffset`, respectively (in other words, the indexed variants are not really helpful until carrier size > 1 byte). ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From mcimadamore at openjdk.java.net Mon Nov 9 13:22:14 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 9 Nov 2020 13:22:14 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v24] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Improve debugging output of TestHandhsake ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/548/files - new: https://git.openjdk.java.net/jdk/pull/548/files/6940f0ac..f5d339a7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=23 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=22-23 Stats: 37 lines in 1 file changed: 11 ins; 1 del; 25 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From erikj at openjdk.java.net Mon Nov 9 13:56:00 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 9 Nov 2020 13:56:00 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project In-Reply-To: References: Message-ID: <9tWC1JJvMV865mtS9HsFZb03EHSccdshjh-Kmf9kntU=.b7150925-2223-458c-9723-4cfd9a0d0689@github.com> On Mon, 9 Nov 2020 12:07:37 GMT, Jie Fu wrote: > Hi all, > > The gitignore setting for netbeans project seems to be incomplete. > Only nbproject/private/ [1] is ignored. > > But there are also other files under nbproject/, which should be ignored too. > ---------------------------- > $ tree -L 1 nbproject/ > nbproject/ > ??? configurations.xml > ??? private > ??? project.xml > ---------------------------- > > It would be better to fix it. > > Thanks. > Best regards, > Jie > > > [1] https://github.com/openjdk/jdk/blob/master/.gitignore#L5 .gitignore line 5: > 3: /.idea/ > 4: /.vscode/ > 5: /nbproject/ I believe the existing nbproject/private/ is meant to catch any private files created by someone opening any of the various nbproject directories that exist in the source base, so it should probably stay. Adding an ignore for any top level nbproject directory in addition to that should be ok. ------------- PR: https://git.openjdk.java.net/jdk/pull/1122 From jiefu at openjdk.java.net Mon Nov 9 15:25:08 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 9 Nov 2020 15:25:08 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project [v2] In-Reply-To: References: Message-ID: > Hi all, > > The gitignore setting for netbeans project seems to be incomplete. > Only nbproject/private/ [1] is ignored. > > But there are also other files under nbproject/, which should be ignored too. > ---------------------------- > $ tree -L 1 nbproject/ > nbproject/ > ??? configurations.xml > ??? private > ??? project.xml > ---------------------------- > > It would be better to fix it. > > Thanks. > Best regards, > Jie > > > [1] https://github.com/openjdk/jdk/blob/master/.gitignore#L5 Jie Fu has updated the pull request incrementally with one additional commit since the last revision: Don't remove nbproject/private/ ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1122/files - new: https://git.openjdk.java.net/jdk/pull/1122/files/573612e4..f48474eb Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1122&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1122&range=00-01 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1122.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1122/head:pull/1122 PR: https://git.openjdk.java.net/jdk/pull/1122 From mcimadamore at openjdk.java.net Mon Nov 9 15:26:13 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 9 Nov 2020 15:26:13 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v25] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Further improve output of TestHandshake ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/548/files - new: https://git.openjdk.java.net/jdk/pull/548/files/f5d339a7..27677a13 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=24 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=23-24 Stats: 8 lines in 1 file changed: 4 ins; 2 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From jiefu at openjdk.java.net Mon Nov 9 15:27:58 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 9 Nov 2020 15:27:58 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project [v2] In-Reply-To: <9tWC1JJvMV865mtS9HsFZb03EHSccdshjh-Kmf9kntU=.b7150925-2223-458c-9723-4cfd9a0d0689@github.com> References: <9tWC1JJvMV865mtS9HsFZb03EHSccdshjh-Kmf9kntU=.b7150925-2223-458c-9723-4cfd9a0d0689@github.com> Message-ID: On Mon, 9 Nov 2020 13:53:35 GMT, Erik Joelsson wrote: >> Jie Fu has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't remove nbproject/private/ > > .gitignore line 5: > >> 3: /.idea/ >> 4: /.vscode/ >> 5: /nbproject/ > > I believe the existing nbproject/private/ is meant to catch any private files created by someone opening any of the various nbproject directories that exist in the source base, so it should probably stay. Adding an ignore for any top level nbproject directory in addition to that should be ok. Thanks @erikj79 for your review. Updated. Any comments? Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/1122 From mcimadamore at openjdk.java.net Mon Nov 9 16:07:13 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Mon, 9 Nov 2020 16:07:13 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v26] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Add more output in TestHandhsake.java ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/548/files - new: https://git.openjdk.java.net/jdk/pull/548/files/27677a13..a22b6b5b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=25 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=24-25 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From erikj at openjdk.java.net Mon Nov 9 18:05:56 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 9 Nov 2020 18:05:56 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project [v2] In-Reply-To: References: <9tWC1JJvMV865mtS9HsFZb03EHSccdshjh-Kmf9kntU=.b7150925-2223-458c-9723-4cfd9a0d0689@github.com> Message-ID: <-4dB2aO2qpIRecSft4uehI_IxY6wbdlNC4f00PJdKUY=.dc1beb62-07eb-47fb-a420-10bfb23c4d5f@github.com> On Mon, 9 Nov 2020 15:25:28 GMT, Jie Fu wrote: >> .gitignore line 5: >> >>> 3: /.idea/ >>> 4: /.vscode/ >>> 5: /nbproject/ >> >> I believe the existing nbproject/private/ is meant to catch any private files created by someone opening any of the various nbproject directories that exist in the source base, so it should probably stay. Adding an ignore for any top level nbproject directory in addition to that should be ok. > > Thanks @erikj79 for your review. > Updated. > > Any comments? > Thanks. You misunderstand me. Adding the following line is fine and will make git ignore any nbproject dir in the root of the repository, which is what I assume you want: But removing the following existing line is not, as this matches any potential private part of already existing nbproject directories: nbproject/private/ ------------- PR: https://git.openjdk.java.net/jdk/pull/1122 From erikj at openjdk.java.net Mon Nov 9 18:05:56 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 9 Nov 2020 18:05:56 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project [v2] In-Reply-To: <-4dB2aO2qpIRecSft4uehI_IxY6wbdlNC4f00PJdKUY=.dc1beb62-07eb-47fb-a420-10bfb23c4d5f@github.com> References: <9tWC1JJvMV865mtS9HsFZb03EHSccdshjh-Kmf9kntU=.b7150925-2223-458c-9723-4cfd9a0d0689@github.com> <-4dB2aO2qpIRecSft4uehI_IxY6wbdlNC4f00PJdKUY=.dc1beb62-07eb-47fb-a420-10bfb23c4d5f@github.com> Message-ID: On Mon, 9 Nov 2020 18:02:32 GMT, Erik Joelsson wrote: >> Thanks @erikj79 for your review. >> Updated. >> >> Any comments? >> Thanks. > > You misunderstand me. Adding the following line is fine and will make git ignore any nbproject dir in the root of the repository, which is what I assume you want: > > > But removing the following existing line is not, as this matches any potential private part of already existing nbproject directories: > nbproject/private/ Adding the below will make git ignore a bunch of already existing nbproject files. nbproject/configurations.xml nbproject/project.xml ------------- PR: https://git.openjdk.java.net/jdk/pull/1122 From coleenp at openjdk.java.net Mon Nov 9 20:43:06 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 9 Nov 2020 20:43:06 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Thu, 5 Nov 2020 12:42:40 GMT, Coleen Phillimore wrote: >> Thank you for the update, Coleen! >> I leave it for you to decide to refactor the gc_notification or not. >> Thanks, >> Serguei > > Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change. This change also passes tier 7,8 testing. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From jiefu at openjdk.java.net Mon Nov 9 23:31:10 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 9 Nov 2020 23:31:10 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project [v3] In-Reply-To: References: <9tWC1JJvMV865mtS9HsFZb03EHSccdshjh-Kmf9kntU=.b7150925-2223-458c-9723-4cfd9a0d0689@github.com> <-4dB2aO2qpIRecSft4uehI_IxY6wbdlNC4f00PJdKUY=.dc1beb62-07eb-47fb-a420-10bfb23c4d5f@github.com> Message-ID: On Mon, 9 Nov 2020 18:03:31 GMT, Erik Joelsson wrote: >> You misunderstand me. Adding the following line is fine and will make git ignore any nbproject dir in the root of the repository, which is what I assume you want: >> >> >> But removing the following existing line is not, as this matches any potential private part of already existing nbproject directories: >> nbproject/private/ > > Adding the below will make git ignore a bunch of already existing nbproject files. > nbproject/configurations.xml > nbproject/project.xml Got it. Thanks @erikj79 for your kind explanation. Updated. ------------- PR: https://git.openjdk.java.net/jdk/pull/1122 From jiefu at openjdk.java.net Mon Nov 9 23:31:09 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 9 Nov 2020 23:31:09 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project [v3] In-Reply-To: References: Message-ID: > Hi all, > > The gitignore setting for netbeans project seems to be incomplete. > Only nbproject/private/ [1] is ignored. > > But there are also other files under nbproject/, which should be ignored too. > ---------------------------- > $ tree -L 1 nbproject/ > nbproject/ > ??? configurations.xml > ??? private > ??? project.xml > ---------------------------- > > It would be better to fix it. > > Thanks. > Best regards, > Jie > > > [1] https://github.com/openjdk/jdk/blob/master/.gitignore#L5 Jie Fu has updated the pull request incrementally with one additional commit since the last revision: Updated v3 ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1122/files - new: https://git.openjdk.java.net/jdk/pull/1122/files/f48474eb..06d0c766 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1122&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1122&range=01-02 Stats: 3 lines in 1 file changed: 1 ins; 2 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1122.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1122/head:pull/1122 PR: https://git.openjdk.java.net/jdk/pull/1122 From erikj at openjdk.java.net Mon Nov 9 23:50:58 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 9 Nov 2020 23:50:58 GMT Subject: RFR: 8256048: Incomplete gitignore setting for netbeans project [v3] In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 23:31:09 GMT, Jie Fu wrote: >> Hi all, >> >> The gitignore setting for netbeans project seems to be incomplete. >> Only nbproject/private/ [1] is ignored. >> >> But there are also other files under nbproject/, which should be ignored too. >> ---------------------------- >> $ tree -L 1 nbproject/ >> nbproject/ >> ??? configurations.xml >> ??? private >> ??? project.xml >> ---------------------------- >> >> It would be better to fix it. >> >> Thanks. >> Best regards, >> Jie >> >> >> [1] https://github.com/openjdk/jdk/blob/master/.gitignore#L5 > > Jie Fu has updated the pull request incrementally with one additional commit since the last revision: > > Updated v3 Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1122 From shade at openjdk.java.net Tue Nov 10 07:22:07 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 10 Nov 2020 07:22:07 GMT Subject: RFR: 8255822: Zero: improve build-time JVMTI handling [v3] In-Reply-To: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> References: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> Message-ID: > Current Zero interpreter has the optimization for JVMTI support. It recognizes that JVMTI is disabled most of the time, and that JVMTI checks in the interpreter code slows it down considerably. (I measured it myself when working on this patch: removing this optimization yields about 20% hit in build times). > > Current optimization works as follows. At build time, an XSLT transform is performed on `bytecodeInterpreter.cpp`, yielding `bytecodeInterpreterWithChecks.cpp`. In that new compilation unit, `VM_JVMTI` macro is defined, and a new entry point -- `BytecodeInterpreter::withChecks` -- is defined. Then, both compilation units are compiled. In one of them, `JVMTI` hooks are stripped out. In another, they persist. Then, callers have to choose which entry point to use. > > I believe this can be rewritten to use C++ templates instead of XLST and defines dance. This also allows to clean up JVMTI checks a bit. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug build with `-jvmti` > - [x] Linux x86_64 Zero fastdebug/release build times are not regressing Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: - Simplify a few JVMTI_ENABLED blocks - Merge branch 'master' into JDK-8255822-zero-jvmti-rework - Fix build error - Merge branch 'master' into JDK-8255822-zero-jvmti-rework - Revert one dubious change - 8255822: Zero: improve build-time JVMTI handling Summary: use C++ templates instead of XSLT transforms ------------- Changes: https://git.openjdk.java.net/jdk/pull/1061/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1061&range=02 Stats: 188 lines in 6 files changed: 5 ins; 128 del; 55 mod Patch: https://git.openjdk.java.net/jdk/pull/1061.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1061/head:pull/1061 PR: https://git.openjdk.java.net/jdk/pull/1061 From serb at openjdk.java.net Tue Nov 10 08:31:04 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Tue, 10 Nov 2020 08:31:04 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU Message-ID: This is a review request for the bug particularly fixed some time ago: https://mail.openjdk.java.net/pipermail/2d-dev/2015-May/005425.html In that review request it was found that the old fix does not work well in all cases, see: https://mail.openjdk.java.net/pipermail/2d-dev/2015-August/005611.html The current fix updates an embedded plist.info, so the java will not require discrete graphics by default, same as for any other applications. The discrete card will be used: 1. If macOS decided to enable it for some reason 2. If the java app sets/uses a full-screen window 3. If the user disable "automatic graphics switching" in the system preferences 4. If an external monitor is connected to the laptop In other cases, the integrated graphics will be used, which should be fine in most cases since this graphic is used/tested in the mbp 13/etc. This is not only about rendering performance but also about startup performance, on my current new laptop mbp 16 + Cataline 10.15.7 the switching discrete/integrated causes unexpected delays up to 10 seconds. Note that the new "metal" pipeline also does not require discrete graphics. The documentation for NSSupportsAutomaticGraphicsSwitching: https://developer.apple.com/documentation/bundleresources/information_property_list/nssupportsautomaticgraphicsswitching I'll create a release note after approval. Performance numbers: https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010979.html Old review request: https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010973.html ------------- Commit messages: - Initial fix Changes: https://git.openjdk.java.net/jdk/pull/1139/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1139&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8251854 Stats: 6 lines in 3 files changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1139.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1139/head:pull/1139 PR: https://git.openjdk.java.net/jdk/pull/1139 From beurba at microsoft.com Tue Nov 10 10:01:55 2020 From: beurba at microsoft.com (Bernhard Urban-Forster) Date: Tue, 10 Nov 2020 10:01:55 +0000 Subject: Preliminary review for new WINENV support In-Reply-To: References: , Message-ID: Hello Magnus, Sorry for the late reply! This is awesome work and I don't want to see that to bit rot :-) I rebased your changes on top of current master: https://github.com/openjdk/jdk/compare/master...lewurm:winenv-testing This branch is by no means ready to be reviewed, it's just whatever I've used to do my experiments. Also note that it was a bit painful to merge the changes in toolchain_windows.m4 due to the renaming. While I agree that toolchain_microsoft.m4 is the better name, I suggest to defer that change to a later point. I've tested the following scenarios: 1. cygwin on x86_64 to build x86_64 2. cygwin on x86_64 to build arm64 3. WSL1 on x86_64 to build x86_64 4. WSL1 on x86_64 to build arm64 5. WSL2 on x86_64 to build x86_64 6. WSL1 on arm64 to build arm64 Some context: Microsoft doesn't ship native Arm64 binaries for the VS toolchain (yet). Instead one can run x86 binaries via xtajit (a binary translator), but as one can imagine this comes with some performance penalty. Cygwin is also not available on Arm64. Cygwin already being slow, it made sense to avoid running Cygwin x86 on Arm64, and instead go the cross-compilation route in the OpenJDK build for Windows+Arm64. Previously the OpenJDK build did not support cross-compiling on Windows. Even today, on openjdk/jdk:master a dirty workaround for fixpath.exe is needed to build Windows+Arm64 ( https://github.com/openjdk/jdk/pull/212#issuecomment-695024586 ). So we have been very excited to see this WINENV patch :-) Here are comments for each scenario. ====================================== ## 1. cygwin on x86_64 to build x86_64 ====================================== ``` $ bash configure --with-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-build-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-boot-jdk=/cygdrive/c/work/jdk-16+22 --openjdk-target=aarch64-unknown-cygwin configure: Configuration created at Mon Nov 9 21:33:42 CET 2020. checking for basename... /usr/bin/basename checking for dirname... /usr/bin/dirname checking for file... /usr/bin/file checking for ldd... /usr/bin/ldd checking for bash... /usr/bin/bash checking for cat... /usr/bin/cat checking for chmod... /usr/bin/chmod checking for cp... /usr/bin/cp checking for cut... /usr/bin/cut checking for date... /usr/bin/date checking for gdiff... /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: /cygdrive/c/Program: binary operator expected /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: /cygdrive/c/Program: binary operator expected /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments [not found] [...] ``` I didn't investigate it any further, as this has already been reported on the mailing list. ===================================== ## 2. cygwin on x86_64 to build arm64 ===================================== ``` $ bash configure --with-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-build-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-boot-jdk=/cygdrive/c/work/jdk-16+22 --openjdk-target=aarch64-unknown-cygwin ``` Same as scenario 1) ==================================== ## 3. WSL1 on x86_64 to build x86_64 ==================================== I do get some weird permission errors, so I run everything with sudo (chmod -R $user doesn't help): ``` $ sudo bash configure --with-build-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-boot-jdk="/mnt/c/work/jdk-16+22" [...] Configuration summary: * Debug level: release * HS debug level: product * JVM variants: server * JVM features: server: 'aot cds compiler1 compiler2 epsilongc g1gc graal jfr jni-check jvmci jvmti management nmt parallelgc serialgc services shenandoahgc vm-structs zgc' * OpenJDK target: OS: windows, CPU architecture: x86, address length: 64 * Version string: 16-internal+0-adhoc.root.winenv-wsl1 (16-internal) Tools summary: * Environment: wsl1 version 4.4.0-19041-Microsoft, #488-Microsoft Mon Sep 01 13:43:00 PST 2020 (Ubuntu 20.04.1 LTS); windows version 10.0.19041.572; prefix "/mnt"; root "[unavailable]" * Boot JDK: openjdk version "16" 2021-03-16 OpenJDK Runtime Environment AdoptOpenJDK (build 16+22-202011040337) OpenJDK 64-Bit Server VM AdoptOpenJDK (build 16+22-202011040337, mixed mode, sharing) (at /mnt/c/work/jdk-16~1) * Toolchain: microsoft (Microsoft Visual Studio 2019 16.6.1 (devkit)) * C Compiler: Version 19.26.28806 (at /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/x64/cl.exe) * C++ Compiler: Version 19.26.28806 (at /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/x64/cl.exe) Build performance summary: * Cores to use: 12 * Memory limit: 32605 MB ``` The build fails eventually: ``` $ sudo make jdk-image JOBS=12 LOG=info [...] /mnt/c/work/winenv-wsl1/make/scripts/fixpath.sh: line 405: /mnt/c/work/winenv-wsl1/build/windows-x86_64-server-release/jdk/bin/java: No such file or directory make[3]: *** [ExplodedImageOptimize.gmk:41: /mnt/c/work/winenv-wsl1/build/windows-x86_64-server-release/jdk/_optimize_image_exec.marker] Error 127 make[2]: *** [make/Main.gmk:437: exploded-image-optimize] Error 2 make[2]: *** Waiting for unfinished jobs.... ``` Which looks like a path issue, that should be relatively easy to fix (I didn't investigate). HotSpot and parts of the JDK built successfully though: ``` $ ./build/windows-x86_64-server-release/jdk/bin/java.exe -version openjdk version "16-internal" 2021-03-16 OpenJDK Runtime Environment (build 16-internal+0-adhoc.root.winenv-wsl1) OpenJDK 64-Bit Server VM (build 16-internal+0-adhoc.root.winenv-wsl1, mixed mode) ``` So I declare success on this one :-) =================================== ## 4. WSL1 on x86_64 to build arm64 =================================== ``` $ bash configure --with-build-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-boot-jdk="/mnt/c/work/jdk-16+22" --openjdk-target=aarch64-unknown-cygwin [...] checking for link... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/arm64/link.exe checking if the found link.exe is actually the Visual Studio linker... yes configure: Using microsoft linker version 14.26.28806.0 [Microsoft (R) Incremental Linker Version 14.26.28806.0] checking for aarch64-unknown-cygwin-ml... [not found] checking for ml... [not found] configure: WARNING: using cross tools not prefixed with host triplet configure: WARNING: Ignoring value of AS from the environment. Use command line variables instead. checking for ml... [not found] checking for aarch64-unknown-cygwin-lib... [not found] checking for lib... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/arm64/lib.exe checking for aarch64-unknown-cygwin-mt... [not found] checking for mt... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/10/bin/x64/mt.exe checking for aarch64-unknown-cygwin-rc... [not found] checking for rc... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/10/bin/x64/rc.exe checking for aarch64-unknown-cygwin-dumpbin... [not found] checking for dumpbin... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/arm64/dumpbin.exe checking for aarch64-unknown-cygwin-gobjdump... [not found] checking for gobjdump... [not found] checking for aarch64-unknown-cygwin-objdump... [not found] checking for objdump... /usr/bin/objdump checking for build platform devkit... Microsoft Visual Studio 2019 16.6.1 (devkit) in /mnt/c/work/vs2019-16.6.1-devkit checking for cl... fixpath: failure: Cannot locate cygpath or wslpath configure: error: Could not find required tool for BUILD_CC /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 12: sed: command not found /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 38: sort: command not found /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 35: sed: command not found /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 53: sort: command not found /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 77: cat: command not found /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 40: rm: command not found ``` It looks like it's still not suitable for cross compilation. I might have messed up resolving some merge conflicts. I haven't investigated that further, but that would probably be a scenario that would be nice to have eventually. ==================================== ## 5. WSL2 on x86_64 to build x86_64 ==================================== I second the clock skew observations :-( I had a problem like that on WSL2 in a different context before https://github.com/microsoft/WSL/issues/4898 Apart from that, crossing the "file system boundary" got worse with WSL2, and as long checking out the sources on a Windows visible path is a requirment, I'm not sure if it's worth going down this road. ================================== ## 6. WSL1 on arm64 to build arm64 ================================== Same problem with permissions. I must be doing something wrong. ``` $ sudo bash configure --with-devkit="/mnt/c/lewurm/VS2019-16.6.1-devkit" --with-boot-jdk="/mnt/c/lewurm/jdk-16-ea+19-windows-aarch64" --disable-precompiled-headers [...] Configuration summary: * Debug level: release * HS debug level: product * JVM variants: server * JVM features: server: 'cds compiler1 compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc serialgc services shenandoahgc vm-structs zgc' * OpenJDK target: OS: windows, CPU architecture: aarch64, address length: 64 * Version string: 16-internal+0-adhoc.root.openjdk-jdk (16-internal) Tools summary: * Environment: wsl1 version 4.4.0-19607-Microsoft, #1000-Microsoft Wed Apr 09 15:02:00 PST 2020 (Ubuntu 20.04 LTS); windows version 10.0.19607.1000; prefix "/mnt"; root "[unavailable]" * Boot JDK: openjdk version "16-ea" 2020-10-09 OpenJDK Runtime Environment Microsoft (build 16-ea+19-windows-aarch64) OpenJDK 64-Bit Server VM Microsoft (build 16-ea+19-windows-aarch64, mixed mode) (at /mnt/c/lewurm/jdk-16~1) * Toolchain: microsoft (Microsoft Visual Studio 2019 16.6.1 (devkit)) * C Compiler: Version 19.27.29112 (at /mnt/c/lewurm/vs2019-16.6.1-devkit/vc/bin/arm64/cl.exe) * C++ Compiler: Version 19.27.29112 (at /mnt/c/lewurm/vs2019-16.6.1-devkit/vc/bin/arm64/cl.exe) Build performance summary: * Cores to use: 64 * Memory limit: 524204 MB ``` So a couple of things that I hacked up: * I manually copied x86 MSVC binaries into /mnt/c/lewurm/vs2019-16.6.1-devkit/vc/bin/arm64 * I manually copied arm64 binaries from the Windows SDK into /mnt/c/lewurm/vs2019-16.6.1-devkit/10/bin/arm64 Eventually I need to update the devkit script to create a proper archive and/or update the build script to be aware that x86 MSVC binaries must be used on arm64 (the hope is that this eventually won't be true anymore). * add detection for WSL on aarch64: https://github.com/lewurm/openjdk/commit/4da4b6df16d05c97b7f48bef166e6d2080d37511 * The boot-jdk is from https://github.com/microsoft/openjdk-aarch64/releases * `--disable-precompiled-headers` is needed because otherwise a bug in the binary translator (xtajit) is triggered when running x86 cl.exe with it. Note to myself: File a bug report. Let's build a jdk image: ``` $ sudo make jdk-image JOBS=64 LOG=info [...] /mnt/c/lewurm/openjdk-jdk/make/scripts/fixpath.sh: line 405: /mnt/c/lewurm/openjdk-jdk/build/windows-aarch64-server-release/jdk/bin/java: No such file or directory make[3]: *** [ExplodedImageOptimize.gmk:41: /mnt/c/lewurm/openjdk-jdk/build/windows-aarch64-server-release/jdk/_optimize_image_exec.marker] Error 127 make[2]: *** [make/Main.gmk:437: exploded-image-optimize] Error 2 $ ./build/windows-aarch64-server-release/jdk/bin/java.exe -version openjdk version "16-internal" 2021-03-16 OpenJDK Runtime Environment (build 16-internal+0-adhoc.root.openjdk-jdk) OpenJDK 64-Bit Server VM (build 16-internal+0-adhoc.root.openjdk-jdk, mixed mode) ``` So same issue as in scenario 3 (WSL1 on x86_64 building x86_64), which is _great_ news :-) ============== ## Performance ============== I did a comparision run on WSL1 on x86_64 to build x86_64 (scenario 3), once with the WINENV patches applied and once without it. Overall it seems there is some performance regression, but not too bad. The profile logs aren't very meaningful to me, but if I would need to guess the difference is due to having a native `fixpath.exe` vs. `fixpath.sh`. Logs are here: https://gist.github.com/lewurm/2fbf2971a55a3a4a4f5933b105925976 ========= ## Epilog ========= I hope this has met your bar of exotic setup. Unfortunately I can't provide much feedback on the automagic. I would love to see your work landing eventually! Thank you, -Bernhard ________________________________________ From: build-dev on behalf of Magnus Ihse Bursie Sent: Friday, August 28, 2020 11:09 To: Ludovic Henry; Yasumasa Suenaga; Monica Beckwith; build-dev Subject: Re: Preliminary review for new WINENV support On 2020-08-24 19:21, Ludovic Henry wrote: > Hi Magnus, > > I want to follow up on this work of yours, as we've particular interest in it for the Windows-AArch64 port. > > Let me know how I could assist you in this effort. The big blockers for this fix is basically these two issues: >> I have also temporarily disabled the javac server, and building without absolute paths. I believe a better way forward is to address these with improved functionality in separate patches, instead of trying to work around the issues introduced by them in this patch. Until that is fixed, any integration of this patch is a no-go. I have been working on those, but a lot of other stuff has kept me from gaining much progress; everything from long vacations to preparations to the switch to github. Unfortunately, I don't think there's an easy way for you to help on those two blockers. However, there are two more things that I'd like to look into, if possible, before integrating my patch, and there you might be able to help. One is performance. I got a somewhat different performance profile with my patch than without -- some things were faster, and some were slower. I'd like to understand why some things got slower, and if it is possible to do anything about. (Why things got faster I don't care about, I'm just happy :)) So maybe you can help me with some performance analysis? With the limited availability of tooling for makefiles, this is not really trivial. What I usually do is I run with JOBS=1 (to be able to track each task individually without effects of concurrency), and LOG=info,profile. This will give you time stamps for each individual execution in build.log. And see if there is any interesting differences with and without the patch. The other thing is that WSL2 works awfully bad. :( Maybe there's nothing to be done about it, if the thing being shipped is just too buggy, but maybe we do things incorrectly, or that unnecessarily triggers bugs. Or maybe there are workarounds we could use. But I think it could be worth some more effort to try to get it working. Maybe you can help with this? /Magnus From alanb at openjdk.java.net Tue Nov 10 11:03:55 2020 From: alanb at openjdk.java.net (Alan Bateman) Date: Tue, 10 Nov 2020 11:03:55 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v26] In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 16:07:13 GMT, Maurizio Cimadamore wrote: >> This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: >> >> * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads >> * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually >> * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. >> >> A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. >> >> This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). >> >> A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. >> >> A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. >> >> Thanks >> Maurizio >> >> Javadoc: >> >> http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html >> >> Specdiff: >> >> http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html >> >> CSR: >> >> https://bugs.openjdk.java.net/browse/JDK-8254163 >> >> >> >> ### API Changes >> >> * `MemorySegment` >> * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) >> * added a no-arg factory for a native restricted segment representing entire native heap >> * rename `withOwnerThread` to `handoff` >> * add new `share` method, to create shared segments >> * add new `registerCleaner` method, to register a segment against a cleaner >> * add more helpers to create arrays from a segment e.g. `toIntArray` >> * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) >> * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) >> * `MemoryAddress` >> * drop `segment` accessor >> * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment >> * `MemoryAccess` >> * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). >> * `MemoryHandles` >> * drop `withOffset` combinator >> * drop `withStride` combinator >> * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. >> * `Addressable` >> * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. >> * `MemoryLayouts` >> * A new layout, for machine addresses, has been added to the mix. >> >> >> >> ### Implementation changes >> >> There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. >> >> #### Shared segments >> >> The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. >> >> After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. >> >> Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). >> >> The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. >> >> As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. >> >> In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. >> >> To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). >> >> Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). >> >> `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. >> >> The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. >> >> #### Memory access var handles overhaul >> >> The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. >> >> This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. >> >> This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. >> >> #### Test changes >> >> Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. >> >> [1] - https://openjdk.java.net/jeps/393 >> [2] - https://openjdk.java.net/jeps/389 >> [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html >> [4] - https://openjdk.java.net/jeps/312 > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Add more output in TestHandhsake.java Marked as reviewed by alanb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From magnus.ihse.bursie at oracle.com Tue Nov 10 11:30:21 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 10 Nov 2020 12:30:21 +0100 Subject: Preliminary review for new WINENV support In-Reply-To: References: Message-ID: Hi Bernhard, Thank you for you efforts in keeping this patch viable and getting it closer to integration! I have no intention of letting this bitrot. The patch is, still, blocked by major simplifications I made: ?* There is no support for the javac server ?* There is no support for building without absolute pathnames This functionality is a must-have. The good news is that there is a relative clear way forward for me to achieve both of these, which in both cases involves fixing the code base: to simplify how we pass arguments to the javac server and how we build without absolute pathnames on Windows. And in both cases these will be good additions in their own right. After that, all irregularities must be fixed, so there is no regression in terms of building functionality or major performance regressions on already supported platforms. Expanding to support new compilation combinations is certainly a nice addition, but not one that's required for an initial checkin. I apologize for the slow progress on this patch. Due to personal and family issues, I am unfortunately unable to work on the OpenJDK build as much as I'd like, and the time I can spend is somewhat unpredictable. But I can assure you that this patch is important for me as well, and I'm trying to prioritize it as much as I can. /Magnus On 2020-11-10 11:01, Bernhard Urban-Forster wrote: > Hello Magnus, > > Sorry for the late reply! This is awesome work and I don't want to see that to bit rot :-) > > I rebased your changes on top of current master: https://github.com/openjdk/jdk/compare/master...lewurm:winenv-testing > > This branch is by no means ready to be reviewed, it's just whatever I've used to do my experiments. > > Also note that it was a bit painful to merge the changes in toolchain_windows.m4 due to the renaming. While I agree that toolchain_microsoft.m4 is the better name, I suggest to defer that change to a later point. > > I've tested the following scenarios: > > 1. cygwin on x86_64 to build x86_64 > 2. cygwin on x86_64 to build arm64 > 3. WSL1 on x86_64 to build x86_64 > 4. WSL1 on x86_64 to build arm64 > 5. WSL2 on x86_64 to build x86_64 > 6. WSL1 on arm64 to build arm64 > > Some context: Microsoft doesn't ship native Arm64 binaries for the VS toolchain (yet). Instead one can run x86 binaries via xtajit (a binary translator), but as one can imagine this comes with some performance penalty. > Cygwin is also not available on Arm64. Cygwin already being slow, it made sense to avoid running Cygwin x86 on Arm64, and instead go the cross-compilation route in the OpenJDK build for Windows+Arm64. > Previously the OpenJDK build did not support cross-compiling on Windows. Even today, on openjdk/jdk:master a dirty workaround for fixpath.exe is needed to build Windows+Arm64 ( https://github.com/openjdk/jdk/pull/212#issuecomment-695024586 ). So we have been very excited to see this WINENV patch :-) > > Here are comments for each scenario. > > ====================================== > ## 1. cygwin on x86_64 to build x86_64 > ====================================== > > ``` > $ bash configure --with-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-build-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-boot-jdk=/cygdrive/c/work/jdk-16+22 --openjdk-target=aarch64-unknown-cygwin > configure: Configuration created at Mon Nov 9 21:33:42 CET 2020. > checking for basename... /usr/bin/basename > checking for dirname... /usr/bin/dirname > checking for file... /usr/bin/file > checking for ldd... /usr/bin/ldd > checking for bash... /usr/bin/bash > checking for cat... /usr/bin/cat > checking for chmod... /usr/bin/chmod > checking for cp... /usr/bin/cp > checking for cut... /usr/bin/cut > checking for date... /usr/bin/date > checking for gdiff... /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: /cygdrive/c/Program: binary operator expected > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: /cygdrive/c/Program: binary operator expected > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10043: test: too many arguments > /cygdrive/c/work/winenv-cygwin/build/.configure-support/generated-configure.sh: line 10047: test: too many arguments > [not found] > [...] > ``` > > I didn't investigate it any further, as this has already been reported on the mailing list. > > ===================================== > ## 2. cygwin on x86_64 to build arm64 > ===================================== > ``` > $ bash configure --with-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-build-devkit=/cygdrive/c/work/VS2019-16.6.1-devkit --with-boot-jdk=/cygdrive/c/work/jdk-16+22 --openjdk-target=aarch64-unknown-cygwin > ``` > Same as scenario 1) > > ==================================== > ## 3. WSL1 on x86_64 to build x86_64 > ==================================== > I do get some weird permission errors, so I run everything with sudo (chmod -R $user doesn't help): > ``` > $ sudo bash configure --with-build-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-boot-jdk="/mnt/c/work/jdk-16+22" > [...] > > Configuration summary: > * Debug level: release > * HS debug level: product > * JVM variants: server > * JVM features: server: 'aot cds compiler1 compiler2 epsilongc g1gc graal jfr jni-check jvmci jvmti management nmt parallelgc serialgc services shenandoahgc vm-structs zgc' > * OpenJDK target: OS: windows, CPU architecture: x86, address length: 64 > * Version string: 16-internal+0-adhoc.root.winenv-wsl1 (16-internal) > > Tools summary: > * Environment: wsl1 version 4.4.0-19041-Microsoft, #488-Microsoft Mon Sep 01 13:43:00 PST 2020 (Ubuntu 20.04.1 LTS); windows version 10.0.19041.572; prefix "/mnt"; root "[unavailable]" > * Boot JDK: openjdk version "16" 2021-03-16 OpenJDK Runtime Environment AdoptOpenJDK (build 16+22-202011040337) OpenJDK 64-Bit Server VM AdoptOpenJDK (build 16+22-202011040337, mixed mode, sharing) (at /mnt/c/work/jdk-16~1) > * Toolchain: microsoft (Microsoft Visual Studio 2019 16.6.1 (devkit)) > * C Compiler: Version 19.26.28806 (at /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/x64/cl.exe) > * C++ Compiler: Version 19.26.28806 (at /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/x64/cl.exe) > > Build performance summary: > * Cores to use: 12 > * Memory limit: 32605 MB > ``` > > The build fails eventually: > ``` > $ sudo make jdk-image JOBS=12 LOG=info > [...] > /mnt/c/work/winenv-wsl1/make/scripts/fixpath.sh: line 405: /mnt/c/work/winenv-wsl1/build/windows-x86_64-server-release/jdk/bin/java: No such file or directory > make[3]: *** [ExplodedImageOptimize.gmk:41: /mnt/c/work/winenv-wsl1/build/windows-x86_64-server-release/jdk/_optimize_image_exec.marker] Error 127 > make[2]: *** [make/Main.gmk:437: exploded-image-optimize] Error 2 > make[2]: *** Waiting for unfinished jobs.... > ``` > Which looks like a path issue, that should be relatively easy to fix (I didn't investigate). HotSpot and parts of the JDK built successfully though: > ``` > $ ./build/windows-x86_64-server-release/jdk/bin/java.exe -version > openjdk version "16-internal" 2021-03-16 > OpenJDK Runtime Environment (build 16-internal+0-adhoc.root.winenv-wsl1) > OpenJDK 64-Bit Server VM (build 16-internal+0-adhoc.root.winenv-wsl1, mixed mode) > ``` > So I declare success on this one :-) > > =================================== > ## 4. WSL1 on x86_64 to build arm64 > =================================== > ``` > $ bash configure --with-build-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-devkit="/mnt/c/work/VS2019-16.6.1-devkit" --with-boot-jdk="/mnt/c/work/jdk-16+22" --openjdk-target=aarch64-unknown-cygwin > [...] > checking for link... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/arm64/link.exe > checking if the found link.exe is actually the Visual Studio linker... yes > configure: Using microsoft linker version 14.26.28806.0 [Microsoft (R) Incremental Linker Version 14.26.28806.0] > checking for aarch64-unknown-cygwin-ml... [not found] > checking for ml... [not found] > configure: WARNING: using cross tools not prefixed with host triplet > configure: WARNING: Ignoring value of AS from the environment. Use command line variables instead. > checking for ml... [not found] > checking for aarch64-unknown-cygwin-lib... [not found] > checking for lib... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/arm64/lib.exe > checking for aarch64-unknown-cygwin-mt... [not found] > checking for mt... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/10/bin/x64/mt.exe > checking for aarch64-unknown-cygwin-rc... [not found] > checking for rc... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/10/bin/x64/rc.exe > checking for aarch64-unknown-cygwin-dumpbin... [not found] > checking for dumpbin... $FIXPATH /mnt/c/work/vs2019-16.6.1-devkit/vc/bin/arm64/dumpbin.exe > checking for aarch64-unknown-cygwin-gobjdump... [not found] > checking for gobjdump... [not found] > checking for aarch64-unknown-cygwin-objdump... [not found] > checking for objdump... /usr/bin/objdump > checking for build platform devkit... Microsoft Visual Studio 2019 16.6.1 (devkit) in /mnt/c/work/vs2019-16.6.1-devkit > checking for cl... fixpath: failure: Cannot locate cygpath or wslpath > > configure: error: Could not find required tool for BUILD_CC > /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 12: sed: command not found > /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 38: sort: command not found > /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 35: sed: command not found > /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 53: sort: command not found > /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 77: cat: command not found > /mnt/c/work/winenv-wsl1/build/.configure-support/generated-configure.sh: line 40: rm: command not found > ``` > It looks like it's still not suitable for cross compilation. I might have messed up resolving some merge conflicts. I haven't investigated that further, but that would probably be a scenario that would be nice to have eventually. > > ==================================== > ## 5. WSL2 on x86_64 to build x86_64 > ==================================== > > I second the clock skew observations :-( I had a problem like that on WSL2 in a different context before https://github.com/microsoft/WSL/issues/4898 > > Apart from that, crossing the "file system boundary" got worse with WSL2, and as long checking out the sources on a Windows visible path is a requirment, I'm not sure if it's worth going down this road. > > ================================== > ## 6. WSL1 on arm64 to build arm64 > ================================== > Same problem with permissions. I must be doing something wrong. > ``` > $ sudo bash configure --with-devkit="/mnt/c/lewurm/VS2019-16.6.1-devkit" --with-boot-jdk="/mnt/c/lewurm/jdk-16-ea+19-windows-aarch64" --disable-precompiled-headers > [...] > Configuration summary: > * Debug level: release > * HS debug level: product > * JVM variants: server > * JVM features: server: 'cds compiler1 compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc serialgc services shenandoahgc vm-structs zgc' > * OpenJDK target: OS: windows, CPU architecture: aarch64, address length: 64 > * Version string: 16-internal+0-adhoc.root.openjdk-jdk (16-internal) > > Tools summary: > * Environment: wsl1 version 4.4.0-19607-Microsoft, #1000-Microsoft Wed Apr 09 15:02:00 PST 2020 (Ubuntu 20.04 LTS); windows version 10.0.19607.1000; prefix "/mnt"; root "[unavailable]" > * Boot JDK: openjdk version "16-ea" 2020-10-09 OpenJDK Runtime Environment Microsoft (build 16-ea+19-windows-aarch64) OpenJDK 64-Bit Server VM Microsoft (build 16-ea+19-windows-aarch64, mixed mode) (at /mnt/c/lewurm/jdk-16~1) > * Toolchain: microsoft (Microsoft Visual Studio 2019 16.6.1 (devkit)) > * C Compiler: Version 19.27.29112 (at /mnt/c/lewurm/vs2019-16.6.1-devkit/vc/bin/arm64/cl.exe) > * C++ Compiler: Version 19.27.29112 (at /mnt/c/lewurm/vs2019-16.6.1-devkit/vc/bin/arm64/cl.exe) > > Build performance summary: > * Cores to use: 64 > * Memory limit: 524204 MB > ``` > So a couple of things that I hacked up: > * I manually copied x86 MSVC binaries into /mnt/c/lewurm/vs2019-16.6.1-devkit/vc/bin/arm64 > * I manually copied arm64 binaries from the Windows SDK into /mnt/c/lewurm/vs2019-16.6.1-devkit/10/bin/arm64 Eventually I need to update the devkit script to create a proper archive and/or update the build script to be aware that x86 MSVC binaries must be used on arm64 (the hope is that this eventually won't be true anymore). > * add detection for WSL on aarch64: https://github.com/lewurm/openjdk/commit/4da4b6df16d05c97b7f48bef166e6d2080d37511 > * The boot-jdk is from https://github.com/microsoft/openjdk-aarch64/releases > * `--disable-precompiled-headers` is needed because otherwise a bug in the binary translator (xtajit) is triggered when running x86 cl.exe with it. Note to myself: File a bug report. > > > Let's build a jdk image: > ``` > $ sudo make jdk-image JOBS=64 LOG=info > [...] > /mnt/c/lewurm/openjdk-jdk/make/scripts/fixpath.sh: line 405: /mnt/c/lewurm/openjdk-jdk/build/windows-aarch64-server-release/jdk/bin/java: No such file or directory > make[3]: *** [ExplodedImageOptimize.gmk:41: /mnt/c/lewurm/openjdk-jdk/build/windows-aarch64-server-release/jdk/_optimize_image_exec.marker] Error 127 > make[2]: *** [make/Main.gmk:437: exploded-image-optimize] Error 2 > $ ./build/windows-aarch64-server-release/jdk/bin/java.exe -version > openjdk version "16-internal" 2021-03-16 > OpenJDK Runtime Environment (build 16-internal+0-adhoc.root.openjdk-jdk) > OpenJDK 64-Bit Server VM (build 16-internal+0-adhoc.root.openjdk-jdk, mixed mode) > ``` > So same issue as in scenario 3 (WSL1 on x86_64 building x86_64), which is _great_ news :-) > > ============== > ## Performance > ============== > I did a comparision run on WSL1 on x86_64 to build x86_64 (scenario 3), once with the WINENV patches applied and once without it. > Overall it seems there is some performance regression, but not too bad. The profile logs aren't very meaningful to me, but if I would need to guess the difference is due to having a native `fixpath.exe` vs. `fixpath.sh`. > > Logs are here: https://gist.github.com/lewurm/2fbf2971a55a3a4a4f5933b105925976 > > > ========= > ## Epilog > ========= > > I hope this has met your bar of exotic setup. Unfortunately I can't provide much feedback on the automagic. I would love to see your work landing eventually! > > Thank you, > -Bernhard > > ________________________________________ > From: build-dev on behalf of Magnus Ihse Bursie > Sent: Friday, August 28, 2020 11:09 > To: Ludovic Henry; Yasumasa Suenaga; Monica Beckwith; build-dev > Subject: Re: Preliminary review for new WINENV support > > On 2020-08-24 19:21, Ludovic Henry wrote: >> Hi Magnus, >> >> I want to follow up on this work of yours, as we've particular interest in it for the Windows-AArch64 port. >> >> Let me know how I could assist you in this effort. > The big blockers for this fix is basically these two issues: > >>> I have also temporarily disabled the javac server, and building without absolute paths. I believe a better way forward is to address these with improved functionality in separate patches, instead of trying to work around the issues introduced by them in this patch. > Until that is fixed, any integration of this patch is a no-go. > > I have been working on those, but a lot of other stuff has kept me from > gaining much progress; everything from long vacations to preparations to > the switch to github. Unfortunately, I don't think there's an easy way > for you to help on those two blockers. > > However, there are two more things that I'd like to look into, if > possible, before integrating my patch, and there you might be able to help. > > One is performance. I got a somewhat different performance profile with > my patch than without -- some things were faster, and some were slower. > I'd like to understand why some things got slower, and if it is possible > to do anything about. (Why things got faster I don't care about, I'm > just happy :)) > > So maybe you can help me with some performance analysis? With the > limited availability of tooling for makefiles, this is not really > trivial. What I usually do is I run with JOBS=1 (to be able to track > each task individually without effects of concurrency), and > LOG=info,profile. This will give you time stamps for each individual > execution in build.log. And see if there is any interesting differences > with and without the patch. > > The other thing is that WSL2 works awfully bad. :( Maybe there's nothing > to be done about it, if the thing being shipped is just too buggy, but > maybe we do things incorrectly, or that unnecessarily triggers bugs. Or > maybe there are workarounds we could use. But I think it could be worth > some more effort to try to get it working. Maybe you can help with this? > > /Magnus From jiefu at openjdk.java.net Tue Nov 10 12:19:59 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Tue, 10 Nov 2020 12:19:59 GMT Subject: Integrated: 8256048: Incomplete gitignore setting for netbeans project In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 12:07:37 GMT, Jie Fu wrote: > Hi all, > > The gitignore setting for netbeans project seems to be incomplete. > Only nbproject/private/ [1] is ignored. > > But there are also other files under nbproject/, which should be ignored too. > ---------------------------- > $ tree -L 1 nbproject/ > nbproject/ > ??? configurations.xml > ??? private > ??? project.xml > ---------------------------- > > It would be better to fix it. > > Thanks. > Best regards, > Jie > > > [1] https://github.com/openjdk/jdk/blob/master/.gitignore#L5 This pull request has now been integrated. Changeset: 52805f52 Author: Jie Fu URL: https://git.openjdk.java.net/jdk/commit/52805f52 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod 8256048: Incomplete gitignore setting for netbeans project Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1122 From dholmes at openjdk.java.net Tue Nov 10 12:35:59 2020 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 10 Nov 2020 12:35:59 GMT Subject: RFR: 8255822: Zero: improve build-time JVMTI handling [v3] In-Reply-To: References: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> Message-ID: <83OnOrWG5dCsK_f6OUm7517vxV4uSg76a-PhTQZJjlM=.687f44d9-d4c2-440a-86dd-efa5da75800b@github.com> On Tue, 10 Nov 2020 07:22:07 GMT, Aleksey Shipilev wrote: >> Current Zero interpreter has the optimization for JVMTI support. It recognizes that JVMTI is disabled most of the time, and that JVMTI checks in the interpreter code slows it down considerably. (I measured it myself when working on this patch: removing this optimization yields about 20% hit in build times). >> >> Current optimization works as follows. At build time, an XSLT transform is performed on `bytecodeInterpreter.cpp`, yielding `bytecodeInterpreterWithChecks.cpp`. In that new compilation unit, `VM_JVMTI` macro is defined, and a new entry point -- `BytecodeInterpreter::withChecks` -- is defined. Then, both compilation units are compiled. In one of them, `JVMTI` hooks are stripped out. In another, they persist. Then, callers have to choose which entry point to use. >> >> I believe this can be rewritten to use C++ templates instead of XLST and defines dance. This also allows to clean up JVMTI checks a bit. >> >> Additional testing: >> - [x] Linux x86_64 Zero fastdebug build with `-jvmti` >> - [x] Linux x86_64 Zero fastdebug/release build times are not regressing > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Simplify a few JVMTI_ENABLED blocks > - Merge branch 'master' into JDK-8255822-zero-jvmti-rework > - Fix build error > - Merge branch 'master' into JDK-8255822-zero-jvmti-rework > - Revert one dubious change > - 8255822: Zero: improve build-time JVMTI handling > Summary: use C++ templates instead of XSLT transforms Still good. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1061 From kcr at openjdk.java.net Tue Nov 10 12:53:55 2020 From: kcr at openjdk.java.net (Kevin Rushforth) Date: Tue, 10 Nov 2020 12:53:55 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 08:19:13 GMT, Sergey Bylokhov wrote: > This is a review request for the bug particularly fixed some time ago: > https://mail.openjdk.java.net/pipermail/2d-dev/2015-May/005425.html > > In that review request it was found that the old fix does not work well in all cases, see: > https://mail.openjdk.java.net/pipermail/2d-dev/2015-August/005611.html > > The current fix updates an embedded plist.info, so the java will not require > discrete graphics by default, same as for any other applications. > > The discrete card will be used: > 1. If macOS decided to enable it for some reason > 2. If the java app sets/uses a full-screen window > 3. If the user disable "automatic graphics switching" in the system preferences > 4. If an external monitor is connected to the laptop > > In other cases, the integrated graphics will be used, which should be fine in most cases since this graphic is used/tested in the mbp 13/etc. This is not only about rendering performance but also about startup performance, on my current new laptop mbp 16 + Cataline 10.15.7 the switching discrete/integrated causes unexpected delays up to 10 seconds. > > > Note that the new "metal" pipeline also does not require discrete graphics. > > The documentation for NSSupportsAutomaticGraphicsSwitching: > https://developer.apple.com/documentation/bundleresources/information_property_list/nssupportsautomaticgraphicsswitching > > I'll create a release note after approval. > > Performance numbers: > https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010979.html > Old review request: > https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010973.html I'm still somewhat unsure whether we want to do this in all cases. As mentioned offline, the discrete GPU will be unused (might as well not be there) for almost all Java applications when using a single screen. When the MacBook Pro is running on battery, this seems like a good thing, but when it is plugged in, it seems like we wasting the discrete GPU. It's too bad Apple doesn't provide a way for an application to hint whether they would like to use the discrete GPU if available. I want to run some performance tests with JavaFX, since it will likely impact JavaFX applications. ------------- PR: https://git.openjdk.java.net/jdk/pull/1139 From erikj at openjdk.java.net Tue Nov 10 13:26:57 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 10 Nov 2020 13:26:57 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 08:19:13 GMT, Sergey Bylokhov wrote: > This is a review request for the bug particularly fixed some time ago: > https://mail.openjdk.java.net/pipermail/2d-dev/2015-May/005425.html > > In that review request it was found that the old fix does not work well in all cases, see: > https://mail.openjdk.java.net/pipermail/2d-dev/2015-August/005611.html > > The current fix updates an embedded plist.info, so the java will not require > discrete graphics by default, same as for any other applications. > > The discrete card will be used: > 1. If macOS decided to enable it for some reason > 2. If the java app sets/uses a full-screen window > 3. If the user disable "automatic graphics switching" in the system preferences > 4. If an external monitor is connected to the laptop > > In other cases, the integrated graphics will be used, which should be fine in most cases since this graphic is used/tested in the mbp 13/etc. This is not only about rendering performance but also about startup performance, on my current new laptop mbp 16 + Cataline 10.15.7 the switching discrete/integrated causes unexpected delays up to 10 seconds. > > > Note that the new "metal" pipeline also does not require discrete graphics. > > The documentation for NSSupportsAutomaticGraphicsSwitching: > https://developer.apple.com/documentation/bundleresources/information_property_list/nssupportsautomaticgraphicsswitching > > I'll create a release note after approval. > > Performance numbers: > https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010979.html > Old review request: > https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010973.html Change looks ok from a build point of view, but I can't comment on the validity and implications of using this key. ------------- Marked as reviewed by erikj (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1139 From kcr at openjdk.java.net Tue Nov 10 16:40:58 2020 From: kcr at openjdk.java.net (Kevin Rushforth) Date: Tue, 10 Nov 2020 16:40:58 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU In-Reply-To: References: Message-ID: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> On Tue, 10 Nov 2020 13:24:21 GMT, Erik Joelsson wrote: >> This is a review request for the bug particularly fixed some time ago: >> https://mail.openjdk.java.net/pipermail/2d-dev/2015-May/005425.html >> >> In that review request it was found that the old fix does not work well in all cases, see: >> https://mail.openjdk.java.net/pipermail/2d-dev/2015-August/005611.html >> >> The current fix updates an embedded plist.info, so the java will not require >> discrete graphics by default, same as for any other applications. >> >> The discrete card will be used: >> 1. If macOS decided to enable it for some reason >> 2. If the java app sets/uses a full-screen window >> 3. If the user disable "automatic graphics switching" in the system preferences >> 4. If an external monitor is connected to the laptop >> >> In other cases, the integrated graphics will be used, which should be fine in most cases since this graphic is used/tested in the mbp 13/etc. This is not only about rendering performance but also about startup performance, on my current new laptop mbp 16 + Cataline 10.15.7 the switching discrete/integrated causes unexpected delays up to 10 seconds. >> >> >> Note that the new "metal" pipeline also does not require discrete graphics. >> >> The documentation for NSSupportsAutomaticGraphicsSwitching: >> https://developer.apple.com/documentation/bundleresources/information_property_list/nssupportsautomaticgraphicsswitching >> >> I'll create a release note after approval. >> >> Performance numbers: >> https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010979.html >> Old review request: >> https://mail.openjdk.java.net/pipermail/2d-dev/2020-August/010973.html > > Change looks ok from a build point of view, but I can't comment on the validity and implications of using this key. I ran a 3D lighting test that is designed to be a GPU stress test. It's a worst case, to be sure, but it take 10 times as long to render with the integrated GPU as it does with the discrete GPU: **attenuation.LightingSample: 500 large quads** discrete GPU: 23.5 fps integrated GPU: 2.34 fps In a more realistic example of drawing a large number of 2D vectors, it runs 35% slower with the integrated GPU: **Vector charting test, oval clip** discrete GPU: 41.1 fps integrated GPU: 26.6 fps I see similar results in the performance numbers you listed above. An application developer who packages up their JDK, for example, using jpackage, can make the decision for themselves. Application developers who rely on the JDK as delivered will get whatever we choose as the default. So we need to be sure that the benefit of doing this justifies the performance hit. ------------- PR: https://git.openjdk.java.net/jdk/pull/1139 From serb at openjdk.java.net Tue Nov 10 17:17:58 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Tue, 10 Nov 2020 17:17:58 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU In-Reply-To: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> References: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> Message-ID: On Tue, 10 Nov 2020 16:38:25 GMT, Kevin Rushforth wrote: >> Change looks ok from a build point of view, but I can't comment on the validity and implications of using this key. > > I ran a 3D lighting test that is designed to be a GPU stress test. It's a worst case, to be sure, but it take 10 times as long to render with the integrated GPU as it does with the discrete GPU: > > **attenuation.LightingSample: 500 large quads** > discrete GPU: 23.5 fps > integrated GPU: 2.34 fps > > In a more realistic example of drawing a large number of 2D vectors, it runs 35% slower with the integrated GPU: > > **Vector charting test, oval clip** > discrete GPU: 41.1 fps > integrated GPU: 26.6 fps > > I see similar results in the performance numbers you listed above. > > An application developer who packages up their JDK, for example, using jpackage, can make the decision for themselves. Application developers who rely on the JDK as delivered will get whatever we choose as the default. So we need to be sure that the benefit of doing this justifies the performance hit. The difference between the two is that, if the integrated card is default then it is possible to force the discrete card if needed, but it is not possible to force the integrated card if discrete is the default. In the end, it is a laptop, it will work longer on an integrated card. ------------- PR: https://git.openjdk.java.net/jdk/pull/1139 From shade at openjdk.java.net Tue Nov 10 17:26:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 10 Nov 2020 17:26:00 GMT Subject: Integrated: 8255822: Zero: improve build-time JVMTI handling In-Reply-To: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> References: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> Message-ID: On Wed, 4 Nov 2020 16:29:27 GMT, Aleksey Shipilev wrote: > Current Zero interpreter has the optimization for JVMTI support. It recognizes that JVMTI is disabled most of the time, and that JVMTI checks in the interpreter code slows it down considerably. (I measured it myself when working on this patch: removing this optimization yields about 20% hit in build times). > > Current optimization works as follows. At build time, an XSLT transform is performed on `bytecodeInterpreter.cpp`, yielding `bytecodeInterpreterWithChecks.cpp`. In that new compilation unit, `VM_JVMTI` macro is defined, and a new entry point -- `BytecodeInterpreter::withChecks` -- is defined. Then, both compilation units are compiled. In one of them, `JVMTI` hooks are stripped out. In another, they persist. Then, callers have to choose which entry point to use. > > I believe this can be rewritten to use C++ templates instead of XLST and defines dance. This also allows to clean up JVMTI checks a bit. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug build with `-jvmti` > - [x] Linux x86_64 Zero fastdebug/release build times are not regressing This pull request has now been integrated. Changeset: 643969a1 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/643969a1 Stats: 188 lines in 6 files changed: 5 ins; 128 del; 55 mod 8255822: Zero: improve build-time JVMTI handling Reviewed-by: dholmes, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1061 From shade at openjdk.java.net Tue Nov 10 17:25:59 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 10 Nov 2020 17:25:59 GMT Subject: RFR: 8255822: Zero: improve build-time JVMTI handling [v3] In-Reply-To: <83OnOrWG5dCsK_f6OUm7517vxV4uSg76a-PhTQZJjlM=.687f44d9-d4c2-440a-86dd-efa5da75800b@github.com> References: <7-Y8_X8c04i5dRiPOWnSV-Wqige6jXxb1WPW1pIO7CE=.f66482a9-f624-4897-a68f-1e131a6abf8f@github.com> <83OnOrWG5dCsK_f6OUm7517vxV4uSg76a-PhTQZJjlM=.687f44d9-d4c2-440a-86dd-efa5da75800b@github.com> Message-ID: On Tue, 10 Nov 2020 12:33:34 GMT, David Holmes wrote: >> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - Simplify a few JVMTI_ENABLED blocks >> - Merge branch 'master' into JDK-8255822-zero-jvmti-rework >> - Fix build error >> - Merge branch 'master' into JDK-8255822-zero-jvmti-rework >> - Revert one dubious change >> - 8255822: Zero: improve build-time JVMTI handling >> Summary: use C++ templates instead of XSLT transforms > > Still good. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1061 From jjg at openjdk.java.net Wed Nov 11 02:25:04 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Wed, 11 Nov 2020 02:25:04 GMT Subject: RFR: JDK-8250768: javac should be adapted to changes in JEP 12 [v11] In-Reply-To: References: Message-ID: On Fri, 6 Nov 2020 18:41:15 GMT, Jan Lahoda wrote: >> This is an update to javac and javadoc, to introduce support for Preview APIs, and generally improve javac and javadoc behavior to more closely adhere to JEP 12. >> >> The notable changes are: >> >> * adding support for Preview APIs (javac until now supported primarily only preview language features, and APIs associated with preview language features). This includes: >> * the @PreviewFeature annotation has boolean attribute "reflective", which should be true for reflective Preview APIs, false otherwise. This replaces the existing "essentialAPI" attribute with roughly inverted meaning. >> * the preview warnings for preview APIs are auto-suppressed as described in the JEP 12. E.g. the module that declares the preview API is free to use it without warnings >> * improving error/warning messages. Please see [1] for a list of cases/examples. >> * class files are only marked as preview if they are using a preview feature. [1] also shows if a class file is marked as preview or not. >> * the PreviewFeature annotation has been moved to jdk.internal.javac package (originally was in the jdk.internal package). >> * Preview API in JDK's javadoc no longer needs to specify @preview tag in the source files. javadoc will auto-generate a note for @PreviewFeature elements, see e.g. [2] and [3] (non-reflective and reflective API, respectively). A summary of preview elements is also provided [4]. Existing uses of @preview have been updated/removed. >> * non-JDK javadoc is also enhanced to auto-generate preview notes for uses of Preview elements, and for declaring elements using preview language features [5]. >> >> Please also see the CSR [6] for more information. >> >> [1] http://cr.openjdk.java.net/~jlahoda/8250768/JEP12-errors-warnings-v6.html >> [2] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.base/java/lang/Record.html >> [3] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/java.compiler/javax/lang/model/element/RecordComponentElement.html >> [4] http://cr.openjdk.java.net/~jlahoda/8250768/jdk.javadoc.00/api/preview-list.html >> [5] http://cr.openjdk.java.net/~jlahoda/8250768/test.javadoc.00/ >> [6] https://bugs.openjdk.java.net/browse/JDK-8250769 > > Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision: > > Fixing navigator for the PREVIEW page. I have a mild queasiness about this new overloaded use of the word "Summary", since "summary tables" are normally the summary of the contents of a declaration, like fields and methods of a class. That being said, the usage is primarily internal, and I have no overwhelmingly wonderful alternative, and (overloading aside) the term is accurate. So, OK for now. We can change it later if we want to. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Utils.java line 2949: > 2947: > 2948: /** > 2949: * Return the set of preview language features used to declare the given element. "Returns" ------------- Marked as reviewed by jjg (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/703 From shade at openjdk.java.net Wed Nov 11 10:49:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 10:49:00 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe Message-ID: Update the docs with the (f)actual recipe used in JDK-8256127. ------------- Commit messages: - 8256182: Update qemu-debootstrap cross-compilation recipe Changes: https://git.openjdk.java.net/jdk/pull/1160/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1160&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256182 Stats: 42 lines in 2 files changed: 32 ins; 0 del; 10 mod Patch: https://git.openjdk.java.net/jdk/pull/1160.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1160/head:pull/1160 PR: https://git.openjdk.java.net/jdk/pull/1160 From mcimadamore at openjdk.java.net Wed Nov 11 11:40:12 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 11 Nov 2020 11:40:12 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v27] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 34 commits: - Merge branch 'master' into 8254162 - Add more output in TestHandhsake.java - Further improve output of TestHandshake - Improve debugging output of TestHandhsake - Remove endianness-aware byte getter/setter in MemoryAccess Remove index-based version of byte getter/setter in MemoryAccess - Fix post-merge issues caused by 8219014 - Merge branch 'master' into 8254162 - Addess remaining feedback from @AlanBateman and @mrserb - Address comments from @AlanBateman - Merge branch 'master' into 8254162 - ... and 24 more: https://git.openjdk.java.net/jdk/compare/432c387e...8444c633 ------------- Changes: https://git.openjdk.java.net/jdk/pull/548/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=26 Stats: 7600 lines in 82 files changed: 4791 ins; 1590 del; 1219 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From ihse at openjdk.java.net Wed Nov 11 11:45:55 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 11 Nov 2020 11:45:55 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe In-Reply-To: References: Message-ID: <_X3RS1OgN3bhSTvnBvVQWvPw2rmsu0-MXfCEjb8fVKU=.3b4d7a88-4b2f-4bd5-943d-0a8bc94c0c4d@github.com> On Wed, 11 Nov 2020 09:02:27 GMT, Aleksey Shipilev wrote: > Update the docs with the (f)actual recipe used in JDK-8256127. Thanks for keeping the documentation up to date! I have no objection to the actual content, just these minor formatting comments. doc/building.md line 1100: > 1098: --include=fakeroot,symlinks,build-essential,libx11-dev,libxext-dev,libxrender-dev,libxrandr-dev,libxtst-dev,libxt-dev,libcups2-dev,libfontconfig1-dev,libasound2-dev,libfreetype6-dev,libpng-dev \ > 1099: --resolve-deps \ > 1100: buster \ I'm guessing `--resolve-deps buster` is the full option; if so, it should be on a single line, otherwise it reads very strange. doc/building.md line 1107: > 1105: * Make sure the symlinks inside the newly created chroot point to proper locations: > 1106: > 1107: ``` I think it is important that you do not have an empty line between the bullet line and the code block, to get nicely formatted html. I know markdown is typically fiddly in such cases, and I notice that this is how the other bullets are written. ------------- Changes requested by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1160 From mcimadamore at openjdk.java.net Wed Nov 11 11:50:14 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Wed, 11 Nov 2020 11:50:14 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v28] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Invert condition in memory access var handle `withInvokeBehavior' ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/548/files - new: https://git.openjdk.java.net/jdk/pull/548/files/8444c633..eae57b4d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=27 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=26-27 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From shade at openjdk.java.net Wed Nov 11 11:57:14 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 11:57:14 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v2] In-Reply-To: References: Message-ID: > Update the docs with the (f)actual recipe used in JDK-8256127. Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Remove newline before code block ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1160/files - new: https://git.openjdk.java.net/jdk/pull/1160/files/ac27b8e3..187b2962 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1160&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1160&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1160.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1160/head:pull/1160 PR: https://git.openjdk.java.net/jdk/pull/1160 From shade at openjdk.java.net Wed Nov 11 11:57:15 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 11:57:15 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v2] In-Reply-To: <_X3RS1OgN3bhSTvnBvVQWvPw2rmsu0-MXfCEjb8fVKU=.3b4d7a88-4b2f-4bd5-943d-0a8bc94c0c4d@github.com> References: <_X3RS1OgN3bhSTvnBvVQWvPw2rmsu0-MXfCEjb8fVKU=.3b4d7a88-4b2f-4bd5-943d-0a8bc94c0c4d@github.com> Message-ID: On Wed, 11 Nov 2020 11:39:56 GMT, Magnus Ihse Bursie wrote: >> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove newline before code block > > doc/building.md line 1100: > >> 1098: --include=fakeroot,symlinks,build-essential,libx11-dev,libxext-dev,libxrender-dev,libxrandr-dev,libxtst-dev,libxt-dev,libcups2-dev,libfontconfig1-dev,libasound2-dev,libfreetype6-dev,libpng-dev \ >> 1099: --resolve-deps \ >> 1100: buster \ > > I'm guessing `--resolve-deps buster` is the full option; if so, it should be on a single line, otherwise it reads very strange. Not really: `buster` is the `SUITE`, it is the argument on its own. > doc/building.md line 1107: > >> 1105: * Make sure the symlinks inside the newly created chroot point to proper locations: >> 1106: >> 1107: ``` > > I think it is important that you do not have an empty line between the bullet line and the code block, to get nicely formatted html. I know markdown is typically fiddly in such cases, and I notice that this is how the other bullets are written. Right! Fixed that. ------------- PR: https://git.openjdk.java.net/jdk/pull/1160 From shade at openjdk.java.net Wed Nov 11 11:59:58 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 11:59:58 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v2] In-Reply-To: References: <_X3RS1OgN3bhSTvnBvVQWvPw2rmsu0-MXfCEjb8fVKU=.3b4d7a88-4b2f-4bd5-943d-0a8bc94c0c4d@github.com> Message-ID: On Wed, 11 Nov 2020 11:54:27 GMT, Aleksey Shipilev wrote: >> doc/building.md line 1107: >> >>> 1105: * Make sure the symlinks inside the newly created chroot point to proper locations: >>> 1106: >>> 1107: ``` >> >> I think it is important that you do not have an empty line between the bullet line and the code block, to get nicely formatted html. I know markdown is typically fiddly in such cases, and I notice that this is how the other bullets are written. > > Right! Fixed that. Well, and `make update-build-docs` made no changes after this, so I guess pandoc did not care. Maybe it is missing indents... ------------- PR: https://git.openjdk.java.net/jdk/pull/1160 From shade at openjdk.java.net Wed Nov 11 12:04:11 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 12:04:11 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v3] In-Reply-To: References: Message-ID: > Update the docs with the (f)actual recipe used in JDK-8256127. Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Properly indent the code blocks ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1160/files - new: https://git.openjdk.java.net/jdk/pull/1160/files/187b2962..fcb48697 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1160&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1160&range=01-02 Stats: 43 lines in 2 files changed: 1 ins; 7 del; 35 mod Patch: https://git.openjdk.java.net/jdk/pull/1160.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1160/head:pull/1160 PR: https://git.openjdk.java.net/jdk/pull/1160 From shade at openjdk.java.net Wed Nov 11 12:04:11 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 12:04:11 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v3] In-Reply-To: References: <_X3RS1OgN3bhSTvnBvVQWvPw2rmsu0-MXfCEjb8fVKU=.3b4d7a88-4b2f-4bd5-943d-0a8bc94c0c4d@github.com> Message-ID: On Wed, 11 Nov 2020 11:56:57 GMT, Aleksey Shipilev wrote: >> Right! Fixed that. > > Well, and `make update-build-docs` made no changes after this, so I guess pandoc did not care. Maybe it is missing indents... Yes, indents were incorrect. Fixed, and now generated html looks fine. ------------- PR: https://git.openjdk.java.net/jdk/pull/1160 From glaubitz at openjdk.java.net Wed Nov 11 12:27:00 2020 From: glaubitz at openjdk.java.net (John Paul Adrian Glaubitz) Date: Wed, 11 Nov 2020 12:27:00 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v3] In-Reply-To: References: <_X3RS1OgN3bhSTvnBvVQWvPw2rmsu0-MXfCEjb8fVKU=.3b4d7a88-4b2f-4bd5-943d-0a8bc94c0c4d@github.com> Message-ID: <-0RMjqigN2OYO7i5woxAZOadW1XHrL12ju-hdy6qQN0=.0bf85506-8071-41e8-90f9-1fc67697276d@github.com> On Wed, 11 Nov 2020 11:53:39 GMT, Aleksey Shipilev wrote: >> doc/building.md line 1100: >> >>> 1098: --include=fakeroot,symlinks,build-essential,libx11-dev,libxext-dev,libxrender-dev,libxrandr-dev,libxtst-dev,libxt-dev,libcups2-dev,libfontconfig1-dev,libasound2-dev,libfreetype6-dev,libpng-dev \ >>> 1099: --resolve-deps \ >>> 1100: buster \ >> >> I'm guessing `--resolve-deps buster` is the full option; if so, it should be on a single line, otherwise it reads very strange. > > Not really: `buster` is the `SUITE`, it is the argument on its own. Correct, ```buster``` is the suite (= Debian release). You might want to use ```stable``` instead of ```buster```, then you don't need to adjust that part all the time as ```buster``` is a symlink to ```buster``` on the Debian FTP servers. ------------- PR: https://git.openjdk.java.net/jdk/pull/1160 From ihse at openjdk.java.net Wed Nov 11 12:31:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 11 Nov 2020 12:31:56 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v3] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 12:04:11 GMT, Aleksey Shipilev wrote: >> Update the docs with the (f)actual recipe used in JDK-8256127. > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Properly indent the code blocks Looks good now. Sorry about me jumping to conclusions about `--resolve-deps`. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1160 From shade at openjdk.java.net Wed Nov 11 12:31:57 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 12:31:57 GMT Subject: RFR: 8256182: Update qemu-debootstrap cross-compilation recipe [v3] In-Reply-To: <-0RMjqigN2OYO7i5woxAZOadW1XHrL12ju-hdy6qQN0=.0bf85506-8071-41e8-90f9-1fc67697276d@github.com> References: <_X3RS1OgN3bhSTvnBvVQWvPw2rmsu0-MXfCEjb8fVKU=.3b4d7a88-4b2f-4bd5-943d-0a8bc94c0c4d@github.com> <-0RMjqigN2OYO7i5woxAZOadW1XHrL12ju-hdy6qQN0=.0bf85506-8071-41e8-90f9-1fc67697276d@github.com> Message-ID: <7Ut3kBZjHb0_eaZ6tjwwPiZWblaqUZmVB3F3-sUTndY=.1a246f0a-84c8-462a-b10b-877605773768@github.com> On Wed, 11 Nov 2020 12:23:48 GMT, John Paul Adrian Glaubitz wrote: >> Not really: `buster` is the `SUITE`, it is the argument on its own. > > Correct, ```buster``` is the suite (= Debian release). > > You might want to use ```stable``` instead of ```buster```, then you don't need to adjust that part all the time as ```buster``` is a symlink to ```buster``` on the Debian FTP servers. Thanks. I'd rather leave `buster`, because we know that would work with the package list the line above requires. I believe I had some problems with `libpng-dev` before. So changing the suite later would also test the include package list. ------------- PR: https://git.openjdk.java.net/jdk/pull/1160 From mbaesken at openjdk.java.net Wed Nov 11 12:41:09 2020 From: mbaesken at openjdk.java.net (Matthias Baesken) Date: Wed, 11 Nov 2020 12:41:09 GMT Subject: RFR: JDK-8256188: Adjust output of make/autoconf/configure Message-ID: The script make/autoconf/configure should give a bit more output in case various errors occur (e.g. missing autoconf). The new output in case of missing autoconf adds the PATH; additionally we show on AIX some info on how to install autoconf. bash /jdk_6/jdk/jdk/configure Runnable configure script is not present ... Autoconf is not found on the PATH (/opt/freeware/bin:/usr/bin:/etc:/usr/sbin), and AUTOCONF is not set. You need autoconf to be able to generate a runnable configure script. You might be able to fix this by installing autoconf from the 'AIX Toolbox for Linux Applications' (or compile it from the sources). Error: Cannot find autoconf ------------- Commit messages: - JDK-8256188 Changes: https://git.openjdk.java.net/jdk/pull/1164/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1164&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256188 Stats: 6 lines in 1 file changed: 3 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1164.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1164/head:pull/1164 PR: https://git.openjdk.java.net/jdk/pull/1164 From shade at openjdk.java.net Wed Nov 11 13:52:01 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 13:52:01 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow Message-ID: It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. ------------- Commit messages: - Remove special directories from chroot to make it archivable - 8256127: Add cross-compiled foreign architectures builds to submit workflow Changes: https://git.openjdk.java.net/jdk/pull/1147/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256127 Stats: 557 lines in 1 file changed: 556 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1147.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1147/head:pull/1147 PR: https://git.openjdk.java.net/jdk/pull/1147 From mdoerr at openjdk.java.net Wed Nov 11 15:20:58 2020 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Wed, 11 Nov 2020 15:20:58 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 19:05:31 GMT, Aleksey Shipilev wrote: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. > > This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. > > As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. > > The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. Awesome! Thanks a lot for doing this! ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Wed Nov 11 16:08:57 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 16:08:57 GMT Subject: Integrated: 8256182: Update qemu-debootstrap cross-compilation recipe In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 09:02:27 GMT, Aleksey Shipilev wrote: > Update the docs with the (f)actual recipe used in JDK-8256127. This pull request has now been integrated. Changeset: 421a7c3b Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/421a7c3b Stats: 52 lines in 2 files changed: 25 ins; 0 del; 27 mod 8256182: Update qemu-debootstrap cross-compilation recipe Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1160 From shade at openjdk.java.net Wed Nov 11 16:16:09 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 16:16:09 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v2] In-Reply-To: References: Message-ID: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. > > This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. > > As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. > > The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Merge branch 'master' into JDK-8256127-foreign-build - Remove special directories from chroot to make it archivable - 8256127: Add cross-compiled foreign architectures builds to submit workflow ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1147/files - new: https://git.openjdk.java.net/jdk/pull/1147/files/98cbf2f2..c69b529e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=00-01 Stats: 298 lines in 10 files changed: 260 ins; 1 del; 37 mod Patch: https://git.openjdk.java.net/jdk/pull/1147.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1147/head:pull/1147 PR: https://git.openjdk.java.net/jdk/pull/1147 From clanger at openjdk.java.net Wed Nov 11 16:31:57 2020 From: clanger at openjdk.java.net (Christoph Langer) Date: Wed, 11 Nov 2020 16:31:57 GMT Subject: RFR: JDK-8256188: Adjust output of make/autoconf/configure In-Reply-To: References: Message-ID: <5I56b-GJEuV30AII8urdJEQRpgfSG6msrrMHFE7yTt4=.5ce376af-1f19-44f7-a8de-119b71bd6bf3@github.com> On Wed, 11 Nov 2020 12:35:07 GMT, Matthias Baesken wrote: > The script make/autoconf/configure should give a bit more output in case various errors occur (e.g. missing autoconf). > The new output in case of missing autoconf adds the PATH; additionally we show on AIX some info on how to install autoconf. > > bash /jdk_6/jdk/jdk/configure > Runnable configure script is not present > ... > Autoconf is not found on the PATH (/opt/freeware/bin:/usr/bin:/etc:/usr/sbin), and AUTOCONF is not set. > You need autoconf to be able to generate a runnable configure script. > You might be able to fix this by installing autoconf from the 'AIX Toolbox for Linux Applications' (or compile it from the sources). > Error: Cannot find autoconf This looks good to me. Obviously somebody from the build folks needs to review this, too. ------------- Marked as reviewed by clanger (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1164 From shade at openjdk.java.net Wed Nov 11 17:15:12 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 17:15:12 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v3] In-Reply-To: References: Message-ID: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. > > This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. > > As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. > > The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Incomplete fix: remove special dirs ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1147/files - new: https://git.openjdk.java.net/jdk/pull/1147/files/c69b529e..eb6b10a3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=01-02 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/1147.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1147/head:pull/1147 PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Wed Nov 11 19:09:09 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 19:09:09 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v4] In-Reply-To: References: Message-ID: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. > > This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. > > As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. > > The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - Merge branch 'master' into JDK-8256127-foreign-build - Incomplete fix: remove special dirs - Merge branch 'master' into JDK-8256127-foreign-build - Remove special directories from chroot to make it archivable - 8256127: Add cross-compiled foreign architectures builds to submit workflow ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1147/files - new: https://git.openjdk.java.net/jdk/pull/1147/files/eb6b10a3..f0170df7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=02-03 Stats: 2530 lines in 28 files changed: 617 ins; 1697 del; 216 mod Patch: https://git.openjdk.java.net/jdk/pull/1147.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1147/head:pull/1147 PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Wed Nov 11 19:58:10 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 11 Nov 2020 19:58:10 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v5] In-Reply-To: References: Message-ID: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. > > This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. > > As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. > > The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: - Merge branch 'master' into JDK-8256127-foreign-build - Merge branch 'master' into JDK-8256127-foreign-build - Incomplete fix: remove special dirs - Merge branch 'master' into JDK-8256127-foreign-build - Remove special directories from chroot to make it archivable - 8256127: Add cross-compiled foreign architectures builds to submit workflow ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1147/files - new: https://git.openjdk.java.net/jdk/pull/1147/files/f0170df7..8509843e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=03-04 Stats: 2 lines in 2 files changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1147.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1147/head:pull/1147 PR: https://git.openjdk.java.net/jdk/pull/1147 From ihse at openjdk.java.net Wed Nov 11 20:19:59 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 11 Nov 2020 20:19:59 GMT Subject: RFR: 8256216: Enable reproducible builds in jib-profiles Message-ID: The time has come to actually turn on the support for reproducible builds by default in the jib profiles. ------------- Commit messages: - 8256216: Enable reproducible builds in jib-profiles Changes: https://git.openjdk.java.net/jdk/pull/1176/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1176&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256216 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1176.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1176/head:pull/1176 PR: https://git.openjdk.java.net/jdk/pull/1176 From ihse at openjdk.java.net Wed Nov 11 20:56:53 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 11 Nov 2020 20:56:53 GMT Subject: RFR: JDK-8256188: Adjust output of make/autoconf/configure In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 12:35:07 GMT, Matthias Baesken wrote: > The script make/autoconf/configure should give a bit more output in case various errors occur (e.g. missing autoconf). > The new output in case of missing autoconf adds the PATH; additionally we show on AIX some info on how to install autoconf. > > bash /jdk_6/jdk/jdk/configure > Runnable configure script is not present > ... > Autoconf is not found on the PATH (/opt/freeware/bin:/usr/bin:/etc:/usr/sbin), and AUTOCONF is not set. > You need autoconf to be able to generate a runnable configure script. > You might be able to fix this by installing autoconf from the 'AIX Toolbox for Linux Applications' (or compile it from the sources). > Error: Cannot find autoconf Looks good to me. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1164 From ihse at openjdk.java.net Wed Nov 11 21:05:00 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 11 Nov 2020 21:05:00 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v5] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 19:58:10 GMT, Aleksey Shipilev wrote: >> It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. >> >> This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. >> >> As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. >> >> The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: > > - Merge branch 'master' into JDK-8256127-foreign-build > - Merge branch 'master' into JDK-8256127-foreign-build > - Incomplete fix: remove special dirs > - Merge branch 'master' into JDK-8256127-foreign-build > - Remove special directories from chroot to make it archivable > - 8256127: Add cross-compiled foreign architectures builds to submit workflow Aleksey, I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for *all* changes seem like the wrong way to proceed. My concern vary all the way from increased build time, to increased usage of the Github CPU quota (which, as I understand it, is fairly large but not unlimited), to the elephant in the room: the question on who the burden lays to fix bugs in these exotic platforms. It is not at all obvious that these builds should be tested before commit. I think this change warrants a much wider and deeper discussion about the purpose of the Github commit hook actions, how the free Github CPU quota should be spent, and on whom the responsibility lays to either make sure no. commit is pushed that breaks an exotic platform, or to rectify the problem with a follow-up issue. I recommend you withdraw this PR and instead initiate a wider discussion on a suitable mailing list, e.g. jdk-dev. ------------- Changes requested by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1147 From mdoerr at openjdk.java.net Wed Nov 11 22:00:57 2020 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Wed, 11 Nov 2020 22:00:57 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v5] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 21:02:38 GMT, Magnus Ihse Bursie wrote: >> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: >> >> - Merge branch 'master' into JDK-8256127-foreign-build >> - Merge branch 'master' into JDK-8256127-foreign-build >> - Incomplete fix: remove special dirs >> - Merge branch 'master' into JDK-8256127-foreign-build >> - Remove special directories from chroot to make it archivable >> - 8256127: Add cross-compiled foreign architectures builds to submit workflow > > Aleksey, > > I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for *all* changes seem like the wrong way to proceed. > > My concern vary all the way from increased build time, to increased usage of the Github CPU quota (which, as I understand it, is fairly large but not unlimited), to the elephant in the room: the question on who the burden lays to fix bugs in these exotic platforms. It is not at all obvious that these builds should be tested before commit. > > I think this change warrants a much wider and deeper discussion about the purpose of the Github commit hook actions, how the free Github CPU quota should be spent, and on whom the responsibility lays to either make sure no. commit is pushed that breaks an exotic platform, or to rectify the problem with a follow-up issue. > > I recommend you withdraw this PR and instead initiate a wider discussion on a suitable mailing list, e.g. jdk-dev. Maybe it would be possible to build hotspot only as this is the part where things often break? This would save a lot of CPU time. E.g. I think it'd be valueable to have at least one Big Endian platform included. I noticed that some Oracle people use cross builds to check their changes on these platforms. It would save a bit of their time as well. ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From mbaesken at openjdk.java.net Thu Nov 12 06:33:54 2020 From: mbaesken at openjdk.java.net (Matthias Baesken) Date: Thu, 12 Nov 2020 06:33:54 GMT Subject: Integrated: JDK-8256188: Adjust output of make/autoconf/configure In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 12:35:07 GMT, Matthias Baesken wrote: > The script make/autoconf/configure should give a bit more output in case various errors occur (e.g. missing autoconf). > The new output in case of missing autoconf adds the PATH; additionally we show on AIX some info on how to install autoconf. > > bash /jdk_6/jdk/jdk/configure > Runnable configure script is not present > ... > Autoconf is not found on the PATH (/opt/freeware/bin:/usr/bin:/etc:/usr/sbin), and AUTOCONF is not set. > You need autoconf to be able to generate a runnable configure script. > You might be able to fix this by installing autoconf from the 'AIX Toolbox for Linux Applications' (or compile it from the sources). > Error: Cannot find autoconf This pull request has now been integrated. Changeset: ec08b3f2 Author: Matthias Baesken URL: https://git.openjdk.java.net/jdk/commit/ec08b3f2 Stats: 6 lines in 1 file changed: 3 ins; 0 del; 3 mod 8256188: Adjust output of make/autoconf/configure Reviewed-by: clanger, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1164 From shade at openjdk.java.net Thu Nov 12 06:50:08 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 12 Nov 2020 06:50:08 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v6] In-Reply-To: References: Message-ID: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). Those cross-compilation targets frequently find arch-specific build bugs. > > This improvement adds several foreign architectures to GH actions workflow to provide a safety net against build breakages on foreign architectures. We cannot easily _test_ those foreign architectures (that is another can of worms, not easily doable without exploding the pipeline time), but even having the builds cover a significant part of follow-up bugs for JDK/VM work. > > As workflow performance optimization, foreign architectures reuse the Linux x86_64 release build as build JDK. Otherwise, they would need to compile it again for every config, thus doubling the build times. This means that foreign architectures builds are running in parallel with test jobs, which might prolong the critical path length for the entire batch to complete. These builds still complete before Windows and MacOS testing catches up. On the upside, x86_64 act as "pilot builds" for foreign arches: if x86_64 fails, then foreign arches would not be compiled at all. > > The created sysroot is also cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. The regeneration takes about 10 minutes, and it produces about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Build only hotspot reduced build ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1147/files - new: https://git.openjdk.java.net/jdk/pull/1147/files/8509843e..81e07fe2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=04-05 Stats: 20 lines in 1 file changed: 4 ins; 4 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/1147.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1147/head:pull/1147 PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Thu Nov 12 09:37:54 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 12 Nov 2020 09:37:54 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v5] In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 21:57:47 GMT, Martin Doerr wrote: >> Aleksey, >> >> I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for *all* changes seem like the wrong way to proceed. >> >> My concern vary all the way from increased build time, to increased usage of the Github CPU quota (which, as I understand it, is fairly large but not unlimited), to the elephant in the room: the question on who the burden lays to fix bugs in these exotic platforms. It is not at all obvious that these builds should be tested before commit. >> >> I think this change warrants a much wider and deeper discussion about the purpose of the Github commit hook actions, how the free Github CPU quota should be spent, and on whom the responsibility lays to either make sure no. commit is pushed that breaks an exotic platform, or to rectify the problem with a follow-up issue. >> >> I recommend you withdraw this PR and instead initiate a wider discussion on a suitable mailing list, e.g. jdk-dev. > > Maybe it would be possible to build hotspot only as this is the part where things often break? This would save a lot of CPU time. > E.g. I think it'd be valueable to have at least one Big Endian platform included. > I noticed that some Oracle people use cross builds to check their changes on these platforms. It would save a bit of their time as well. > I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for _all_ changes seem like the wrong way to proceed. I understand the first reaction, but please read the updated description. It includes the discussion what bugs it tries to catch, and updated cost estimates. Submit workflow is a convenience thing for better testing coverage. Having the early warnings for fixable issues in foreign architectures helps to avoid follow-up churn without increasing the costs all too much. Do note that it does not require contributors to fix the runtime issues in foreign architectures, it only asks for passing a rather low bar of _not breaking the builds_ for them, which is, in my experience, an easily achievable goal. It is easily achievable because -- I am speaking from experience of fixing a lot of them over the years! -- the overwhelming majority of arch-specific build breakages are about silly things that are obvious from the build logs. So I am leaving this PR open, letting others to chime in. If there is really a need for a wider discussion here, I could start a thread at jdk-dev@, but so far it does not look as a good use of our collective time. Please read through the description, and see if that soothes your concerns. Aside: the issues about costs were [raised before](https://mail.openjdk.java.net/pipermail/skara-dev/2020-October/003581.html), but I think all of them resolve as "enjoy your free tier". If we target to optimize the GH action costs, IMO we should instead consider to avoid triggering them on every push, as [I hinted before](https://mail.openjdk.java.net/pipermail/jdk-dev/2020-September/004739.html), and which I think requires better Skara integration (i.e. for `/test` command). Meanwhile, this PR gives us extended build coverage at fraction of the additional cost. ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From simonis at openjdk.java.net Thu Nov 12 10:10:58 2020 From: simonis at openjdk.java.net (Volker Simonis) Date: Thu, 12 Nov 2020 10:10:58 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v5] In-Reply-To: References: Message-ID: <_0MkfVxIjGdbfkLg8XxWK_g9-GAQkILpZiL2TzElA5k=.4c752739-3ecd-485e-a123-f1fbb4e70b25@github.com> On Thu, 12 Nov 2020 09:34:51 GMT, Aleksey Shipilev wrote: >> Maybe it would be possible to build hotspot only as this is the part where things often break? This would save a lot of CPU time. >> E.g. I think it'd be valueable to have at least one Big Endian platform included. >> I noticed that some Oracle people use cross builds to check their changes on these platforms. It would save a bit of their time as well. > >> I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for _all_ changes seem like the wrong way to proceed. > > I understand the first reaction, but please read the updated description. It includes the discussion what bugs it tries to catch, and updated cost estimates. > > Submit workflow is a convenience thing for better testing coverage. Having the early warnings for fixable issues in foreign architectures helps to avoid follow-up churn without increasing the costs all too much. Do note that it does not require contributors to fix the runtime issues in foreign architectures, it only asks for passing a rather low bar of _not breaking the builds_ for them, which is, in my experience, an easily achievable goal. It is easily achievable because -- I am speaking from experience of fixing a lot of them over the years! -- the overwhelming majority of arch-specific build breakages are about silly things that are obvious from the build logs. > > So I am leaving this PR open, letting others to chime in. If there is really a need for a wider discussion here, I could start a thread at jdk-dev@, but so far it does not look as a good use of our collective time. Please read through the description, and see if that soothes your concerns. > > Aside: the issues about costs were [raised before](https://mail.openjdk.java.net/pipermail/skara-dev/2020-October/003581.html), but I think all of them resolve as "enjoy your free tier". If we target to optimize the GH action costs, IMO we should instead consider to avoid triggering them on every push, as [I hinted before](https://mail.openjdk.java.net/pipermail/jdk-dev/2020-September/004739.html), and which I think requires better Skara integration (i.e. for `/test` command). Meanwhile, this PR gives us extended build coverage at fraction of the additional cost. I think this is a really useful feature and I'm a little surprised that it gets rejected. Please find my comments inline. > Aleksey, > > I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for _all_ changes seem like the wrong way to proceed. > > My concern vary all the way from increased build time, to increased usage of the Github CPU quota (which, as I understand it, is fairly large but not unlimited), to the elephant in the room: the question on who the burden lays to fix bugs in these exotic platforms. It is not at all obvious that these builds should be tested before commit. > I think "on who's the burden to fix issues" on *exotic* platforms and "testing changes on all platforms" at commit time are two totally distinct questions. While I agree that problems on a *exotic* platform shouldn't block reviewed changes for an unreasonable amount of time, I also think it is very valuable for both, authors of changes and maintainers of *exotic* platforms to at least get early notifications of problems on other platforms. My experience as a maintainer of the AIX7PowerPC/s390 ports has always been very positive with regards to the collaboration with authors of changes which impact these ports. Authors are anxious to not break other platforms but usually don't have the possibility to test on these platforms. Aleksey's change comes in very handy here. It will help authors to fix trivial build problems right in their initial submission which I'm sure they'll appreciate. For more complex problems it gives Authors the possibility to notify port maintainers early, before a change gets pushed. I'm sure this alerting could even be automatized in the future if builds/tests fail on certain platforms. Of course such problems shouldn't block changes which are otherwise reviewed and ready to push for an unreasonable amount of time, so I have nothing against making the test on such platform optional submit requirements. But overall I think Aleksey's enhancement are a great means to improve the development experience and to keep the OpenJDK repos clean of trivial follow-up fixes. PS: I specially emphasized *exotic platforms* because it seems to me that your comment implies that all not Oracle-supported platforms are considered *exotic* for you. I think in an open source project it should be decided by the community which platforms are considered *main* (or *primary*) and which are considered *secondary* platforms. > I think this change warrants a much wider and deeper discussion about the purpose of the Github commit hook actions, how the free Github CPU quota should be spent, and on whom the responsibility lays to either make sure no. commit is pushed that breaks an exotic platform, or to rectify the problem with a follow-up issue. > I agree that we should we frugal with resources, especially if these resources are being offered to us at no charge. But from my understanding, the resources for GitHub actions are accounted on a personal account base for every developer and not to the upstream project. So every developer who clones OpenJDK and submits changes to his fork will use his personal "GitHub Actions" budget. According to [the documentation](https://docs.github.com/en/free-pro-team at latest/actions/reference/usage-limits-billing-and-administration#usage-limits) and [@rwestberg's explanations on skara-dev](https://mail.openjdk.java.net/pipermail/skara-dev/2020-October/003581.html) this budget contains 20 parallel jobs (with up to 6 hours runtime) 24/7. Even with Aleksey's changes I think we're well in this budget. Are you afraid tha GitHub/Microsoft will cut down this free-tier in general or for for OpenJDK users? If that's the case we should probably try to get an offical statement from GitHub/Microsoft on this topic before unnecessarily restricting ourselves. As Aleksey explained, the additional builds should still finish before the corresponding Windows/MacOS builds and tests terminate, so there shouldn't be any usability issue either. > I recommend you withdraw this PR and instead initiate a wider discussion on a suitable mailing list, e.g. jdk-dev. I really like this change and I'd appreciate if we could get it merged. I'm of course always open to discussion on details (e.g. if the new biuilds/tests should be mandatory or. optional, how to notify port maintainers, etc...). ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From mcimadamore at openjdk.java.net Thu Nov 12 12:18:09 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 12 Nov 2020 12:18:09 GMT Subject: RFR: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) [v29] In-Reply-To: References: Message-ID: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 36 commits: - Merge branch 'master' into 8254162 - Invert condition in memory access var handle `withInvokeBehavior' - Merge branch 'master' into 8254162 - Add more output in TestHandhsake.java - Further improve output of TestHandshake - Improve debugging output of TestHandhsake - Remove endianness-aware byte getter/setter in MemoryAccess Remove index-based version of byte getter/setter in MemoryAccess - Fix post-merge issues caused by 8219014 - Merge branch 'master' into 8254162 - Addess remaining feedback from @AlanBateman and @mrserb - ... and 26 more: https://git.openjdk.java.net/jdk/compare/ec08b3f2...0b81a39e ------------- Changes: https://git.openjdk.java.net/jdk/pull/548/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=548&range=28 Stats: 7600 lines in 82 files changed: 4791 ins; 1590 del; 1219 mod Patch: https://git.openjdk.java.net/jdk/pull/548.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/548/head:pull/548 PR: https://git.openjdk.java.net/jdk/pull/548 From coleenp at openjdk.java.net Thu Nov 12 16:23:23 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 12 Nov 2020 16:23:23 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v8] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Add logging to event posting in case of pauses. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/6dec83d8..0487b84c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=06-07 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From mcimadamore at openjdk.java.net Thu Nov 12 16:41:01 2020 From: mcimadamore at openjdk.java.net (Maurizio Cimadamore) Date: Thu, 12 Nov 2020 16:41:01 GMT Subject: Integrated: 8254162: Implementation of Foreign-Memory Access API (Third Incubator) In-Reply-To: References: Message-ID: On Wed, 7 Oct 2020 17:13:22 GMT, Maurizio Cimadamore wrote: > This patch contains the changes associated with the third incubation round of the foreign memory access API incubation (see JEP 393 [1]). This iteration focus on improving the usability of the API in 3 main ways: > > * first, by providing a way to obtain truly *shared* segments, which can be accessed and closed concurrently from multiple threads > * second, by providing a way to register a memory segment against a `Cleaner`, so as to have some (optional) guarantee that the memory will be deallocated, eventually > * third, by not requiring users to dive deep into var handles when they first pick up the API; a new `MemoryAccess` class has been added, which defines several useful dereference routines; these are really just thin wrappers around memory access var handles, but they make the barrier of entry for using this API somewhat lower. > > A big conceptual shift that comes with this API refresh is that the role of `MemorySegment` and `MemoryAddress` is not the same as it used to be; it used to be the case that a memory address could (sometimes, not always) have a back link to the memory segment which originated it; additionally, memory access var handles used `MemoryAddress` as a basic unit of dereference. > > This has all changed as per this API refresh; now a `MemoryAddress` is just a dumb carrier which wraps a pair of object/long addressing coordinates; `MemorySegment` has become the star of the show, as far as dereferencing memory is concerned. You cannot dereference memory if you don't have a segment. This improves usability in a number of ways - first, it is a lot easier to wrap native addresses (`long`, essentially) into a `MemoryAddress`; secondly, it is crystal clear what a client has to do in order to dereference memory: if a client has a segment, it can use that; otherwise, if the client only has an address, it will have to create a segment *unsafely* (this can be done by calling `MemoryAddress::asSegmentRestricted`). > > A list of the API, implementation and test changes is provided below. If you have any questions, or need more detailed explanations, I (and the rest of the Panama team) will be happy to point at existing discussions, and/or to provide the feedback required. > > A big thank to Erik Osterlund, Vladimir Ivanov and David Holmes, without whom the work on shared memory segment would not have been possible; also I'd like to thank Paul Sandoz, whose insights on API design have been very helpful in this journey. > > Thanks > Maurizio > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/javadoc/jdk/incubator/foreign/package-summary.html > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8254162_v1/specdiff/jdk/incubator/foreign/package-summary.html > > CSR: > > https://bugs.openjdk.java.net/browse/JDK-8254163 > > > > ### API Changes > > * `MemorySegment` > * drop factory for restricted segment (this has been moved to `MemoryAddress`, see below) > * added a no-arg factory for a native restricted segment representing entire native heap > * rename `withOwnerThread` to `handoff` > * add new `share` method, to create shared segments > * add new `registerCleaner` method, to register a segment against a cleaner > * add more helpers to create arrays from a segment e.g. `toIntArray` > * add some `asSlice` overloads (to make up for the fact that now segments are more frequently used as cursors) > * rename `baseAddress` to `address` (so that `MemorySegment` can implement `Addressable`) > * `MemoryAddress` > * drop `segment` accessor > * drop `rebase` method and replace it with `segmentOffset` which returns the offset (a `long`) of this address relative to a given segment > * `MemoryAccess` > * New class supporting several static dereference helpers; the helpers are organized by carrier and access mode, where a carrier is one of the usual suspect (a Java primitive, minus `boolean`); the access mode can be simple (e.g. access base address of given segment), or indexed, in which case the accessor takes a segment and either a low-level byte offset,or a high level logical index. The classification is reflected in the naming scheme (e.g. `getByte` vs. `getByteAtOffset` vs `getByteAtIndex`). > * `MemoryHandles` > * drop `withOffset` combinator > * drop `withStride` combinator > * the basic memory access handle factory now returns a var handle which takes a `MemorySegment` and a `long` - from which it is easy to derive all the other handles using plain var handle combinators. > * `Addressable` > * This is a new interface which is attached to entities which can be projected to a `MemoryAddress`. For now, both `MemoryAddress` and `MemorySegment` implement it; we have plans, with JEP 389 [2] to add more implementations. Clients can largely ignore this interface, which comes in really handy when defining native bindings with tools like `jextract`. > * `MemoryLayouts` > * A new layout, for machine addresses, has been added to the mix. > > > > ### Implementation changes > > There are two main things to discuss here: support for shared segments, and the general simplification of the memory access var handle support. > > #### Shared segments > > The support for shared segments cuts in pretty deep in the VM. Support for shared segments is notoriously hard to achieve, at least in a way that guarantees optimal access performances. This is caused by the fact that, if a segment is shared, it would be possible for a thread to close it while another is accessing it. > > After considering several options (see [3]), we zeroed onto an approach which is inspired by an happy idea that Andrew Haley had (and that he reminded me of at this year OpenJDK committer workshop - thanks!). The idea is that if we could *freeze* the world (e.g. with a GC pause), while a segment is closed, we could then prevent segments from being accessed concurrently to a close operation. For this to work, it is crucial that no GC safepoints can occur between a segment liveness check and the access itself (otherwise it would be possible for the accessing thread to stop just right before an unsafe call). It also relies on the fact that hotspot/C2 should not be able to propagate loads across safepoints. > > Sadly, none of these conditions seems to be valid in the current implementation, so we needed to resort to a bit of creativity. First, we noted that, if we could mark so called *scoped* method with an annotation, it would be very simply to check as to whether a thread was in the middle of a scoped method when we stopped the world for a close operation (btw, instead of stopping the world, we do a much more efficient, thread-local polling, thanks to JEP 312 [4]). > > The question is, then, once we detect that a thread is accessing the very segment we're about to close, what should happen? We first experimented with a solution which would install an *asynchronous* exception on the accessing thread, thus making it fail. This solution has some desirable properties, in that a `close` operation always succeeds. Unfortunately the machinery for async exceptions is a bit fragile (e.g. not all the code in hotspot checks for async exceptions); to minimize risks, we decided to revert to a simpler strategy, where `close` might fail when it finds that another thread is accessing the segment being closed. > > As written in the javadoc, this doesn't mean that clients should just catch and try again; an exception on `close` is a bug in the user code, likely arising from lack of synchronization, and should be treated as such. > > In terms of gritty implementation, we needed to centralize memory access routines in a single place, so that we could have a set of routines closely mimicking the primitives exposed by `Unsafe` but which, in addition, also provided a liveness check. This way we could mark all these routines with the special `@Scoped` annotation, which tells the VM that something important is going on. > > To achieve this, we created a new (autogenerated) class, called `ScopedMemoryAccess`. This class contains all the main memory access primitives (including bulk access, like `copyMemory`, or `setMemory`), and accepts, in addition to the access coordinates, also a scope object, which is tested before access. A reachability fence is also thrown in the mix to make sure that the scope is kept alive during access (which is important when registering segments against cleaners). > > Of course, to make memory access safe, memory access var handles, byte buffer var handles, and byte buffer API should use the new `ScopedMemoryAccess` class instead of unsafe, so that a liveness check can be triggered (in case a scope is present). > > `ScopedMemoryAccess` has a `closeScope` method, which initiates the thread-local handshakes, and returns `true` if the handshake completed successfully. > > The implementation of `MemoryScope` (now significantly simplified from what we had before), has two implementations, one for confined segments and one for shared segments; the main difference between the two is what happens when the scope is closed; a confined segment sets a boolean flag to false, and returns, whereas a shared segment goes into a `CLOSING` state, then starts the handshake, and then updates the state again, to either `CLOSED` or `ALIVE` depending on whether the handshake was successful or not. Note that when a shared segment is in the `CLOSING` state, `MemorySegment::isAlive` will still return `true`, while the liveness check upon memory access will fail. > > #### Memory access var handles overhaul > > The key realization here was that if all memory access var handles took a coordinate pair of `MemorySegment` and `long`, all other access types could be derived from this basic var handle form. > > This allowed us to remove the on-the-fly var handle generation, and to simply derive structural access var handles (such as those obtained by calling `MemoryLayout::varHandle`) using *plain* var handle combinators, so that e.g. additional offset is injected into a base memory access var handle. > > This also helped in simplifying the implementation by removing the special `withStride` and `withOffset` combinators, which previously needed low-level access on the innards of the memory access var handle. All that code is now gone. > > #### Test changes > > Not much to see here - most of the tests needed to be updated because of the API changes. Some were beefed up (like the array test, since now segments can be projected into many different kinds of arrays). A test has been added to test the `Cleaner` functionality, and another stress test has been added for shared segments (`TestHandshake`). Some of the microbenchmarks also needed some tweaks - and some of them were also updated to also test performance in the shared segment case. > > [1] - https://openjdk.java.net/jeps/393 > [2] - https://openjdk.java.net/jeps/389 > [3] - https://mail.openjdk.java.net/pipermail/panama-dev/2020-May/009004.html > [4] - https://openjdk.java.net/jeps/312 This pull request has now been integrated. Changeset: 3e70aac5 Author: Maurizio Cimadamore URL: https://git.openjdk.java.net/jdk/commit/3e70aac5 Stats: 7600 lines in 82 files changed: 4791 ins; 1590 del; 1219 mod 8254162: Implementation of Foreign-Memory Access API (Third Incubator) Reviewed-by: erikj, psandoz, alanb ------------- PR: https://git.openjdk.java.net/jdk/pull/548 From ihse at openjdk.java.net Thu Nov 12 22:23:02 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 12 Nov 2020 22:23:02 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file Message-ID: Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) ------------- Commit messages: - 8256308: Send arguments to javac server in a config file Changes: https://git.openjdk.java.net/jdk/pull/1195/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256308 Stats: 118 lines in 3 files changed: 48 ins; 33 del; 37 mod Patch: https://git.openjdk.java.net/jdk/pull/1195.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1195/head:pull/1195 PR: https://git.openjdk.java.net/jdk/pull/1195 From ihse at openjdk.java.net Thu Nov 12 22:54:13 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 12 Nov 2020 22:54:13 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v2] In-Reply-To: References: Message-ID: > Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. > > Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. > > So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. > > Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. > > I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Fix race where multiple compilations overwrite the config file ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1195/files - new: https://git.openjdk.java.net/jdk/pull/1195/files/26e5efdd..0a613f55 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=00-01 Stats: 6 lines in 1 file changed: 2 ins; 3 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1195.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1195/head:pull/1195 PR: https://git.openjdk.java.net/jdk/pull/1195 From ihse at openjdk.java.net Fri Nov 13 00:12:12 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 13 Nov 2020 00:12:12 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v3] In-Reply-To: References: Message-ID: > Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. > > Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. > > So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. > > Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. > > I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Paths on Windows must be converted before being written to the config file ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1195/files - new: https://git.openjdk.java.net/jdk/pull/1195/files/0a613f55..c47f7c2a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=01-02 Stats: 8 lines in 1 file changed: 5 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1195.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1195/head:pull/1195 PR: https://git.openjdk.java.net/jdk/pull/1195 From ihse at openjdk.java.net Fri Nov 13 01:04:15 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 13 Nov 2020 01:04:15 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v4] In-Reply-To: References: Message-ID: > Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. > > Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. > > So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. > > Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. > > I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Some rules had identical names which still caused races, so move config file ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1195/files - new: https://git.openjdk.java.net/jdk/pull/1195/files/c47f7c2a..ba312950 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1195.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1195/head:pull/1195 PR: https://git.openjdk.java.net/jdk/pull/1195 From ihse at openjdk.java.net Fri Nov 13 01:08:09 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 13 Nov 2020 01:08:09 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v5] In-Reply-To: References: Message-ID: > Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. > > Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. > > So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. > > Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. > > I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: MODULE_SUBDIR must be defined before using ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1195/files - new: https://git.openjdk.java.net/jdk/pull/1195/files/ba312950..44507d25 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=03-04 Stats: 8 lines in 1 file changed: 4 ins; 4 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1195.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1195/head:pull/1195 PR: https://git.openjdk.java.net/jdk/pull/1195 From ihse at openjdk.java.net Fri Nov 13 09:48:17 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 13 Nov 2020 09:48:17 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v6] In-Reply-To: References: Message-ID: On Thu, 12 Nov 2020 06:50:08 GMT, Aleksey Shipilev wrote: >> It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). >> >> These cross-compilation targets frequently find arch-specific build bugs. Hotspot is rather well insulated from the fundamental problems in arch-support, so the overwhelming majority of those arch-specific bugs are simple omissions of `#include`-s, inconsistent `#ifdef`-ing, missed method renames / signature changes, or incorrect removals of methods that are used by arch-specific code. Those are straightforward to fix, if the contributor knows about them. My experience says the common attitude to these fixes is: "Oh, I would have fixed it in the original change if I knew about this." >> >> This improvement adds several foreign architectures to GH actions workflow to provide the automatic safety net to give early warning for these cases. In fact, many Hotspot contributors already build AArch64, ARM, PPC64 targets as pre-integration sanity checks, and this does the important checks automatically for them. >> >> To optimize workflow costs, the change does the following: >> - The build is only done for hotspot-debug-no-pch, as the most frequent place where arch-specific build bugs show up. >> - There are no tests, and in fact the arch-specific binary does not even run. The build is purely about the build. >> - Foreign architectures reuse the Linux x86_64 release build as build JDK. This avoids building "host" build JDK as it would otherwise happen with cross-compiling. >> - The created sysroot is cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. This saves about 10 minutes per arch. >> >> Space-wise, sysroots in GH cache take about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. >> >> Time-wise, ball-parking the current workflow budget [looks like this](https://github.com/shipilev/jdk/runs/1389151071): >> - Linux x86_64: >> - builds: 120m >> - tests: 200m >> - Linux x86_32: >> - builds: 75m >> - tests: 235m >> - Windows x86_64 >> - builds: 120m >> - tests: 290m >> - MacOS x86_64 >> - builds: 75m >> - tests: 165m >> - Linux aarch64: >> - builds: 25m (+10m to create uncached sysroot) >> - Linux arm: >> - builds: 20m (+10m to create uncached sysroot) >> - Linux ppc64le: >> - builds: 20m (+15m to create uncached sysroot) >> - Linux s390x: >> - builds: 20m (+10m to create uncached sysroot) >> >> In other words, new workflow takes about 715 Linux-host-minutes, 410 Windows-host-minutes, 240 Mac-host-minutes. Out of which new jobs take about 85 Linux-host-minutes. So the cost of new jobs is roughly: >> - 11.9% of Linux-host-minutes >> - 6.2% of all-host-minutes >> - 2.2% of runner-minutes, if you [weigh in the proportional cost of non-Linux runners](https://docs.github.com/en/free-pro-team at latest/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions) >> >> In other words, the additional runner costs are pale in comparison with what is already done in build and test jobs. > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Build only hotspot reduced build Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From ihse at openjdk.java.net Fri Nov 13 09:48:18 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 13 Nov 2020 09:48:18 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v5] In-Reply-To: <_0MkfVxIjGdbfkLg8XxWK_g9-GAQkILpZiL2TzElA5k=.4c752739-3ecd-485e-a123-f1fbb4e70b25@github.com> References: <_0MkfVxIjGdbfkLg8XxWK_g9-GAQkILpZiL2TzElA5k=.4c752739-3ecd-485e-a123-f1fbb4e70b25@github.com> Message-ID: On Thu, 12 Nov 2020 10:08:39 GMT, Volker Simonis wrote: >>> I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for _all_ changes seem like the wrong way to proceed. >> >> I understand the first reaction, but please read the updated description. It includes the discussion what bugs it tries to catch, and updated cost estimates. >> >> Submit workflow is a convenience thing for better testing coverage. Having the early warnings for fixable issues in foreign architectures helps to avoid follow-up churn without increasing the costs all too much. Do note that it does not require contributors to fix the runtime issues in foreign architectures, it only asks for passing a rather low bar of _not breaking the builds_ for them, which is, in my experience, an easily achievable goal. It is easily achievable because -- I am speaking from experience of fixing a lot of them over the years! -- the overwhelming majority of arch-specific build breakages are about silly things that are obvious from the build logs. >> >> So I am leaving this PR open, letting others to chime in. If there is really a need for a wider discussion here, I could start a thread at jdk-dev@, but so far it does not look as a good use of our collective time. Please read through the description, and see if that soothes your concerns. >> >> Aside: the issues about costs were [raised before](https://mail.openjdk.java.net/pipermail/skara-dev/2020-October/003581.html), but I think all of them resolve as "enjoy your free tier". If we target to optimize the GH action costs, IMO we should instead consider to avoid triggering them on every push, as [I hinted before](https://mail.openjdk.java.net/pipermail/jdk-dev/2020-September/004739.html), and which I think requires better Skara integration (i.e. for `/test` command). Meanwhile, this PR gives us extended build coverage at fraction of the additional cost. > > I think this is a really useful feature and I'm a little surprised that it gets rejected. Please find my comments inline. > >> Aleksey, >> >> I'm not at all sure I want to accept these changes. I understand that you care about these exotic platforms, and so do I, but adding them to the submit workflow for _all_ changes seem like the wrong way to proceed. >> >> My concern vary all the way from increased build time, to increased usage of the Github CPU quota (which, as I understand it, is fairly large but not unlimited), to the elephant in the room: the question on who the burden lays to fix bugs in these exotic platforms. It is not at all obvious that these builds should be tested before commit. >> > > I think "on who's the burden to fix issues" on *exotic* platforms and "testing changes on all platforms" at commit time are two totally distinct questions. While I agree that problems on a *exotic* platform shouldn't block reviewed changes for an unreasonable amount of time, I also think it is very valuable for both, authors of changes and maintainers of *exotic* platforms to at least get early notifications of problems on other platforms. > > My experience as a maintainer of the AIX7PowerPC/s390 ports has always been very positive with regards to the collaboration with authors of changes which impact these ports. Authors are anxious to not break other platforms but usually don't have the possibility to test on these platforms. Aleksey's change comes in very handy here. It will help authors to fix trivial build problems right in their initial submission which I'm sure they'll appreciate. For more complex problems it gives Authors the possibility to notify port maintainers early, before a change gets pushed. I'm sure this alerting could even be automatized in the future if builds/tests fail on certain platforms. > > Of course such problems shouldn't block changes which are otherwise reviewed and ready to push for an unreasonable amount of time, so I have nothing against making the test on such platform optional submit requirements. > > But overall I think Aleksey's enhancement are a great means to improve the development experience and to keep the OpenJDK repos clean of trivial follow-up fixes. > > > PS: I specially emphasized *exotic platforms* because it seems to me that your comment implies that all not Oracle-supported platforms are considered *exotic* for you. I think in an open source project it should be decided by the community which platforms are considered *main* (or *primary*) and which are considered *secondary* platforms. > >> I think this change warrants a much wider and deeper discussion about the purpose of the Github commit hook actions, how the free Github CPU quota should be spent, and on whom the responsibility lays to either make sure no. commit is pushed that breaks an exotic platform, or to rectify the problem with a follow-up issue. >> > > I agree that we should we frugal with resources, especially if these resources are being offered to us at no charge. But from my understanding, the resources for GitHub actions are accounted on a personal account base for every developer and not to the upstream project. So every developer who clones OpenJDK and submits changes to his fork will use his personal "GitHub Actions" budget. According to [the documentation](https://docs.github.com/en/free-pro-team at latest/actions/reference/usage-limits-billing-and-administration#usage-limits) and [@rwestberg's explanations on skara-dev](https://mail.openjdk.java.net/pipermail/skara-dev/2020-October/003581.html) this budget contains 20 parallel jobs (with up to 6 hours runtime) 24/7. Even with Aleksey's changes I think we're well in this budget. > > Are you afraid tha GitHub/Microsoft will cut down this free-tier in general or for for OpenJDK users? If that's the case we should probably try to get an offical statement from GitHub/Microsoft on this topic before unnecessarily restricting ourselves. > > As Aleksey explained, the additional builds should still finish before the corresponding Windows/MacOS builds and tests terminate, so there shouldn't be any usability issue either. > >> I recommend you withdraw this PR and instead initiate a wider discussion on a suitable mailing list, e.g. jdk-dev. > > I really like this change and I'd appreciate if we could get it merged. I'm of course always open to discussion on details (e.g. if the new biuilds/tests should be mandatory or. optional, how to notify port maintainers, etc...). The change to only build hotspot is good. It alleviates some of my concern over this patch. I still think there is a wide difference in how different contributors view these Github Actions submit hooks. Some would never consider to push anything that is not green everywhere, while you think it could be okay to push with broken platform if it takes an unreasonable amount of time to fix. I do think we need to align ourself as a community on how to relate to these submit hook tests. Perhaps getting some ground rules down in the new Developer's Guide, if we can agree on such ground rules. (It's not like there's an old history of oral tradition on how to do things that needs to be written down like the rest of the Guide, it's more like we decide now on how to deal with them.) Finally, I'm worried about reproducibility and debuggability of the Github Actions. We've seen that Github Actions can be updated right under our feet, and have already had trouble with that. And if the logs are not enough, there is no simple way to reproduce the build environment locally do examine the situation. But these are general concerns about the Github Actions themselves, and not specifically aimed at this patch. And I've initiated an off-list discussion on how we can try to address some of these issues. ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From rwestberg at openjdk.java.net Fri Nov 13 10:09:58 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 13 Nov 2020 10:09:58 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v6] In-Reply-To: References: Message-ID: On Thu, 12 Nov 2020 06:50:08 GMT, Aleksey Shipilev wrote: >> It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). >> >> These cross-compilation targets frequently find arch-specific build bugs. Hotspot is rather well insulated from the fundamental problems in arch-support, so the overwhelming majority of those arch-specific bugs are simple omissions of `#include`-s, inconsistent `#ifdef`-ing, missed method renames / signature changes, or incorrect removals of methods that are used by arch-specific code. Those are straightforward to fix, if the contributor knows about them. My experience says the common attitude to these fixes is: "Oh, I would have fixed it in the original change if I knew about this." >> >> This improvement adds several foreign architectures to GH actions workflow to provide the automatic safety net to give early warning for these cases. In fact, many Hotspot contributors already build AArch64, ARM, PPC64 targets as pre-integration sanity checks, and this does the important checks automatically for them. >> >> To optimize workflow costs, the change does the following: >> - The build is only done for hotspot-debug-no-pch, as the most frequent place where arch-specific build bugs show up. >> - There are no tests, and in fact the arch-specific binary does not even run. The build is purely about the build. >> - Foreign architectures reuse the Linux x86_64 release build as build JDK. This avoids building "host" build JDK as it would otherwise happen with cross-compiling. >> - The created sysroot is cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. This saves about 10 minutes per arch. >> >> Space-wise, sysroots in GH cache take about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. >> >> Time-wise, ball-parking the current workflow budget [looks like this](https://github.com/shipilev/jdk/runs/1389151071): >> - Linux x86_64: >> - builds: 120m >> - tests: 200m >> - Linux x86_32: >> - builds: 75m >> - tests: 235m >> - Windows x86_64 >> - builds: 120m >> - tests: 290m >> - MacOS x86_64 >> - builds: 75m >> - tests: 165m >> - Linux aarch64: >> - builds: 25m (+10m to create uncached sysroot) >> - Linux arm: >> - builds: 20m (+10m to create uncached sysroot) >> - Linux ppc64le: >> - builds: 20m (+15m to create uncached sysroot) >> - Linux s390x: >> - builds: 20m (+10m to create uncached sysroot) >> >> In other words, new workflow takes about 715 Linux-host-minutes, 410 Windows-host-minutes, 240 Mac-host-minutes. Out of which new jobs take about 85 Linux-host-minutes. So the cost of new jobs is roughly: >> - 11.9% of Linux-host-minutes >> - 6.2% of all-host-minutes >> - 2.2% of runner-minutes, if you [weigh in the proportional cost of non-Linux runners](https://docs.github.com/en/free-pro-team at latest/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions) >> >> In other words, the additional runner costs are pale in comparison with what is already done in build and test jobs. > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > Build only hotspot reduced build Looks good! I did a similar attempt at cross building a while ago, but never got around to finishing it, so it's nice to see it materializing! I do have a general comment on reducing the amount of duplicated content though. Since all these cross-build platforms share the same prerequisites, they can be expressed as matrix builds. Here's what I did: https://github.com/rwestberg/jdk/blob/947c934621c3013c055152356615e0120382cedf/.github/workflows/submit.yml#L102 You'd have to adjust the details obviously, but I think it could help with future maintainability. Another minor comment is that it may be faster to use http://debian-archive.trafficmanager.net/debian/ instead of http://httpredir.debian.org/debian/ (the former is Azure-specific but I don't think it's part of the list that the latter uses). But since it will be cached after first use it probably doesn't matter much. ------------- Marked as reviewed by rwestberg (Committer). PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Fri Nov 13 10:57:10 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 10:57:10 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v6] In-Reply-To: References: Message-ID: On Fri, 13 Nov 2020 10:07:26 GMT, Robin Westberg wrote: > Looks good! I did a similar attempt at cross building a while ago, but never got around to finishing it, so it's nice to see it materializing! I do have a general comment on reducing the amount of duplicated content though. Since all these cross-build platforms share the same prerequisites, they can be expressed as matrix builds. Here's what I did: https://github.com/rwestberg/jdk/blob/947c934621c3013c055152356615e0120382cedf/.github/workflows/submit.yml#L102 Right. AFAIU your code, it bootstraps the chroot and builds x86_64 build JDK for every config and every run, something this PR is able to avoid. We'd need to figure that out. I think that is pretty doable, but it would require a few days worth of pipeline testing to work out the kinks. So, would you mind we do that in the follow-ups? ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From rwestberg at openjdk.java.net Fri Nov 13 11:07:02 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 13 Nov 2020 11:07:02 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v6] In-Reply-To: References: Message-ID: On Fri, 13 Nov 2020 10:54:01 GMT, Aleksey Shipilev wrote: >> Looks good! I did a similar attempt at cross building a while ago, but never got around to finishing it, so it's nice to see it materializing! I do have a general comment on reducing the amount of duplicated content though. Since all these cross-build platforms share the same prerequisites, they can be expressed as matrix builds. Here's what I did: https://github.com/rwestberg/jdk/blob/947c934621c3013c055152356615e0120382cedf/.github/workflows/submit.yml#L102 >> >> You'd have to adjust the details obviously, but I think it could help with future maintainability. >> >> Another minor comment is that it may be faster to use http://debian-archive.trafficmanager.net/debian/ instead of http://httpredir.debian.org/debian/ (the former is Azure-specific but I don't think it's part of the list that the latter uses). But since it will be cached after first use it probably doesn't matter much. > >> Looks good! I did a similar attempt at cross building a while ago, but never got around to finishing it, so it's nice to see it materializing! I do have a general comment on reducing the amount of duplicated content though. Since all these cross-build platforms share the same prerequisites, they can be expressed as matrix builds. Here's what I did: https://github.com/rwestberg/jdk/blob/947c934621c3013c055152356615e0120382cedf/.github/workflows/submit.yml#L102 > > Right. AFAIU your code, it bootstraps the chroot and builds x86_64 build JDK for every config and every run, something this PR is able to avoid. We'd need to figure that out. I think that is pretty doable, but it would require a few days worth of pipeline testing to work out the kinks. So, would you mind we do that in the follow-ups? Yeah, it would certainly need a bit of adaptation to capture the differences properly, so perfectly fine to look into later! ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Fri Nov 13 11:19:30 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 11:19:30 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v7] In-Reply-To: References: Message-ID: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). > > These cross-compilation targets frequently find arch-specific build bugs. Hotspot is rather well insulated from the fundamental problems in arch-support, so the overwhelming majority of those arch-specific bugs are simple omissions of `#include`-s, inconsistent `#ifdef`-ing, missed method renames / signature changes, or incorrect removals of methods that are used by arch-specific code. Those are straightforward to fix, if the contributor knows about them. My experience says the common attitude to these fixes is: "Oh, I would have fixed it in the original change if I knew about this." > > This improvement adds several foreign architectures to GH actions workflow to provide the automatic safety net to give early warning for these cases. In fact, many Hotspot contributors already build AArch64, ARM, PPC64 targets as pre-integration sanity checks, and this does the important checks automatically for them. > > To optimize workflow costs, the change does the following: > - The build is only done for hotspot-debug-no-pch, as the most frequent place where arch-specific build bugs show up. > - There are no tests, and in fact the arch-specific binary does not even run. The build is purely about the build. > - Foreign architectures reuse the Linux x86_64 release build as build JDK. This avoids building "host" build JDK as it would otherwise happen with cross-compiling. > - The created sysroot is cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. This saves about 10 minutes per arch. > > Space-wise, sysroots in GH cache take about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. > > Time-wise, ball-parking the current workflow budget [looks like this](https://github.com/shipilev/jdk/runs/1389151071): > - Linux x86_64: > - builds: 120m > - tests: 200m > - Linux x86_32: > - builds: 75m > - tests: 235m > - Windows x86_64 > - builds: 120m > - tests: 290m > - MacOS x86_64 > - builds: 75m > - tests: 165m > - Linux aarch64: > - builds: 25m (+10m to create uncached sysroot) > - Linux arm: > - builds: 20m (+10m to create uncached sysroot) > - Linux ppc64le: > - builds: 20m (+15m to create uncached sysroot) > - Linux s390x: > - builds: 20m (+10m to create uncached sysroot) > > In other words, new workflow takes about 715 Linux-host-minutes, 410 Windows-host-minutes, 240 Mac-host-minutes. Out of which new jobs take about 85 Linux-host-minutes. So the cost of new jobs is roughly: > - 11.9% of Linux-host-minutes > - 6.2% of all-host-minutes > - 2.2% of runner-minutes, if you [weigh in the proportional cost of non-Linux runners](https://docs.github.com/en/free-pro-team at latest/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions) > > In other words, the additional runner costs are pale in comparison with what is already done in build and test jobs. Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: - Merge branch 'master' into JDK-8256127-foreign-build - Build only hotspot reduced build - Merge branch 'master' into JDK-8256127-foreign-build - Merge branch 'master' into JDK-8256127-foreign-build - Incomplete fix: remove special dirs - Merge branch 'master' into JDK-8256127-foreign-build - Remove special directories from chroot to make it archivable - 8256127: Add cross-compiled foreign architectures builds to submit workflow ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1147/files - new: https://git.openjdk.java.net/jdk/pull/1147/files/81e07fe2..b0dfab5f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1147&range=05-06 Stats: 13479 lines in 272 files changed: 7889 ins; 3996 del; 1594 mod Patch: https://git.openjdk.java.net/jdk/pull/1147.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1147/head:pull/1147 PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Fri Nov 13 11:19:31 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 11:19:31 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v6] In-Reply-To: References: Message-ID: <64lg1TwimDtzP4fhzogjvk0fV1oaUWe064twCS2YcRA=.9e1b5342-c35b-4960-9bb2-0009bfbe1ce3@github.com> On Fri, 13 Nov 2020 11:04:38 GMT, Robin Westberg wrote: >>> Looks good! I did a similar attempt at cross building a while ago, but never got around to finishing it, so it's nice to see it materializing! I do have a general comment on reducing the amount of duplicated content though. Since all these cross-build platforms share the same prerequisites, they can be expressed as matrix builds. Here's what I did: https://github.com/rwestberg/jdk/blob/947c934621c3013c055152356615e0120382cedf/.github/workflows/submit.yml#L102 >> >> Right. AFAIU your code, it bootstraps the chroot and builds x86_64 build JDK for every config and every run, something this PR is able to avoid. We'd need to figure that out. I think that is pretty doable, but it would require a few days worth of pipeline testing to work out the kinks. So, would you mind we do that in the follow-ups? > > Yeah, it would certainly need a bit of adaptation to capture the differences properly, so perfectly fine to look into later! Thanks folks! I'll merge the master and see if it is still green, and if it is, I'll integrate. ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From rwestberg at openjdk.java.net Fri Nov 13 11:56:05 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 13 Nov 2020 11:56:05 GMT Subject: RFR: 8256277: Github Action build on macOS should define OS and Xcode versions Message-ID: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and Xcode versions used from latest (currently 10.15) / default (currently 12.0) to 10.15 / 11.3.1. ------------- Commit messages: - Select explicit versions of macOS and Xcode Changes: https://git.openjdk.java.net/jdk/pull/1201/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1201&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256277 Stats: 8 lines in 1 file changed: 6 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1201.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1201/head:pull/1201 PR: https://git.openjdk.java.net/jdk/pull/1201 From shade at openjdk.java.net Fri Nov 13 12:01:01 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 12:01:01 GMT Subject: RFR: 8256277: Github Action build on macOS should define OS and Xcode versions In-Reply-To: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> References: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> Message-ID: On Fri, 13 Nov 2020 11:48:31 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and Xcode versions used from latest (currently 10.15) / default (currently 12.0) to 10.15 / 11.3.1. Looks fine to me. I was wondering if we should do the same for `ubuntu-latest` and `windows-latest`. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1201 From rwestberg at openjdk.java.net Fri Nov 13 12:12:59 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 13 Nov 2020 12:12:59 GMT Subject: RFR: 8256277: Github Action build on macOS should define OS and Xcode versions In-Reply-To: References: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> Message-ID: <0LPM5aFMED3YEfALstwA913BetQVh4B0BGUSK1B_FH0=.b46c443c-a3ef-4a33-ae47-86266162e031@github.com> On Fri, 13 Nov 2020 11:58:19 GMT, Aleksey Shipilev wrote: >> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and Xcode versions used from latest (currently 10.15) / default (currently 12.0) to 10.15 / 11.3.1. > > Looks fine to me. I was wondering if we should do the same for `ubuntu-latest` and `windows-latest`. Thanks for reviewing! Yeah, planning to do something similar for the other platforms as well! ------------- PR: https://git.openjdk.java.net/jdk/pull/1201 From ehelin at openjdk.java.net Fri Nov 13 12:12:57 2020 From: ehelin at openjdk.java.net (Erik Helin) Date: Fri, 13 Nov 2020 12:12:57 GMT Subject: RFR: 8256277: Github Action build on macOS should define OS and Xcode versions In-Reply-To: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> References: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> Message-ID: On Fri, 13 Nov 2020 11:48:31 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and Xcode versions used from latest (currently 10.15) / default (currently 12.0) to 10.15 / 11.3.1. Looks good! ------------- Marked as reviewed by ehelin (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1201 From shade at openjdk.java.net Fri Nov 13 12:39:57 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 12:39:57 GMT Subject: RFR: 8256127: Add cross-compiled foreign architectures builds to submit workflow [v6] In-Reply-To: <64lg1TwimDtzP4fhzogjvk0fV1oaUWe064twCS2YcRA=.9e1b5342-c35b-4960-9bb2-0009bfbe1ce3@github.com> References: <64lg1TwimDtzP4fhzogjvk0fV1oaUWe064twCS2YcRA=.9e1b5342-c35b-4960-9bb2-0009bfbe1ce3@github.com> Message-ID: <0NdLyILXb-03p9hCdc7tpROS1jTdPbmEpgqPf-NfP4U=.3d3e3fd1-0b93-40b7-b9b3-a9caf8d7b9ef@github.com> On Fri, 13 Nov 2020 11:14:42 GMT, Aleksey Shipilev wrote: >> Yeah, it would certainly need a bit of adaptation to capture the differences properly, so perfectly fine to look into later! > > Thanks folks! I'll merge the master and see if it is still green, and if it is, I'll integrate. Cross-compiled builds still look green. Integrating. ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From shade at openjdk.java.net Fri Nov 13 12:39:58 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 12:39:58 GMT Subject: Integrated: 8256127: Add cross-compiled foreign architectures builds to submit workflow In-Reply-To: References: Message-ID: On Tue, 10 Nov 2020 19:05:31 GMT, Aleksey Shipilev wrote: > It is [possible](https://github.com/openjdk/jdk/blob/master/doc/building.md#creating-and-using-sysroots-with-qemu-deboostrap) to efficiently cross-compile to foreign architectures on current GH actions that are driven by Ubuntu. I have been using this method for years to produce binaries at [builds.shipilev.net](https://builds.shipilev.net). > > These cross-compilation targets frequently find arch-specific build bugs. Hotspot is rather well insulated from the fundamental problems in arch-support, so the overwhelming majority of those arch-specific bugs are simple omissions of `#include`-s, inconsistent `#ifdef`-ing, missed method renames / signature changes, or incorrect removals of methods that are used by arch-specific code. Those are straightforward to fix, if the contributor knows about them. My experience says the common attitude to these fixes is: "Oh, I would have fixed it in the original change if I knew about this." > > This improvement adds several foreign architectures to GH actions workflow to provide the automatic safety net to give early warning for these cases. In fact, many Hotspot contributors already build AArch64, ARM, PPC64 targets as pre-integration sanity checks, and this does the important checks automatically for them. > > To optimize workflow costs, the change does the following: > - The build is only done for hotspot-debug-no-pch, as the most frequent place where arch-specific build bugs show up. > - There are no tests, and in fact the arch-specific binary does not even run. The build is purely about the build. > - Foreign architectures reuse the Linux x86_64 release build as build JDK. This avoids building "host" build JDK as it would otherwise happen with cross-compiling. > - The created sysroot is cached and keyed on `submit.yml` hash. So it would regenerate only if the workflow itself changes. This saves about 10 minutes per arch. > > Space-wise, sysroots in GH cache take about 580M uncompressed, and 270M zstd-compressed bundle. In the end, this adds 4x270 = 1080M into local cache. > > Time-wise, ball-parking the current workflow budget [looks like this](https://github.com/shipilev/jdk/runs/1389151071): > - Linux x86_64: > - builds: 120m > - tests: 200m > - Linux x86_32: > - builds: 75m > - tests: 235m > - Windows x86_64 > - builds: 120m > - tests: 290m > - MacOS x86_64 > - builds: 75m > - tests: 165m > - Linux aarch64: > - builds: 25m (+10m to create uncached sysroot) > - Linux arm: > - builds: 20m (+10m to create uncached sysroot) > - Linux ppc64le: > - builds: 20m (+15m to create uncached sysroot) > - Linux s390x: > - builds: 20m (+10m to create uncached sysroot) > > In other words, new workflow takes about 715 Linux-host-minutes, 410 Windows-host-minutes, 240 Mac-host-minutes. Out of which new jobs take about 85 Linux-host-minutes. So the cost of new jobs is roughly: > - 11.9% of Linux-host-minutes > - 6.2% of all-host-minutes > - 2.2% of runner-minutes, if you [weigh in the proportional cost of non-Linux runners](https://docs.github.com/en/free-pro-team at latest/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions) > > In other words, the additional runner costs are pale in comparison with what is already done in build and test jobs. This pull request has now been integrated. Changeset: e9956fec Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/e9956fec Stats: 557 lines in 1 file changed: 556 ins; 0 del; 1 mod 8256127: Add cross-compiled foreign architectures builds to submit workflow Reviewed-by: ihse, rwestberg ------------- PR: https://git.openjdk.java.net/jdk/pull/1147 From erikj at openjdk.java.net Fri Nov 13 14:19:56 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 13 Nov 2020 14:19:56 GMT Subject: RFR: 8256277: Github Action build on macOS should define OS and Xcode versions In-Reply-To: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> References: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> Message-ID: On Fri, 13 Nov 2020 11:48:31 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and Xcode versions used from latest (currently 10.15) / default (currently 12.0) to 10.15 / 11.3.1. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1201 From erikj at openjdk.java.net Fri Nov 13 14:32:03 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 13 Nov 2020 14:32:03 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v5] In-Reply-To: References: Message-ID: On Fri, 13 Nov 2020 01:08:09 GMT, Magnus Ihse Bursie wrote: >> Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. >> >> Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. >> >> So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. >> >> Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. >> >> I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > MODULE_SUBDIR must be defined before using make/common/JavaCompilation.gmk line 237: > 235: $1_JAVAC_SERVER_CMD := $$(JAVA_DETACH) $$($1_JAVA_FLAGS) $$($1_JAVAC) > 236: > 237: $1_CONFIG_VARDEPS :=$$($1_JAVAC_PORT_FILE) $$($1_JAVAC_SERVER_CMD) Space make/common/JavaCompilation.gmk line 239: > 237: $1_CONFIG_VARDEPS :=$$($1_JAVAC_PORT_FILE) $$($1_JAVAC_SERVER_CMD) > 238: $1_CONFIG_VARDEPS_FILE := $$(call DependOnVariable, $1_CONFIG_VARDEPS, \ > 239: $$($1_BIN)$$($1_MODULE_SUBDIR)/_the.$1.config_vardeps) Indentation 4 make/common/JavaCompilation.gmk line 246: > 244: $1_ECHO_COMMAND := $(ECHO) > 245: endif > 246: $$($1_JAVAC_SERVER_CONFIG): $$($1_CONFIG_VARDEPS_FILE) Indentation? ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From erikj at openjdk.java.net Fri Nov 13 14:31:59 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 13 Nov 2020 14:31:59 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file In-Reply-To: References: Message-ID: On Thu, 12 Nov 2020 22:13:37 GMT, Magnus Ihse Bursie wrote: > Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. > > Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. > > So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. > > Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. > > I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) Looks good in general, just some whitespace issues. ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From rwestberg at openjdk.java.net Fri Nov 13 17:33:19 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 13 Nov 2020 17:33:19 GMT Subject: RFR: 8256354: Github Action build on Windows should define OS and MSVC versions Message-ID: <2mo8iYBgUIHeRclGe6zu4bviqrSsm0qm-em8zNsoDe4=.8c14a5bf-f63b-4dc9-9981-c6135f09cf00@github.com> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and MSVC versions used from latest (currently 2019) / default (currently 14.28) to 2019 / 14.27. ------------- Commit messages: - Use Windows 2019 and MSVC 14.27 Changes: https://git.openjdk.java.net/jdk/pull/1207/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1207&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256354 Stats: 9 lines in 1 file changed: 7 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1207.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1207/head:pull/1207 PR: https://git.openjdk.java.net/jdk/pull/1207 From erikj at openjdk.java.net Fri Nov 13 18:30:57 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 13 Nov 2020 18:30:57 GMT Subject: RFR: 8256354: Github Action build on Windows should define OS and MSVC versions In-Reply-To: <2mo8iYBgUIHeRclGe6zu4bviqrSsm0qm-em8zNsoDe4=.8c14a5bf-f63b-4dc9-9981-c6135f09cf00@github.com> References: <2mo8iYBgUIHeRclGe6zu4bviqrSsm0qm-em8zNsoDe4=.8c14a5bf-f63b-4dc9-9981-c6135f09cf00@github.com> Message-ID: On Fri, 13 Nov 2020 17:26:28 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and MSVC versions used from latest (currently 2019) / default (currently 14.28) to 2019 / 14.27. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1207 From shade at openjdk.java.net Fri Nov 13 18:38:56 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 13 Nov 2020 18:38:56 GMT Subject: RFR: 8256354: Github Action build on Windows should define OS and MSVC versions In-Reply-To: <2mo8iYBgUIHeRclGe6zu4bviqrSsm0qm-em8zNsoDe4=.8c14a5bf-f63b-4dc9-9981-c6135f09cf00@github.com> References: <2mo8iYBgUIHeRclGe6zu4bviqrSsm0qm-em8zNsoDe4=.8c14a5bf-f63b-4dc9-9981-c6135f09cf00@github.com> Message-ID: On Fri, 13 Nov 2020 17:26:28 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and MSVC versions used from latest (currently 2019) / default (currently 14.28) to 2019 / 14.27. Looks good to me. I wondered if https://github.com/marketplace/actions/setup-msbuild could have been used instead, but this is fine as well. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1207 From rwestberg at openjdk.java.net Mon Nov 16 08:04:55 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 16 Nov 2020 08:04:55 GMT Subject: Integrated: 8256354: Github Action build on Windows should define OS and MSVC versions In-Reply-To: <2mo8iYBgUIHeRclGe6zu4bviqrSsm0qm-em8zNsoDe4=.8c14a5bf-f63b-4dc9-9981-c6135f09cf00@github.com> References: <2mo8iYBgUIHeRclGe6zu4bviqrSsm0qm-em8zNsoDe4=.8c14a5bf-f63b-4dc9-9981-c6135f09cf00@github.com> Message-ID: On Fri, 13 Nov 2020 17:26:28 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and MSVC versions used from latest (currently 2019) / default (currently 14.28) to 2019 / 14.27. This pull request has now been integrated. Changeset: 1103e337 Author: Robin Westberg URL: https://git.openjdk.java.net/jdk/commit/1103e337 Stats: 9 lines in 1 file changed: 7 ins; 0 del; 2 mod 8256354: Github Action build on Windows should define OS and MSVC versions Reviewed-by: erikj, shade ------------- PR: https://git.openjdk.java.net/jdk/pull/1207 From rwestberg at openjdk.java.net Mon Nov 16 08:04:56 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 16 Nov 2020 08:04:56 GMT Subject: Integrated: 8256277: Github Action build on macOS should define OS and Xcode versions In-Reply-To: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> References: <8UB1dssvbjSJqs3RdH3w7HihdR6WRNclTKjuX9DX7ao=.05c415b1-4c9b-41a5-9ac8-ffdd8b7db8b4@github.com> Message-ID: On Fri, 13 Nov 2020 11:48:31 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and Xcode versions used from latest (currently 10.15) / default (currently 12.0) to 10.15 / 11.3.1. This pull request has now been integrated. Changeset: 588caab0 Author: Robin Westberg URL: https://git.openjdk.java.net/jdk/commit/588caab0 Stats: 8 lines in 1 file changed: 6 ins; 0 del; 2 mod 8256277: Github Action build on macOS should define OS and Xcode versions Reviewed-by: shade, ehelin, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1201 From ihse at openjdk.java.net Mon Nov 16 12:44:08 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 16 Nov 2020 12:44:08 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v6] In-Reply-To: References: Message-ID: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> > Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. > > Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. > > So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. > > Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. > > I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Fix whitespace issues ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1195/files - new: https://git.openjdk.java.net/jdk/pull/1195/files/44507d25..3015d5e4 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1195&range=04-05 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1195.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1195/head:pull/1195 PR: https://git.openjdk.java.net/jdk/pull/1195 From rwestberg at openjdk.java.net Mon Nov 16 12:58:34 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 16 Nov 2020 12:58:34 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions Message-ID: We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. ------------- Commit messages: - Select Ubuntu 20.04 and GCC 10 Changes: https://git.openjdk.java.net/jdk/pull/1225/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1225&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256393 Stats: 16 lines in 1 file changed: 3 ins; 0 del; 13 mod Patch: https://git.openjdk.java.net/jdk/pull/1225.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1225/head:pull/1225 PR: https://git.openjdk.java.net/jdk/pull/1225 From shade at openjdk.java.net Mon Nov 16 13:17:08 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 16 Nov 2020 13:17:08 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 12:51:25 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. Hold on a sec. `ubuntu-latest` is `ubuntu-18.04`, as per [GH manual](https://docs.github.com/en/free-pro-team at latest/actions/reference/specifications-for-github-hosted-runners). So this effectively upgrades the whole thing to Ubuntu 20.04, and upgrades GCC then? I think we better stick to current defaults, i.e. `ubuntu-18.04` and its GCC. In JDK-8256277, we did not upgrade MacOS target either... .github/workflows/submit.yml line 91: > 89: > 90: - name: Build jtreg > 91: run: sh make/build-all.sh ${JAVA_HOME_8_X64} What is this change? ------------- Changes requested by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1225 From shade at openjdk.java.net Mon Nov 16 13:41:07 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 16 Nov 2020 13:41:07 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: References: Message-ID: <8_ikms2yhsjCkgCoa5UDEUBgw-_DTGTCqiir9hfild0=.ebe30951-d0bb-4d16-93c9-85ae5d70a609@github.com> On Mon, 16 Nov 2020 12:51:25 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. Marked as reviewed by shade (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From rwestberg at openjdk.java.net Mon Nov 16 13:41:09 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 16 Nov 2020 13:41:09 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: References: Message-ID: <7s_qv_ufzjqr8Ur6-O8nusnlovn-jkjrvCzUkJVcmVA=.ecba0775-53c9-42df-bafc-f34c1b4867be@github.com> On Mon, 16 Nov 2020 13:13:52 GMT, Aleksey Shipilev wrote: >> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. > > Hold on a sec. `ubuntu-latest` is `ubuntu-18.04`, as per [GH manual](https://docs.github.com/en/free-pro-team at latest/actions/reference/specifications-for-github-hosted-runners). So this effectively upgrades the whole thing to Ubuntu 20.04, and upgrades GCC then? I think we better stick to current defaults, i.e. `ubuntu-18.04` and its GCC. > > In JDK-8256277, we did not upgrade MacOS target either... Right, currently ubuntu-latest means 18.04, but that is only true for another two weeks (see https://github.com/actions/virtual-environments/issues/1816 - originally it was also planned for next week). So I think we should go straight for the upcoming latest. > .github/workflows/submit.yml line 91: > >> 89: >> 90: - name: Build jtreg >> 91: run: sh make/build-all.sh ${JAVA_HOME_8_X64} > > What is this change? On ubuntu-20.04 the default Java installation is now set to 11, but jtreg still requires Java 8 for building. ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From shade at openjdk.java.net Mon Nov 16 13:41:09 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 16 Nov 2020 13:41:09 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: <8_ikms2yhsjCkgCoa5UDEUBgw-_DTGTCqiir9hfild0=.ebe30951-d0bb-4d16-93c9-85ae5d70a609@github.com> References: <8_ikms2yhsjCkgCoa5UDEUBgw-_DTGTCqiir9hfild0=.ebe30951-d0bb-4d16-93c9-85ae5d70a609@github.com> Message-ID: On Mon, 16 Nov 2020 13:36:57 GMT, Aleksey Shipilev wrote: >> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. > > Marked as reviewed by shade (Reviewer). > Right, currently ubuntu-latest means 18.04, but that is only true for another two weeks (see [actions/virtual-environments#1816](https://github.com/actions/virtual-environments/issues/1816) - originally it was also planned for next week). So I think we should go straight for the upcoming latest. All right, changing this today makes sense to me. ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From shade at openjdk.java.net Mon Nov 16 13:41:09 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 16 Nov 2020 13:41:09 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: <7s_qv_ufzjqr8Ur6-O8nusnlovn-jkjrvCzUkJVcmVA=.ecba0775-53c9-42df-bafc-f34c1b4867be@github.com> References: <7s_qv_ufzjqr8Ur6-O8nusnlovn-jkjrvCzUkJVcmVA=.ecba0775-53c9-42df-bafc-f34c1b4867be@github.com> Message-ID: On Mon, 16 Nov 2020 13:34:58 GMT, Robin Westberg wrote: >> .github/workflows/submit.yml line 91: >> >>> 89: >>> 90: - name: Build jtreg >>> 91: run: sh make/build-all.sh ${JAVA_HOME_8_X64} >> >> What is this change? > > On ubuntu-20.04 the default Java installation is now set to 11, but jtreg still requires Java 8 for building. I see! Okay then. ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From erikj at openjdk.java.net Mon Nov 16 13:43:07 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 16 Nov 2020 13:43:07 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v6] In-Reply-To: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> References: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> Message-ID: On Mon, 16 Nov 2020 12:44:08 GMT, Magnus Ihse Bursie wrote: >> Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. >> >> Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. >> >> So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. >> >> Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. >> >> I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace issues Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From erikj at openjdk.java.net Mon Nov 16 13:54:03 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 16 Nov 2020 13:54:03 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 12:51:25 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From ihse at openjdk.java.net Mon Nov 16 14:30:04 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 16 Nov 2020 14:30:04 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: References: Message-ID: <-6nNHhzc934ck7eLnQJwo0yt0kyVtvWrU6Ma2573lu8=.1fa9c0a1-3986-4051-a881-bd7dc5f7ab1d@github.com> On Mon, 16 Nov 2020 12:51:25 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. .github/workflows/submit.yml line 190: > 188: run: | > 189: sudo apt-get install libxrandr-dev libxtst-dev libcups2-dev libasound2-dev > 190: sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 --slave /usr/bin/g++ g++ /usr/bin/g++-10 Maybe we should use apt-get functionality to install a specific version of packages? I'm not sure how relevant it is for the X and alsa libraries since they change very seldom, but perhaps for gcc, to get a specific point release of the compiler. ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From ihse at openjdk.java.net Mon Nov 16 15:16:06 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 16 Nov 2020 15:16:06 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: <-6nNHhzc934ck7eLnQJwo0yt0kyVtvWrU6Ma2573lu8=.1fa9c0a1-3986-4051-a881-bd7dc5f7ab1d@github.com> References: <-6nNHhzc934ck7eLnQJwo0yt0kyVtvWrU6Ma2573lu8=.1fa9c0a1-3986-4051-a881-bd7dc5f7ab1d@github.com> Message-ID: On Mon, 16 Nov 2020 14:27:39 GMT, Magnus Ihse Bursie wrote: >> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. > > .github/workflows/submit.yml line 190: > >> 188: run: | >> 189: sudo apt-get install libxrandr-dev libxtst-dev libcups2-dev libasound2-dev >> 190: sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 --slave /usr/bin/g++ g++ /usr/bin/g++-10 > > Maybe we should use apt-get functionality to install a specific version of packages? I'm not sure how relevant it is for the X and alsa libraries since they change very seldom, but perhaps for gcc, to get a specific point release of the compiler. Something along the lines of `sudo apt-get install gcc-10=10.2.0-5ubuntu1~20`, which I believe should match quite well the version used internally in the Oracle CI builds. ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From tiago.daitx at canonical.com Mon Nov 16 15:32:05 2020 From: tiago.daitx at canonical.com (Tiago Daitx) Date: Mon, 16 Nov 2020 12:32:05 -0300 Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: <7s_qv_ufzjqr8Ur6-O8nusnlovn-jkjrvCzUkJVcmVA=.ecba0775-53c9-42df-bafc-f34c1b4867be@github.com> References: <7s_qv_ufzjqr8Ur6-O8nusnlovn-jkjrvCzUkJVcmVA=.ecba0775-53c9-42df-bafc-f34c1b4867be@github.com> Message-ID: Hi, > >> 89: > >> 90: - name: Build jtreg > >> 91: run: sh make/build-all.sh ${JAVA_HOME_8_X64} > > > > What is this change? > > On ubuntu-20.04 the default Java installation is now set to 11, but jtreg still requires Java 8 for building. Why does jtreg need OpenJDK-8 for building? On Debian/Ubuntu we have been using the 'release' parameter to build jtreg using OpenJDK 11 while maintaining OpenJDK 8 compatibility. See https://sources.debian.org/src/jtreg/5.1-b01-2/debian/patches/use-release-instead-of-source-target.patch/ for how we do it. Please note that while we replace "source/target" for "release", one can specify both "source/target" and "release" for a javac task: ant is smart enough to pick one and ignore the other, depending on it being run on JDK9+ or not. See 'release' description at https://ant.apache.org/manual/Tasks/javac.html Cheers! -- Tiago St?rmer Daitx Software Engineer tiago.daitx at canonical.com PGP Key: 4096R/F5B213BE (hkp://keyserver.ubuntu.com) Fingerprint = 45D0 FE5A 8109 1E91 866E 8CA4 1931 8D5E F5B2 13BE From coleenp at openjdk.java.net Mon Nov 16 17:12:32 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 17:12:32 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v9] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with three additional commits since the last revision: - Add shenandoah set_needs_cleaning but this doesn't work. - fix vmTestbase/nsk/jvmti tests - improve tagmap cleanup and objectfree event posting ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/0487b84c..283696f6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=08 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=07-08 Stats: 223 lines in 16 files changed: 174 ins; 12 del; 37 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From iignatyev at openjdk.java.net Mon Nov 16 18:33:13 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Mon, 16 Nov 2020 18:33:13 GMT Subject: RFR: 8256414: add optimized build to submit workflow Message-ID: Hi all, Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? Thanks, -- Igor ------------- Commit messages: - add linux-x64-optimized build Changes: https://git.openjdk.java.net/jdk/pull/1233/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1233&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256414 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1233.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1233/head:pull/1233 PR: https://git.openjdk.java.net/jdk/pull/1233 From vlivanov at openjdk.java.net Mon Nov 16 18:38:10 2020 From: vlivanov at openjdk.java.net (Vladimir Ivanov) Date: Mon, 16 Nov 2020 18:38:10 GMT Subject: RFR: 8256414: add optimized build to submit workflow In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 18:26:18 GMT, Igor Ignatyev wrote: > Hi all, > > Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? > > Thanks, > -- Igor Thanks a lot, Igor! Looks good. ------------- Marked as reviewed by vlivanov (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1233 From shade at openjdk.java.net Mon Nov 16 18:43:03 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 16 Nov 2020 18:43:03 GMT Subject: RFR: 8256414: add optimized build to submit workflow In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 18:26:18 GMT, Igor Ignatyev wrote: > Hi all, > > Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? > > Thanks, > -- Igor Looks fine, but wouldn't you like to add `--disable-precompiled-headers` as well? ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1233 From kvn at openjdk.java.net Mon Nov 16 18:47:06 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 16 Nov 2020 18:47:06 GMT Subject: RFR: 8256414: add optimized build to submit workflow In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 18:26:18 GMT, Igor Ignatyev wrote: > Hi all, > > Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? > > Thanks, > -- Igor Marked as reviewed by kvn (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1233 From iignatyev at openjdk.java.net Mon Nov 16 18:53:20 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Mon, 16 Nov 2020 18:53:20 GMT Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: > Hi all, > > Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? > > Thanks, > -- Igor Igor Ignatyev has updated the pull request incrementally with one additional commit since the last revision: added --disable-precompiled-headers ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1233/files - new: https://git.openjdk.java.net/jdk/pull/1233/files/bad0582f..f8189b0a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1233&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1233&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1233.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1233/head:pull/1233 PR: https://git.openjdk.java.net/jdk/pull/1233 From iignatyev at openjdk.java.net Mon Nov 16 18:53:21 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Mon, 16 Nov 2020 18:53:21 GMT Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 18:40:27 GMT, Aleksey Shipilev wrote: > Looks fine, but wouldn't you like to add `--disable-precompiled-headers` as well? sure, make sense. ------------- PR: https://git.openjdk.java.net/jdk/pull/1233 From shade at openjdk.java.net Mon Nov 16 19:01:07 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 16 Nov 2020 19:01:07 GMT Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 18:53:20 GMT, Igor Ignatyev wrote: >> Hi all, >> >> Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? >> >> Thanks, >> -- Igor > > Igor Ignatyev has updated the pull request incrementally with one additional commit since the last revision: > > added --disable-precompiled-headers Marked as reviewed by shade (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1233 From iignatyev at openjdk.java.net Mon Nov 16 19:34:06 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Mon, 16 Nov 2020 19:34:06 GMT Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 18:58:37 GMT, Aleksey Shipilev wrote: >> Igor Ignatyev has updated the pull request incrementally with one additional commit since the last revision: >> >> added --disable-precompiled-headers > > Marked as reviewed by shade (Reviewer). Thanks for the reviews, folks. the build looks green, integrating. ------------- PR: https://git.openjdk.java.net/jdk/pull/1233 From iignatyev at openjdk.java.net Mon Nov 16 19:34:09 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Mon, 16 Nov 2020 19:34:09 GMT Subject: Integrated: 8256414: add optimized build to submit workflow In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 18:26:18 GMT, Igor Ignatyev wrote: > Hi all, > > Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? > > Thanks, > -- Igor This pull request has now been integrated. Changeset: 68fd71d2 Author: Igor Ignatyev URL: https://git.openjdk.java.net/jdk/commit/68fd71d2 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod 8256414: add optimized build to submit workflow add linux-x64-optimized to submit workflow Reviewed-by: vlivanov, shade, kvn ------------- PR: https://git.openjdk.java.net/jdk/pull/1233 From dcubed at openjdk.java.net Mon Nov 16 20:34:59 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 16 Nov 2020 20:34:59 GMT Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: <3OXruv8DJOopa0weeyBUGCY8uw244AJFdLL8a4CkQ3M=.cc7ae99e-a8b2-4a9c-95bc-06167d365936@github.com> On Mon, 16 Nov 2020 19:29:59 GMT, Igor Ignatyev wrote: >> Marked as reviewed by shade (Reviewer). > > Thanks for the reviews, folks. the build looks green, integrating. @iignatev - did you also change Mach5? I don't have workflow builds enabled by default since I typically do Mach5 builds... ------------- PR: https://git.openjdk.java.net/jdk/pull/1233 From iignatyev at openjdk.java.net Mon Nov 16 20:41:03 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Mon, 16 Nov 2020 20:41:03 GMT Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 19:29:59 GMT, Igor Ignatyev wrote: >> Marked as reviewed by shade (Reviewer). > > Thanks for the reviews, folks. the build looks green, integrating. > @iignatev - did you also change Mach5? I don't have workflow builds enabled > by default since I typically do Mach5 builds... Hi Dan, no, not yet. I?m going to change jib profile and tier definitions by a separate RFE. ------------- PR: https://git.openjdk.java.net/jdk/pull/1233 From thomas.stuefe at gmail.com Mon Nov 16 20:42:47 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 16 Nov 2020 21:42:47 +0100 Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: <3OXruv8DJOopa0weeyBUGCY8uw244AJFdLL8a4CkQ3M=.cc7ae99e-a8b2-4a9c-95bc-06167d365936@github.com> References: <3OXruv8DJOopa0weeyBUGCY8uw244AJFdLL8a4CkQ3M=.cc7ae99e-a8b2-4a9c-95bc-06167d365936@github.com> Message-ID: Hi, just curious, was the optimized build not earmarked for removal? I dimly remember reading discussions about this. Thanks, Thomas On Mon, Nov 16, 2020 at 9:35 PM Daniel D.Daugherty wrote: > On Mon, 16 Nov 2020 19:29:59 GMT, Igor Ignatyev > wrote: > > >> Marked as reviewed by shade (Reviewer). > > > > Thanks for the reviews, folks. the build looks green, integrating. > > @iignatev - did you also change Mach5? I don't have workflow builds enabled > by default since I typically do Mach5 builds... > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/1233 > From igor.ignatyev at oracle.com Mon Nov 16 20:55:03 2020 From: igor.ignatyev at oracle.com (igor.ignatyev at oracle.com) Date: Mon, 16 Nov 2020 12:55:03 -0800 Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: Hi Thomas, There is indeed was a discussion about the future of optimized flavor, and the consensus was to remove it. We, however, haven?t done that yet (8183287[*] is filed for that work). meanwhile, people still use optimized builds from time to time so we need to ensure they are at least buildable. [*]: https://bugs.openjdk.java.net/browse/JDK-8183287 ? Igor > On Nov 16, 2020, at 12:43 PM, Thomas St?fe wrote: > > ? > Hi, > > just curious, was the optimized build not earmarked for removal? I dimly remember reading discussions about this. > > Thanks, Thomas > > On Mon, Nov 16, 2020 at 9:35 PM Daniel D.Daugherty > wrote: > On Mon, 16 Nov 2020 19:29:59 GMT, Igor Ignatyev > wrote: > > >> Marked as reviewed by shade (Reviewer). > > > > Thanks for the reviews, folks. the build looks green, integrating. > > @iignatev - did you also change Mach5? I don't have workflow builds enabled > by default since I typically do Mach5 builds... > > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/1233 From thomas.stuefe at gmail.com Mon Nov 16 21:08:49 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 16 Nov 2020 22:08:49 +0100 Subject: RFR: 8256414: add optimized build to submit workflow [v2] In-Reply-To: References: Message-ID: Hi Igor, thanks for the update and the reference. Yes, makes sense to test it automatically as long as it?s still there. Cheers, Thomas On Mon 16. Nov 2020 at 21:55, wrote: > Hi Thomas, > > There is indeed was a discussion about the future of optimized flavor, and > the consensus was to remove it. We, however, haven?t done that yet > (8183287[*] is filed for that work). meanwhile, people still use optimized > builds from time to time so we need to ensure they are at least buildable. > > [*]: https://bugs.openjdk.java.net/browse/JDK-8183287 > > ? Igor > > On Nov 16, 2020, at 12:43 PM, Thomas St?fe > wrote: > > ? > > Hi, > > just curious, was the optimized build not earmarked for removal? I dimly > remember reading discussions about this. > > Thanks, Thomas > > On Mon, Nov 16, 2020 at 9:35 PM Daniel D.Daugherty < > dcubed at openjdk.java.net> wrote: > >> On Mon, 16 Nov 2020 19:29:59 GMT, Igor Ignatyev >> wrote: >> >> >> Marked as reviewed by shade (Reviewer). >> > >> > Thanks for the reviews, folks. the build looks green, integrating. >> >> @iignatev - did you also change Mach5? I don't have workflow builds >> enabled >> by default since I typically do Mach5 builds... >> >> ------------- >> >> PR: https://git.openjdk.java.net/jdk/pull/1233 >> > From coleenp at openjdk.java.net Mon Nov 16 23:10:21 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:10:21 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v10] In-Reply-To: References: Message-ID: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com> > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits: - Reverse remove_dead_entries_locked function names. - Merge branch 'master' into jvmti-table - Add shenandoah set_needs_cleaning but this doesn't work. - fix vmTestbase/nsk/jvmti tests - improve tagmap cleanup and objectfree event posting - Add logging to event posting in case of pauses. - Merge branch 'master' into jvmti-table - Add back WeakProcessorPhases::Phase enum. - Serguei 1. - Code review comments from Kim and Albert. - ... and 5 more: https://git.openjdk.java.net/jdk/compare/0357db35...daaa13fe ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=09 Stats: 1884 lines in 49 files changed: 768 ins; 993 del; 123 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 16 23:19:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:19:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v4] In-Reply-To: References: <172AiTMoD9T5iKu5xEVQ3AEixTDRx4gaAx0pQFfR57k=.d1d7648c-fb6f-4e87-b515-295c4e6187f7@github.com> Message-ID: On Thu, 5 Nov 2020 14:36:44 GMT, Erik ?sterlund wrote: >> Ok, so there were many test failures with other approaches. Having GC trigger the posting was the most reliable way to post the events when the tests (and presumably the jvmti customers) expected the events to be posted. We could revisit during event disabling if a customer complains about GC pause times. > > The point of this change was not necessarily to be lazy about updating the tagmap, until someone uses it. The point was to get rid of the last annoying serial GC phase. Doing it all lazily would certainly also achieve that. But it would also lead to situations where no event is ever posted from GC to GC. So you would get the event 20 GCs later, which might come as a surprise. It did come as a surprise to some tests, so it is reasonable to assume it would come as a surprise to users too. And I don't think we want such surprises unless we couldn't deal with them. And we can. Kim's change to post the events from the service thread or before other JVMTI operations removes posting events from the gc_notification, which was the objection. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 16 23:19:13 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:19:13 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v6] In-Reply-To: References: Message-ID: On Mon, 9 Nov 2020 20:39:40 GMT, Coleen Phillimore wrote: >> Thanks @sspitsyn . I'm going to leave the gc_notification code because structurally the two sides of the if statement are different and it's not a long function. Thank you for reviewing the change. > > This change also passes tier 7,8 testing. does this work? I've added two commits from @kimbarrett that defer the ObjectFree posting to the service thread or to a place where it could be removed before posting. I also remerged and added the call JvmtiTagMap::set_needs_cleaning() to shenandoah which works after merging the latest code from shenandoah. Testing tiers 1-6 currently. jvmti/jdi tests pass with G1 and ZGC stress options, and JVMTI tests pass with shenandoah. Thanks! Coleen ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Mon Nov 16 23:30:25 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Mon, 16 Nov 2020 23:30:25 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Fix minimal build. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/daaa13fe..1940eaf1 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=10 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=09-10 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From iignatyev at openjdk.java.net Tue Nov 17 00:36:10 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Tue, 17 Nov 2020 00:36:10 GMT Subject: RFR: 8256430: add linux-x64-optimized to tier1 Message-ID: Hi all, [8256414](https://bugs.openjdk.java.net/browse/JDK-8256414) / #1233 added similar profile to submit workflow, this patch defines `linux-x64-optimized` profile in jib-profile so it can be used by mach5 and added to tier1? Thanks -- Igor cc-ing @dcubed-ojdk ------------- Commit messages: - 8256430: add linux-x64-optimized to tier1 Changes: https://git.openjdk.java.net/jdk/pull/1244/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1244&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256430 Stats: 12 lines in 1 file changed: 12 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1244.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1244/head:pull/1244 PR: https://git.openjdk.java.net/jdk/pull/1244 From kvn at openjdk.java.net Tue Nov 17 01:17:02 2020 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Tue, 17 Nov 2020 01:17:02 GMT Subject: RFR: 8256430: add linux-x64-optimized to tier1 In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 00:31:24 GMT, Igor Ignatyev wrote: > Hi all, > > > [8256414](https://bugs.openjdk.java.net/browse/JDK-8256414) / #1233 added similar profile to submit workflow, this patch defines `linux-x64-optimized` profile in jib-profile so it can be used by mach5 and added to tier1? > > Thanks > -- Igor > > cc-ing @dcubed-ojdk Good. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1244 From dcubed at openjdk.java.net Tue Nov 17 01:43:01 2020 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Tue, 17 Nov 2020 01:43:01 GMT Subject: RFR: 8256430: add linux-x64-optimized to tier1 In-Reply-To: References: Message-ID: <3XYPJS0vv9aGfb613MCbkJw0VvYIdwvE4T2iwCXhDCg=.602bf1fa-f90b-496a-a9be-ddafb13e153a@github.com> On Tue, 17 Nov 2020 00:31:24 GMT, Igor Ignatyev wrote: > Hi all, > > > [8256414](https://bugs.openjdk.java.net/browse/JDK-8256414) / #1233 added similar profile to submit workflow, this patch defines `linux-x64-optimized` profile in jib-profile so it can be used by mach5 and added to tier1? > > Thanks > -- Igor > > cc-ing @dcubed-ojdk Thumbs up. ------------- Marked as reviewed by dcubed (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1244 From kbarrett at openjdk.java.net Tue Nov 17 03:52:14 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 17 Nov 2020 03:52:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 23:30:25 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Fix minimal build. Marked as reviewed by kbarrett (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From rwestberg at openjdk.java.net Tue Nov 17 09:27:03 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Tue, 17 Nov 2020 09:27:03 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 13:51:29 GMT, Erik Joelsson wrote: >> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. > > Marked as reviewed by erikj (Reviewer). > Why does jtreg need OpenJDK-8 for building? Fair enough, the 1.8 requirement comes from using build-all.sh which checks this explicitly. But I'd rather leave changing how we build jtreg for a separate change.. ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From magnus.ihse.bursie at oracle.com Tue Nov 17 09:37:31 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 10:37:31 +0100 Subject: RFR: 8256414: add optimized build to submit workflow In-Reply-To: References: Message-ID: <9ffae379-72ea-f830-c6da-763522e0ed5f@oracle.com> Hi Igor, There is a long-standing bug with the intent to remove optimized builds (https://bugs.openjdk.java.net/browse/JDK-8183287). Given that it does not seem that popular, I wonder if it really is necessary to burden the submit workflow (and tier1 testing, as requested in https://bugs.openjdk.java.net/browse/JDK-8256430) with this. At the very least, I'd like to get some input from more Hotspot developers to hear if they think it is a worthy cause to spend our resources at. Otherwise, I believe a better way forward is to follow through on JDK-8183287, viz. to split up optimized builds into the two extra components it actually provides: enable diagnostic code in normal release builds (https://bugs.openjdk.java.net/browse/JDK-8183288) and enable tracing with INCLUDE_PRINT (https://bugs.openjdk.java.net/browse/JDK-8202283). /Magnus On 2020-11-16 19:33, Igor Ignatyev wrote: > Hi all, > > Could you please review this small and trivial patch which adds `linux-x64-optimized` build to submit workflow so breakages of this build flavor would be easier to spot? > > Thanks, > -- Igor > > ------------- > > Commit messages: > - add linux-x64-optimized build > > Changes: https://git.openjdk.java.net/jdk/pull/1233/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1233&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8256414 > Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod > Patch: https://git.openjdk.java.net/jdk/pull/1233.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/1233/head:pull/1233 > > PR: https://git.openjdk.java.net/jdk/pull/1233 From magnus.ihse.bursie at oracle.com Tue Nov 17 09:47:26 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 10:47:26 +0100 Subject: RFR: 8256414: add optimized build to submit workflow In-Reply-To: <9ffae379-72ea-f830-c6da-763522e0ed5f@oracle.com> References: <9ffae379-72ea-f830-c6da-763522e0ed5f@oracle.com> Message-ID: <282bed4c-56b5-3da2-1bbe-a9ff54edf91b@oracle.com> I see now that this PR was already integrated. I think that any change to the submit workflow (if they add additional testing, and not just fix bugs) is a non-trivial change which needs careful consideration. We have already had a huge influx of additional build platforms in a very short time. Each additional platform is subject to any kind of build issues, not all of which might be related to the actual patch, and we therefore need to weight the benefits of getting additional testing of build platforms to the risks that this might cause unnecessary road blocks for developers. Also, I believe it is good practice when changing build code to make sure that at least one reviewer is a member of the Build Group (https://openjdk.java.net/census#build). This is not something Skara can enforce, so it is dependent on the good will of committers (who should notify the correct set of reviewers), and of JDK Reviewers to specify if they believe additional reviewers from any particular area is needed. /Magnus On 2020-11-17 10:37, Magnus Ihse Bursie wrote: > Hi Igor, > > There is a long-standing bug with the intent to remove optimized > builds (https://bugs.openjdk.java.net/browse/JDK-8183287). Given that > it does not seem that popular, I wonder if it really is necessary to > burden the submit workflow (and tier1 testing, as requested in > https://bugs.openjdk.java.net/browse/JDK-8256430) with this. > > At the very least, I'd like to get some input from more Hotspot > developers to hear if they think it is a worthy cause to spend our > resources at. > > Otherwise, I believe a better way forward is to follow through on > JDK-8183287, viz. to split up optimized builds into the two extra > components it actually provides: enable diagnostic code in normal > release builds (https://bugs.openjdk.java.net/browse/JDK-8183288) and > enable tracing with INCLUDE_PRINT > (https://bugs.openjdk.java.net/browse/JDK-8202283). > > /Magnus > > On 2020-11-16 19:33, Igor Ignatyev wrote: >> Hi all, >> >> Could you please review this small and trivial patch which adds >> `linux-x64-optimized` build to submit workflow so breakages of this >> build flavor would be easier to spot? >> >> Thanks, >> -- Igor >> >> ------------- >> >> Commit messages: >> ? - add linux-x64-optimized build >> >> Changes: https://git.openjdk.java.net/jdk/pull/1233/files >> ? Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1233&range=00 >> ?? Issue: https://bugs.openjdk.java.net/browse/JDK-8256414 >> ?? Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod >> ?? Patch: https://git.openjdk.java.net/jdk/pull/1233.diff >> ?? Fetch: git fetch https://git.openjdk.java.net/jdk >> pull/1233/head:pull/1233 >> >> PR: https://git.openjdk.java.net/jdk/pull/1233 > From vlivanov at openjdk.java.net Tue Nov 17 10:49:04 2020 From: vlivanov at openjdk.java.net (Vladimir Ivanov) Date: Tue, 17 Nov 2020 10:49:04 GMT Subject: RFR: 8256430: add linux-x64-optimized to tier1 In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 00:31:24 GMT, Igor Ignatyev wrote: > Hi all, > > > [8256414](https://bugs.openjdk.java.net/browse/JDK-8256414) / #1233 added similar profile to submit workflow, this patch defines `linux-x64-optimized` profile in jib-profile so it can be used by mach5 and added to tier1? > > Thanks > -- Igor > > cc-ing @dcubed-ojdk Looks good. ------------- Marked as reviewed by vlivanov (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1244 From vladimir.x.ivanov at oracle.com Tue Nov 17 11:02:18 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 17 Nov 2020 14:02:18 +0300 Subject: RFR: 8256414: add optimized build to submit workflow In-Reply-To: <9ffae379-72ea-f830-c6da-763522e0ed5f@oracle.com> References: <9ffae379-72ea-f830-c6da-763522e0ed5f@oracle.com> Message-ID: <58ab29bb-9170-78cd-97d1-87505ecb36bb@oracle.com> > There is a long-standing bug with the intent to remove optimized builds > (https://bugs.openjdk.java.net/browse/JDK-8183287). Given that it does > not seem that popular, I wonder if it really is necessary to burden the > submit workflow (and tier1 testing, as requested in > https://bugs.openjdk.java.net/browse/JDK-8256430) with this. > > At the very least, I'd like to get some input from more Hotspot > developers to hear if they think it is a worthy cause to spend our > resources at. > > Otherwise, I believe a better way forward is to follow through on > JDK-8183287, viz. to split up optimized builds into the two extra > components it actually provides: enable diagnostic code in normal > release builds (https://bugs.openjdk.java.net/browse/JDK-8183288) and > enable tracing with INCLUDE_PRINT > (https://bugs.openjdk.java.net/browse/JDK-8202283). I find !PRODUCT vs ASSERT distinction confusing, but irrespective of the way the relevant code is guarded (!PRODUCT or INCLUDE_PRINT), it has to be built regularly to avoid the rot. So, once the bugs you mentioned are addressed, optimized build can be replaced with release build + tracing configuration. Regarding the most appropriate tier to put it, I don't think it has to be part of tier1. IMO later tiers are fine as well. But having it in tier1 doesn't look like a significant waste of resources. I don't think there's a notion of tiers in submit workflow, so I'm strongly in favor of having optimized configuration built there. Best regards, Vladimir Ivanov > On 2020-11-16 19:33, Igor Ignatyev wrote: >> Hi all, >> >> Could you please review this small and trivial patch which adds >> `linux-x64-optimized` build to submit workflow so breakages of this >> build flavor would be easier to spot? >> >> Thanks, >> -- Igor >> >> ------------- >> >> Commit messages: >> ? - add linux-x64-optimized build >> >> Changes: https://git.openjdk.java.net/jdk/pull/1233/files >> ? Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1233&range=00 >> ?? Issue: https://bugs.openjdk.java.net/browse/JDK-8256414 >> ?? Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod >> ?? Patch: https://git.openjdk.java.net/jdk/pull/1233.diff >> ?? Fetch: git fetch https://git.openjdk.java.net/jdk >> pull/1233/head:pull/1233 >> >> PR: https://git.openjdk.java.net/jdk/pull/1233 > From ihse at openjdk.java.net Tue Nov 17 14:10:21 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 14:10:21 GMT Subject: RFR: 8256216: Enable reproducible builds in jib-profiles [v2] In-Reply-To: References: Message-ID: > The time has come to actually turn on the support for reproducible builds by default in the jib profiles. Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Only enable reproducible builds for release buils. Also explicitly set --enable-reproducible-build and not just let --with-source-date set it implicitly. - 8256216: Enable reproducible builds in jib-profiles ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1176/files - new: https://git.openjdk.java.net/jdk/pull/1176/files/ba537433..35809f3d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1176&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1176&range=00-01 Stats: 19388 lines in 420 files changed: 12098 ins; 4830 del; 2460 mod Patch: https://git.openjdk.java.net/jdk/pull/1176.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1176/head:pull/1176 PR: https://git.openjdk.java.net/jdk/pull/1176 From ihse at openjdk.java.net Tue Nov 17 14:10:21 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 14:10:21 GMT Subject: RFR: 8256216: Enable reproducible builds in jib-profiles In-Reply-To: References: Message-ID: On Wed, 11 Nov 2020 20:13:55 GMT, Magnus Ihse Bursie wrote: > The time has come to actually turn on the support for reproducible builds by default in the jib profiles. Due to feedback from Erik in a private discussion, I have now changed the implementation so that reproducible builds are only enabled for release builds. I have also added an explicit `--enable-reproducible-build` and not just relying on the implicit behavior of `--with-source-date`. ------------- PR: https://git.openjdk.java.net/jdk/pull/1176 From ihse at openjdk.java.net Tue Nov 17 14:37:22 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 14:37:22 GMT Subject: RFR: 8256216: Enable reproducible builds in jib-profiles [v3] In-Reply-To: References: Message-ID: > The time has come to actually turn on the support for reproducible builds by default in the jib profiles. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Change formatting as requested ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1176/files - new: https://git.openjdk.java.net/jdk/pull/1176/files/35809f3d..5b51de95 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1176&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1176&range=01-02 Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1176.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1176/head:pull/1176 PR: https://git.openjdk.java.net/jdk/pull/1176 From erikj at openjdk.java.net Tue Nov 17 14:50:10 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 17 Nov 2020 14:50:10 GMT Subject: RFR: 8256216: Enable reproducible builds in jib-profiles [v3] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 14:37:22 GMT, Magnus Ihse Bursie wrote: >> The time has come to actually turn on the support for reproducible builds by default in the jib profiles. > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Change formatting as requested Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1176 From ihse at openjdk.java.net Tue Nov 17 15:33:07 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 15:33:07 GMT Subject: Integrated: 8256216: Enable reproducible builds in jib-profiles In-Reply-To: References: Message-ID: <_DwHFs_6O1lzOm7_rLif7KBitziIF4AN_njwKiPpBsg=.8f165ebe-58b2-435b-8bd7-c9e7c22d5b4c@github.com> On Wed, 11 Nov 2020 20:13:55 GMT, Magnus Ihse Bursie wrote: > The time has come to actually turn on the support for reproducible builds by default in the jib profiles. This pull request has now been integrated. Changeset: c255b18c Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/c255b18c Stats: 14 lines in 1 file changed: 14 ins; 0 del; 0 mod 8256216: Enable reproducible builds in jib-profiles Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1176 From fw at deneb.enyo.de Tue Nov 17 17:01:17 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Tue, 17 Nov 2020 18:01:17 +0100 Subject: [OpenJDK 2D-Dev] RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> (Phil Race's message of "Mon, 2 Nov 2020 04:29:10 GMT") References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> Message-ID: <87o8jv6hpe.fsf@mid.deneb.enyo.de> * Phil Race: > This upgrades JDK to import the current 2.7.2 version of harfbuzz - > an OpenType text shaping library > > https://bugs.openjdk.java.net/browse/JDK-8247872 > > This has passed building and headless and headful automated tests on > all platforms. Is it just me, or does the new Harfbuzz build *significantly* slower? It adds about 20 seconds to the build, compared to using the system Harfbuzz, whereas before it was around 5 seconds. From philip.race at oracle.com Tue Nov 17 17:56:47 2020 From: philip.race at oracle.com (Philip Race) Date: Tue, 17 Nov 2020 09:56:47 -0800 Subject: [OpenJDK 2D-Dev] RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <87o8jv6hpe.fsf@mid.deneb.enyo.de> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <87o8jv6hpe.fsf@mid.deneb.enyo.de> Message-ID: <5FB40EDF.4090300@oracle.com> There is more code in the newer version but not 4 times as much ! Harfbuzz now requires c++11 features (-std=c++11) Possibly the C++ compiler you are using (you don't mention platform) is slower in this mode. -phil. On 11/17/20, 9:01 AM, Florian Weimer wrote: > * Phil Race: > >> This upgrades JDK to import the current 2.7.2 version of harfbuzz - >> an OpenType text shaping library >> >> https://bugs.openjdk.java.net/browse/JDK-8247872 >> >> This has passed building and headless and headful automated tests on >> all platforms. > Is it just me, or does the new Harfbuzz build *significantly* slower? > > It adds about 20 seconds to the build, compared to using the system > Harfbuzz, whereas before it was around 5 seconds. From fw at deneb.enyo.de Tue Nov 17 18:00:02 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Tue, 17 Nov 2020 19:00:02 +0100 Subject: [OpenJDK 2D-Dev] RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <5FB40EDF.4090300@oracle.com> (Philip Race's message of "Tue, 17 Nov 2020 09:56:47 -0800") References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <87o8jv6hpe.fsf@mid.deneb.enyo.de> <5FB40EDF.4090300@oracle.com> Message-ID: <87y2iz50f1.fsf@mid.deneb.enyo.de> * Philip Race: > There is more code in the newer version but not 4 times as much ! > Harfbuzz now requires c++11 features (-std=c++11) > Possibly the C++ compiler you are using (you don't mention platform) is > slower in this mode. It's GCC 8 on Debian buster, which defaults to C++14. And it's possible to write slow-to-compile C++ in all language modes. Is the Harfbuzz build parallelized? From philip.race at oracle.com Tue Nov 17 18:08:08 2020 From: philip.race at oracle.com (Philip Race) Date: Tue, 17 Nov 2020 10:08:08 -0800 Subject: [OpenJDK 2D-Dev] RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <87y2iz50f1.fsf@mid.deneb.enyo.de> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <87o8jv6hpe.fsf@mid.deneb.enyo.de> <5FB40EDF.4090300@oracle.com> <87y2iz50f1.fsf@mid.deneb.enyo.de> Message-ID: <5FB41188.30704@oracle.com> So maybe the actual *usage* of C++11 features slows it down. I found this :- https://stackoverflow.com/questions/34179434/is-compiling-with-c11-way-slower-than-with-c98 I really did not even think to measure the build time and I don't think there's any JDK build parallelism in building a specific native library. The build is parallel at a higher level but not at that lower level. -phil On 11/17/20, 10:00 AM, Florian Weimer wrote: > * Philip Race: > >> There is more code in the newer version but not 4 times as much ! >> Harfbuzz now requires c++11 features (-std=c++11) >> Possibly the C++ compiler you are using (you don't mention platform) is >> slower in this mode. > It's GCC 8 on Debian buster, which defaults to C++14. And it's > possible to write slow-to-compile C++ in all language modes. > > Is the Harfbuzz build parallelized? From vdyakov at openjdk.java.net Tue Nov 17 18:22:06 2020 From: vdyakov at openjdk.java.net (Victor Dyakov) Date: Tue, 17 Nov 2020 18:22:06 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU In-Reply-To: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> References: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> Message-ID: On Tue, 10 Nov 2020 16:38:25 GMT, Kevin Rushforth wrote: >> Change looks ok from a build point of view, but I can't comment on the validity and implications of using this key. > > I ran a 3D lighting test that is designed to be a GPU stress test. It's a worst case, to be sure, but it take 10 times as long to render with the integrated GPU as it does with the discrete GPU: > > **attenuation.LightingSample: 500 large quads** > discrete GPU: 23.5 fps > integrated GPU: 2.34 fps > > In a more realistic example of drawing a large number of 2D vectors, it runs 35% slower with the integrated GPU: > > **Vector charting test, oval clip** > discrete GPU: 41.1 fps > integrated GPU: 26.6 fps > > I see similar results in the performance numbers you listed above. > > An application developer who packages up their JDK, for example, using jpackage, can make the decision for themselves. Application developers who rely on the JDK as delivered will get whatever we choose as the default. So we need to be sure that the benefit of doing this justifies the performance hit. @kevinrushforth @prrace could you please review? ------------- PR: https://git.openjdk.java.net/jdk/pull/1139 From prr at openjdk.java.net Tue Nov 17 18:44:08 2020 From: prr at openjdk.java.net (Phil Race) Date: Tue, 17 Nov 2020 18:44:08 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU In-Reply-To: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> References: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> Message-ID: On Tue, 10 Nov 2020 16:38:25 GMT, Kevin Rushforth wrote: >> Change looks ok from a build point of view, but I can't comment on the validity and implications of using this key. > > I ran a 3D lighting test that is designed to be a GPU stress test. It's a worst case, to be sure, but it take 10 times as long to render with the integrated GPU as it does with the discrete GPU: > > **attenuation.LightingSample: 500 large quads** > discrete GPU: 23.5 fps > integrated GPU: 2.34 fps > > In a more realistic example of drawing a large number of 2D vectors, it runs 35% slower with the integrated GPU: > > **Vector charting test, oval clip** > discrete GPU: 41.1 fps > integrated GPU: 26.6 fps > > I see similar results in the performance numbers you listed above. > > An application developer who packages up their JDK, for example, using jpackage, can make the decision for themselves. Application developers who rely on the JDK as delivered will get whatever we choose as the default. So we need to be sure that the benefit of doing this justifies the performance hit. > @kevinrushforth @prrace could you please review? As we discussed yesterday, it is reviewed but not ready to be approved. We are waiting for a reply from Apple. ------------- PR: https://git.openjdk.java.net/jdk/pull/1139 From serb at openjdk.java.net Tue Nov 17 18:59:04 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Tue, 17 Nov 2020 18:59:04 GMT Subject: RFR: 8251854: [macosx] Java forces the use of discrete GPU In-Reply-To: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> References: <0fKDXwkuSKcZPDRMyxmYG91a_weT0KEh7lobfkCPDY0=.e2b8fce5-4b73-47f1-b676-f8b9184a6a85@github.com> Message-ID: On Tue, 10 Nov 2020 16:38:25 GMT, Kevin Rushforth wrote: >> Change looks ok from a build point of view, but I can't comment on the validity and implications of using this key. > > I ran a 3D lighting test that is designed to be a GPU stress test. It's a worst case, to be sure, but it take 10 times as long to render with the integrated GPU as it does with the discrete GPU: > > **attenuation.LightingSample: 500 large quads** > discrete GPU: 23.5 fps > integrated GPU: 2.34 fps > > In a more realistic example of drawing a large number of 2D vectors, it runs 35% slower with the integrated GPU: > > **Vector charting test, oval clip** > discrete GPU: 41.1 fps > integrated GPU: 26.6 fps > > I see similar results in the performance numbers you listed above. > > An application developer who packages up their JDK, for example, using jpackage, can make the decision for themselves. Application developers who rely on the JDK as delivered will get whatever we choose as the default. So we need to be sure that the benefit of doing this justifies the performance hit. > > @kevinrushforth @prrace could you please review? > > As we discussed yesterday, it is reviewed but not ready to be approved. > We are waiting for a reply from Apple. @prrace We are waiting for it for three months already. If it is possible then not sure why other applications did not found any possible ways to force one specific GPU. What we are wating for? ------------- PR: https://git.openjdk.java.net/jdk/pull/1139 From iignatyev at openjdk.java.net Tue Nov 17 19:31:16 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Tue, 17 Nov 2020 19:31:16 GMT Subject: RFR: 8256430: add linux-x64-optimized to regular testing [v2] In-Reply-To: References: Message-ID: > Hi all, > > > [8256414](https://bugs.openjdk.java.net/browse/JDK-8256414) / #1233 added similar profile to submit workflow, this patch defines `linux-x64-optimized` profile in jib-profile so it can be used by mach5 and added to tier1? > > Thanks > -- Igor > > cc-ing @dcubed-ojdk Igor Ignatyev has updated the pull request incrementally with one additional commit since the last revision: build only hotspot for optimized ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1244/files - new: https://git.openjdk.java.net/jdk/pull/1244/files/05c735be..2d861d14 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1244&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1244&range=00-01 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1244.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1244/head:pull/1244 PR: https://git.openjdk.java.net/jdk/pull/1244 From erikj at openjdk.java.net Tue Nov 17 19:55:12 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 17 Nov 2020 19:55:12 GMT Subject: RFR: 8256430: add linux-x64-optimized to regular testing [v2] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:31:16 GMT, Igor Ignatyev wrote: >> Hi all, >> >> >> [8256414](https://bugs.openjdk.java.net/browse/JDK-8256414) / #1233 added similar profile to submit workflow, this patch defines `linux-x64-optimized` profile in jib-profile so it can be used by mach5 and added to tier1? >> >> Thanks >> -- Igor >> >> cc-ing @dcubed-ojdk > > Igor Ignatyev has updated the pull request incrementally with one additional commit since the last revision: > > build only hotspot for optimized Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1244 From iignatyev at openjdk.java.net Tue Nov 17 20:04:08 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Tue, 17 Nov 2020 20:04:08 GMT Subject: RFR: 8256430: add linux-x64-optimized to regular testing [v2] In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:52:49 GMT, Erik Joelsson wrote: >> Igor Ignatyev has updated the pull request incrementally with one additional commit since the last revision: >> >> build only hotspot for optimized > > Marked as reviewed by erikj (Reviewer). folks, thanks for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/1244 From iignatyev at openjdk.java.net Tue Nov 17 20:04:09 2020 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Tue, 17 Nov 2020 20:04:09 GMT Subject: Integrated: 8256430: add linux-x64-optimized to regular testing In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 00:31:24 GMT, Igor Ignatyev wrote: > Hi all, > > > [8256414](https://bugs.openjdk.java.net/browse/JDK-8256414) / #1233 added similar profile to submit workflow, this patch defines `linux-x64-optimized` profile in jib-profile so it can be used by mach5 and added to tier1? > > Thanks > -- Igor > > cc-ing @dcubed-ojdk This pull request has now been integrated. Changeset: d9dbd5de Author: Igor Ignatyev URL: https://git.openjdk.java.net/jdk/commit/d9dbd5de Stats: 13 lines in 1 file changed: 13 ins; 0 del; 0 mod 8256430: add linux-x64-optimized to regular testing Reviewed-by: kvn, dcubed, vlivanov, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1244 From jlaskey at openjdk.java.net Tue Nov 17 20:04:11 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 17 Nov 2020 20:04:11 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators Message-ID: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . ------------- Commit messages: - 8248862: Implement Enhanced Pseudo-Random Number Generators - Merge branch 'master' into 8248862 - Merge branch 'master' into 8248862 - 8248862: Implement Enhanced Pseudo-Random Number Generators - Merge branch 'master' into 8248862 - 8248862: Implement Enhanced Pseudo-Random Number Generators - 8248862; Implement Enhanced Pseudo-Random Number Generators - Merge branch 'master' into 8248862 - 8248862: Implement Enhanced Pseudo-Random Number Generators - Merge branch 'master' into 8248862 - ... and 27 more: https://git.openjdk.java.net/jdk/compare/9efbb463...7469ca58 Changes: https://git.openjdk.java.net/jdk/pull/1273/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1273&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8248862 Stats: 14875 lines in 31 files changed: 11094 ins; 3704 del; 77 mod Patch: https://git.openjdk.java.net/jdk/pull/1273.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1273/head:pull/1273 PR: https://git.openjdk.java.net/jdk/pull/1273 From erikj at openjdk.java.net Tue Nov 17 20:18:08 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 17 Nov 2020 20:18:08 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> Message-ID: On Tue, 17 Nov 2020 19:58:47 GMT, Jim Laskey wrote: > This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . It looks like you have deleted the doc directory. That can't be right? ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Tue Nov 17 20:43:03 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 17 Nov 2020 20:43:03 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> Message-ID: On Tue, 17 Nov 2020 20:15:34 GMT, Erik Joelsson wrote: >> This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . > > It looks like you have deleted the doc directory. That can't be right? @erikj79 That's possible I had an intermediate doc folder before the master doc was added. I though I was careful to resolve conflict by selecting master but it might be that I picked the wrong one. Will try and correct in next round,. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Tue Nov 17 21:03:18 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 17 Nov 2020 21:03:18 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> Message-ID: <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> On Tue, 17 Nov 2020 20:15:34 GMT, Erik Joelsson wrote: >> This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . > > It looks like you have deleted the doc directory. That can't be right? @erikj79 my local branch seems to have the right sources for doc @erikj79 https://github.com/JimLaskey/jdk/tree/8248862 ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Tue Nov 17 21:03:18 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 17 Nov 2020 21:03:18 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v2] In-Reply-To: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> Message-ID: <_AerudoiREbI3Qj9VRi5zoiogsiCWPD17HhkIvTTCaw=.be7f1fa8-3c1c-49ca-90ba-96b010033087@github.com> > This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . Jim Laskey has updated the pull request incrementally with one additional commit since the last revision: 8248862: Implement Enhanced Pseudo-Random Number Generators Updated RandomGeneratorFactory javadoc. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1273/files - new: https://git.openjdk.java.net/jdk/pull/1273/files/7469ca58..b24024bb Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1273&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1273&range=00-01 Stats: 39 lines in 6 files changed: 16 ins; 0 del; 23 mod Patch: https://git.openjdk.java.net/jdk/pull/1273.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1273/head:pull/1273 PR: https://git.openjdk.java.net/jdk/pull/1273 From erikj at openjdk.java.net Tue Nov 17 21:13:08 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Tue, 17 Nov 2020 21:13:08 GMT Subject: RFR: JDK-8256501: libTestMainKeyWindow fails to build with Xcode 12.2 Message-ID: <2Opj59OMzdr_XjiC2SftWkXLKbjOnfBPst26kNvH8Gk=.b1214b22-0137-4495-a7ed-7abc6c4c36e1@github.com> In Xcode 12.2, the framework JavaVM was removed in the sysroot and all (relevant) frameworks inside were just moved one step up so we still find things like JavaNativeFoundation and JavaRuntimeSupport. There is however one test that currently links explicitly to JavaVM, which fails the build. The fix seems to be to simply remove this from the linker flags. ------------- Commit messages: - Stop linking to framework JavaVM Changes: https://git.openjdk.java.net/jdk/pull/1275/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1275&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256501 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1275.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1275/head:pull/1275 PR: https://git.openjdk.java.net/jdk/pull/1275 From ihse at openjdk.java.net Tue Nov 17 21:13:08 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 21:13:08 GMT Subject: RFR: JDK-8256501: libTestMainKeyWindow fails to build with Xcode 12.2 In-Reply-To: <2Opj59OMzdr_XjiC2SftWkXLKbjOnfBPst26kNvH8Gk=.b1214b22-0137-4495-a7ed-7abc6c4c36e1@github.com> References: <2Opj59OMzdr_XjiC2SftWkXLKbjOnfBPst26kNvH8Gk=.b1214b22-0137-4495-a7ed-7abc6c4c36e1@github.com> Message-ID: On Tue, 17 Nov 2020 21:04:08 GMT, Erik Joelsson wrote: > In Xcode 12.2, the framework JavaVM was removed in the sysroot and all (relevant) frameworks inside were just moved one step up so we still find things like JavaNativeFoundation and JavaRuntimeSupport. There is however one test that currently links explicitly to JavaVM, which fails the build. The fix seems to be to simply remove this from the linker flags. Looks good. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1275 From kcr at openjdk.java.net Tue Nov 17 21:14:06 2020 From: kcr at openjdk.java.net (Kevin Rushforth) Date: Tue, 17 Nov 2020 21:14:06 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> Message-ID: On Tue, 17 Nov 2020 20:56:52 GMT, Jim Laskey wrote: > my local branch seems to have the right sources for doc Maybe, but your branch on GitHub does not. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From serb at openjdk.java.net Tue Nov 17 21:30:06 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Tue, 17 Nov 2020 21:30:06 GMT Subject: RFR: JDK-8256501: libTestMainKeyWindow fails to build with Xcode 12.2 In-Reply-To: <2Opj59OMzdr_XjiC2SftWkXLKbjOnfBPst26kNvH8Gk=.b1214b22-0137-4495-a7ed-7abc6c4c36e1@github.com> References: <2Opj59OMzdr_XjiC2SftWkXLKbjOnfBPst26kNvH8Gk=.b1214b22-0137-4495-a7ed-7abc6c4c36e1@github.com> Message-ID: On Tue, 17 Nov 2020 21:04:08 GMT, Erik Joelsson wrote: > In Xcode 12.2, the framework JavaVM was removed in the sysroot and all (relevant) frameworks inside were just moved one step up so we still find things like JavaNativeFoundation and JavaRuntimeSupport. There is however one test that currently links explicitly to JavaVM, which fails the build. The fix seems to be to simply remove this from the linker flags. Marked as reviewed by serb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1275 From ihse at openjdk.java.net Tue Nov 17 21:38:04 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 21:38:04 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> Message-ID: On Tue, 17 Nov 2020 21:10:48 GMT, Kevin Rushforth wrote: >> @erikj79 my local branch seems to have the right sources for doc > >> my local branch seems to have the right sources for doc > > Maybe, but your branch on GitHub does not. This PR looks seriously messed up: JimLaskey added 30 commits on Oct 9 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators 515a5a4 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 7c25f2d @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 642022d @JimLaskey Merge branch 'master' into 8248862 b537e0f @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? ca90d01 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? ec2941a @JimLaskey Merge branch 'master' into 8248862 dbe142c @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? cca6f0f @JimLaskey Merge branch 'master' into 8248862 93ee16f @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 84d4704 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? b50f2ee @JimLaskey Merge branch 'master' into 8248862 b1d6abe @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 8afe7a2 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 2f99696 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 7c58819 @JimLaskey Merge branch 'master' into 8248862 655ad08 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? d5860b9 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 3ac78f8 @JimLaskey Merge branch 'master' into 8248862 e3b6e64 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 8ec7f87 @JimLaskey Merge branch 'master' into 8248862 ed959f8 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 72195e8 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 84730c3 @JimLaskey Merge branch 'master' into 8248862 05f0477 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 7dd81ac @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 38f534b @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? f428b61 @JimLaskey Merge branch 'master' into 8248862 5e33872 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 0bc8e3d @JimLaskey Merge branch 'master' into 8248862 679f1b5 JimLaskey added 7 commits 4 days ago @JimLaskey 8248862; Implement Enhanced Pseudo-Random Number Generators ? 17972a9 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 4f0338f @JimLaskey Merge branch 'master' into 8248862 7c6b9fa @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? c67c729 @JimLaskey Merge branch 'master' into 8248862 108ea50 @JimLaskey Merge branch 'master' into 8248862 1b31205 @JimLaskey 8248862: Implement Enhanced Pseudo-Random Number Generators ? 7469ca5 It might be better if you close this, rebase your actual change on top of a current master, and open a new PR instead. The only reason this was flagged with the `build` label was due to the incorrect deletion of doc, so if you intend to go on with this PR, please remove the label after you've fixed the doc issue. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From ihse at openjdk.java.net Tue Nov 17 21:38:04 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 21:38:04 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> Message-ID: On Tue, 17 Nov 2020 21:33:13 GMT, Magnus Ihse Bursie wrote: >>> my local branch seems to have the right sources for doc >> >> Maybe, but your branch on GitHub does not. > > This PR looks seriously messed up: > > JimLaskey added 30 commits on Oct 9 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators > 515a5a4 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 7c25f2d > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 642022d > @JimLaskey > Merge branch 'master' into 8248862 > b537e0f > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > ca90d01 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > ec2941a > @JimLaskey > Merge branch 'master' into 8248862 > dbe142c > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > cca6f0f > @JimLaskey > Merge branch 'master' into 8248862 > 93ee16f > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 84d4704 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > b50f2ee > @JimLaskey > Merge branch 'master' into 8248862 > b1d6abe > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 8afe7a2 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 2f99696 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 7c58819 > @JimLaskey > Merge branch 'master' into 8248862 > 655ad08 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > d5860b9 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 3ac78f8 > @JimLaskey > Merge branch 'master' into 8248862 > e3b6e64 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 8ec7f87 > @JimLaskey > Merge branch 'master' into 8248862 > ed959f8 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 72195e8 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 84730c3 > @JimLaskey > Merge branch 'master' into 8248862 > 05f0477 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 7dd81ac > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 38f534b > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > f428b61 > @JimLaskey > Merge branch 'master' into 8248862 > 5e33872 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 0bc8e3d > @JimLaskey > Merge branch 'master' into 8248862 > 679f1b5 > JimLaskey added 7 commits 4 days ago > @JimLaskey > 8248862; Implement Enhanced Pseudo-Random Number Generators ? > 17972a9 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 4f0338f > @JimLaskey > Merge branch 'master' into 8248862 > 7c6b9fa > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > c67c729 > @JimLaskey > Merge branch 'master' into 8248862 > 108ea50 > @JimLaskey > Merge branch 'master' into 8248862 > 1b31205 > @JimLaskey > 8248862: Implement Enhanced Pseudo-Random Number Generators ? > 7469ca5 > > It might be better if you close this, rebase your actual change on top of a current master, and open a new PR instead. > > The only reason this was flagged with the `build` label was due to the incorrect deletion of doc, so if you intend to go on with this PR, please remove the label after you've fixed the doc issue. Ah, sorry, I see now that I expanded the commit comments that it was not just the same commit applied over and over again, as I assumed, but iterative steps from your branch while developing, that just had the same first line... sorry for the confusion about that. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From magnus.ihse.bursie at oracle.com Tue Nov 17 21:40:33 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 17 Nov 2020 22:40:33 +0100 Subject: [OpenJDK 2D-Dev] RFR: 8247872: Upgrade HarfBuzz to the latest 2.7.2 In-Reply-To: <5FB41188.30704@oracle.com> References: <17xsLO4swc44fnvWNGvTiAIoAeOlQ6dHzTY_ZPQQXQ4=.0a151ce3-1533-469e-a5fd-6631ccf101cc@github.com> <87o8jv6hpe.fsf@mid.deneb.enyo.de> <5FB40EDF.4090300@oracle.com> <87y2iz50f1.fsf@mid.deneb.enyo.de> <5FB41188.30704@oracle.com> Message-ID: <608e514f-8299-46bd-6821-be17ff7aedf5@oracle.com> On 2020-11-17 19:08, Philip Race wrote: > So maybe the actual *usage* of C++11 features slows it down. > > I found this :- > https://stackoverflow.com/questions/34179434/is-compiling-with-c11-way-slower-than-with-c98 > > > I really did not even think to measure the build time and I don't > think there's any JDK build parallelism in building a specific native > library. > The build is parallel at a higher level but not at that lower level. There is build parallelism on multiple levels. When building an individual library, make is free to parallelize the compile task; only the link task is done singlethreaded. When determining which libraries to build, make is free to chose those available depending on tasks already finished. For instance, to build java.desktop-libs, we need to have finished java.desktop-java (since javac generates the header files needed by JNI integration). But after that, it can be compiled whenever make think it's suitable. This freedom for make to reorganize the order also means that it is hard to determine the additional cost for building a specific part. You could try to build java.desktop-java, and then run "make LOG=info java.desktop-libs" JOBS=1 to get a somewhat proper assessment of the time spent building all native libs for java.desktop. If this changes with the HarfBuzz upgrade, the HB upgrade is likely to blame. /Magnus > > -phil > > On 11/17/20, 10:00 AM, Florian Weimer wrote: >> * Philip Race: >> >>> There is more code in the newer version but not 4 times as much ! >>> Harfbuzz now requires c++11 features (-std=c++11) >>> Possibly the C++ compiler you are using (you don't mention platform) is >>> slower in this mode. >> It's GCC 8 on Debian buster, which defaults to C++14.? And it's >> possible to write slow-to-compile C++ in all language modes. >> >> Is the Harfbuzz build parallelized? From jlaskey at openjdk.java.net Tue Nov 17 22:02:11 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 17 Nov 2020 22:02:11 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> Message-ID: On Tue, 17 Nov 2020 21:10:48 GMT, Kevin Rushforth wrote: >> @erikj79 my local branch seems to have the right sources for doc > >> my local branch seems to have the right sources for doc > > Maybe, but your branch on GitHub does not. @kevinrushforth What is the recommended approach to remove the doc edit/commit? ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Tue Nov 17 22:15:03 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 17 Nov 2020 22:15:03 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> Message-ID: On Tue, 17 Nov 2020 21:59:04 GMT, Jim Laskey wrote: >>> my local branch seems to have the right sources for doc >> >> Maybe, but your branch on GitHub does not. > > @kevinrushforth What is the recommended approach to remove the doc edit/commit? javadoc can be found at http://cr.openjdk.java.net/~jlaskey/prng/doc/api/java.base/java/util/random/package-summary.html ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Tue Nov 17 22:21:18 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 17 Nov 2020 22:21:18 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> Message-ID: <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> > This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits: - Merge branch 'master' into 8248862 - 8248862: Implement Enhanced Pseudo-Random Number Generators Update package-info.java - 8248862: Implement Enhanced Pseudo-Random Number Generators Updated RandomGeneratorFactory javadoc. - 8248862: Implement Enhanced Pseudo-Random Number Generators Updated documentation for RandomGeneratorFactory. - Merge branch 'master' into 8248862 - Merge branch 'master' into 8248862 - 8248862: Implement Enhanced Pseudo-Random Number Generators Move RandomGeneratorProperty - Merge branch 'master' into 8248862 - 8248862: Implement Enhanced Pseudo-Random Number Generators Clear up javadoc - 8248862; Implement Enhanced Pseudo-Random Number Generators remove RandomGeneratorProperty from API - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 ------------- Changes: https://git.openjdk.java.net/jdk/pull/1273/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1273&range=02 Stats: 14891 lines in 31 files changed: 11110 ins; 3704 del; 77 mod Patch: https://git.openjdk.java.net/jdk/pull/1273.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1273/head:pull/1273 PR: https://git.openjdk.java.net/jdk/pull/1273 From kcr at openjdk.java.net Tue Nov 17 22:54:06 2020 From: kcr at openjdk.java.net (Kevin Rushforth) Date: Tue, 17 Nov 2020 22:54:06 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <3vhrOZBGrwfQVyPxA_qi7l_gimy0RqkPIRDHR7rm3SE=.79a40273-6ed4-474f-9b1e-30bd6845d303@github.com> Message-ID: On Tue, 17 Nov 2020 22:12:19 GMT, Jim Laskey wrote: >> @kevinrushforth What is the recommended approach to remove the doc edit/commit? > > javadoc can be found at http://cr.openjdk.java.net/~jlaskey/prng/doc/api/java.base/java/util/random/package-summary.html Presuming your master branch is in sync with the upstream jdk repo, something like the following: git checkout master -- doc git add doc git commit -m "restore doc files" ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jjg at openjdk.java.net Tue Nov 17 23:03:08 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Tue, 17 Nov 2020 23:03:08 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v6] In-Reply-To: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> References: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> Message-ID: <2dyPy7yPtxW7Xo0R1yFx958RE2viLNNd5eWMS-SICMk=.cb7adb60-9e0c-43d0-a945-88d3ede6953c@github.com> On Mon, 16 Nov 2020 12:44:08 GMT, Magnus Ihse Bursie wrote: >> Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. >> >> Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. >> >> So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. >> >> Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. >> >> I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace issues There seems to be much more here than there needs to be. If the issue is just long command lines, then the obvious/conventional solution would be to use @files, which would be a tiny change to the sjavac source code, to insert a single call to `CommandLine.parse` to expand any @file arguments on the command line. So, before reading all the various details in this proposed change, is there any reason why the simple @file solution cannot be used? ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From forax at univ-mlv.fr Tue Nov 17 23:18:16 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 18 Nov 2020 00:18:16 +0100 (CET) Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> Message-ID: <676865722.1878668.1605655096153.JavaMail.zimbra@u-pem.fr> An honest question, why do we need so many interfaces for the different categories of RandomGenerator ? My fear is that we are encoding the state of our knowledge of the different kinds of random generators now so it will not be pretty in the future when new categories of random generator are discovered/invented. If we can take example of the past to predict the future, 20 years ago, what should have been the hierarchy at that time. Is it not reasonable to think that we will need new kinds of random generator in the future ? I wonder if it's not better to have one interface and several optional methods like we have with the collections, it means that we are loosing the possibilities to precisely type a method that only works with a precise type of generator but it will be more future proof. R?mi ----- Mail original ----- > De: "Jim Laskey" > ?: "build-dev" , "core-libs-dev" , > security-dev at openjdk.java.net > Envoy?: Mardi 17 Novembre 2020 23:21:18 > Objet: Re: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] >> This PR is to introduce a new random number API for the JDK. The primary API is >> found in RandomGenerator and RandomGeneratorFactory. Further description can be >> found in the JEP https://openjdk.java.net/jeps/356 . > > Jim Laskey has updated the pull request with a new target base due to a merge or > a rebase. The pull request now contains 40 commits: > > - Merge branch 'master' into 8248862 > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Update package-info.java > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Updated RandomGeneratorFactory javadoc. > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Updated documentation for RandomGeneratorFactory. > - Merge branch 'master' into 8248862 > - Merge branch 'master' into 8248862 > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Move RandomGeneratorProperty > - Merge branch 'master' into 8248862 > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Clear up javadoc > - 8248862; Implement Enhanced Pseudo-Random Number Generators > > remove RandomGeneratorProperty from API > - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 > > ------------- > > Changes: https://git.openjdk.java.net/jdk/pull/1273/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1273&range=02 > Stats: 14891 lines in 31 files changed: 11110 ins; 3704 del; 77 mod > Patch: https://git.openjdk.java.net/jdk/pull/1273.diff > Fetch: git fetch https://git.openjdk.java.net/jdk pull/1273/head:pull/1273 > > PR: https://git.openjdk.java.net/jdk/pull/1273 From naoto at openjdk.java.net Tue Nov 17 23:28:21 2020 From: naoto at openjdk.java.net (Naoto Sato) Date: Tue, 17 Nov 2020 23:28:21 GMT Subject: RFR: 8251317: Support for CLDR version 38 Message-ID: Hi, Please review the changes for upgrading the CLDR data to version 38. The vast majority of the changes are simply the changes in CLDR upstream, and others are mainly test changes due to the locale data change. ------------- Commit messages: - Updated the version in `cldr.md` files - Merge branch 'master' into cldr - 8251317: Support for CLDR version 38 Changes: https://git.openjdk.java.net/jdk/pull/1279/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1279&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8251317 Stats: 57338 lines in 209 files changed: 41958 ins; 4932 del; 10448 mod Patch: https://git.openjdk.java.net/jdk/pull/1279.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1279/head:pull/1279 PR: https://git.openjdk.java.net/jdk/pull/1279 From ihse at openjdk.java.net Wed Nov 18 00:07:11 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 00:07:11 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v6] In-Reply-To: <2dyPy7yPtxW7Xo0R1yFx958RE2viLNNd5eWMS-SICMk=.cb7adb60-9e0c-43d0-a945-88d3ede6953c@github.com> References: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> <2dyPy7yPtxW7Xo0R1yFx958RE2viLNNd5eWMS-SICMk=.cb7adb60-9e0c-43d0-a945-88d3ede6953c@github.com> Message-ID: On Tue, 17 Nov 2020 23:00:26 GMT, Jonathan Gibbons wrote: >> Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix whitespace issues > > There seems to be much more here than there needs to be. > > If the issue is just long command lines, then the obvious/conventional solution would be to use @files, which would be a tiny change to the sjavac source code, to insert a single call to `CommandLine.parse` to expand any @file arguments on the command line. > > So, before reading all the various details in this proposed change, is there any reason why the simple @file solution cannot be used? The main issue is not the long command lines. Getting shorter command lines is nice, but more a side-effect. The real driver here is the awkward encoding of the command line, with spaces replaced by %20. This makes it impossible to parse the --server option to Java and properly replace unix paths with Windows paths, in the way that is needed by the upcoming changes to how we build on Windows. ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From psandoz at openjdk.java.net Wed Nov 18 00:54:18 2020 From: psandoz at openjdk.java.net (Paul Sandoz) Date: Wed, 18 Nov 2020 00:54:18 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> Message-ID: <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> On Tue, 17 Nov 2020 22:21:18 GMT, Jim Laskey wrote: >> This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . > > Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits: > > - Merge branch 'master' into 8248862 > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Update package-info.java > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Updated RandomGeneratorFactory javadoc. > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Updated documentation for RandomGeneratorFactory. > - Merge branch 'master' into 8248862 > - Merge branch 'master' into 8248862 > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Move RandomGeneratorProperty > - Merge branch 'master' into 8248862 > - 8248862: Implement Enhanced Pseudo-Random Number Generators > > Clear up javadoc > - 8248862; Implement Enhanced Pseudo-Random Number Generators > > remove RandomGeneratorProperty from API > - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 I am unsure if the intent is also to support external libraries providing `RandomGenerator` implementations. Currently there is an implicit contract for properties (reflectively invoking a method returning a map with a set of entries with known keys). Since the service provider implementation is the `RandomGenerator` itself, rather than `RandomGeneratorFactory` it is harder expose the meta-data with a clearer contract. src/java.base/share/classes/java/util/Random.java line 592: > 590: > 591: @Override > 592: public Spliterator.OfInt makeIntsSpliterator(long index, long fence, int origin, int bound) { Unsure if this and the other two methods are intended to be public or not, since they are at the end of the class and override methods of a module private class. In principle there is nothing wrong with such `Spliterator` factories, but wonder if they are really needed given the `Stream` returning methods. The arrangement of classes makes it awkward to hide these methods. src/java.base/share/classes/java/util/SplittableRandom.java line 171: > 169: * RandomGenerator properties. > 170: */ > 171: static Map getProperties() { With records exiting preview in 16 this map of properties could i think be represented as a record instance, with better type safety, where `RandomSupport.RandomGeneratorProperty` enum values become typed fields declared on the record class. Something to consider after integration perhaps? src/java.base/share/classes/java/util/SplittableRandom.java line 211: > 209: * http://zimbry.blogspot.com/2011/09/better-bit-mixing-improving-on.html > 210: */ > 211: private static long mix64(long z) { Usages be replaced with calls to `RandomSupport.mixStafford13`? src/java.base/share/classes/module-info.java line 250: > 248: exports jdk.internal.util.xml.impl to > 249: jdk.jfr; > 250: exports jdk.internal.util.random; Unqualified export, should this be `to jdk.random`? src/jdk.random/share/classes/module-info.java line 50: > 48: */ > 49: module jdk.random { > 50: uses java.util.random.RandomGenerator; Are these `uses` declarations needed? `ServiceLoader` is not used by this module, and `RandomSupport` is not a service interface. src/jdk.random/share/classes/module-info.java line 53: > 51: uses RandomSupport; > 52: > 53: exports jdk.random to Why is this needed? src/java.base/share/classes/java/util/random/package-info.java line 50: > 48: * given its name. > 49: * > 50: *

The principal supporting class is {@link RandomGenertatorFactor}. This s/RandomGenertatorFactor/RandomGenertatorFactory src/java.base/share/classes/java/util/random/package-info.java line 140: > 138: * > 139: *

For applications with no special requirements, > 140: * "L64X128MixRandom" has a good balance among speed, space, The documentation assumes that the `jdk.random` module is present in the JDK image. Perhaps we need to spit the specifics to `jdk.random`? src/java.base/share/classes/jdk/internal/util/random/RandomSupport.java line 1211: > 1209: Udiff = -Udiff; > 1210: U2 = U1; > 1211: U1 -= Udiff; Updated `U1` never used (recommend running the code through a checker e.g. use IntelliJ) src/java.base/share/classes/jdk/internal/util/random/RandomSupport.java line 331: > 329: } > 330: // Finally, we need to make sure the last z words are not all zero. > 331: search: { Nice! a rarely used feature :-) src/java.base/share/classes/jdk/internal/util/random/RandomSupport.java line 1157: > 1155: /* > 1156: * The tables themselves, as well as a number of associated parameters, are > 1157: * defined in class java.util.DoubleZigguratTables, which is automatically `DoubleZigguratTables` is an inner class of `RandomSupport` src/java.base/share/classes/jdk/internal/util/random/RandomSupport.java line 2895: > 2893: * distribution: 0.0330 > 2894: */ > 2895: static class DoubleZigguratTables { make `final` src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 167: > 165: * Return the properties map for the specified provider. > 166: * > 167: * @param provider provider to locate. Method has no such parameter src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 173: > 171: @SuppressWarnings("unchecked") > 172: private Map getProperties() { > 173: if (properties == null) { `properties` needs to be marked volatile, and it needs to be assigned at line 182 or line 184. src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 148: > 146: */ > 147: private static Map> getFactoryMap() { > 148: if (factoryMap == null) { `factoryMap` needs to be marked volatile when using the double checked locking idiom. src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 320: > 318: } > 319: } > 320: } Add an `assert` statement that `ctor`, `ctorLong` and `ctorBytes` are all non-null? src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 331: > 329: */ > 330: private void ensureConstructors() { > 331: if (ctor == null) { This check occurs outside of the synchronized block, field may need to be marked volatile. Unsure about the other dependent fields. Might need to store values from loop in `getConstructors` in locals and then assign in appropriate order, assigning the volatile field last. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From naoto at openjdk.java.net Wed Nov 18 02:07:01 2020 From: naoto at openjdk.java.net (Naoto Sato) Date: Wed, 18 Nov 2020 02:07:01 GMT Subject: RFR: 8251317: Support for CLDR version 38 In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 23:19:23 GMT, Naoto Sato wrote: > Hi, > > Please review the changes for upgrading the CLDR data to version 38. The vast majority of the changes are simply the changes in CLDR upstream, and others are mainly test changes due to the locale data change. Looks like the generated webrev is empty, possibly due to this issue (https://bugs.openjdk.java.net/browse/SKARA-732). Here is the one manually generated: https://cr.openjdk.java.net/~naoto/cldr/webrev.00/ ------------- PR: https://git.openjdk.java.net/jdk/pull/1279 From shade at openjdk.java.net Wed Nov 18 08:32:10 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 18 Nov 2020 08:32:10 GMT Subject: RFR: 8256499: Zero: enable Epsilon GC Message-ID: Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. Epsilon GC does not require anything else. Other GCs require a bit of fiddling and more testing (see e.g. [JDK-8256497](https://bugs.openjdk.java.net/browse/JDK-8256497)). Additional testing: - [x] Linux x86_64 Zero {fastdebug,release}, `gc/epsilon` tests ------------- Commit messages: - 8256499: Zero: enable Epsilon GC Changes: https://git.openjdk.java.net/jdk/pull/1269/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1269&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256499 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1269.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1269/head:pull/1269 PR: https://git.openjdk.java.net/jdk/pull/1269 From rkennke at openjdk.java.net Wed Nov 18 09:17:04 2020 From: rkennke at openjdk.java.net (Roman Kennke) Date: Wed, 18 Nov 2020 09:17:04 GMT Subject: RFR: 8256499: Zero: enable Epsilon GC In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:11:45 GMT, Aleksey Shipilev wrote: > Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. Epsilon GC does not require anything else. Other GCs require a bit of fiddling and more testing (see e.g. [JDK-8256497](https://bugs.openjdk.java.net/browse/JDK-8256497)). > > Additional testing: > - [x] Linux x86_64 Zero {fastdebug,release}, `gc/epsilon` tests Great! Looks good! ------------- Marked as reviewed by rkennke (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1269 From ihse at openjdk.java.net Wed Nov 18 09:53:09 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 09:53:09 GMT Subject: RFR: 8256538: Fix annoying awk warning in configure for java versions Message-ID: The change from grep to awk in JDK-8244248 and further bug fixed in JDK-8244756 still has invalid syntax. This causes some awk (most notably gawk, the most commonly used) to complain: gawk: cmd. line:1: warning: regexp escape sequence `"' is not a known regexp operator This is annoying and should be fixed. ------------- Commit messages: - 8256538: Fix annoying awk warning in configure for java versions Changes: https://git.openjdk.java.net/jdk/pull/1285/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1285&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256538 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1285.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1285/head:pull/1285 PR: https://git.openjdk.java.net/jdk/pull/1285 From ihse at openjdk.java.net Wed Nov 18 09:55:05 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 09:55:05 GMT Subject: RFR: 8256499: Zero: enable Epsilon GC In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:11:45 GMT, Aleksey Shipilev wrote: > Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. Epsilon GC does not require anything else. Other GCs require a bit of fiddling and more testing (see e.g. [JDK-8256497](https://bugs.openjdk.java.net/browse/JDK-8256497)). > > Additional testing: > - [x] Linux x86_64 Zero {fastdebug,release}, `gc/epsilon` tests Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1269 From ihse at openjdk.java.net Wed Nov 18 10:42:13 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 10:42:13 GMT Subject: RFR: 8256240: Reproducible builds should turn on the "deterministic" flag for Visual Studio Message-ID: The microsoft toolchain now supports a new "deterministic" build mode. This goes a long way towards building properly deterministic. Another important feature is that with the deterministic build mode enabled, the -pathmap flag, similar to gcc's -fmacro-prefix-map, is enabled. ------------- Commit messages: - 8256240: Reproducible builds should turn on the "deterministic" flag for Visual Studio Changes: https://git.openjdk.java.net/jdk/pull/1287/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1287&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256240 Stats: 68 lines in 4 files changed: 48 ins; 14 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/1287.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1287/head:pull/1287 PR: https://git.openjdk.java.net/jdk/pull/1287 From ihse at openjdk.java.net Wed Nov 18 12:28:05 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 12:28:05 GMT Subject: RFR: 8256538: Fix annoying awk warning in configure for java versions In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 09:47:58 GMT, Magnus Ihse Bursie wrote: > The change from grep to awk in JDK-8244248 and further bug fixed in JDK-8244756 still has invalid syntax. This causes some awk (most notably gawk, the most commonly used) to complain: > > gawk: cmd. line:1: warning: regexp escape sequence `"' is not a known regexp operator > > This is annoying and should be fixed. I have gotten confirmation from Martin Doerr that this also works on AIX, which has an awk that previously complained about this pattern. And I've tested it on all platforms used in the Oracle CI system. ------------- PR: https://git.openjdk.java.net/jdk/pull/1285 From jlaskey at openjdk.java.net Wed Nov 18 13:21:11 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Wed, 18 Nov 2020 13:21:11 GMT Subject: Withdrawn: 8248862: Implement Enhanced Pseudo-Random Number Generators In-Reply-To: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> Message-ID: <0ehbQoBJaglOxi_fA4XXjjVbL4iXPwj8wz-W9aoW6I8=.5a76bd44-9a20-4e29-b9d9-5f21548df415@github.com> On Tue, 17 Nov 2020 19:58:47 GMT, Jim Laskey wrote: > This PR is to introduce a new random number API for the JDK. The primary API is found in RandomGenerator and RandomGeneratorFactory. Further description can be found in the JEP https://openjdk.java.net/jeps/356 . This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Wed Nov 18 13:21:10 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Wed, 18 Nov 2020 13:21:10 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> Message-ID: On Wed, 18 Nov 2020 00:51:43 GMT, Paul Sandoz wrote: >> Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits: >> >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Update package-info.java >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated RandomGeneratorFactory javadoc. >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated documentation for RandomGeneratorFactory. >> - Merge branch 'master' into 8248862 >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Move RandomGeneratorProperty >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Clear up javadoc >> - 8248862; Implement Enhanced Pseudo-Random Number Generators >> >> remove RandomGeneratorProperty from API >> - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 > > I am unsure if the intent is also to support external libraries providing `RandomGenerator` implementations. Currently there is an implicit contract for properties (reflectively invoking a method returning a map with a set of entries with known keys). Since the service provider implementation is the `RandomGenerator` itself, rather than `RandomGeneratorFactory` it is harder expose the meta-data with a clearer contract. Need rebase ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From erikj at openjdk.java.net Wed Nov 18 13:38:07 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 18 Nov 2020 13:38:07 GMT Subject: RFR: 8256538: Fix annoying awk warning in configure for java versions In-Reply-To: References: Message-ID: <5Z0hT6tZxqhvzdhbRMpnXvhVqnkHGL-ysG-TzDs03fc=.1a5d5fda-198a-4377-910f-9b4425fd43fb@github.com> On Wed, 18 Nov 2020 09:47:58 GMT, Magnus Ihse Bursie wrote: > The change from grep to awk in JDK-8244248 and further bug fixed in JDK-8244756 still has invalid syntax. This causes some awk (most notably gawk, the most commonly used) to complain: > > gawk: cmd. line:1: warning: regexp escape sequence `"' is not a known regexp operator > > This is annoying and should be fixed. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1285 From rkennke at openjdk.java.net Wed Nov 18 13:47:19 2020 From: rkennke at openjdk.java.net (Roman Kennke) Date: Wed, 18 Nov 2020 13:47:19 GMT Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:02:03 GMT, Aleksey Shipilev wrote: > Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling. > > Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops) > - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC` Looks good to me! Thanks! ------------- Marked as reviewed by rkennke (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1268 From shade at openjdk.java.net Wed Nov 18 13:47:19 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 18 Nov 2020 13:47:19 GMT Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs Message-ID: Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling. Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah. Additional testing: - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops) - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC` ------------- Commit messages: - Remove TODO - 8256497: Zero: enable G1 and Shenandoah GCs Changes: https://git.openjdk.java.net/jdk/pull/1268/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1268&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256497 Stats: 72 lines in 5 files changed: 54 ins; 16 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1268.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1268/head:pull/1268 PR: https://git.openjdk.java.net/jdk/pull/1268 From jlaskey at openjdk.java.net Wed Nov 18 13:48:04 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Wed, 18 Nov 2020 13:48:04 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> Message-ID: <5IW2VsaVaqCMqkD6qkNxdWpL1qJM8daN1yf9a-Ivxk8=.d15e95c1-183d-4d50-98ff-e112beadf602@github.com> On Wed, 18 Nov 2020 13:18:30 GMT, Jim Laskey wrote: >> I am unsure if the intent is also to support external libraries providing `RandomGenerator` implementations. Currently there is an implicit contract for properties (reflectively invoking a method returning a map with a set of entries with known keys). Since the service provider implementation is the `RandomGenerator` itself, rather than `RandomGeneratorFactory` it is harder expose the meta-data with a clearer contract. > > Need rebase Created new PR because of forced push: https://github.com/openjdk/jdk/pull/1292 ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From erikj at openjdk.java.net Wed Nov 18 14:14:07 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 18 Nov 2020 14:14:07 GMT Subject: Integrated: JDK-8256501: libTestMainKeyWindow fails to build with Xcode 12.2 In-Reply-To: <2Opj59OMzdr_XjiC2SftWkXLKbjOnfBPst26kNvH8Gk=.b1214b22-0137-4495-a7ed-7abc6c4c36e1@github.com> References: <2Opj59OMzdr_XjiC2SftWkXLKbjOnfBPst26kNvH8Gk=.b1214b22-0137-4495-a7ed-7abc6c4c36e1@github.com> Message-ID: On Tue, 17 Nov 2020 21:04:08 GMT, Erik Joelsson wrote: > In Xcode 12.2, the framework JavaVM was removed in the sysroot and all (relevant) frameworks inside were just moved one step up so we still find things like JavaNativeFoundation and JavaRuntimeSupport. There is however one test that currently links explicitly to JavaVM, which fails the build. The fix seems to be to simply remove this from the linker flags. This pull request has now been integrated. Changeset: 4e5116c4 Author: Erik Joelsson URL: https://git.openjdk.java.net/jdk/commit/4e5116c4 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8256501: libTestMainKeyWindow fails to build with Xcode 12.2 Reviewed-by: ihse, serb ------------- PR: https://git.openjdk.java.net/jdk/pull/1275 From burban at openjdk.java.net Wed Nov 18 14:16:08 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Wed, 18 Nov 2020 14:16:08 GMT Subject: RFR: 8256538: Fix annoying awk warning in configure for java versions In-Reply-To: <5Z0hT6tZxqhvzdhbRMpnXvhVqnkHGL-ysG-TzDs03fc=.1a5d5fda-198a-4377-910f-9b4425fd43fb@github.com> References: <5Z0hT6tZxqhvzdhbRMpnXvhVqnkHGL-ysG-TzDs03fc=.1a5d5fda-198a-4377-910f-9b4425fd43fb@github.com> Message-ID: On Wed, 18 Nov 2020 13:35:14 GMT, Erik Joelsson wrote: >> The change from grep to awk in JDK-8244248 and further bug fixed in JDK-8244756 still has invalid syntax. This causes some awk (most notably gawk, the most commonly used) to complain: >> >> gawk: cmd. line:1: warning: regexp escape sequence `"' is not a known regexp operator >> >> This is annoying and should be fixed. > > Marked as reviewed by erikj (Reviewer). Thank you Magnus, this is something that has bugged me as well. FWIW the same problem exists here with the build JDK detection: https://github.com/openjdk/jdk/blob/9d5fd4fcabf7bdf580e2fff8e12c2cf130ef44c9/make/autoconf/boot-jdk.m4#L519 Would you mind fixing it as part of this PR too? ------------- PR: https://git.openjdk.java.net/jdk/pull/1285 From sspitsyn at openjdk.java.net Wed Nov 18 19:34:12 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Wed, 18 Nov 2020 19:34:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: Message-ID: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> On Mon, 16 Nov 2020 23:30:25 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: > > Fix minimal build. Hi Coleen, It looks good to me. Just a couple of nits below. src/hotspot/share/prims/jvmtiTagMap.cpp: There is a double-check for _needs_cleaning, so the one at line 136 can be removed: 136 if (_needs_cleaning && 137 post_events && 138 env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { 139 remove_dead_entries(true /* post_object_free */); 1158 void JvmtiTagMap::remove_dead_entries(bool post_object_free) { 1159 assert(is_locked(), "precondition"); 1160 if (_needs_cleaning) { 1161 log_info(jvmti, table)("TagMap table needs cleaning%s", 1162 (post_object_free ? " and posting" : "")); 1163 hashmap()->remove_dead_entries(env(), post_object_free); 1164 _needs_cleaning = false; 1165 } 1166 } test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); + + /* Flush any pending ObjectFree events, which may set success to 1 */ + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, + JVMTI_EVENT_OBJECT_FREE, + NULL) != JVMTI_ERROR_NONE) { + success = 0; + } + + nsk_aod_agentFinished(jni, agentName, success); } ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/967 From jjg at openjdk.java.net Wed Nov 18 20:09:15 2020 From: jjg at openjdk.java.net (Jonathan Gibbons) Date: Wed, 18 Nov 2020 20:09:15 GMT Subject: RFR: JDK-6251738: Want a top-level summary page that itemizes all spec documents referenced from javadocs (OEM spec) [v3] In-Reply-To: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> References: <6T_odbmLeZsI2_G8H7bcfL8jLi8TUAxc_u6g6OvUBjU=.76eff40f-10f7-4b16-a605-718f528b3e03@github.com> Message-ID: > This introduces support for a new `@spec` tag that can be used as either an inline tag or as a block tag. It is used to identify references to external specifications, in such a way that the references can be collected together on a new summary page, called "Other Specifications", available from either the static INDEX page or the interactive search box. > > As an inline tag, the format is `{@spec url label}`, which is roughly translated to `label` in the generated docs. > > As a block tag, the format is simply > > @spec url label > > which is handled in a manner analogous to > > @see label > > The tag is notable for being the first standard/supported tag that can be used as either an inline or block tag. (We have had support for bimodal non-standard/custom tags for a while, such as `{@jls}` and `{@jvms}`.) To support bimodal standard tags, some changes to `DocCommentParser` are incorporated here. > > This change is only the _support_ for the new tag; it does _not_ include any changes to API docs to _use_ the new tag. Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits: - Merge master - Fix merge issues; review feedback - Merge with master - allow rich content in createAnchorAndSearchIndex - update Docs.gmk to stop disabling spec tag - fix TestSpecTag.testEncodedURI - fix tests - remove support to workaround legacy @spec tag - Merge remote-tracking branch 'upstream/master' into new-spec-tag - fix trailing whitespace in test - ... and 2 more: https://git.openjdk.java.net/jdk/compare/300cbaa6...69595e04 ------------- Changes: https://git.openjdk.java.net/jdk/pull/790/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=790&range=02 Stats: 1374 lines in 42 files changed: 1337 ins; 14 del; 23 mod Patch: https://git.openjdk.java.net/jdk/pull/790.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/790/head:pull/790 PR: https://git.openjdk.java.net/jdk/pull/790 From ihse at openjdk.java.net Wed Nov 18 21:58:15 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 21:58:15 GMT Subject: RFR: 8256538: Fix annoying awk warning in configure for java versions [v2] In-Reply-To: References: Message-ID: > The change from grep to awk in JDK-8244248 and further bug fixed in JDK-8244756 still has invalid syntax. This causes some awk (most notably gawk, the most commonly used) to complain: > > gawk: cmd. line:1: warning: regexp escape sequence `"' is not a known regexp operator > > This is annoying and should be fixed. Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Also fix bad code as copied in BOOTJDK_CHECK_BUILD_JDK ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1285/files - new: https://git.openjdk.java.net/jdk/pull/1285/files/9d5fd4fc..842287e0 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1285&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1285&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1285.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1285/head:pull/1285 PR: https://git.openjdk.java.net/jdk/pull/1285 From ihse at openjdk.java.net Wed Nov 18 21:58:16 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 21:58:16 GMT Subject: RFR: 8256538: Fix annoying awk warning in configure for java versions [v2] In-Reply-To: References: <5Z0hT6tZxqhvzdhbRMpnXvhVqnkHGL-ysG-TzDs03fc=.1a5d5fda-198a-4377-910f-9b4425fd43fb@github.com> Message-ID: On Wed, 18 Nov 2020 14:13:03 GMT, Bernhard Urban-Forster wrote: >> Marked as reviewed by erikj (Reviewer). > > Thank you Magnus, this is something that has bugged me as well. > > FWIW the same problem exists here with the build JDK detection: https://github.com/openjdk/jdk/blob/9d5fd4fcabf7bdf580e2fff8e12c2cf130ef44c9/make/autoconf/boot-jdk.m4#L519 > > Would you mind fixing it as part of this PR too? @lewurm Thanks! I had not noticed that one. (In fact, I'm unsure when we got such a huge chunk of duplicated code... *sigh* Should look into it some day, I think.) ------------- PR: https://git.openjdk.java.net/jdk/pull/1285 From ihse at openjdk.java.net Wed Nov 18 21:58:16 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Wed, 18 Nov 2020 21:58:16 GMT Subject: Integrated: 8256538: Fix annoying awk warning in configure for java versions In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 09:47:58 GMT, Magnus Ihse Bursie wrote: > The change from grep to awk in JDK-8244248 and further bug fixed in JDK-8244756 still has invalid syntax. This causes some awk (most notably gawk, the most commonly used) to complain: > > gawk: cmd. line:1: warning: regexp escape sequence `"' is not a known regexp operator > > This is annoying and should be fixed. This pull request has now been integrated. Changeset: 3110d589 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/3110d589 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8256538: Fix annoying awk warning in configure for java versions Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1285 From erikj at openjdk.java.net Wed Nov 18 22:32:03 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Wed, 18 Nov 2020 22:32:03 GMT Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:02:03 GMT, Aleksey Shipilev wrote: > Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling. > > Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops) > - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC` Build change looks ok. ------------- Marked as reviewed by erikj (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1268 From coleenp at openjdk.java.net Wed Nov 18 23:24:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 18 Nov 2020 23:24:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v10] In-Reply-To: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com> References: <4wSXIq_YvsFFLWWNjVgLCfNqbrxHDajiQGrKPuwcP3A=.5427b09b-48e2-409f-8099-9e44fbdc339d@github.com> Message-ID: <6MQhIGtA1LWvXvSFSZWRkyU6d9AXf_LVdkE0zAq0ekc=.151e621a-eec4-4366-8ffa-729eb1d5bb63@github.com> On Mon, 16 Nov 2020 23:10:21 GMT, Coleen Phillimore wrote: >> This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. >> >> The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. >> >> The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: >> >> 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. >> 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. >> >> Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. >> >> To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. >> >> Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. >> >> It has also been tested with tier1-6. >> >> Thank you to Stefan, Erik and Kim for their help with this change. > > Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 15 commits: > > - Reverse remove_dead_entries_locked function names. > - Merge branch 'master' into jvmti-table > - Add shenandoah set_needs_cleaning but this doesn't work. > - fix vmTestbase/nsk/jvmti tests > - improve tagmap cleanup and objectfree event posting > - Add logging to event posting in case of pauses. > - Merge branch 'master' into jvmti-table > - Add back WeakProcessorPhases::Phase enum. > - Serguei 1. > - Code review comments from Kim and Albert. > - ... and 5 more: https://git.openjdk.java.net/jdk/compare/0357db35...daaa13fe test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp line 72: > 70: Java_nsk_jvmti_AttachOnDemand_attach021_attach021Target_shutdownAgent(JNIEnv * jni, > 71: jclass klass) { > 72: @kimbarrett ? I'm not sure why you made this change. See Serguei's comment. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Wed Nov 18 23:29:10 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 18 Nov 2020 23:29:10 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: On Wed, 18 Nov 2020 19:31:44 GMT, Serguei Spitsyn wrote: > There is a double-check for _needs_cleaning, so the one at line 136 can be removed: I want to leave _needs_cleaning at 136 because even though the boolean is checked twice, it doesn't hurt performance and it has a nice symmetry in that function. I asked Kim about the other change. Thank you for reviewing, Serguei! ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 00:20:24 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 00:20:24 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v12] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: - Merge branch 'master' into jvmti-table - Fix minimal build. - Reverse remove_dead_entries_locked function names. - Merge branch 'master' into jvmti-table - Add shenandoah set_needs_cleaning but this doesn't work. - fix vmTestbase/nsk/jvmti tests - improve tagmap cleanup and objectfree event posting - Add logging to event posting in case of pauses. - Merge branch 'master' into jvmti-table - Add back WeakProcessorPhases::Phase enum. - ... and 7 more: https://git.openjdk.java.net/jdk/compare/2b155713...9ef44f28 ------------- Changes: https://git.openjdk.java.net/jdk/pull/967/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=11 Stats: 1884 lines in 49 files changed: 768 ins; 992 del; 124 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From kbarrett at openjdk.java.net Thu Nov 19 00:43:09 2020 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Thu, 19 Nov 2020 00:43:09 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: On Wed, 18 Nov 2020 19:31:44 GMT, Serguei Spitsyn wrote: > test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: > > The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: > > ``` > - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); > + > + /* Flush any pending ObjectFree events, which may set success to 1 */ > + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, > + JVMTI_EVENT_OBJECT_FREE, > + NULL) != JVMTI_ERROR_NONE) { > + success = 0; > + } > + > + nsk_aod_agentFinished(jni, agentName, success); > } > ``` This change really is needed. The success variable in the test is a global, initially 0, set to 1 by the ObjectFree handler. In the old code, if the ObjectFree event hasn't been posted yet, we pass the initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where it's a local variable (so unaffected by any changes to the variable in the test), so stays 0 through to the call to nsk_aod_agentFinished. So the test fails. The split in the change causes the updated post-ObjectFree event success value of 1 to be passed to agentFinished. So the test passes. That required some head scratching to find at the time. That's the point of the comment about flushing pending events. Maybe the comment should be improved. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From shade at openjdk.java.net Thu Nov 19 08:05:04 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 19 Nov 2020 08:05:04 GMT Subject: Integrated: 8256499: Zero: enable Epsilon GC In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:11:45 GMT, Aleksey Shipilev wrote: > Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. Epsilon GC does not require anything else. Other GCs require a bit of fiddling and more testing (see e.g. [JDK-8256497](https://bugs.openjdk.java.net/browse/JDK-8256497)). > > Additional testing: > - [x] Linux x86_64 Zero {fastdebug,release}, `gc/epsilon` tests This pull request has now been integrated. Changeset: cb2676c8 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/cb2676c8 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8256499: Zero: enable Epsilon GC Reviewed-by: rkennke, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1269 From shade at openjdk.java.net Thu Nov 19 08:05:03 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 19 Nov 2020 08:05:03 GMT Subject: RFR: 8256499: Zero: enable Epsilon GC In-Reply-To: References: Message-ID: <6ht_05zHDXPoQomASZrKIDxai2TkkYe1br__Ng2_hJc=.525248c7-48b3-4601-9fa7-c4b5d52911d7@github.com> On Wed, 18 Nov 2020 09:52:22 GMT, Magnus Ihse Bursie wrote: >> Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. Epsilon GC does not require anything else. Other GCs require a bit of fiddling and more testing (see e.g. [JDK-8256497](https://bugs.openjdk.java.net/browse/JDK-8256497)). >> >> Additional testing: >> - [x] Linux x86_64 Zero {fastdebug,release}, `gc/epsilon` tests > > Marked as reviewed by ihse (Reviewer). Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1269 From shade at openjdk.java.net Thu Nov 19 08:32:14 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 19 Nov 2020 08:32:14 GMT Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs [v2] In-Reply-To: References: Message-ID: > Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling. > > Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops) > - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC` Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: - Merge branch 'master' into JDK-8256497-zero-g1-shenandoah - Remove TODO - 8256497: Zero: enable G1 and Shenandoah GCs ------------- Changes: https://git.openjdk.java.net/jdk/pull/1268/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1268&range=01 Stats: 72 lines in 5 files changed: 54 ins; 16 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1268.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1268/head:pull/1268 PR: https://git.openjdk.java.net/jdk/pull/1268 From ihse at openjdk.java.net Thu Nov 19 09:58:05 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 19 Nov 2020 09:58:05 GMT Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs [v2] In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 08:32:14 GMT, Aleksey Shipilev wrote: >> Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling. >> >> Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah. >> >> Additional testing: >> - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops) >> - [ ] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC` > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: > > - Merge branch 'master' into JDK-8256497-zero-g1-shenandoah > - Remove TODO > - 8256497: Zero: enable G1 and Shenandoah GCs Build changes look good. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1268 From sspitsyn at openjdk.java.net Thu Nov 19 10:13:13 2020 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 19 Nov 2020 10:13:13 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: On Thu, 19 Nov 2020 00:39:52 GMT, Kim Barrett wrote: >> Hi Coleen, >> It looks good to me. >> Just a couple of nits below. >> >> src/hotspot/share/prims/jvmtiTagMap.cpp: >> >> There is a double-check for _needs_cleaning, so the one at line 136 can be removed: >> 136 if (_needs_cleaning && >> 137 post_events && >> 138 env()->is_enabled(JVMTI_EVENT_OBJECT_FREE)) { >> 139 remove_dead_entries(true /* post_object_free */); >> 1158 void JvmtiTagMap::remove_dead_entries(bool post_object_free) { >> 1159 assert(is_locked(), "precondition"); >> 1160 if (_needs_cleaning) { >> 1161 log_info(jvmti, table)("TagMap table needs cleaning%s", >> 1162 (post_object_free ? " and posting" : "")); >> 1163 hashmap()->remove_dead_entries(env(), post_object_free); >> 1164 _needs_cleaning = false; >> 1165 } >> 1166 } >> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: >> >> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: >> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); >> + >> + /* Flush any pending ObjectFree events, which may set success to 1 */ >> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, >> + JVMTI_EVENT_OBJECT_FREE, >> + NULL) != JVMTI_ERROR_NONE) { >> + success = 0; >> + } >> + >> + nsk_aod_agentFinished(jni, agentName, success); >> } > >> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: >> >> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: >> >> ``` >> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); >> + >> + /* Flush any pending ObjectFree events, which may set success to 1 */ >> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, >> + JVMTI_EVENT_OBJECT_FREE, >> + NULL) != JVMTI_ERROR_NONE) { >> + success = 0; >> + } >> + >> + nsk_aod_agentFinished(jni, agentName, success); >> } >> ``` > > This change really is needed. > > The success variable in the test is a global, initially 0, set to 1 by the > ObjectFree handler. > > In the old code, if the ObjectFree event hasn't been posted yet, we pass the > initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where > it's a local variable (so unaffected by any changes to the variable in the > test), so stays 0 through to the call to nsk_aod_agentFinished. So the test > fails. > > The split in the change causes the updated post-ObjectFree event success > value of 1 to be passed to agentFinished. So the test passes. > > That required some head scratching to find at the time. That's the point of > the comment about flushing pending events. Maybe the comment should be > improved. @kimbarrett Okay, thank you for explanation. I agree, it'd make sense to improve the comment a little bit. Thanks, Serguei ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:39:14 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:39:14 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> Message-ID: <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> On Thu, 19 Nov 2020 10:10:06 GMT, Serguei Spitsyn wrote: >>> test/hotspot/jtreg/vmTestbase/nsk/jvmti/AttachOnDemand/attach021/attach021Agent00.cpp: >>> >>> The change below is not needed as the call to nsk_jvmti_aod_disableEventAndFinish() does exactly the same: >>> >>> ``` >>> - nsk_jvmti_aod_disableEventAndFinish(agentName, JVMTI_EVENT_OBJECT_FREE, success, jvmti, jni); >>> + >>> + /* Flush any pending ObjectFree events, which may set success to 1 */ >>> + if (jvmti->SetEventNotificationMode(JVMTI_DISABLE, >>> + JVMTI_EVENT_OBJECT_FREE, >>> + NULL) != JVMTI_ERROR_NONE) { >>> + success = 0; >>> + } >>> + >>> + nsk_aod_agentFinished(jni, agentName, success); >>> } >>> ``` >> >> This change really is needed. >> >> The success variable in the test is a global, initially 0, set to 1 by the >> ObjectFree handler. >> >> In the old code, if the ObjectFree event hasn't been posted yet, we pass the >> initial 0 value of success to nsk_jvmti_aod_disabledEventAndFinish, where >> it's a local variable (so unaffected by any changes to the variable in the >> test), so stays 0 through to the call to nsk_aod_agentFinished. So the test >> fails. >> >> The split in the change causes the updated post-ObjectFree event success >> value of 1 to be passed to agentFinished. So the test passes. >> >> That required some head scratching to find at the time. That's the point of >> the comment about flushing pending events. Maybe the comment should be >> improved. > > @kimbarrett > Okay, thank you for explanation. > I agree, it'd make sense to improve the comment a little bit. > Thanks, > Serguei So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change? ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:57:25 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:57:25 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> Message-ID: On Thu, 19 Nov 2020 12:36:46 GMT, Coleen Phillimore wrote: >> @kimbarrett >> Okay, thank you for explanation. >> I agree, it'd make sense to improve the comment a little bit. >> Thanks, >> Serguei > > So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change? > Ok, with this change, it's not posted yet and the success variable for nsk_aod_agentFinished is the local variable. We should fix this in an RFE filed: https://bugs.openjdk.java.net/browse/JDK-8256651 /* Flush any pending ObjectFree events, which will set global success variable to 1 for any pending ObjectFree events. */ How about this? The word 'global' helps me. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:57:24 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:57:24 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v13] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Update comment in jvmti test. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/9ef44f28..40168e63 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=12 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=11-12 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 12:57:25 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 12:57:25 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> Message-ID: On Thu, 19 Nov 2020 12:51:11 GMT, Coleen Phillimore wrote: >> So should nsk_jvmti_aod_disableEventAndFinish pass the address of success instead? Why didn't it fail before this change? >> Ok, with this change, it's not posted yet and the success variable for nsk_aod_agentFinished is the local variable. We should fix this in an RFE filed: https://bugs.openjdk.java.net/browse/JDK-8256651 > > /* Flush any pending ObjectFree events, which will set global success variable to 1 > for any pending ObjectFree events. */ > How about this? The word 'global' helps me. With remerging into shenandoah, all the jdi tests pass with shenandoah also. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 13:01:22 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 13:01:22 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v14] In-Reply-To: References: Message-ID: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. Coleen Phillimore has updated the pull request incrementally with one additional commit since the last revision: Fix copyrights in test changes. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/967/files - new: https://git.openjdk.java.net/jdk/pull/967/files/40168e63..589e4c5b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=13 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=967&range=12-13 Stats: 5 lines in 5 files changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/967.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/967/head:pull/967 PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 14:33:16 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 14:33:16 GMT Subject: Integrated: 8212879: Make JVMTI TagMap table concurrent In-Reply-To: References: Message-ID: On Fri, 30 Oct 2020 20:23:04 GMT, Coleen Phillimore wrote: > This change turns the HashTable that JVMTI uses for object tagging into a regular Hotspot hashtable - the one in hashtable.hpp with resizing and rehashing. Instead of pointing directly to oops so that GC has to walk the table to follow oops and then to rehash the table, this table points to WeakHandle. GC walks the backing OopStorages concurrently. > > The hash function for the table is a hash of the lower 32 bits of the address. A flag is set during GC (gc_notification if in a safepoint, and through a call to JvmtiTagMap::needs_processing()) so that the table is rehashed at the next use. > > The gc_notification mechanism of weak oop processing is used to notify Jvmti to post ObjectFree events. In concurrent GCs there can be a window of time between weak oop marking where the oop is unmarked, so dead (the phantom load in peek returns NULL) but the gc_notification hasn't been done yet. In this window, a heap walk or GetObjectsWithTags call would not find an object before the ObjectFree event is posted. This is dealt with in two ways: > > 1. In the Heap walk, there's an unconditional table walk to post events if events are needed to post. > 2. For GetObjectWithTags, if a dead oop is found in the table and posting is required, we use the VM thread to post the event. > > Event posting cannot be done in a JavaThread because the posting needs to be done while holding the table lock, so that the JvmtiEnv state doesn't change before posting is done. ObjectFree callbacks are limited in what they can do as per the JVMTI Specification. The allowed callbacks to the VM already have code to allow NonJava threads. > > To avoid rehashing, I also tried to use object->identity_hash() but this breaks because entries can be added to the table during heapwalk, where the objects use marking. The starting markWord is saved and restored. Adding a hashcode during this operation makes restoring the former markWord (locked, inflated, etc) too complicated. Plus we don't want all these objects to have hashcodes because locking operations after tagging would have to always use inflated locks. > > Much of this change is to remove serial weak oop processing for the weakProcessor, ZGC and Shenandoah. The GCs have been stress tested with jvmti code. > > It has also been tested with tier1-6. > > Thank you to Stefan, Erik and Kim for their help with this change. This pull request has now been integrated. Changeset: ba721f5f Author: Coleen Phillimore URL: https://git.openjdk.java.net/jdk/commit/ba721f5f Stats: 1891 lines in 49 files changed: 769 ins; 992 del; 130 mod 8212879: Make JVMTI TagMap table concurrent Co-authored-by: Kim Barrett Co-authored-by: Coleen Phillimore Reviewed-by: stefank, ihse, zgu, eosterlund, sspitsyn, kbarrett ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From coleenp at openjdk.java.net Thu Nov 19 14:33:12 2020 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Thu, 19 Nov 2020 14:33:12 GMT Subject: RFR: 8212879: Make JVMTI TagMap table concurrent [v11] In-Reply-To: References: <8m4pLTYDq8LLEZ3MRVfnZsKducSHJLX8WK-FxGaqgQw=.f426b0f8-0a50-4383-b037-24a925d9cf7e@github.com> <6Rg4SwfEMn0metyXsBl4pGfdP5zfspPuBLjFP82bGic=.f3ba30ad-ea01-4484-ae5f-1b6e3ce5b12a@github.com> Message-ID: On Thu, 19 Nov 2020 12:54:23 GMT, Coleen Phillimore wrote: >> /* Flush any pending ObjectFree events, which will set global success variable to 1 >> for any pending ObjectFree events. */ >> How about this? The word 'global' helps me. > > With remerging into shenandoah, all the jdi tests pass with shenandoah also. Thank you to all the reviewers. ------------- PR: https://git.openjdk.java.net/jdk/pull/967 From rwestberg at openjdk.java.net Thu Nov 19 16:16:21 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Thu, 19 Nov 2020 16:16:21 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions [v2] In-Reply-To: References: Message-ID: <9eRYhD5PmfiZRsRAVZgGNrYVRjmnJTG23ac9XEfV6U0=.010b1392-f328-46e3-906e-3303dbf5f8ab@github.com> > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: Select gcc/g++ package versions explicitly ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1225/files - new: https://git.openjdk.java.net/jdk/pull/1225/files/87ab0729..3dcb26d6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1225&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1225&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1225.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1225/head:pull/1225 PR: https://git.openjdk.java.net/jdk/pull/1225 From rwestberg at openjdk.java.net Thu Nov 19 16:16:21 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Thu, 19 Nov 2020 16:16:21 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions [v2] In-Reply-To: References: <-6nNHhzc934ck7eLnQJwo0yt0kyVtvWrU6Ma2573lu8=.1fa9c0a1-3986-4051-a881-bd7dc5f7ab1d@github.com> Message-ID: On Mon, 16 Nov 2020 15:12:25 GMT, Magnus Ihse Bursie wrote: >> .github/workflows/submit.yml line 190: >> >>> 188: run: | >>> 189: sudo apt-get install libxrandr-dev libxtst-dev libcups2-dev libasound2-dev >>> 190: sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 --slave /usr/bin/g++ g++ /usr/bin/g++-10 >> >> Maybe we should use apt-get functionality to install a specific version of packages? I'm not sure how relevant it is for the X and alsa libraries since they change very seldom, but perhaps for gcc, to get a specific point release of the compiler. > > Something along the lines of `sudo apt-get install gcc-10=10.2.0-5ubuntu1~20`, which I believe should match quite well the version used internally in the Oracle CI builds. That sounds reasonable, I don't know how often these change in Ubuntu LTS, but can't hurt to be explicit here as well. ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From rwestberg at openjdk.java.net Thu Nov 19 16:22:20 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Thu, 19 Nov 2020 16:22:20 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions [v3] In-Reply-To: References: Message-ID: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: Use proper package version ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1225/files - new: https://git.openjdk.java.net/jdk/pull/1225/files/3dcb26d6..12b1ffe7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1225&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1225&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1225.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1225/head:pull/1225 PR: https://git.openjdk.java.net/jdk/pull/1225 From ihse at openjdk.java.net Thu Nov 19 16:38:06 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 19 Nov 2020 16:38:06 GMT Subject: RFR: 8256393: Github Actions build on Linux should define OS and GCC versions [v3] In-Reply-To: References: Message-ID: <_npV6pgdrCqkDkhRXKSiM9ww-O8QUqctX8VHvP0NP1Y=.b4393e2a-23fc-4d1d-a21d-6b49494466fc@github.com> On Thu, 19 Nov 2020 16:22:20 GMT, Robin Westberg wrote: >> We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. > > Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: > > Use proper package version LGTMN. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1225 From ihse at openjdk.java.net Fri Nov 20 00:31:17 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 20 Nov 2020 00:31:17 GMT Subject: RFR: 8255526: Enable jcheck whitespace checking of build system files [v2] In-Reply-To: <4KS9T_q3TFpdbE-8tp0zMwBy_aNk465Wcd5B5xNJx0E=.555adb95-c050-4580-8fd6-98f322b74f40@github.com> References: <4KS9T_q3TFpdbE-8tp0zMwBy_aNk465Wcd5B5xNJx0E=.555adb95-c050-4580-8fd6-98f322b74f40@github.com> Message-ID: <4rTNMitonzJfb5qHzj4QkV8mt9THcgfH_ZqKsFoWq2Q=.862e6173-31bb-4c36-899c-4ff5b294a1e9@github.com> > With the new Skara tooling, we can finally enable whitespace testing for build system files (makefiles and autoconf files). Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: Add ignore-tabs ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/898/files - new: https://git.openjdk.java.net/jdk/pull/898/files/236d8360..5aa2766b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=898&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=898&range=00-01 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/898.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/898/head:pull/898 PR: https://git.openjdk.java.net/jdk/pull/898 From ihse at openjdk.java.net Fri Nov 20 00:31:17 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 20 Nov 2020 00:31:17 GMT Subject: RFR: 8255526: Enable jcheck whitespace checking of build system files [v2] In-Reply-To: References: <4KS9T_q3TFpdbE-8tp0zMwBy_aNk465Wcd5B5xNJx0E=.555adb95-c050-4580-8fd6-98f322b74f40@github.com> Message-ID: On Thu, 29 Oct 2020 00:54:41 GMT, Magnus Ihse Bursie wrote: >> I need to put this on hold for a while. It turned out that the Skara jcheck disallows initial tabs, which makes sense for sane source code formats (java, c++) but is unfortunately an incorrect assumption for makefiles. >> >> (I tested that the modification to jcheck detected trailing whitespaces, but it never occurred to me that I needed to modify a line starting with tab to show that this fix would not work...) > > Fix for Skara in PR https://github.com/openjdk/skara/pull/929. Now the Skara fix is deployed, and I've added instructions to the jcheck configure file to allow tabs in Makefiles, so this is finally ready to be integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/898 From shade at openjdk.java.net Fri Nov 20 06:32:04 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 20 Nov 2020 06:32:04 GMT Subject: RFR: 8255526: Enable jcheck whitespace checking of build system files [v2] In-Reply-To: <4rTNMitonzJfb5qHzj4QkV8mt9THcgfH_ZqKsFoWq2Q=.862e6173-31bb-4c36-899c-4ff5b294a1e9@github.com> References: <4KS9T_q3TFpdbE-8tp0zMwBy_aNk465Wcd5B5xNJx0E=.555adb95-c050-4580-8fd6-98f322b74f40@github.com> <4rTNMitonzJfb5qHzj4QkV8mt9THcgfH_ZqKsFoWq2Q=.862e6173-31bb-4c36-899c-4ff5b294a1e9@github.com> Message-ID: <1TVG-z3W9m7BD2Ykpka2_7ow6xLEZZi4KTiw01j4fNc=.8d60575d-5690-4501-9075-c4c42413a40d@github.com> On Fri, 20 Nov 2020 00:31:17 GMT, Magnus Ihse Bursie wrote: >> With the new Skara tooling, we can finally enable whitespace testing for build system files (makefiles and autoconf files). > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Add ignore-tabs Still good! ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/898 From rwestberg at openjdk.java.net Fri Nov 20 08:08:04 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 20 Nov 2020 08:08:04 GMT Subject: Integrated: 8256393: Github Actions build on Linux should define OS and GCC versions In-Reply-To: References: Message-ID: On Mon, 16 Nov 2020 12:51:25 GMT, Robin Westberg wrote: > We should be more explicit about OS and compiler versions used in the GitHub Actions builds, to avoid problems caused by unexpected changes to the defaults. This patch changes the OS and GCC versions used from ubuntu-latest (currently 18.04, but will change to 20.04 sometime soon) / default (currently 9.3.0) to 20.04 / 10.2.0. This pull request has now been integrated. Changeset: c45ab1aa Author: Robin Westberg URL: https://git.openjdk.java.net/jdk/commit/c45ab1aa Stats: 16 lines in 1 file changed: 3 ins; 0 del; 13 mod 8256393: Github Actions build on Linux should define OS and GCC versions Reviewed-by: shade, erikj, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1225 From ihse at openjdk.java.net Fri Nov 20 10:05:05 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 20 Nov 2020 10:05:05 GMT Subject: Integrated: 8255526: Enable jcheck whitespace checking of build system files In-Reply-To: <4KS9T_q3TFpdbE-8tp0zMwBy_aNk465Wcd5B5xNJx0E=.555adb95-c050-4580-8fd6-98f322b74f40@github.com> References: <4KS9T_q3TFpdbE-8tp0zMwBy_aNk465Wcd5B5xNJx0E=.555adb95-c050-4580-8fd6-98f322b74f40@github.com> Message-ID: On Wed, 28 Oct 2020 10:16:36 GMT, Magnus Ihse Bursie wrote: > With the new Skara tooling, we can finally enable whitespace testing for build system files (makefiles and autoconf files). This pull request has now been integrated. Changeset: 8e7a855e Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/8e7a855e Stats: 4 lines in 3 files changed: 1 ins; 0 del; 3 mod 8255526: Enable jcheck whitespace checking of build system files Reviewed-by: shade, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/898 From erikj at openjdk.java.net Fri Nov 20 13:36:02 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 20 Nov 2020 13:36:02 GMT Subject: RFR: 8256240: Reproducible builds should turn on the "deterministic" flag for Visual Studio In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 10:35:58 GMT, Magnus Ihse Bursie wrote: > The microsoft toolchain now supports a new "deterministic" build mode. This goes a long way towards building properly deterministic. Another important feature is that with the deterministic build mode enabled, the -pathmap flag, similar to gcc's -fmacro-prefix-map, is enabled. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1287 From rwestberg at openjdk.java.net Fri Nov 20 14:31:14 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 20 Nov 2020 14:31:14 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing Message-ID: Currently Linux x64 testing in GitHub Actions depends on a few non-relevant hotspot build-only jobs (such as zero) that prevents testing from being run if those build were to fail. As the tests only require the x64 release and debug builds to run and provide interesting feedback, we should separate these other builds into a different job. To reduce duplicated steps, all the existing hotspot-only builds have been consolidated into a single job. ------------- Commit messages: - Separate hotspot-only builds into a separate job Changes: https://git.openjdk.java.net/jdk/pull/1350/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1350&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256747 Stats: 517 lines in 1 file changed: 32 ins; 438 del; 47 mod Patch: https://git.openjdk.java.net/jdk/pull/1350.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1350/head:pull/1350 PR: https://git.openjdk.java.net/jdk/pull/1350 From erikj at openjdk.java.net Fri Nov 20 14:40:11 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 20 Nov 2020 14:40:11 GMT Subject: RFR: JDK-8256751: Incremental rebuild with precompiled header fails when touching a header file Message-ID: When building with --disable-absolute-paths-in-output and precompiled header enabled (both of which are default in release builds), then incremental builds of Hotspot fails if a header file is touched. This is caused by the deps file the compiler generates is now containing relative paths, but those relative paths will not resolve when make imports the file. For all normal compilation units, we post process the file to rewrite these paths to absolute paths, but that call was missed for the precompiled header compilation. ------------- Commit messages: - Fix incremental build of precomp header Changes: https://git.openjdk.java.net/jdk/pull/1351/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1351&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256751 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1351.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1351/head:pull/1351 PR: https://git.openjdk.java.net/jdk/pull/1351 From shade at openjdk.java.net Fri Nov 20 14:49:17 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 20 Nov 2020 14:49:17 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 14:24:12 GMT, Robin Westberg wrote: > Currently Linux x64 testing in GitHub Actions depends on a few non-relevant hotspot build-only jobs (such as zero) that prevents testing from being run if those build were to fail. As the tests only require the x64 release and debug builds to run and provide interesting feedback, we should separate these other builds into a different job. > > To reduce duplicated steps, all the existing hotspot-only builds have been consolidated into a single job. I like the cleanup! I do wonder though about phasing, probably unrelated to this patch. It seems to me that if some builds fail, then it is highly likely the contributor would push another update. So letting the tests run before all builds complete might be a waste of time on that failure path. It is probably a rare occasion, as normally everything is green, and thus unblocking the tests when base Linux x64 builds are done is a good optimization. .github/workflows/submit.yml line 380: > 378: continue-on-error: true > 379: > 380: linux_x64_hotspot_only: This is a weird name for a job that also cross-compiles non-x64 versions. Maybe `linux_aux_builds` or some such? .github/workflows/submit.yml line 469: > 467: name: transient_jdk-linux-x64_${{ needs.prerequisites.outputs.bundle_id }} > 468: path: ~/jdk-linux-x64 > 469: if: steps.build_restore.outcome == 'failure' I may be blind, but I don't see any step with id `build_restore` in new file... .github/workflows/submit.yml line 506: > 504: run: > > 505: sudo qemu-debootstrap > 506: --arch=${{ matrix.debian-arch}} Style: `${{ matrix.debian-arch }}` (space before closing braces) -- also in other places. .github/workflows/submit.yml line 532: > 530: run: > > 531: bash configure > 532: --with-conf-name=linux-x64-hotspot That does not look a proper config name for cross-compiled jobs? ------------- Changes requested by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1350 From rwestberg at openjdk.java.net Fri Nov 20 16:14:19 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 20 Nov 2020 16:14:19 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: > Currently Linux x64 testing in GitHub Actions depends on a few non-relevant hotspot build-only jobs (such as zero) that prevents testing from being run if those build were to fail. As the tests only require the x64 release and debug builds to run and provide interesting feedback, we should separate these other builds into a different job. > > To reduce duplicated steps, all the existing hotspot-only builds have been consolidated into a single job. Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: Improve naming, fix style issues ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1350/files - new: https://git.openjdk.java.net/jdk/pull/1350/files/9cf0c80d..c79e6ab5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1350&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1350&range=00-01 Stats: 21 lines in 1 file changed: 0 ins; 0 del; 21 mod Patch: https://git.openjdk.java.net/jdk/pull/1350.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1350/head:pull/1350 PR: https://git.openjdk.java.net/jdk/pull/1350 From rwestberg at openjdk.java.net Fri Nov 20 16:14:21 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Fri, 20 Nov 2020 16:14:21 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 14:39:54 GMT, Aleksey Shipilev wrote: >> Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: >> >> Improve naming, fix style issues > > .github/workflows/submit.yml line 380: > >> 378: continue-on-error: true >> 379: >> 380: linux_x64_hotspot_only: > > This is a weird name for a job that also cross-compiles non-x64 versions. Maybe `linux_aux_builds` or some such? Yeah this can probably be improved, how about the updated version? > .github/workflows/submit.yml line 469: > >> 467: name: transient_jdk-linux-x64_${{ needs.prerequisites.outputs.bundle_id }} >> 468: path: ~/jdk-linux-x64 >> 469: if: steps.build_restore.outcome == 'failure' > > I may be blind, but I don't see any step with id `build_restore` in new file... It already existed in the step right above so I didn't have to add it.. This follows the same pattern as other artifact downloads (I've seen them fail a fair amount of time in the past). > .github/workflows/submit.yml line 506: > >> 504: run: > >> 505: sudo qemu-debootstrap >> 506: --arch=${{ matrix.debian-arch}} > > Style: `${{ matrix.debian-arch }}` (space before closing braces) -- also in other places. Ah right, good catch, fixed these. > .github/workflows/submit.yml line 532: > >> 530: run: > >> 531: bash configure >> 532: --with-conf-name=linux-x64-hotspot > > That does not look a proper config name for cross-compiled jobs? No you are right, I didn't really care since it's not visible anywhere, but can certainly update it a bit. ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From ihse at openjdk.java.net Fri Nov 20 17:30:02 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 20 Nov 2020 17:30:02 GMT Subject: RFR: JDK-8256751: Incremental rebuild with precompiled header fails when touching a header file In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 14:35:27 GMT, Erik Joelsson wrote: > When building with --disable-absolute-paths-in-output and precompiled header enabled (both of which are default in release builds), then incremental builds of Hotspot fails if a header file is touched. > > This is caused by the deps file the compiler generates is now containing relative paths, but those relative paths will not resolve when make imports the file. For all normal compilation units, we post process the file to rewrite these paths to absolute paths, but that call was missed for the precompiled header compilation. Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1351 From shade at openjdk.java.net Fri Nov 20 17:35:06 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 20 Nov 2020 17:35:06 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 16:14:19 GMT, Robin Westberg wrote: >> Currently Linux x64 testing in GitHub Actions depends on a few non-relevant hotspot build-only jobs (such as zero) that prevents testing from being run if those build were to fail. As the tests only require the x64 release and debug builds to run and provide interesting feedback, we should separate these other builds into a different job. >> >> To reduce duplicated steps, all the existing hotspot-only builds have been consolidated into a single job. > > Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: > > Improve naming, fix style issues Looks good! A few minor nits, if you want to address them. .github/workflows/submit.yml line 526: > 524: echo "CC=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}-gcc-10" >> $GITHUB_ENV > 525: echo "CXX=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}-g++-10" >> $GITHUB_ENV > 526: echo "cross_flags=--openjdk-target=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}} --with-sysroot=${HOME}/sysroot-${{ matrix.debian-arch }}/ --with-toolchain-path=${HOME}/sysroot-${{ matrix.debian-arch }}/ --with-freetype-lib=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/lib/${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}/ --with-freetype-include=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/include/freetype2/ --x-libraries=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/lib/${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}/" >> $GITHUB_ENV No chance to break this line by option? It is kinda hard to read and thus verify. It is still okay if we cannot. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1350 From shade at openjdk.java.net Fri Nov 20 17:35:07 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 20 Nov 2020 17:35:07 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 16:09:59 GMT, Robin Westberg wrote: >> .github/workflows/submit.yml line 469: >> >>> 467: name: transient_jdk-linux-x64_${{ needs.prerequisites.outputs.bundle_id }} >>> 468: path: ~/jdk-linux-x64 >>> 469: if: steps.build_restore.outcome == 'failure' >> >> I may be blind, but I don't see any step with id `build_restore` in new file... > > It already existed in the step right above so I didn't have to add it.. This follows the same pattern as other artifact downloads (I've seen them fail a fair amount of time in the past). Right. Got myself confused with GitHub UI. ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From shade at openjdk.java.net Fri Nov 20 17:40:06 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 20 Nov 2020 17:40:06 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 17:31:51 GMT, Aleksey Shipilev wrote: >> Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: >> >> Improve naming, fix style issues > > Looks good! A few minor nits, if you want to address them. Ah wait, I now see "Linux additional" is the column name in testing table, because it is the name of the job! Eh... It was nicer to have columns per arch. Does it make sense to split the "Linux x64 (other)" and "Linux Foreign" then? It might also simplify the coding as we would not have to check for `matrix.debian_arch`. EDIT: Or maybe it is fine to have "Linux additional", as it stand now. Really undecided here :) ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From naoto at openjdk.java.net Fri Nov 20 18:01:14 2020 From: naoto at openjdk.java.net (Naoto Sato) Date: Fri, 20 Nov 2020 18:01:14 GMT Subject: RFR: 8247432: Update IANA Language Subtag Registry to Version 2020-09-29 Message-ID: Hi, Please review the changes to the subject issue. This is to incorporate the latest language subtag registry definition into the JDK. ------------- Commit messages: - LSR 2020-09-29 - LSR 2020-07-17 Changes: https://git.openjdk.java.net/jdk/pull/1357/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1357&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8247432 Stats: 52 lines in 2 files changed: 44 ins; 5 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1357.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1357/head:pull/1357 PR: https://git.openjdk.java.net/jdk/pull/1357 From erikj at openjdk.java.net Fri Nov 20 18:17:01 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Fri, 20 Nov 2020 18:17:01 GMT Subject: Integrated: JDK-8256751: Incremental rebuild with precompiled header fails when touching a header file In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 14:35:27 GMT, Erik Joelsson wrote: > When building with --disable-absolute-paths-in-output and precompiled header enabled (both of which are default in release builds), then incremental builds of Hotspot fails if a header file is touched. > > This is caused by the deps file the compiler generates is now containing relative paths, but those relative paths will not resolve when make imports the file. For all normal compilation units, we post process the file to rewrite these paths to absolute paths, but that call was missed for the precompiled header compilation. This pull request has now been integrated. Changeset: 19b28986 Author: Erik Joelsson URL: https://git.openjdk.java.net/jdk/commit/19b28986 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod 8256751: Incremental rebuild with precompiled header fails when touching a header file Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1351 From joehw at openjdk.java.net Fri Nov 20 21:48:05 2020 From: joehw at openjdk.java.net (Joe Wang) Date: Fri, 20 Nov 2020 21:48:05 GMT Subject: RFR: 8247432: Update IANA Language Subtag Registry to Version 2020-09-29 In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 17:55:55 GMT, Naoto Sato wrote: > Hi, > > Please review the changes to the subject issue. This is to incorporate the latest language subtag registry definition into the JDK. Marked as reviewed by joehw (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1357 From erikj at openjdk.java.net Sat Nov 21 00:18:11 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Sat, 21 Nov 2020 00:18:11 GMT Subject: RFR: JDK-8256810: Incremental rebuild broken on Macosx Message-ID: After fixing JDK-8256751, I noticed that the fix-deps-file macro isn't actually doing the right thing at all. It seems clang on Macosx is inconsistent with outputting relative or absolute paths in the deps files, so some files end up with double WORKSPACE_ROOT in the paths. It's also not handling the trailing slash after WORKSPACE_ROOT correctly, so some files are missing the slash between WORKSPACE_ROOT and the original relative path. This time I have checked the output carefully. ------------- Commit messages: - Rewrite fix-deps-file macro Changes: https://git.openjdk.java.net/jdk/pull/1360/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1360&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256810 Stats: 8 lines in 1 file changed: 5 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1360.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1360/head:pull/1360 PR: https://git.openjdk.java.net/jdk/pull/1360 From mikael at openjdk.java.net Sat Nov 21 00:45:10 2020 From: mikael at openjdk.java.net (Mikael Vidstedt) Date: Sat, 21 Nov 2020 00:45:10 GMT Subject: RFR: JDK-8256810: Incremental rebuild broken on Macosx In-Reply-To: References: Message-ID: On Sat, 21 Nov 2020 00:12:30 GMT, Erik Joelsson wrote: > After fixing JDK-8256751, I noticed that the fix-deps-file macro isn't actually doing the right thing at all. It seems clang on Macosx is inconsistent with outputting relative or absolute paths in the deps files, so some files end up with double WORKSPACE_ROOT in the paths. It's also not handling the trailing slash after WORKSPACE_ROOT correctly, so some files are missing the slash between WORKSPACE_ROOT and the original relative path. > > This time I have checked the output carefully. Marked as reviewed by mikael (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1360 From shade at openjdk.java.net Sun Nov 22 14:24:31 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Sun, 22 Nov 2020 14:24:31 GMT Subject: RFR: 8256829: GNU hash style is not available on MIPS Message-ID: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: collect2: error: ld returned 1 exit status I believe it is a regression in 16, as GNU hash style was forced with [JDK-8200738](https://bugs.openjdk.java.net/browse/JDK-8200738). The way out is to special-case MIPS hash-style to `sysv`. This enumerates all MIPS targets that `make/autoconf/platform.m4` knows about. Testing: - [x] Linux mipsel zero fastdebug build (requires additional unrelated fixes) - [x] Linux mips64el zero fastdebug build ------------- Commit messages: - 8256829: GNU hash style is not available on MIPS Changes: https://git.openjdk.java.net/jdk/pull/1374/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1374&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256829 Stats: 11 lines in 1 file changed: 10 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1374.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1374/head:pull/1374 PR: https://git.openjdk.java.net/jdk/pull/1374 From shade at openjdk.java.net Sun Nov 22 14:45:31 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Sun, 22 Nov 2020 14:45:31 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 Message-ID: If you try to build `linux-mipsel-zero-fastdebug`, this happens: I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. Attention @DamonFool, who must be running into this problem for their MIPS builds? Testing: - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) ------------- Commit messages: - 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 Changes: https://git.openjdk.java.net/jdk/pull/1375/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1375&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256831 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1375.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1375/head:pull/1375 PR: https://git.openjdk.java.net/jdk/pull/1375 From glaubitz at openjdk.java.net Sun Nov 22 14:49:23 2020 From: glaubitz at openjdk.java.net (John Paul Adrian Glaubitz) Date: Sun, 22 Nov 2020 14:49:23 GMT Subject: RFR: 8256829: GNU hash style is not available on MIPS In-Reply-To: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> References: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> Message-ID: On Sun, 22 Nov 2020 14:19:04 GMT, Aleksey Shipilev wrote: > For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: > > > collect2: error: ld returned 1 exit status > > I believe it is a regression in 16, as GNU hash style was forced with [JDK-8200738](https://bugs.openjdk.java.net/browse/JDK-8200738). The way out is to special-case MIPS hash-style to `sysv`. This enumerates all MIPS targets that `make/autoconf/platform.m4` knows about. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel zero fastdebug build (requires additional unrelated fixes) > - [x] Linux mips64el zero fastdebug build Hi Aleksey! > For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: Thanks for looking to fixing the non-mainstream architectures. I'm surprised we don't need the sigset patch anymore for MIPS, looks like that has been upstreamed already. If you are looking into fixing more of such issues, you can have a look at the patches we currently carry in Debian to address various of such issues: > https://git.launchpad.net/~openjdk/ubuntu/+source/openjdk/+git/openjdk/tree/debian/patches I'm currently busy with LLVM but I'm planning to get more active with OpenJDK in the future and get more of such fixes upstreamed. Again, thanks for fixing this. We actually use OpenJDK on MIPS targets in Debian. ------------- PR: https://git.openjdk.java.net/jdk/pull/1374 From shade at openjdk.java.net Sun Nov 22 18:12:41 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Sun, 22 Nov 2020 18:12:41 GMT Subject: Integrated: 8256497: Zero: enable G1 and Shenandoah GCs In-Reply-To: References: Message-ID: On Tue, 17 Nov 2020 19:02:03 GMT, Aleksey Shipilev wrote: > Following the [JDK-8255796](https://bugs.openjdk.java.net/browse/JDK-8255796) improvement that ditched the inline contiguous alloc use from Zero, we can now rely on GC interface to hook the GCs properly. G1 and Shenandoah are a bit special here, because they require special `Reference.get` handling. > > Note that it does not change the default GC for Zero, because Zero is implicitly `NeverActAsServerMachine`, which still selects Serial GC by default. After this change, Zero users can opt-in to G1 or Shenandoah. > > Additional testing: > - [x] Linux x86_64 Zero fastdebug `hotspot_gc_shenandoah` (some lingering failures about non-enabled compressed oops) > - [x] Linux x86_64 Zero fastdebug `tier1` with `-XX:+UseG1GC` This pull request has now been integrated. Changeset: e06a6839 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/e06a6839 Stats: 72 lines in 5 files changed: 54 ins; 16 del; 2 mod 8256497: Zero: enable G1 and Shenandoah GCs Reviewed-by: rkennke, erikj, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1268 From shade at openjdk.java.net Sun Nov 22 18:12:39 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Sun, 22 Nov 2020 18:12:39 GMT Subject: RFR: 8256497: Zero: enable G1 and Shenandoah GCs [v2] In-Reply-To: References: Message-ID: On Thu, 19 Nov 2020 09:55:04 GMT, Magnus Ihse Bursie wrote: >> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: >> >> - Merge branch 'master' into JDK-8256497-zero-g1-shenandoah >> - Remove TODO >> - 8256497: Zero: enable G1 and Shenandoah GCs > > Build changes look good. Thanks! By now I completed more thorough tests with Shenandoah, and those look fine. ------------- PR: https://git.openjdk.java.net/jdk/pull/1268 From jiefu at openjdk.java.net Mon Nov 23 08:23:03 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 23 Nov 2020 08:23:03 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 In-Reply-To: References: Message-ID: On Sun, 22 Nov 2020 14:40:12 GMT, Aleksey Shipilev wrote: > If you try to build `linux-mipsel-zero-fastdebug`, this happens: > > > > > > > > I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. > > Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. > > The alternative would be to massage the otherwise generic Zero code to unimplement the 8-byte CAS. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) Thanks @shipilev for fixing it. I don't have a mips machine at hand. Maybe, people from Loongson can verify this fix. @theaoqi , @xiangzhai Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From jiefu at openjdk.java.net Mon Nov 23 08:26:05 2020 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 23 Nov 2020 08:26:05 GMT Subject: RFR: 8256829: GNU hash style is not available on MIPS In-Reply-To: References: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> Message-ID: <-nulvZ5uSPmCJxv_4gK-C7LAE1Sf9qOfQlpWyxCH0ns=.312236a9-077f-4a81-914e-26e200ec1933@github.com> On Sun, 22 Nov 2020 14:47:07 GMT, John Paul Adrian Glaubitz wrote: >> For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: >> >> >> collect2: error: ld returned 1 exit status >> >> I believe it is a regression in 16, as GNU hash style was forced with [JDK-8200738](https://bugs.openjdk.java.net/browse/JDK-8200738). The way out is to special-case MIPS hash-style to `sysv`. This enumerates all MIPS targets that `make/autoconf/platform.m4` knows about. >> >> Attention @DamonFool, who must be running into this problem for their MIPS builds? >> >> Testing: >> - [x] Linux mipsel zero fastdebug build (requires additional unrelated fixes) >> - [x] Linux mips64el zero fastdebug build > > Hi Aleksey! > >> For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: > > Thanks for looking to fixing the non-mainstream architectures. I'm surprised we don't need the sigset patch anymore for MIPS, looks like that has been upstreamed already. > > If you are looking into fixing more of such issues, you can have a look at the patches we currently carry in Debian to address various of such issues: > >> https://git.launchpad.net/~openjdk/ubuntu/+source/openjdk/+git/openjdk/tree/debian/patches > > I'm currently busy with LLVM but I'm planning to get more active with OpenJDK in the future and get more of such fixes upstreamed. > > Again, thanks for fixing this. We actually use OpenJDK on MIPS targets in Debian. @theaoqi and @xiangzhai would you like to verify this fix? Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/1374 From aoqi at openjdk.java.net Mon Nov 23 08:46:57 2020 From: aoqi at openjdk.java.net (Ao Qi) Date: Mon, 23 Nov 2020 08:46:57 GMT Subject: RFR: 8256829: GNU hash style is not available on MIPS In-Reply-To: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> References: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> Message-ID: On Sun, 22 Nov 2020 14:19:04 GMT, Aleksey Shipilev wrote: > For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: > > > collect2: error: ld returned 1 exit status > > I believe it is a regression in 16, as GNU hash style was forced with [JDK-8200738](https://bugs.openjdk.java.net/browse/JDK-8200738). The way out is to special-case MIPS hash-style to `sysv`. This enumerates all MIPS targets that `make/autoconf/platform.m4` knows about. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel zero fastdebug build (requires additional unrelated fixes) > - [x] Linux mips64el zero fastdebug build @shipilev Thanks for fixing this! I am testing the patch with native builds. When there is a result, I will reply. ------------- PR: https://git.openjdk.java.net/jdk/pull/1374 From burban at openjdk.java.net Mon Nov 23 09:41:04 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 23 Nov 2020 09:41:04 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow Message-ID: This adds the cross-compiled build only, as no Windows+Arm64 machines are available on GitHub Action that we could use to run the tests. Due to cross-compilation a build JDK is required. Initially I added EA builds to be downloaded from https://jdk.java.net/16/ and used for that, but then I saw how @shipiliv attempted it for the linux cross-compilation builds in https://github.com/openjdk/jdk/pull/1147. That is using the JDK image produced by the x64 variant. This however add more stress to the "critical path", as now two more jobs depend on the x64 build first. Let's see how it works out in the long-run. A Windows+AArch64 build takes 40-50min. ------------- Commit messages: - 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow Changes: https://git.openjdk.java.net/jdk/pull/1379/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1379&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256657 Stats: 164 lines in 1 file changed: 163 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1379.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1379/head:pull/1379 PR: https://git.openjdk.java.net/jdk/pull/1379 From rwestberg at openjdk.java.net Mon Nov 23 09:55:11 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 23 Nov 2020 09:55:11 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v3] In-Reply-To: References: Message-ID: > Currently Linux x64 testing in GitHub Actions depends on a few non-relevant hotspot build-only jobs (such as zero) that prevents testing from being run if those build were to fail. As the tests only require the x64 release and debug builds to run and provide interesting feedback, we should separate these other builds into a different job. > > To reduce duplicated steps, all the existing hotspot-only builds have been consolidated into a single job. Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: Split long line ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1350/files - new: https://git.openjdk.java.net/jdk/pull/1350/files/c79e6ab5..75f1e659 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1350&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1350&range=01-02 Stats: 13 lines in 1 file changed: 11 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1350.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1350/head:pull/1350 PR: https://git.openjdk.java.net/jdk/pull/1350 From rwestberg at openjdk.java.net Mon Nov 23 09:57:55 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 23 Nov 2020 09:57:55 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> On Fri, 20 Nov 2020 17:35:12 GMT, Aleksey Shipilev wrote: >> Looks good! A few minor nits, if you want to address them. > > Ah wait, I now see "Linux additional" is the column name in testing table, because it is the name of the job! Eh... It was nicer to have columns per arch. Does it make sense to split the "Linux x64 (other)" and "Linux Foreign" then? It might also simplify the coding as we would not have to check for `matrix.debian_arch`. > > EDIT: Or maybe it is fine to have "Linux additional", as it stand now. Really undecided here :) I personally think I prefer to just use a single column for the additional cross compile builds, because the table becomes very big otherwise (and the rows get split up). You still see the full name of the build in case anything fails, so not sure it adds that much value to get an additional box for each cross-compiled architecture, I guess in 99% of cases they will just pass and then you don't really care exactly which platforms you didn't break.. But I'm no UX expert. :) ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From rwestberg at openjdk.java.net Mon Nov 23 09:57:57 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 23 Nov 2020 09:57:57 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 17:31:08 GMT, Aleksey Shipilev wrote: >> Robin Westberg has updated the pull request incrementally with one additional commit since the last revision: >> >> Improve naming, fix style issues > > .github/workflows/submit.yml line 526: > >> 524: echo "CC=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}-gcc-10" >> $GITHUB_ENV >> 525: echo "CXX=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}-g++-10" >> $GITHUB_ENV >> 526: echo "cross_flags=--openjdk-target=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}} --with-sysroot=${HOME}/sysroot-${{ matrix.debian-arch }}/ --with-toolchain-path=${HOME}/sysroot-${{ matrix.debian-arch }}/ --with-freetype-lib=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/lib/${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}/ --with-freetype-include=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/include/freetype2/ --x-libraries=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/lib/${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}/" >> $GITHUB_ENV > > No chance to break this line by option? It is kinda hard to read and thus verify. It is still okay if we cannot. Sure, I agree, perhaps just putting that as a separate step would increase readability. ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From shade at openjdk.java.net Mon Nov 23 10:01:58 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 10:01:58 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> References: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> Message-ID: On Mon, 23 Nov 2020 09:54:52 GMT, Robin Westberg wrote: >> Ah wait, I now see "Linux additional" is the column name in testing table, because it is the name of the job! Eh... It was nicer to have columns per arch. Does it make sense to split the "Linux x64 (other)" and "Linux Foreign" then? It might also simplify the coding as we would not have to check for `matrix.debian_arch`. >> >> EDIT: Or maybe it is fine to have "Linux additional", as it stand now. Really undecided here :) > > I personally think I prefer to just use a single column for the additional cross compile builds, because the table becomes very big otherwise (and the rows get split up). You still see the full name of the build in case anything fails, so not sure it adds that much value to get an additional box for each cross-compiled architecture, I guess in 99% of cases they will just pass and then you don't really care exactly which platforms you didn't break.. But I'm no UX expert. :) Yeah, bike-shedding aside, "Linux additional" is probably fine. So if we accept #1379, then it would also add "Windows additional"? ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From shade at openjdk.java.net Mon Nov 23 10:02:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 10:02:00 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: Message-ID: On Mon, 23 Nov 2020 09:55:34 GMT, Robin Westberg wrote: >> .github/workflows/submit.yml line 526: >> >>> 524: echo "CC=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}-gcc-10" >> $GITHUB_ENV >>> 525: echo "CXX=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}-g++-10" >> $GITHUB_ENV >>> 526: echo "cross_flags=--openjdk-target=${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}} --with-sysroot=${HOME}/sysroot-${{ matrix.debian-arch }}/ --with-toolchain-path=${HOME}/sysroot-${{ matrix.debian-arch }}/ --with-freetype-lib=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/lib/${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}/ --with-freetype-include=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/include/freetype2/ --x-libraries=${HOME}/sysroot-${{ matrix.debian-arch }}/usr/lib/${{ matrix.gnu-arch }}-linux-gnu${{ matrix.gnu-flavor}}/" >> $GITHUB_ENV >> >> No chance to break this line by option? It is kinda hard to read and thus verify. It is still okay if we cannot. > > Sure, I agree, perhaps just putting that as a separate step would increase readability. The update looks fine. ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From rwestberg at openjdk.java.net Mon Nov 23 10:11:57 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Mon, 23 Nov 2020 10:11:57 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> Message-ID: <8YwwhqUReVn9jy_yPA_6NB0T4AKpIanNe0t0XKI1zV4=.3d43df3f-2754-4a24-b1c5-8da962de9f6e@github.com> On Mon, 23 Nov 2020 09:59:39 GMT, Aleksey Shipilev wrote: >> I personally think I prefer to just use a single column for the additional cross compile builds, because the table becomes very big otherwise (and the rows get split up). You still see the full name of the build in case anything fails, so not sure it adds that much value to get an additional box for each cross-compiled architecture, I guess in 99% of cases they will just pass and then you don't really care exactly which platforms you didn't break.. But I'm no UX expert. :) > > Yeah, bike-shedding aside, "Linux additional" is probably fine. So if we accept #1379, then it would also add "Windows additional"? Ah yeah, just saw that one. I guess just one additional column will still make the table too wide.. Perhaps all the cross builds should just share a single column. :) Going forward, I do think that the table should perhaps be removed though, and the information should instead be moved down to the "checks" box (just as GitHub actions show them natively - but perhaps condense them a bit just as the current table). The constant updating of the table is probably the major source of the "edit war" you have noticed can occur if you want to edit the PR body yourself while any GitHub actions are still in progress. ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From thomas.stuefe at gmail.com Mon Nov 23 11:11:25 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 23 Nov 2020 12:11:25 +0100 Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow In-Reply-To: References: Message-ID: Hi Bernhard, just some drive-by comments. - 40-50 min builds seem to be excessive for just the hotspot build, do you know what exactly takes that long? Is this for release and debug each or both combined? - Is it worth the cycles to build both release *and* debug? How probable is it that a non-win-aarch-dev will break one but not the other? I presume developers on the windows aarch project will have tested the build locally before pushing. - fixpath: Having the gh actions download a patch from your personal repos seems odd I feel at some point we need to have a talk about the growing number of platforms in github actions. E.g. which platforms should be mandatory green before pushing. I also am unsure about the associated cost for each developer (I know atm everyone has a free quota, but that is not limitless. If it is eaten up, strictly speaking one would not be able to push anymore since no submit tests can be ran until the quota is refilled). Cheers, Thomas On Mon, Nov 23, 2020 at 10:41 AM Bernhard Urban-Forster < burban at openjdk.java.net> wrote: > This adds the cross-compiled build only, as no Windows+Arm64 machines are > available on GitHub Action that we could use to run the tests. > > Due to cross-compilation a build JDK is required. Initially I added EA > builds to be downloaded from https://jdk.java.net/16/ and used for that, > but then I saw how @shipiliv attempted it for the linux cross-compilation > builds in https://github.com/openjdk/jdk/pull/1147. That is using the > JDK image produced by the x64 variant. This however add more stress to the > "critical path", as now two more jobs depend on the x64 build first. > > Let's see how it works out in the long-run. A Windows+AArch64 build takes > 40-50min. > > ------------- > > Commit messages: > - 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow > > Changes: https://git.openjdk.java.net/jdk/pull/1379/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1379&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8256657 > Stats: 164 lines in 1 file changed: 163 ins; 0 del; 1 mod > Patch: https://git.openjdk.java.net/jdk/pull/1379.diff > Fetch: git fetch https://git.openjdk.java.net/jdk > pull/1379/head:pull/1379 > > PR: https://git.openjdk.java.net/jdk/pull/1379 > From ihse at openjdk.java.net Mon Nov 23 11:26:58 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 11:26:58 GMT Subject: RFR: 8256829: GNU hash style is not available on MIPS In-Reply-To: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> References: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> Message-ID: On Sun, 22 Nov 2020 14:19:04 GMT, Aleksey Shipilev wrote: > For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: > > > collect2: error: ld returned 1 exit status > > I believe it is a regression in 16, as GNU hash style was forced with [JDK-8200738](https://bugs.openjdk.java.net/browse/JDK-8200738). The way out is to special-case MIPS hash-style to `sysv`. This enumerates all MIPS targets that `make/autoconf/platform.m4` knows about. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel zero fastdebug build (requires additional unrelated fixes) > - [x] Linux mips64el zero fastdebug build Given that it works on MIPS, it looks good to me. Too bad gnu hash style is not universally accepted. :( ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1374 From ihse at openjdk.java.net Mon Nov 23 11:30:55 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 11:30:55 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 In-Reply-To: References: Message-ID: <3XndVjuvdhoOCP6RZJDwpcUmoNcnErBq5WVd_aQdWOs=.ddaa5839-d6a1-4dc4-9920-744584e074f8@github.com> On Sun, 22 Nov 2020 14:40:12 GMT, Aleksey Shipilev wrote: > If you try to build `linux-mipsel-zero-fastdebug`, this happens: > > > > > > > > I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. > > Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. > > The alternative would be to massage the otherwise generic Zero code to unimplement the 8-byte CAS. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) This is incorrect. The `-l` prefix indicates a library to link with. As such, it belongs to LIBS, not LDFLAGS. I'm not sure if we still have a global LIBS variable that is added to all compile lines. We used to have since Solaris Studio required `-lc` for every non-trivial file, but it might have been purged with the Solaris platform. Otherwise this looks like something that belong in LIBJVM LIBS. In fact, if it is *only* needed for the hotspot build, it is really where it belong. And even if it's needed in an additional library or two, it should be added on those places. Only if like more or less all libraries require this should it be added as a global flag. ------------- Changes requested by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1375 From ihse at openjdk.java.net Mon Nov 23 11:36:55 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 11:36:55 GMT Subject: Integrated: 8256240: Reproducible builds should turn on the "deterministic" flag for Visual Studio In-Reply-To: References: Message-ID: On Wed, 18 Nov 2020 10:35:58 GMT, Magnus Ihse Bursie wrote: > The microsoft toolchain now supports a new "deterministic" build mode. This goes a long way towards building properly deterministic. Another important feature is that with the deterministic build mode enabled, the -pathmap flag, similar to gcc's -fmacro-prefix-map, is enabled. This pull request has now been integrated. Changeset: b0485b96 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/b0485b96 Stats: 68 lines in 4 files changed: 48 ins; 14 del; 6 mod 8256240: Reproducible builds should turn on the "deterministic" flag for Visual Studio Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1287 From shade at openjdk.java.net Mon Nov 23 11:39:55 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 11:39:55 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 In-Reply-To: <3XndVjuvdhoOCP6RZJDwpcUmoNcnErBq5WVd_aQdWOs=.ddaa5839-d6a1-4dc4-9920-744584e074f8@github.com> References: <3XndVjuvdhoOCP6RZJDwpcUmoNcnErBq5WVd_aQdWOs=.ddaa5839-d6a1-4dc4-9920-744584e074f8@github.com> Message-ID: On Mon, 23 Nov 2020 11:28:05 GMT, Magnus Ihse Bursie wrote: >> If you try to build `linux-mipsel-zero-fastdebug`, this happens: >> >> >> >> >> >> >> >> I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. >> >> Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. >> >> The alternative would be to massage the otherwise generic Zero code to unimplement the 8-byte CAS. >> >> Attention @DamonFool, who must be running into this problem for their MIPS builds? >> >> Testing: >> - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) > > This is incorrect. The `-l` prefix indicates a library to link with. As such, it belongs to LIBS, not LDFLAGS. > > I'm not sure if we still have a global LIBS variable that is added to all compile lines. We used to have since Solaris Studio required `-lc` for every non-trivial file, but it might have been purged with the Solaris platform. > > Otherwise this looks like something that belong in LIBJVM LIBS. In fact, if it is *only* needed for the hotspot build, it is really where it belong. And even if it's needed in an additional library or two, it should be added on those places. Only if like more or less all libraries require this should it be added as a global flag. > Otherwise this looks like something that belong in LIBJVM LIBS. In fact, if it is _only_ needed for the hotspot build, it is really where it belong. And even if it's needed in an additional library or two, it should be added on those places. Only if like more or less all libraries require this should it be added as a global flag. So, would adding to `BASIC_JVM_LIBS` at `LIB_SETUP_LIBRARIES` step in `make/autoconf/libraries.m4` be a good place then? ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From ihse at openjdk.java.net Mon Nov 23 11:43:53 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 11:43:53 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 In-Reply-To: References: <3XndVjuvdhoOCP6RZJDwpcUmoNcnErBq5WVd_aQdWOs=.ddaa5839-d6a1-4dc4-9920-744584e074f8@github.com> Message-ID: On Mon, 23 Nov 2020 11:37:14 GMT, Aleksey Shipilev wrote: >> This is incorrect. The `-l` prefix indicates a library to link with. As such, it belongs to LIBS, not LDFLAGS. >> >> I'm not sure if we still have a global LIBS variable that is added to all compile lines. We used to have since Solaris Studio required `-lc` for every non-trivial file, but it might have been purged with the Solaris platform. >> >> Otherwise this looks like something that belong in LIBJVM LIBS. In fact, if it is *only* needed for the hotspot build, it is really where it belong. And even if it's needed in an additional library or two, it should be added on those places. Only if like more or less all libraries require this should it be added as a global flag. > >> Otherwise this looks like something that belong in LIBJVM LIBS. In fact, if it is _only_ needed for the hotspot build, it is really where it belong. And even if it's needed in an additional library or two, it should be added on those places. Only if like more or less all libraries require this should it be added as a global flag. > > So, would adding to `BASIC_JVM_LIBS` at `LIB_SETUP_LIBRARIES` step in `make/autoconf/libraries.m4` be a good place then? @shipilev Looks like you found the place! :-) But make sure to build all the JDK libraries as well, to see if it is needed by elsewhere as well. ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From glaubitz at openjdk.java.net Mon Nov 23 11:50:55 2020 From: glaubitz at openjdk.java.net (John Paul Adrian Glaubitz) Date: Mon, 23 Nov 2020 11:50:55 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 In-Reply-To: References: <3XndVjuvdhoOCP6RZJDwpcUmoNcnErBq5WVd_aQdWOs=.ddaa5839-d6a1-4dc4-9920-744584e074f8@github.com> Message-ID: On Mon, 23 Nov 2020 11:41:08 GMT, Magnus Ihse Bursie wrote: >>> Otherwise this looks like something that belong in LIBJVM LIBS. In fact, if it is _only_ needed for the hotspot build, it is really where it belong. And even if it's needed in an additional library or two, it should be added on those places. Only if like more or less all libraries require this should it be added as a global flag. >> >> So, would adding to `BASIC_JVM_LIBS` at `LIB_SETUP_LIBRARIES` step in `make/autoconf/libraries.m4` be a good place then? > > @shipilev Looks like you found the place! :-) > > But make sure to build all the JDK libraries as well, to see if it is needed by elsewhere as well. As a heads-up. I have a working Loongson 2k board at home now (mips64el). So, in case something needs to be tested natively on MIPS, let me know. ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From ihse at openjdk.java.net Mon Nov 23 11:52:00 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 11:52:00 GMT Subject: RFR: JDK-8256810: Incremental rebuild broken on Macosx In-Reply-To: References: Message-ID: On Sat, 21 Nov 2020 00:12:30 GMT, Erik Joelsson wrote: > After fixing JDK-8256751, I noticed that the fix-deps-file macro isn't actually doing the right thing at all. It seems clang on Macosx is inconsistent with outputting relative or absolute paths in the deps files, so some files end up with double WORKSPACE_ROOT in the paths. It's also not handling the trailing slash after WORKSPACE_ROOT correctly, so some files are missing the slash between WORKSPACE_ROOT and the original relative path. > > This time I have checked the output carefully. Regexps really are write-only code. But I think it looks good. :-) ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1360 From ihse at openjdk.java.net Mon Nov 23 11:54:55 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 11:54:55 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow In-Reply-To: References: Message-ID: <4ccydJjOMYyqfMGYtCVAVHDtVYZd78faYIgJ97v3Tqk=.ea318b26-ea02-488a-949e-fb5581fd02df@github.com> On Mon, 23 Nov 2020 09:35:40 GMT, Bernhard Urban-Forster wrote: > This adds the cross-compiled build only, as no Windows+Arm64 machines are available on GitHub Action that we could use to run the tests. > > Due to cross-compilation a build JDK is required. Initially I added EA builds to be downloaded from https://jdk.java.net/16/ and used for that, but then I saw how @shipiliv attempted it for the linux cross-compilation builds in https://github.com/openjdk/jdk/pull/1147. That is using the JDK image produced by the x64 variant. This however add more stress to the "critical path", as now two more jobs depend on the x64 build first. > > Let's see how it works out in the long-run. A Windows+AArch64 build takes 40-50min. I thought windows-aarch64 was not able to build without a patch? Have you fixed that? ------------- PR: https://git.openjdk.java.net/jdk/pull/1379 From ihse at openjdk.java.net Mon Nov 23 11:57:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 11:57:56 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: <8YwwhqUReVn9jy_yPA_6NB0T4AKpIanNe0t0XKI1zV4=.3d43df3f-2754-4a24-b1c5-8da962de9f6e@github.com> References: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> <8YwwhqUReVn9jy_yPA_6NB0T4AKpIanNe0t0XKI1zV4=.3d43df3f-2754-4a24-b1c5-8da962de9f6e@github.com> Message-ID: On Mon, 23 Nov 2020 10:09:08 GMT, Robin Westberg wrote: >> Yeah, bike-shedding aside, "Linux additional" is probably fine. So if we accept #1379, then it would also add "Windows additional"? > > Ah yeah, just saw that one. I guess just one additional column will still make the table too wide.. Perhaps all the cross builds should just share a single column. :) > > Going forward, I do think that the table should perhaps be removed though, and the information should instead be moved down to the "checks" box (just as GitHub actions show them natively - but perhaps condense them a bit just as the current table). The constant updating of the table is probably the major source of the "edit war" you have noticed can occur if you want to edit the PR body yourself while any GitHub actions are still in progress. @rwestberg That sounds very good indeed! I did not think it was possible for Skara to access the Checks area, but if it does, it's really where this belong. ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From shade at openjdk.java.net Mon Nov 23 12:04:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 12:04:00 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow In-Reply-To: References: Message-ID: On Mon, 23 Nov 2020 09:35:40 GMT, Bernhard Urban-Forster wrote: > This adds the cross-compiled build only, as no Windows+Arm64 machines are available on GitHub Action that we could use to run the tests. > > Due to cross-compilation a build JDK is required. Initially I added EA builds to be downloaded from https://jdk.java.net/16/ and used for that, but then I saw how @shipiliv attempted it for the linux cross-compilation builds in https://github.com/openjdk/jdk/pull/1147. That is using the JDK image produced by the x64 variant. This however add more stress to the "critical path", as now two more jobs depend on the x64 build first. > > Let's see how it works out in the long-run. A Windows+AArch64 build takes 40-50min. I think you need to wait for #1350, and then reconcile this patch with that refactoring. .github/workflows/submit.yml line 1250: > 1248: flavor: > 1249: - build release > 1250: - build debug Other foreign arches build only `debug`, thus optimizing the workflow time, drop `release` here? .github/workflows/submit.yml line 1367: > 1365: git apply p.patch > 1366: working-directory: jdk > 1367: shell: bash This should be in the mainline repo instead. Yes, the absence of this patch blocks this PR, but getting that patch into the mainline first is the right thing to do. .github/workflows/submit.yml line 2021: > 2019: - linux_x64_test > 2020: - linux_x86_test > 2021: - windows_aarch64_build This should move to other `_build` needs, for clarity. ------------- Changes requested by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1379 From ihse at openjdk.java.net Mon Nov 23 12:40:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 12:40:57 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow In-Reply-To: References: Message-ID: On Mon, 23 Nov 2020 11:59:53 GMT, Aleksey Shipilev wrote: >> This adds the cross-compiled build only, as no Windows+Arm64 machines are available on GitHub Action that we could use to run the tests. >> >> Due to cross-compilation a build JDK is required. Initially I added EA builds to be downloaded from https://jdk.java.net/16/ and used for that, but then I saw how @shipiliv attempted it for the linux cross-compilation builds in https://github.com/openjdk/jdk/pull/1147. That is using the JDK image produced by the x64 variant. This however add more stress to the "critical path", as now two more jobs depend on the x64 build first. >> >> Let's see how it works out in the long-run. A Windows+AArch64 build takes 40-50min. > > .github/workflows/submit.yml line 1367: > >> 1365: git apply p.patch >> 1366: working-directory: jdk >> 1367: shell: bash > > This should be in the mainline repo instead. Yes, the absence of this patch blocks this PR, but getting that patch into the mainline first is the right thing to do. I agree. We should not have a build action that requires a patch. ------------- PR: https://git.openjdk.java.net/jdk/pull/1379 From james.laskey at oracle.com Mon Nov 23 13:27:31 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 23 Nov 2020 09:27:31 -0400 Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <676865722.1878668.1605655096153.JavaMail.zimbra@u-pem.fr> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <676865722.1878668.1605655096153.JavaMail.zimbra@u-pem.fr> Message-ID: <740D04D2-6FCB-44B7-BC57-6347B27171B5@oracle.com> [Sorry it took so long. Have been on break.] From Guy: Thanks for the forward. Here are my thoughts: Good question from R?mi. If we consider PRNGs to have started at about the time of von Neumann, circa 1946, then I would say that we have been inventing a new category about once every 25 years or so: jumpable, multi-level jumpable, cryptographically secure, splittable. Twenty years ago we would just have one or more levels of jumping/leaping. I think SecureRandom appeared in 2002 (in J2SE 1.4), and the first version of SplittableRandom was in 2014. So I could be wrong, but I really don?t expect to have to add any more interfaces in the next decade or two. I think we will get more benefit from the better type checking than we would get with optional methods. ?Guy > On Nov 17, 2020, at 7:18 PM, Remi Forax wrote: > > An honest question, > why do we need so many interfaces for the different categories of RandomGenerator ? > > My fear is that we are encoding the state of our knowledge of the different kinds of random generators now so it will not be pretty in the future when new categories of random generator are discovered/invented. > If we can take example of the past to predict the future, 20 years ago, what should have been the hierarchy at that time. > Is it not reasonable to think that we will need new kinds of random generator in the future ? > > I wonder if it's not better to have one interface and several optional methods like we have with the collections, it means that we are loosing the possibilities to precisely type a method that only works with a precise type of generator but it will be more future proof. > > R?mi > > ----- Mail original ----- >> De: "Jim Laskey" >> ?: "build-dev" , "core-libs-dev" , >> security-dev at openjdk.java.net >> Envoy?: Mardi 17 Novembre 2020 23:21:18 >> Objet: Re: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] > >>> This PR is to introduce a new random number API for the JDK. The primary API is >>> found in RandomGenerator and RandomGeneratorFactory. Further description can be >>> found in the JEP https://openjdk.java.net/jeps/356 . >> >> Jim Laskey has updated the pull request with a new target base due to a merge or >> a rebase. The pull request now contains 40 commits: >> >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Update package-info.java >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated RandomGeneratorFactory javadoc. >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated documentation for RandomGeneratorFactory. >> - Merge branch 'master' into 8248862 >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Move RandomGeneratorProperty >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Clear up javadoc >> - 8248862; Implement Enhanced Pseudo-Random Number Generators >> >> remove RandomGeneratorProperty from API >> - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 >> >> ------------- >> >> Changes: https://git.openjdk.java.net/jdk/pull/1273/files >> Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1273&range=02 >> Stats: 14891 lines in 31 files changed: 11110 ins; 3704 del; 77 mod >> Patch: https://git.openjdk.java.net/jdk/pull/1273.diff >> Fetch: git fetch https://git.openjdk.java.net/jdk pull/1273/head:pull/1273 >> >> PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Mon Nov 23 15:01:13 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Mon, 23 Nov 2020 15:01:13 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> Message-ID: On Tue, 17 Nov 2020 21:22:28 GMT, Paul Sandoz wrote: >> Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits: >> >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Update package-info.java >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated RandomGeneratorFactory javadoc. >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated documentation for RandomGeneratorFactory. >> - Merge branch 'master' into 8248862 >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Move RandomGeneratorProperty >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Clear up javadoc >> - 8248862; Implement Enhanced Pseudo-Random Number Generators >> >> remove RandomGeneratorProperty from API >> - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 > > src/java.base/share/classes/java/util/Random.java line 592: > >> 590: >> 591: @Override >> 592: public Spliterator.OfInt makeIntsSpliterator(long index, long fence, int origin, int bound) { > > Unsure if this and the other two methods are intended to be public or not, since they are at the end of the class and override methods of a module private class. In principle there is nothing wrong with such `Spliterator` factories, but wonder if they are really needed given the `Stream` returning methods. The arrangement of classes makes it awkward to hide these methods. Re the properties general comment: I moved properties to RandomSupport based on the notion that the SPI work with come later. Re makeIntsSpliterator: These methods aren't exposed in the java.util.Random API I guess no harm done. The only solution I can think of is to create an intermediate implementor, but that leaves the methods exposed as well. > src/java.base/share/classes/java/util/SplittableRandom.java line 171: > >> 169: * RandomGenerator properties. >> 170: */ >> 171: static Map getProperties() { > > With records exiting preview in 16 this map of properties could i think be represented as a record instance, with better type safety, where `RandomSupport.RandomGeneratorProperty` enum values become typed fields declared on the record class. Something to consider after integration perhaps? Yes. > src/java.base/share/classes/java/util/SplittableRandom.java line 211: > >> 209: * http://zimbry.blogspot.com/2011/09/better-bit-mixing-improving-on.html >> 210: */ >> 211: private static long mix64(long z) { > > Usages be replaced with calls to `RandomSupport.mixStafford13`? We were careful to not change the sequences (from fixed seed) generated by existing prngs. This was an edge case. > src/java.base/share/classes/module-info.java line 250: > >> 248: exports jdk.internal.util.xml.impl to >> 249: jdk.jfr; >> 250: exports jdk.internal.util.random; > > Unqualified export, should this be `to jdk.random`? I guess you are right. Until we have a defined SPI we should restrict. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From mullan at openjdk.java.net Mon Nov 23 15:14:07 2020 From: mullan at openjdk.java.net (Sean Mullan) Date: Mon, 23 Nov 2020 15:14:07 GMT Subject: RFR: 8243559: Remove root certificates with 1024-bit keys Message-ID: <-8utnm55fb3iuZpYtA-RbkbARBwlSRN2HORdsRJ2oJc=.3f5b5d9c-665b-4232-8dfd-a7df00f516c4@github.com> This change removes five root certificates with 1024-bit RSA public keys from the system-wide `cacerts` keystore. These are older VeriSign and Thawte root CA certificates which are no longer necessary to retain and should have minimal compatibility risk if removed. See the CSR for more details: https://bugs.openjdk.java.net/browse/JDK-8256502 ------------- Commit messages: - 8256502: Remove root certificates with 1024-bit keys Changes: https://git.openjdk.java.net/jdk/pull/1387/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1387&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8243559 Stats: 140 lines in 6 files changed: 0 ins; 138 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1387.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1387/head:pull/1387 PR: https://git.openjdk.java.net/jdk/pull/1387 From jlaskey at openjdk.java.net Mon Nov 23 15:20:13 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Mon, 23 Nov 2020 15:20:13 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> Message-ID: On Tue, 17 Nov 2020 23:46:12 GMT, Paul Sandoz wrote: >> Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits: >> >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Update package-info.java >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated RandomGeneratorFactory javadoc. >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated documentation for RandomGeneratorFactory. >> - Merge branch 'master' into 8248862 >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Move RandomGeneratorProperty >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Clear up javadoc >> - 8248862; Implement Enhanced Pseudo-Random Number Generators >> >> remove RandomGeneratorProperty from API >> - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 > > src/jdk.random/share/classes/module-info.java line 53: > >> 51: uses RandomSupport; >> 52: >> 53: exports jdk.random to > > Why is this needed? Removing > src/java.base/share/classes/java/util/random/package-info.java line 50: > >> 48: * given its name. >> 49: * >> 50: *

The principal supporting class is {@link RandomGenertatorFactor}. This > > s/RandomGenertatorFactor/RandomGenertatorFactory fixing > src/java.base/share/classes/java/util/random/package-info.java line 140: > >> 138: * >> 139: *

For applications with no special requirements, >> 140: * "L64X128MixRandom" has a good balance among speed, space, > > The documentation assumes that the `jdk.random` module is present in the JDK image. Perhaps we need to spit the specifics to `jdk.random`? But jdk.random isn't really a public API. > src/java.base/share/classes/jdk/internal/util/random/RandomSupport.java line 1211: > >> 1209: Udiff = -Udiff; >> 1210: U2 = U1; >> 1211: U1 -= Udiff; > > Updated `U1` never used (recommend running the code through a checker e.g. use IntelliJ) I noticed that before. I think it's a symmetry thing - will check with Guy. > src/java.base/share/classes/jdk/internal/util/random/RandomSupport.java line 1157: > >> 1155: /* >> 1156: * The tables themselves, as well as a number of associated parameters, are >> 1157: * defined in class java.util.DoubleZigguratTables, which is automatically > > `DoubleZigguratTables` is an inner class of `RandomSupport` Late change fixing. > src/java.base/share/classes/jdk/internal/util/random/RandomSupport.java line 2895: > >> 2893: * distribution: 0.0330 >> 2894: */ >> 2895: static class DoubleZigguratTables { > > make `final` fixing > src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 167: > >> 165: * Return the properties map for the specified provider. >> 166: * >> 167: * @param provider provider to locate. > > Method has no such parameter Fixing ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From jlaskey at openjdk.java.net Mon Nov 23 15:20:01 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Mon, 23 Nov 2020 15:20:01 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> Message-ID: On Mon, 23 Nov 2020 14:57:59 GMT, Jim Laskey wrote: >> src/java.base/share/classes/module-info.java line 250: >> >>> 248: exports jdk.internal.util.xml.impl to >>> 249: jdk.jfr; >>> 250: exports jdk.internal.util.random; >> >> Unqualified export, should this be `to jdk.random`? > > I guess you are right. Until we have a defined SPI we should restrict. On the other hand: public class Random extends AbstractSpliteratorGenerator ^ error: warnings found and -Werror specified public final class SplittableRandom extends AbstractSplittableGenerator { ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From burban at openjdk.java.net Mon Nov 23 15:22:02 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 23 Nov 2020 15:22:02 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow In-Reply-To: References: Message-ID: On Mon, 23 Nov 2020 12:01:14 GMT, Aleksey Shipilev wrote: >> This adds the cross-compiled build only, as no Windows+Arm64 machines are available on GitHub Action that we could use to run the tests. >> >> Due to cross-compilation a build JDK is required. Initially I added EA builds to be downloaded from https://jdk.java.net/16/ and used for that, but then I saw how @shipiliv attempted it for the linux cross-compilation builds in https://github.com/openjdk/jdk/pull/1147. That is using the JDK image produced by the x64 variant. This however add more stress to the "critical path", as now two more jobs depend on the x64 build first. >> >> Let's see how it works out in the long-run. A Windows+AArch64 build takes 40-50min. > > I think you need to wait for #1350, and then reconcile this patch with that refactoring. Thanks for all the comments. > - 40-50 min builds seem to be excessive for just the hotspot build, do youknow what exactly takes that long? Is this for release and debug each or both combined? It's for each of them. Installing a specific version of MSVC and creating the devkit take each ~10min. Thinking about it, there is a opportunity to cache the devkit and do the MSVC installer invocation only if no cached devkit is available. Here is an example run if you want to have a closer look: https://github.com/lewurm/openjdk/runs/1434118318?check_suite_focus=true > - Is it worth the cycles to build both release *and* debug? How probable is it that a non-win-aarch-dev will break one but not the other? I presume developers on the windows aarch project will have tested the build locally before pushing. Fair, I'll remove one of them (as suggested by Aleksey in another comment, I'll keep the debug one). ------------- PR: https://git.openjdk.java.net/jdk/pull/1379 From burban at openjdk.java.net Mon Nov 23 15:22:03 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Mon, 23 Nov 2020 15:22:03 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow In-Reply-To: References: Message-ID: <1eSwWIzelFVtPZT8E737Vxx1TVNP6tkoFHmLVgKBJHc=.859bdfc2-0b03-4f94-98b4-68a1a126408e@github.com> On Mon, 23 Nov 2020 12:37:47 GMT, Magnus Ihse Bursie wrote: >> .github/workflows/submit.yml line 1367: >> >>> 1365: git apply p.patch >>> 1366: working-directory: jdk >>> 1367: shell: bash >> >> This should be in the mainline repo instead. Yes, the absence of this patch blocks this PR, but getting that patch into the mainline first is the right thing to do. > > I agree. We should not have a build action that requires a patch. That patch in its current form is not suitable for upstream. The initial plan was to wait until the "new WINENV" patch lands [1], which would resolve the problem we face with `fixpath.exe`. But maybe I can come up with a solution for the current situation, given that I've a better understanding of the build system by now. [1] https://mail.openjdk.java.net/pipermail/build-dev/2020-July/027872.html ------------- PR: https://git.openjdk.java.net/jdk/pull/1379 From jlaskey at openjdk.java.net Mon Nov 23 15:24:09 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Mon, 23 Nov 2020 15:24:09 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> Message-ID: On Wed, 18 Nov 2020 00:29:36 GMT, Paul Sandoz wrote: >> Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits: >> >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Update package-info.java >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated RandomGeneratorFactory javadoc. >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated documentation for RandomGeneratorFactory. >> - Merge branch 'master' into 8248862 >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Move RandomGeneratorProperty >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Clear up javadoc >> - 8248862; Implement Enhanced Pseudo-Random Number Generators >> >> remove RandomGeneratorProperty from API >> - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 > > src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 173: > >> 171: @SuppressWarnings("unchecked") >> 172: private Map getProperties() { >> 173: if (properties == null) { > > `properties` needs to be marked volatile, and it needs to be assigned at line 182 or line 184. One of them foggy days. ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From weijun at openjdk.java.net Mon Nov 23 15:49:59 2020 From: weijun at openjdk.java.net (Weijun Wang) Date: Mon, 23 Nov 2020 15:49:59 GMT Subject: RFR: 8243559: Remove root certificates with 1024-bit keys In-Reply-To: <-8utnm55fb3iuZpYtA-RbkbARBwlSRN2HORdsRJ2oJc=.3f5b5d9c-665b-4232-8dfd-a7df00f516c4@github.com> References: <-8utnm55fb3iuZpYtA-RbkbARBwlSRN2HORdsRJ2oJc=.3f5b5d9c-665b-4232-8dfd-a7df00f516c4@github.com> Message-ID: On Mon, 23 Nov 2020 15:08:13 GMT, Sean Mullan wrote: > This change removes five root certificates with 1024-bit RSA public keys from the system-wide `cacerts` keystore. These are older VeriSign and Thawte root CA certificates which are no longer necessary to retain and should have minimal compatibility risk if removed. > > See the CSR for more details: https://bugs.openjdk.java.net/browse/JDK-8256502 Marked as reviewed by weijun (Reviewer). Looks fine. One nit: I see that the `VerifyCACerts.java` test has a whole bunch of `@bug` ids. Maybe we should add this new one as well? ------------- PR: https://git.openjdk.java.net/jdk/pull/1387 From ihse at openjdk.java.net Mon Nov 23 15:59:07 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 23 Nov 2020 15:59:07 GMT Subject: RFR: 8253757: Add LLVM-based backend for hsdis In-Reply-To: <9oXnHULCd76_J69CKMVVZl3FfDte1pnt38y06LVV4Sg=.26a4ab2c-5ff7-4e2f-9428-0d8cd931d243@github.com> References: <91erxiMDb4ftvSomuJYHPi9SX-v8Z2VLD2qEwCbz5tk=.b9ed01b5-f0e0-4ed7-9c1a-b06bc0e64640@github.com> <8Eqswd7tsVaGEXHdKDncXqKpW2tBsSeuY0PV6aTB9_c=.a6cf4957-9d31-4e89-bf44-e7b7852205d5@github.com> <2S00ucaPGiAQLeLOejt1kfXeYEc7ctEPeRCIcq1N0N8=.dbf1ea7a-8de4-48a5-8759-03495e3e3c08@github.com> <9oXnHULCd76_J69CKMVVZl3FfDte1pnt38y06LVV4Sg=.26a4ab2c-5ff7-4e2f-9428-0d8cd931d243@github.com> Message-ID: On Mon, 26 Oct 2020 11:41:16 GMT, Magnus Ihse Bursie wrote: >> Some notes (perhaps most to myself) about how this ties into the existing hsdis implementation, and with JDK-8188073 (Capstone porting). >> >> When printing disassembly from hotspot, the current solution tries to locate and load the hsdis library, which prints disassembly using bfd. The reason for using the separate library approach is, as far as I can understand, perhaps a mix of both incompatible licensing for bfd, and a wish to not burden the jvm library with additional bloat which is needed only for debugging. >> >> The Capstone approach, in the prototype patch presented by Jorn in the issue, is to create a new capstonedis library, and dispatch to it instead of hsdis. >> >> The approach used in this patch is to refactor the existing hsdis library into an abstract base class for hsdis backends, with two concrete implementations, one for bfd and one for llvm. >> >> Unfortunately, I think the resulting code in hsdis.cpp in this patch is hard to read and understand. > > I think a proper solution to both this and the Capstone implementation is to provide a proper framework for selecting the hsdis backend as a first step, and refactor the existing bfd implementation as the first such backend. After that, we can add llvm and capstone as alternative hsdis backend implementations. FWIW, I started working on a framework which would add support for selectable backends for hsdis. Unfortunately it was not as simple as I initially thought, so I had to put it on hold while directing my time to working on the winenv patch instead. I believe the proper way forward is to get the "selectable hsdis backend" framework in place, and then retrofit this patch to add LLVM support in that framework. If this means that this PR should be closed, or kept open until this is done, I don't know. ------------- PR: https://git.openjdk.java.net/jdk/pull/392 From naoto at openjdk.java.net Mon Nov 23 16:44:56 2020 From: naoto at openjdk.java.net (Naoto Sato) Date: Mon, 23 Nov 2020 16:44:56 GMT Subject: Integrated: 8247432: Update IANA Language Subtag Registry to Version 2020-09-29 In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 17:55:55 GMT, Naoto Sato wrote: > Hi, > > Please review the changes to the subject issue. This is to incorporate the latest language subtag registry definition into the JDK. This pull request has now been integrated. Changeset: ae0ca743 Author: Naoto Sato URL: https://git.openjdk.java.net/jdk/commit/ae0ca743 Stats: 52 lines in 2 files changed: 44 ins; 5 del; 3 mod 8247432: Update IANA Language Subtag Registry to Version 2020-09-29 Reviewed-by: joehw ------------- PR: https://git.openjdk.java.net/jdk/pull/1357 From aoqi at openjdk.java.net Mon Nov 23 16:54:54 2020 From: aoqi at openjdk.java.net (Ao Qi) Date: Mon, 23 Nov 2020 16:54:54 GMT Subject: RFR: 8256829: GNU hash style is not available on MIPS In-Reply-To: References: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> Message-ID: On Mon, 23 Nov 2020 11:23:55 GMT, Magnus Ihse Bursie wrote: >> For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: >> >> >> collect2: error: ld returned 1 exit status >> >> I believe it is a regression in 16, as GNU hash style was forced with [JDK-8200738](https://bugs.openjdk.java.net/browse/JDK-8200738). The way out is to special-case MIPS hash-style to `sysv`. This enumerates all MIPS targets that `make/autoconf/platform.m4` knows about. >> >> Attention @DamonFool, who must be running into this problem for their MIPS builds? >> >> Testing: >> - [x] Linux mipsel zero fastdebug build (requires additional unrelated fixes) >> - [x] Linux mips64el zero fastdebug build > > Given that it works on MIPS, it looks good to me. Too bad gnu hash style is not universally accepted. :( Sorry for the late reply. Monday is a busy day and I used a zero jdk as boot jdk, so the test took some time. The native build of linux-mips64el-zero-release has passed with this patch. I noticed that there is some [work of MIPS support for GNU hash](https://sourceware.org/pipermail/binutils/2019-July/107496.html), but I don't know the latest progress. I tried [debian:latest images](https://hub.docker.com/_/debian), GNU hash is not supported yet. I think we should keep this patch until MIPS support for GUN hash is integrated into downstream toolchains one day. @glaubitz , FYI, yes, the sigset problem was [solved](https://hg.openjdk.java.net/jdk/jdk/rev/3086f9259e97) :) ------------- PR: https://git.openjdk.java.net/jdk/pull/1374 From darcy at openjdk.java.net Mon Nov 23 17:23:16 2020 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 23 Nov 2020 17:23:16 GMT Subject: RFR: 8251549: Update docs on building for Git [v2] In-Reply-To: References: Message-ID: > 8251549: Update docs on building for Git Joe Darcy has updated the pull request incrementally with one additional commit since the last revision: Update doc/building.md Co-authored-by: Magnus Ihse Bursie ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/21/files - new: https://git.openjdk.java.net/jdk/pull/21/files/185b04d4..91d91125 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=21&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=21&range=00-01 Stats: 15 lines in 1 file changed: 14 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/21.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/21/head:pull/21 PR: https://git.openjdk.java.net/jdk/pull/21 From darcy at openjdk.java.net Mon Nov 23 17:47:58 2020 From: darcy at openjdk.java.net (Joe Darcy) Date: Mon, 23 Nov 2020 17:47:58 GMT Subject: Integrated: 8251549: Update docs on building for Git In-Reply-To: References: Message-ID: On Sat, 5 Sep 2020 20:13:18 GMT, Joe Darcy wrote: > 8251549: Update docs on building for Git This pull request has now been integrated. Changeset: 042734cc Author: Joe Darcy URL: https://git.openjdk.java.net/jdk/commit/042734cc Stats: 68 lines in 1 file changed: 12 ins; 37 del; 19 mod 8251549: Update docs on building for Git Reviewed-by: lancea ------------- PR: https://git.openjdk.java.net/jdk/pull/21 From shade at openjdk.java.net Mon Nov 23 19:37:54 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 19:37:54 GMT Subject: RFR: 8256829: GNU hash style is not available on MIPS In-Reply-To: References: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> Message-ID: On Mon, 23 Nov 2020 16:52:07 GMT, Ao Qi wrote: >> Given that it works on MIPS, it looks good to me. Too bad gnu hash style is not universally accepted. :( > > Sorry for the late reply. Monday is a busy day and I used a zero jdk as boot jdk, so the test took some time. The native build of linux-mips64el-zero-release has passed with this patch. > > I noticed that there is some [work of MIPS support for GNU hash](https://sourceware.org/pipermail/binutils/2019-July/107496.html), but I don't know the latest progress. I tried [debian:latest images](https://hub.docker.com/_/debian), GNU hash is not supported yet. I think we should keep this patch until MIPS support for GUN hash is integrated into downstream toolchains one day. > > @glaubitz , FYI, yes, the sigset problem was [solved](https://hg.openjdk.java.net/jdk/jdk/rev/3086f9259e97) :) Thanks folks! I rechecked MIPS Zero still builds after today's whirlwind. ------------- PR: https://git.openjdk.java.net/jdk/pull/1374 From shade at openjdk.java.net Mon Nov 23 19:37:56 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 19:37:56 GMT Subject: Integrated: 8256829: GNU hash style is not available on MIPS In-Reply-To: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> References: <6VCwF6acyS0FUIkkdHaVLufHnkzqcziKBvfCT4f_ISg=.8e318d88-fe23-4e2d-9fac-9cee33b5af17@github.com> Message-ID: On Sun, 22 Nov 2020 14:19:04 GMT, Aleksey Shipilev wrote: > For fun, I tried to build `linux-mips64el-zero-fastdebug`, and it cannot be built, because linker complains: > > > collect2: error: ld returned 1 exit status > > I believe it is a regression in 16, as GNU hash style was forced with [JDK-8200738](https://bugs.openjdk.java.net/browse/JDK-8200738). The way out is to special-case MIPS hash-style to `sysv`. This enumerates all MIPS targets that `make/autoconf/platform.m4` knows about. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel zero fastdebug build (requires additional unrelated fixes) > - [x] Linux mips64el zero fastdebug build This pull request has now been integrated. Changeset: 47a78e8a Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/47a78e8a Stats: 11 lines in 1 file changed: 10 ins; 0 del; 1 mod 8256829: GNU hash style is not available on MIPS Reviewed-by: ihse, aoqi, glaubitz ------------- PR: https://git.openjdk.java.net/jdk/pull/1374 From shade at openjdk.java.net Mon Nov 23 19:58:08 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 19:58:08 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v2] In-Reply-To: References: Message-ID: > If you try to build `linux-mipsel-zero-fastdebug`, this happens: > > > > > > > > I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. > > Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. > > The alternative would be to massage the otherwise generic Zero code to unimplement the 8-byte CAS. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: - Move the library addition to LIB_SETUP_LIBRARIES - Merge branch 'master' into JDK-8256831-mips-8-byte-cas - 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 ------------- Changes: https://git.openjdk.java.net/jdk/pull/1375/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1375&range=01 Stats: 8 lines in 1 file changed: 8 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1375.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1375/head:pull/1375 PR: https://git.openjdk.java.net/jdk/pull/1375 From shade at openjdk.java.net Mon Nov 23 19:58:08 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 19:58:08 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v2] In-Reply-To: References: <3XndVjuvdhoOCP6RZJDwpcUmoNcnErBq5WVd_aQdWOs=.ddaa5839-d6a1-4dc4-9920-744584e074f8@github.com> Message-ID: <9gNoZlZq13n-MrPULBWYdh1Q3IwLPNuBDkTQ9DAhbtU=.937092fb-0078-404f-83aa-67b88309ba64@github.com> On Mon, 23 Nov 2020 11:37:14 GMT, Aleksey Shipilev wrote: > So, would adding to `BASIC_JVM_LIBS` at `LIB_SETUP_LIBRARIES` step in `make/autoconf/libraries.m4` be a good place then? This seems to work. It would seem only JVM is needed to be built with `-latomic`, which makes sense, as the problematic use is in VM code). ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From shade at openjdk.java.net Mon Nov 23 19:58:09 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 19:58:09 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v2] In-Reply-To: References: <3XndVjuvdhoOCP6RZJDwpcUmoNcnErBq5WVd_aQdWOs=.ddaa5839-d6a1-4dc4-9920-744584e074f8@github.com> Message-ID: <6L6CauE-TtVG6v0zcnjRSDB1_VigfbUXUyb4zYwGqCg=.cec5aef0-d4cc-4977-8949-c8a2a721da5c@github.com> On Mon, 23 Nov 2020 11:48:27 GMT, John Paul Adrian Glaubitz wrote: > As a heads-up. I have a working Loongson 2k board at home now (mips64el). So, in case something needs to be tested natively on MIPS, let me know. Oh, cool! Can it build `mipsel`, not only `mips64el`? If so, please try to compile `linux-mipsel-zero-fastdebug` there? ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From shade at openjdk.java.net Mon Nov 23 22:35:05 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 23 Nov 2020 22:35:05 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v3] In-Reply-To: References: Message-ID: <9ocjw2dgAzWNw4lLpGvN6c1xrVYSF3QfK4X_H5Z2aG8=.87b9c9b5-55c2-48cf-af1d-91e47cd4ad38@github.com> > If you try to build `linux-mipsel-zero-fastdebug`, this happens: > > > > > > > > I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. > > Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. > > The alternative would be to massage the otherwise generic Zero code to unimplement the 8-byte CAS. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - Merge branch 'master' into JDK-8256831-mips-8-byte-cas - Move the library addition to LIB_SETUP_LIBRARIES - Merge branch 'master' into JDK-8256831-mips-8-byte-cas - 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 ------------- Changes: https://git.openjdk.java.net/jdk/pull/1375/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1375&range=02 Stats: 8 lines in 1 file changed: 8 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1375.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1375/head:pull/1375 PR: https://git.openjdk.java.net/jdk/pull/1375 From ihse at openjdk.java.net Tue Nov 24 07:16:05 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 24 Nov 2020 07:16:05 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v3] In-Reply-To: <9ocjw2dgAzWNw4lLpGvN6c1xrVYSF3QfK4X_H5Z2aG8=.87b9c9b5-55c2-48cf-af1d-91e47cd4ad38@github.com> References: <9ocjw2dgAzWNw4lLpGvN6c1xrVYSF3QfK4X_H5Z2aG8=.87b9c9b5-55c2-48cf-af1d-91e47cd4ad38@github.com> Message-ID: On Mon, 23 Nov 2020 22:35:05 GMT, Aleksey Shipilev wrote: >> If you try to build `linux-mipsel-zero-fastdebug`, this happens: >> >> >> >> >> >> >> >> I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. >> >> Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. >> >> The alternative would be to massage the otherwise generic Zero code to unimplement the 8-byte CAS. >> >> Attention @DamonFool, who must be running into this problem for their MIPS builds? >> >> Testing: >> - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: > > - Merge branch 'master' into JDK-8256831-mips-8-byte-cas > - Move the library addition to LIB_SETUP_LIBRARIES > - Merge branch 'master' into JDK-8256831-mips-8-byte-cas > - 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From jfranck at openjdk.java.net Tue Nov 24 09:57:58 2020 From: jfranck at openjdk.java.net (Joel =?UTF-8?B?Qm9yZ2dyw6luLUZyYW5jaw==?=) Date: Tue, 24 Nov 2020 09:57:58 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v6] In-Reply-To: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> References: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> Message-ID: On Mon, 16 Nov 2020 12:44:08 GMT, Magnus Ihse Bursie wrote: >> Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. >> >> Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. >> >> So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. >> >> Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. >> >> I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) > > Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace issues Marked as reviewed by jfranck (Reviewer). src/jdk.compiler/share/classes/com/sun/tools/sjavac/Util.java line 96: > 94: > 95: public static String extractStringOptionLine(String opName, String s, String deflt) { > 96: return extractStringOptionWithDelimiter(opName, s, deflt, '\n').strip(); Is '\n' going to be problematic due to differences in line endings on various platforms? ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From chegar at openjdk.java.net Tue Nov 24 10:06:09 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Tue, 24 Nov 2020 10:06:09 GMT Subject: RFR: 8256755: Update build.tools.depend.Depend to handle record components in API signatures Message-ID: <3lErqjFlNDVjsKubxd0HVv-PaL100WLOObgSQTXm3m8=.dabdfd0a-d1a2-407e-8172-ce2d1028ced8@github.com> The `Depend` build tool creates a hash of a module's API elements, so that it can determine if downstream modules require recompilation. The build tool fails (throws an exception) when it encounters an "unknown" record attribute/component - the build tool predates records. The components of a public record class, that is a member of an exported package, are logically part of the module's API. The name, type, and order are significant. This issue updates the build tool to include the aforementioned component characteristics. This issue is a blocker to adding any public record types to the JDK - since the build will fail. ------------- Commit messages: - Update the Depend(ency) build tool to handle the record attribute Changes: https://git.openjdk.java.net/jdk/pull/1405/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1405&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256755 Stats: 33 lines in 2 files changed: 31 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1405.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1405/head:pull/1405 PR: https://git.openjdk.java.net/jdk/pull/1405 From jlahoda at openjdk.java.net Tue Nov 24 10:13:56 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Tue, 24 Nov 2020 10:13:56 GMT Subject: RFR: 8256755: Update build.tools.depend.Depend to handle record components in API signatures In-Reply-To: <3lErqjFlNDVjsKubxd0HVv-PaL100WLOObgSQTXm3m8=.dabdfd0a-d1a2-407e-8172-ce2d1028ced8@github.com> References: <3lErqjFlNDVjsKubxd0HVv-PaL100WLOObgSQTXm3m8=.dabdfd0a-d1a2-407e-8172-ce2d1028ced8@github.com> Message-ID: <3Qt6HXmLkLPC3EEKz1NZa00iwS8z2c3bzHTbszLg3n0=.b8798de1-2b51-44b4-8e73-403de15ee98b@github.com> On Tue, 24 Nov 2020 09:59:44 GMT, Chris Hegarty wrote: > The `Depend` build tool creates a hash of a module's API elements, so that it can determine if downstream modules require recompilation. The build tool fails (throws an exception) when it encounters an "unknown" record attribute/component - the build tool predates records. > > The components of a public record class, that is a member of an exported package, are logically part of the module's API. The name, type, and order are significant. This issue updates the build tool to include the aforementioned component characteristics. > > This issue is a blocker to adding any public record types to the JDK - since the build will fail. Looks great, thanks! ------------- Marked as reviewed by jlahoda (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1405 From ihse at openjdk.java.net Tue Nov 24 11:17:06 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 24 Nov 2020 11:17:06 GMT Subject: Integrated: 8256308: Send arguments to javac server in a config file In-Reply-To: References: Message-ID: On Thu, 12 Nov 2020 22:13:37 GMT, Magnus Ihse Bursie wrote: > Currently, to use the javac server, a horrendously long command line option is created, looking like this: `--server:portfile=:sjavac=`, where the sjavac command has had all spaces replaced by %20. Since Project Jigsaw, the set of module arguments needed is huge to begin with, making this command line incomprehensible after mangling. > > Apart from making java command lines hard to read (and copy/paste!) by developers, it also makes it hard for scripts to parse. The upcoming winenv rewrite is dependent on being able to differentiate between path names and other arguments, which is not possible in this mess. > > So, instead, let's write it to a file, without any escaping, and just pass the configuration file name to the server. > > Note that this will change the behavior of the javac server, but as the source code states this is not a documented or externally supported API no CSR is needed. > > I also cleaned up some code in SjavacClient, in particular code relating to the passing of arguments. (We never change poolsize or keepalive when we call it.) This pull request has now been integrated. Changeset: 9e4944f7 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/9e4944f7 Stats: 136 lines in 3 files changed: 59 ins; 40 del; 37 mod 8256308: Send arguments to javac server in a config file Reviewed-by: erikj, jfranck ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From ihse at openjdk.java.net Tue Nov 24 11:17:05 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Tue, 24 Nov 2020 11:17:05 GMT Subject: RFR: 8256308: Send arguments to javac server in a config file [v6] In-Reply-To: References: <7NBBom9s8IWvbkdHOFdk4-uaLeVqobEInNq4g2jL8U4=.c180727d-4fdf-444b-b748-a9854e4115ee@github.com> Message-ID: On Tue, 24 Nov 2020 09:52:34 GMT, Joel Borggr?n-Franck wrote: >> Magnus Ihse Bursie has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix whitespace issues > > src/jdk.compiler/share/classes/com/sun/tools/sjavac/Util.java line 96: > >> 94: >> 95: public static String extractStringOptionLine(String opName, String s, String deflt) { >> 96: return extractStringOptionWithDelimiter(opName, s, deflt, '\n').strip(); > > Is '\n' going to be problematic due to differences in line endings on various platforms? The JDK project is limited to unix-style line endings only. No other styles are supported, and any attempt to use such will break the build (often early on). ------------- PR: https://git.openjdk.java.net/jdk/pull/1195 From darcy at openjdk.java.net Tue Nov 24 21:58:03 2020 From: darcy at openjdk.java.net (Joe Darcy) Date: Tue, 24 Nov 2020 21:58:03 GMT Subject: RFR: 8253753 Enable default constructor warning in client modules Message-ID: With the default constructors warnings in java.desktop and jdk.accessibility now fixed, the warning should be enabled in the build for those modules. ------------- Commit messages: - Merge branch 'JDK-8253753' of https://github.com/jddarcy/jdk into JDK-8253753 - Update build to enable default constructor warning in client modules. - Update build to enable default constructor warning in client modules. Changes: https://git.openjdk.java.net/jdk/pull/1420/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1420&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8253753 Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1420.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1420/head:pull/1420 PR: https://git.openjdk.java.net/jdk/pull/1420 From prr at openjdk.java.net Tue Nov 24 22:05:56 2020 From: prr at openjdk.java.net (Phil Race) Date: Tue, 24 Nov 2020 22:05:56 GMT Subject: RFR: 8253753 Enable default constructor warning in client modules In-Reply-To: References: Message-ID: On Tue, 24 Nov 2020 19:10:04 GMT, Joe Darcy wrote: > With the default constructors warnings in java.desktop and jdk.accessibility now fixed, the warning should be enabled in the build for those modules. I tried this myself a few weeks ago and it was fine. ------------- Marked as reviewed by prr (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1420 From jlaskey at openjdk.java.net Tue Nov 24 22:31:08 2020 From: jlaskey at openjdk.java.net (Jim Laskey) Date: Tue, 24 Nov 2020 22:31:08 GMT Subject: RFR: 8248862: Implement Enhanced Pseudo-Random Number Generators [v3] In-Reply-To: <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> References: <17ZVd3Hqa05Hm2lbHEiXi38RXG__C09KL0QLBqxsb0g=.ef1bca7a-8ec3-4cae-b457-680e739887ea@github.com> <6h9eeR6HFk03eD3DCtXjJo9Wg81zWH1090paXb-Yp1k=.da9e5ecd-377a-488f-a2f6-b43876af6f60@github.com> <2I2HK8tDFSLtPqs0JFLKLgh3QmMTkM1HKuvKQNmsfIg=.a383a449-18e3-413a-a6ba-91dc80fde86a@github.com> Message-ID: On Wed, 18 Nov 2020 00:30:53 GMT, Paul Sandoz wrote: >> Jim Laskey has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 40 commits: >> >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Update package-info.java >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated RandomGeneratorFactory javadoc. >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Updated documentation for RandomGeneratorFactory. >> - Merge branch 'master' into 8248862 >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Move RandomGeneratorProperty >> - Merge branch 'master' into 8248862 >> - 8248862: Implement Enhanced Pseudo-Random Number Generators >> >> Clear up javadoc >> - 8248862; Implement Enhanced Pseudo-Random Number Generators >> >> remove RandomGeneratorProperty from API >> - ... and 30 more: https://git.openjdk.java.net/jdk/compare/f7517386...6fe94c68 > > src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 148: > >> 146: */ >> 147: private static Map> getFactoryMap() { >> 148: if (factoryMap == null) { > > `factoryMap` needs to be marked volatile when using the double checked locking idiom. fixing > src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 320: > >> 318: } >> 319: } >> 320: } > > Add an `assert` statement that `ctor`, `ctorLong` and `ctorBytes` are all non-null? Only `ctor` is required but yes. > src/java.base/share/classes/java/util/random/RandomGeneratorFactory.java line 331: > >> 329: */ >> 330: private void ensureConstructors() { >> 331: if (ctor == null) { > > This check occurs outside of the synchronized block, field may need to be marked volatile. Unsure about the other dependent fields. Might need to store values from loop in `getConstructors` in locals and then assign in appropriate order, assigning the volatile field last. okay ------------- PR: https://git.openjdk.java.net/jdk/pull/1273 From serb at openjdk.java.net Tue Nov 24 23:09:57 2020 From: serb at openjdk.java.net (Sergey Bylokhov) Date: Tue, 24 Nov 2020 23:09:57 GMT Subject: RFR: 8253753 Enable default constructor warning in client modules In-Reply-To: References: Message-ID: On Tue, 24 Nov 2020 19:10:04 GMT, Joe Darcy wrote: > With the default constructors warnings in java.desktop and jdk.accessibility now fixed, the warning should be enabled in the build for those modules. Marked as reviewed by serb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1420 From rwestberg at openjdk.java.net Wed Nov 25 08:31:59 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Wed, 25 Nov 2020 08:31:59 GMT Subject: Integrated: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing In-Reply-To: References: Message-ID: On Fri, 20 Nov 2020 14:24:12 GMT, Robin Westberg wrote: > Currently Linux x64 testing in GitHub Actions depends on a few non-relevant hotspot build-only jobs (such as zero) that prevents testing from being run if those build were to fail. As the tests only require the x64 release and debug builds to run and provide interesting feedback, we should separate these other builds into a different job. > > To reduce duplicated steps, all the existing hotspot-only builds have been consolidated into a single job. This pull request has now been integrated. Changeset: c45725e5 Author: Robin Westberg URL: https://git.openjdk.java.net/jdk/commit/c45725e5 Stats: 517 lines in 1 file changed: 34 ins; 429 del; 54 mod 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From thomas.stuefe at gmail.com Wed Nov 25 08:45:16 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 25 Nov 2020 09:45:16 +0100 Subject: gh actions fail on linux x64 when fetching libsound Message-ID: Hi, all my linux x64 gh builds fail with: "E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb 404 Not Found [IP: 52.147.219.192 80]" https://github.com/tstuefe/jdk/runs/1452221121 Does anyone have an idea? Thanks, Thomas From shade at openjdk.java.net Wed Nov 25 08:45:59 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 25 Nov 2020 08:45:59 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v3] In-Reply-To: References: <9ocjw2dgAzWNw4lLpGvN6c1xrVYSF3QfK4X_H5Z2aG8=.87b9c9b5-55c2-48cf-af1d-91e47cd4ad38@github.com> Message-ID: <5TXFPcFMqqPj2TKkbXb2PwTJEPEDJLzYKZaH_dtlBno=.2435c211-847c-4a47-98ae-9af6144f37cd@github.com> On Tue, 24 Nov 2020 07:13:27 GMT, Magnus Ihse Bursie wrote: >> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: >> >> - Merge branch 'master' into JDK-8256831-mips-8-byte-cas >> - Move the library addition to LIB_SETUP_LIBRARIES >> - Merge branch 'master' into JDK-8256831-mips-8-byte-cas >> - 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 > > Marked as reviewed by ihse (Reviewer). Thanks, I'll wait for either of @glaubitz, @theaoqi or @xiangzhai to confirm this fixes the native `mipsel` builds for them, and then integrate. ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From shade at redhat.com Wed Nov 25 08:49:38 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 25 Nov 2020 09:49:38 +0100 Subject: gh actions fail on linux x64 when fetching libsound In-Reply-To: References: Message-ID: <37d94600-c6f8-06fc-cc56-4baf71615c64@redhat.com> On 11/25/20 9:45 AM, Thomas St?fe wrote: > "E: Failed to fetch > http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb > 404 Not Found [IP: 52.147.219.192 80]" > > https://github.com/tstuefe/jdk/runs/1452221121 > > Does anyone have an idea? I think this usually means the local aptitude index files are outdated. I suspect we have to do "sudo apt-get update" before "sudo apt-get install". Robin, would you like to do that, or should I? -- Thanks, -Aleksey From shade at redhat.com Wed Nov 25 08:55:07 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 25 Nov 2020 09:55:07 +0100 Subject: gh actions fail on linux x64 when fetching libsound In-Reply-To: <37d94600-c6f8-06fc-cc56-4baf71615c64@redhat.com> References: <37d94600-c6f8-06fc-cc56-4baf71615c64@redhat.com> Message-ID: <1ed12224-1e63-03c7-1f73-80b27f4f285d@redhat.com> On 11/25/20 9:49 AM, Aleksey Shipilev wrote: > On 11/25/20 9:45 AM, Thomas St?fe wrote: >> "E: Failed to fetch >> http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb >> 404 Not Found [IP: 52.147.219.192 80]" >> >> https://github.com/tstuefe/jdk/runs/1452221121 >> >> Does anyone have an idea? > > I think this usually means the local aptitude index files are outdated. I suspect we have to do > "sudo apt-get update" before "sudo apt-get install". Robin, would you like to do that, or should I? Doing the update here: https://bugs.openjdk.java.net/browse/JDK-8257056 -- Thanks, -Aleksey From robin.westberg at oracle.com Wed Nov 25 08:58:12 2020 From: robin.westberg at oracle.com (Robin Westberg) Date: Wed, 25 Nov 2020 09:58:12 +0100 Subject: gh actions fail on linux x64 when fetching libsound In-Reply-To: <37d94600-c6f8-06fc-cc56-4baf71615c64@redhat.com> References: <37d94600-c6f8-06fc-cc56-4baf71615c64@redhat.com> Message-ID: Hi Aleksey, > On 25 Nov 2020, at 09:49, Aleksey Shipilev wrote: > > On 11/25/20 9:45 AM, Thomas St?fe wrote: >> "E: Failed to fetch >> http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb >> 404 Not Found [IP: 52.147.219.192 80]" >> https://github.com/tstuefe/jdk/runs/1452221121 >> Does anyone have an idea? > > I think this usually means the local aptitude index files are outdated. I suspect we have to do "sudo apt-get update" before "sudo apt-get install". Robin, would you like to do that, or should I? Yeah I agree, looks like we?re missing an update line for the x64 build (and the Linux additional builds too I think). If you want to fix it, feel free. :) Best regards, Robin > > -- > Thanks, > -Aleksey > From shade at openjdk.java.net Wed Nov 25 09:01:04 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 25 Nov 2020 09:01:04 GMT Subject: RFR: 8257056: Submit workflow should apt-get update to avoid package installation errors Message-ID: For example, current jobs fail with: Get:13 http://azure.archive.ubuntu.com/ubuntu focal/main amd64 libxtst-dev amd64 2:1.2.3-1 [15.2 kB] E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb 404 Not Found [IP: 52.147.219.192 80] Fetched 760 kB in 0s (3154 kB/s) E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ------------- Commit messages: - 8257056: Submit workflow should apt-get update to avoid package installation errors Changes: https://git.openjdk.java.net/jdk/pull/1427/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1427&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8257056 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/1427.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1427/head:pull/1427 PR: https://git.openjdk.java.net/jdk/pull/1427 From rwestberg at openjdk.java.net Wed Nov 25 09:13:57 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Wed, 25 Nov 2020 09:13:57 GMT Subject: RFR: 8257056: Submit workflow should apt-get update to avoid package installation errors In-Reply-To: References: Message-ID: On Wed, 25 Nov 2020 08:56:33 GMT, Aleksey Shipilev wrote: > For example, current jobs fail with: > > Get:13 http://azure.archive.ubuntu.com/ubuntu focal/main amd64 libxtst-dev amd64 2:1.2.3-1 [15.2 kB] > E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb 404 Not Found [IP: 52.147.219.192 80] > Fetched 760 kB in 0s (3154 kB/s) > E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? Looks good! ------------- Marked as reviewed by rwestberg (Committer). PR: https://git.openjdk.java.net/jdk/pull/1427 From shade at openjdk.java.net Wed Nov 25 10:31:55 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 25 Nov 2020 10:31:55 GMT Subject: RFR: 8257056: Submit workflow should apt-get update to avoid package installation errors In-Reply-To: References: Message-ID: <03SZ_yKQDDjzF0qh7WXBa0630kKO5cZ9y_Jrw_USajA=.e75585b8-b12a-424f-8530-81cdeb18fe70@github.com> On Wed, 25 Nov 2020 09:11:26 GMT, Robin Westberg wrote: >> For example, current jobs fail with: >> >> Get:13 http://azure.archive.ubuntu.com/ubuntu focal/main amd64 libxtst-dev amd64 2:1.2.3-1 [15.2 kB] >> E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb 404 Not Found [IP: 52.147.219.192 80] >> Fetched 760 kB in 0s (3154 kB/s) >> E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? > > Looks good! Thanks! By this time, GH actions seem to pass all Linux builds, including the additional ones, except known failures on Linux x86_32. Still need a formal Reviewer ack to integrate. ------------- PR: https://git.openjdk.java.net/jdk/pull/1427 From chegar at openjdk.java.net Wed Nov 25 11:08:19 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 25 Nov 2020 11:08:19 GMT Subject: RFR: 8256755: Update build.tools.depend.Depend to handle record components in API signatures [v2] In-Reply-To: <3lErqjFlNDVjsKubxd0HVv-PaL100WLOObgSQTXm3m8=.dabdfd0a-d1a2-407e-8172-ce2d1028ced8@github.com> References: <3lErqjFlNDVjsKubxd0HVv-PaL100WLOObgSQTXm3m8=.dabdfd0a-d1a2-407e-8172-ce2d1028ced8@github.com> Message-ID: > The `Depend` build tool creates a hash of a module's API elements, so that it can determine if downstream modules require recompilation. The build tool fails (throws an exception) when it encounters an "unknown" record attribute/component - the build tool predates records. > > The components of a public record class, that is a member of an exported package, are logically part of the module's API. The name, type, and order are significant. This issue updates the build tool to include the aforementioned component characteristics. > > This issue is a blocker to adding any public record types to the JDK - since the build will fail. Chris Hegarty has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Additional tests for explicit constructors - Merge branch 'master' into depends - Update the Depend(ency) build tool to handle the record attribute ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1405/files - new: https://git.openjdk.java.net/jdk/pull/1405/files/ce0089d4..9ff3b10c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1405&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1405&range=00-01 Stats: 295440 lines in 2312 files changed: 182698 ins; 87418 del; 25324 mod Patch: https://git.openjdk.java.net/jdk/pull/1405.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1405/head:pull/1405 PR: https://git.openjdk.java.net/jdk/pull/1405 From chegar at openjdk.java.net Wed Nov 25 11:39:55 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Wed, 25 Nov 2020 11:39:55 GMT Subject: Integrated: 8256755: Update build.tools.depend.Depend to handle record components in API signatures In-Reply-To: <3lErqjFlNDVjsKubxd0HVv-PaL100WLOObgSQTXm3m8=.dabdfd0a-d1a2-407e-8172-ce2d1028ced8@github.com> References: <3lErqjFlNDVjsKubxd0HVv-PaL100WLOObgSQTXm3m8=.dabdfd0a-d1a2-407e-8172-ce2d1028ced8@github.com> Message-ID: <66B56fY1kM9aCoTSxCYVs0g_KJsV7lirZv63ujJ32V8=.502db1ba-003a-45eb-9d63-93becab0afe9@github.com> On Tue, 24 Nov 2020 09:59:44 GMT, Chris Hegarty wrote: > The `Depend` build tool creates a hash of a module's API elements, so that it can determine if downstream modules require recompilation. The build tool fails (throws an exception) when it encounters an "unknown" record attribute/component - the build tool predates records. > > The components of a public record class, that is a member of an exported package, are logically part of the module's API. The name, type, and order are significant. This issue updates the build tool to include the aforementioned component characteristics. > > This issue is a blocker to adding any public record types to the JDK - since the build will fail. This pull request has now been integrated. Changeset: b0bd0c24 Author: Chris Hegarty URL: https://git.openjdk.java.net/jdk/commit/b0bd0c24 Stats: 41 lines in 2 files changed: 39 ins; 0 del; 2 mod 8256755: Update build.tools.depend.Depend to handle record components in API signatures Reviewed-by: jlahoda ------------- PR: https://git.openjdk.java.net/jdk/pull/1405 From sgehwolf at openjdk.java.net Wed Nov 25 13:11:58 2020 From: sgehwolf at openjdk.java.net (Severin Gehwolf) Date: Wed, 25 Nov 2020 13:11:58 GMT Subject: RFR: 8257056: Submit workflow should apt-get update to avoid package installation errors In-Reply-To: References: Message-ID: On Wed, 25 Nov 2020 08:56:33 GMT, Aleksey Shipilev wrote: > For example, current jobs fail with: > > Get:13 http://azure.archive.ubuntu.com/ubuntu focal/main amd64 libxtst-dev amd64 2:1.2.3-1 [15.2 kB] > E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb 404 Not Found [IP: 52.147.219.192 80] > Fetched 760 kB in 0s (3154 kB/s) > E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? Ship it! ------------- Marked as reviewed by sgehwolf (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1427 From shade at openjdk.java.net Wed Nov 25 13:17:03 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 25 Nov 2020 13:17:03 GMT Subject: Integrated: 8257056: Submit workflow should apt-get update to avoid package installation errors In-Reply-To: References: Message-ID: On Wed, 25 Nov 2020 08:56:33 GMT, Aleksey Shipilev wrote: > For example, current jobs fail with: > > Get:13 http://azure.archive.ubuntu.com/ubuntu focal/main amd64 libxtst-dev amd64 2:1.2.3-1 [15.2 kB] > E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb 404 Not Found [IP: 52.147.219.192 80] > Fetched 760 kB in 0s (3154 kB/s) > E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? This pull request has now been integrated. Changeset: 911f16d4 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/911f16d4 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod 8257056: Submit workflow should apt-get update to avoid package installation errors Reviewed-by: rwestberg, sgehwolf ------------- PR: https://git.openjdk.java.net/jdk/pull/1427 From shade at openjdk.java.net Wed Nov 25 13:17:02 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 25 Nov 2020 13:17:02 GMT Subject: RFR: 8257056: Submit workflow should apt-get update to avoid package installation errors In-Reply-To: References: Message-ID: On Wed, 25 Nov 2020 13:08:59 GMT, Severin Gehwolf wrote: >> For example, current jobs fail with: >> >> Get:13 http://azure.archive.ubuntu.com/ubuntu focal/main amd64 libxtst-dev amd64 2:1.2.3-1 [15.2 kB] >> E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/a/alsa-lib/libasound2-dev_1.2.2-2.1ubuntu2.1_amd64.deb 404 Not Found [IP: 52.147.219.192 80] >> Fetched 760 kB in 0s (3154 kB/s) >> E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? > > Ship it! Thank you! ------------- PR: https://git.openjdk.java.net/jdk/pull/1427 From ihse at openjdk.java.net Thu Nov 26 08:36:00 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 26 Nov 2020 08:36:00 GMT Subject: RFR: 8253753 Enable default constructor warning in client modules In-Reply-To: References: Message-ID: On Tue, 24 Nov 2020 19:10:04 GMT, Joe Darcy wrote: > With the default constructors warnings in java.desktop and jdk.accessibility now fixed, the warning should be enabled in the build for those modules. Looks good! Thank you for your continued efforts to clean up the code base. ------------- Marked as reviewed by ihse (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1420 From aoqi at openjdk.java.net Thu Nov 26 11:13:55 2020 From: aoqi at openjdk.java.net (Ao Qi) Date: Thu, 26 Nov 2020 11:13:55 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v3] In-Reply-To: <5TXFPcFMqqPj2TKkbXb2PwTJEPEDJLzYKZaH_dtlBno=.2435c211-847c-4a47-98ae-9af6144f37cd@github.com> References: <9ocjw2dgAzWNw4lLpGvN6c1xrVYSF3QfK4X_H5Z2aG8=.87b9c9b5-55c2-48cf-af1d-91e47cd4ad38@github.com> <5TXFPcFMqqPj2TKkbXb2PwTJEPEDJLzYKZaH_dtlBno=.2435c211-847c-4a47-98ae-9af6144f37cd@github.com> Message-ID: On Wed, 25 Nov 2020 08:43:31 GMT, Aleksey Shipilev wrote: >> Marked as reviewed by ihse (Reviewer). > > Thanks, I'll wait for either of @glaubitz, @theaoqi or @xiangzhai to confirm this fixes the native `mipsel` builds for them, and then integrate. The native build of linux-mipsel-zero-release passed. I only have mips64el and 64-bit OS, so I tested the patch by using a mipsel docker image. Some messages during the build: LOAD: .MIPS.abiflags .reginfo .note.gnu.build-id .dynamic .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .init .text .MIPS.stubs .fini .eh_frame However, I think the native build issue has been solved. ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From shade at openjdk.java.net Thu Nov 26 11:28:55 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 26 Nov 2020 11:28:55 GMT Subject: RFR: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 [v3] In-Reply-To: References: <9ocjw2dgAzWNw4lLpGvN6c1xrVYSF3QfK4X_H5Z2aG8=.87b9c9b5-55c2-48cf-af1d-91e47cd4ad38@github.com> <5TXFPcFMqqPj2TKkbXb2PwTJEPEDJLzYKZaH_dtlBno=.2435c211-847c-4a47-98ae-9af6144f37cd@github.com> Message-ID: On Thu, 26 Nov 2020 11:10:43 GMT, Ao Qi wrote: >> Thanks, I'll wait for either of @glaubitz, @theaoqi or @xiangzhai to confirm this fixes the native `mipsel` builds for them, and then integrate. > > The native build of linux-mipsel-zero-release passed. I only have mips64el and 64-bit OS, so I tested the patch by using a mipsel docker image. > > Some messages during the build: > > LOAD: .MIPS.abiflags .reginfo .note.gnu.build-id .dynamic .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .init .text .MIPS.stubs .fini .eh_frame > However, I think the native build issue has been solved. Thanks for testing! ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From shade at openjdk.java.net Thu Nov 26 11:28:56 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 26 Nov 2020 11:28:56 GMT Subject: Integrated: 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 In-Reply-To: References: Message-ID: On Sun, 22 Nov 2020 14:40:12 GMT, Aleksey Shipilev wrote: > If you try to build `linux-mipsel-zero-fastdebug`, this happens: > > > > > > > > I think it relates to [JDK-8253970](https://bugs.openjdk.java.net/browse/JDK-8253970) that introduced `atomic_compare_exchange` on those paths, but maybe the issue exists for longer. > > Various other reports for build failures like this suggest the binary should link with `libatomic`. GCC's `libatomic` is the library that provides runtime support for atomics not supported in hardware, which seems to include 8-byte CAS for MIPS, but not any other OpenJDK platform. > > The alternative would be to massage the otherwise generic Zero code to unimplement the 8-byte CAS. > > Attention @DamonFool, who must be running into this problem for their MIPS builds? > > Testing: > - [x] Linux mipsel Zero fastdebug build (together with JDK-8256829 fix) This pull request has now been integrated. Changeset: 1b3aa3af Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/1b3aa3af Stats: 8 lines in 1 file changed: 8 ins; 0 del; 0 mod 8256831: MIPS Zero builds fail with undefined __atomic_compare_exchange_8 Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1375 From shade at openjdk.java.net Thu Nov 26 15:23:00 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 26 Nov 2020 15:23:00 GMT Subject: RFR: 8257181: s390x builds are very noisy with gc-sections messages Message-ID: Sample build log sizes: PPC64 fastdebug, 200 kilobytes: https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-ppc64le-fastdebug.log S390X fastdebug, 20 megabytes! https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-fastdebug.log S390X slowdebug, 200 megabytes! https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-slowdebug.log JDK-8234525 enabled `gc-sections`. We can and should disable `print-gc-sections` to get sane build logs back. Additional testing: - [x] Linux s390x fastdebug cross-compilation ------------- Commit messages: - 8257181: s390x builds are very noisy with gc-sections messages Changes: https://git.openjdk.java.net/jdk/pull/1459/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1459&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8257181 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/1459.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1459/head:pull/1459 PR: https://git.openjdk.java.net/jdk/pull/1459 From mdoerr at openjdk.java.net Thu Nov 26 15:33:53 2020 From: mdoerr at openjdk.java.net (Martin Doerr) Date: Thu, 26 Nov 2020 15:33:53 GMT Subject: RFR: 8257181: s390x builds are very noisy with gc-sections messages In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 15:18:04 GMT, Aleksey Shipilev wrote: > Sample build log sizes: > > PPC64 fastdebug, 200 kilobytes: > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-ppc64le-fastdebug.log > > S390X fastdebug, 20 megabytes! > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-fastdebug.log > > S390X slowdebug, 200 megabytes! > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-slowdebug.log > > JDK-8234525 enabled `gc-sections`. We can and should disable `print-gc-sections` to get sane build logs back. > > Additional testing: > - [x] Linux s390x fastdebug cross-compilation I agree. This is far too much for the regular build. If anybody wants to check unused functions, it should better be turned on locally. ------------- Marked as reviewed by mdoerr (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1459 From lucy at openjdk.java.net Thu Nov 26 15:58:57 2020 From: lucy at openjdk.java.net (Lutz Schmidt) Date: Thu, 26 Nov 2020 15:58:57 GMT Subject: RFR: 8257181: s390x builds are very noisy with gc-sections messages In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 15:18:04 GMT, Aleksey Shipilev wrote: > Sample build log sizes: > > PPC64 fastdebug, 200 kilobytes: > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-ppc64le-fastdebug.log > > S390X fastdebug, 20 megabytes! > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-fastdebug.log > > S390X slowdebug, 200 megabytes! > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-slowdebug.log > > JDK-8234525 enabled `gc-sections`. We can and should disable `print-gc-sections` to get sane build logs back. > > Additional testing: > - [x] Linux s390x fastdebug cross-compilation This "noise generator" looks like a leftover from way back development times. Thanks for tracking this down. ------------- Marked as reviewed by lucy (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1459 From rwestberg at openjdk.java.net Thu Nov 26 16:10:57 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Thu, 26 Nov 2020 16:10:57 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> <8YwwhqUReVn9jy_yPA_6NB0T4AKpIanNe0t0XKI1zV4=.3d43df3f-2754-4a24-b1c5-8da962de9f6e@github.com> Message-ID: On Mon, 23 Nov 2020 11:54:48 GMT, Magnus Ihse Bursie wrote: > That sounds very good indeed! I did not think it was possible for Skara to access the Checks area, but if it does, it's really where this belong. I've tried this out a bit in the playground now - perhaps something like this: https://github.com/openjdk/playground/pull/77 ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From ihse at openjdk.java.net Thu Nov 26 16:27:57 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 26 Nov 2020 16:27:57 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> <8YwwhqUReVn9jy_yPA_6NB0T4AKpIanNe0t0XKI1zV4=.3d43df3f-2754-4a24-b1c5-8da962de9f6e@github.com> Message-ID: On Thu, 26 Nov 2020 16:07:54 GMT, Robin Westberg wrote: >> @rwestberg That sounds very good indeed! I did not think it was possible for Skara to access the Checks area, but if it does, it's really where this belong. > >> That sounds very good indeed! I did not think it was possible for Skara to access the Checks area, but if it does, it's really where this belong. > > I've tried this out a bit in the playground now - perhaps something like this: https://github.com/openjdk/playground/pull/77 @rwestberg Looks sooo much better! What happens if a build fails? ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From rwestberg at openjdk.java.net Thu Nov 26 16:37:58 2020 From: rwestberg at openjdk.java.net (Robin Westberg) Date: Thu, 26 Nov 2020 16:37:58 GMT Subject: RFR: 8256747: GitHub Actions: decouple the hotspot build-only jobs from Linux x64 testing [v2] In-Reply-To: References: <3VyMHVrNABK4NJHeXoBJEc5VMaHeTEJnGMZV9UluSy8=.af5d4b7f-0bc9-4a7e-b03e-d23b46461431@github.com> <8YwwhqUReVn9jy_yPA_6NB0T4AKpIanNe0t0XKI1zV4=.3d43df3f-2754-4a24-b1c5-8da962de9f6e@github.com> Message-ID: On Thu, 26 Nov 2020 16:25:35 GMT, Magnus Ihse Bursie wrote: >>> That sounds very good indeed! I did not think it was possible for Skara to access the Checks area, but if it does, it's really where this belong. >> >> I've tried this out a bit in the playground now - perhaps something like this: https://github.com/openjdk/playground/pull/77 > > @rwestberg Looks sooo much better! What happens if a build fails? I think there were a few failing test jobs in the current state of the playground, so should be visible in a little while! ------------- PR: https://git.openjdk.java.net/jdk/pull/1350 From shade at openjdk.java.net Thu Nov 26 17:50:58 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 26 Nov 2020 17:50:58 GMT Subject: RFR: 8257181: s390x builds are very noisy with gc-sections messages In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 15:55:52 GMT, Lutz Schmidt wrote: >> Sample build log sizes: >> >> PPC64 fastdebug, 200 kilobytes: >> https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-ppc64le-fastdebug.log >> >> S390X fastdebug, 20 megabytes! >> https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-fastdebug.log >> >> S390X slowdebug, 200 megabytes! >> https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-slowdebug.log >> >> JDK-8234525 enabled `gc-sections`. We can and should disable `print-gc-sections` to get sane build logs back. >> >> Additional testing: >> - [x] Linux s390x fastdebug cross-compilation > > This "noise generator" looks like a leftover from way back development times. Thanks for tracking this down. Thanks. Trivial, right? (So I can push now?) ------------- PR: https://git.openjdk.java.net/jdk/pull/1459 From ihse at openjdk.java.net Thu Nov 26 21:03:01 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Thu, 26 Nov 2020 21:03:01 GMT Subject: RFR: 8256541: Sort out what version of awk is used in the build system Message-ID: For historical reasons, there exists a variety of different implementations of awk: awk (the original implementation), gawk (the GNU version), nawk (new awk, iirc) and the lesser known mawk. Things are complicated by the fact that the original awk is seldom used, but instead gawk or nawk is typically symlinked to be named "awk". In terms of functionality there are very few differences. The original awk is most limited, while nawk and gawk is mostly replaceable. So the conditions for this is somewhat messy, but we manage impressively to mess it up even further. :-) We set up the following definitions: `BASIC_REQUIRE_PROGS(NAWK, [nawk gawk awk])` `BASIC_REQUIRE_SPECIAL(AWK, [AC_PROG_AWK])` and `AC_PROG_AWK`, according to the documentation, "[c]heck for gawk, mawk, nawk, and awk, in that order". So, if you have nawk and awk (but no other) installed, both NAWK and AWK will be set to nawk. If you have only awk, both will be set to awk. The difference is if you have gawk installed, then NAWK will be nawk and AWK will be gawk. As an example, on my mac, I only have the original awk, so both AWK and NAWK will be awk. On my ubuntu box, things are even more confused. I have: $ ls -l /usr/bin/*awk lrwxrwxrwx 1 root root 21 Feb 6 10:36 awk -> /etc/alternatives/awk* -rwxr-xr-x 1 root root 658072 Feb 11 2018 gawk* -rwxr-xr-x 1 root root 3189 Feb 11 2018 igawk* -rwxr-xr-x 1 root root 125416 Apr 3 2018 mawk* lrwxrwxrwx 1 root root 22 Feb 6 10:37 nawk -> /etc/alternatives/nawk* $ ls -l /etc/alternatives/*awk lrwxrwxrwx 1 root root 13 Feb 10 10:56 /etc/alternatives/awk -> /usr/bin/gawk* lrwxrwxrwx 1 root root 13 Feb 10 10:56 /etc/alternatives/nawk -> /usr/bin/gawk* So awk, nawk and gawk all executes the same binary, i.e. gawk. Only mawk is different. So on that machine, AWK would be gawk and NAWK would be nawk, but both will execute gawk. I propose that we remove NAWK, and only use AWK, but we should stop using AC_PROG_AWK and define it in an order that is transparent to us. I recommend [gawk nawk awk], since on Linux systems nawk (as we've seen) is likely to be gawk under disguise anyway, so it's better to be clear about that. This reasoning assumes that the awk scripts we write are portable enough to be executed by any awk. If we run into any problem with this, we might have to restrict the variation of awks we support. To make this work properly, I also needed to get rid of the awk launched by fixpath in CompileCommand. (This only worked. since AWK was not evolved to a full path by `AC_PROG_AWK`, but was only `awk`(or whatever). Otherwise this could not work with fixpath, so it was very much a hack to begin with... ------------- Commit messages: - Remove instance of awk in CompileCommand - 8256541: Sort out what version of awk is used in the build system Changes: https://git.openjdk.java.net/jdk/pull/1470/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1470&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256541 Stats: 35 lines in 15 files changed: 4 ins; 9 del; 22 mod Patch: https://git.openjdk.java.net/jdk/pull/1470.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1470/head:pull/1470 PR: https://git.openjdk.java.net/jdk/pull/1470 From lucy at openjdk.java.net Thu Nov 26 22:00:55 2020 From: lucy at openjdk.java.net (Lutz Schmidt) Date: Thu, 26 Nov 2020 22:00:55 GMT Subject: RFR: 8257181: s390x builds are very noisy with gc-sections messages In-Reply-To: References: Message-ID: <46UxsinsgbAMDPCgLVz4xgHmGGb9RzZ16DE3yP-7ngQ=.e448d8fb-aaf5-4986-b133-5683075aeac2@github.com> On Thu, 26 Nov 2020 17:48:14 GMT, Aleksey Shipilev wrote: >> This "noise generator" looks like a leftover from way back development times. Thanks for tracking this down. > > Thanks. Trivial, right? (So I can push now?) Yes, sure. Push it, if you haven?t already. Didn?t Martin give you a review as well? Sorry, I took a few off-screen (or screen off) hours. Thanks, Lutz From: Aleksey Shipil?v Reply to: openjdk/jdk Date: Thursday, 26. November 2020 at 18:48 To: openjdk/jdk Cc: Lutz Schmidt , Mention Subject: Re: [openjdk/jdk] 8257181: s390x builds are very noisy with gc-sections messages (#1459) Thanks. Trivial, right? (So I can push now?) ? You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. ------------- PR: https://git.openjdk.java.net/jdk/pull/1459 From shade at openjdk.java.net Fri Nov 27 06:51:07 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 27 Nov 2020 06:51:07 GMT Subject: RFR: 8257181: s390x builds are very noisy with gc-sections messages In-Reply-To: <46UxsinsgbAMDPCgLVz4xgHmGGb9RzZ16DE3yP-7ngQ=.e448d8fb-aaf5-4986-b133-5683075aeac2@github.com> References: <46UxsinsgbAMDPCgLVz4xgHmGGb9RzZ16DE3yP-7ngQ=.e448d8fb-aaf5-4986-b133-5683075aeac2@github.com> Message-ID: On Thu, 26 Nov 2020 21:57:58 GMT, Lutz Schmidt wrote: >> Thanks. Trivial, right? (So I can push now?) > > Yes, sure. Push it, if you haven?t already. > Didn?t Martin give you a review as well? > > Sorry, I took a few off-screen (or screen off) hours. > > Thanks, > Lutz > > From: Aleksey Shipil?v > Reply to: openjdk/jdk > Date: Thursday, 26. November 2020 at 18:48 > To: openjdk/jdk > Cc: Lutz Schmidt , Mention > Subject: Re: [openjdk/jdk] 8257181: s390x builds are very noisy with gc-sections messages (#1459) > > > Thanks. Trivial, right? (So I can push now?) > > ? > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or unsubscribe. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1459 From shade at openjdk.java.net Fri Nov 27 06:51:08 2020 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 27 Nov 2020 06:51:08 GMT Subject: Integrated: 8257181: s390x builds are very noisy with gc-sections messages In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 15:18:04 GMT, Aleksey Shipilev wrote: > Sample build log sizes: > > PPC64 fastdebug, 200 kilobytes: > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-ppc64le-fastdebug.log > > S390X fastdebug, 20 megabytes! > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-fastdebug.log > > S390X slowdebug, 200 megabytes! > https://builds.shipilev.net/openjdk-jdk/build-logs/build-linux-s390x-server-slowdebug.log > > JDK-8234525 enabled `gc-sections`. We can and should disable `print-gc-sections` to get sane build logs back. > > Additional testing: > - [x] Linux s390x fastdebug cross-compilation This pull request has now been integrated. Changeset: a3eec39b Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/a3eec39b Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8257181: s390x builds are very noisy with gc-sections messages Reviewed-by: mdoerr, lucy ------------- PR: https://git.openjdk.java.net/jdk/pull/1459 From github.com+43995067+hollowman6 at openjdk.java.net Fri Nov 27 12:27:04 2020 From: github.com+43995067+hollowman6 at openjdk.java.net (Hollow Man) Date: Fri, 27 Nov 2020 12:27:04 GMT Subject: RFR: 8257208: Fix typo in doc/building.md Message-ID: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> overriden -> overridden ------------- Commit messages: - 8257208: Fix typo in doc/building.md Changes: https://git.openjdk.java.net/jdk/pull/1365/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1365&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8257208 Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/1365.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1365/head:pull/1365 PR: https://git.openjdk.java.net/jdk/pull/1365 From ihse at openjdk.java.net Fri Nov 27 12:27:04 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 27 Nov 2020 12:27:04 GMT Subject: RFR: 8257208: Fix typo in doc/building.md In-Reply-To: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> References: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> Message-ID: On Sat, 21 Nov 2020 14:28:23 GMT, Hollow Man wrote: > overriden -> overridden Hi @HollowMan6! Thank you for submitting this PR. As the contribution docs state, you need a sponsor for this PR. I'll be your sponsor. I have opened a bug issue for this fix: https://bugs.openjdk.java.net/browse/JDK-8257208 Please update the PR title to `8257208: Fix typo in doc/building.md`. ------------- PR: https://git.openjdk.java.net/jdk/pull/1365 From github.com+43995067+hollowman6 at openjdk.java.net Fri Nov 27 12:27:04 2020 From: github.com+43995067+hollowman6 at openjdk.java.net (Hollow Man) Date: Fri, 27 Nov 2020 12:27:04 GMT Subject: RFR: 8257208: Fix typo in doc/building.md In-Reply-To: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> References: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> Message-ID: On Sat, 21 Nov 2020 14:28:23 GMT, Hollow Man wrote: > overriden -> overridden > Hi @HollowMan6! > > Thank you for submitting this PR. As the contribution docs state, you need a sponsor for this PR. I'll be your sponsor. > > I have opened a bug issue for this fix: https://bugs.openjdk.java.net/browse/JDK-8257208 > > Please update the PR title to `8257208: Fix typo in doc/building.md`. @magicus Done! Thanks. ------------- PR: https://git.openjdk.java.net/jdk/pull/1365 From ihse at openjdk.java.net Fri Nov 27 14:24:56 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Fri, 27 Nov 2020 14:24:56 GMT Subject: RFR: 8257208: Fix typo in doc/building.md In-Reply-To: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> References: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> Message-ID: On Sat, 21 Nov 2020 14:28:23 GMT, Hollow Man wrote: > overriden -> overridden Marked as reviewed by ihse (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1365 From jlahoda at openjdk.java.net Fri Nov 27 15:45:12 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Fri, 27 Nov 2020 15:45:12 GMT Subject: RFR: JDK-8256950: Add record attribute support to symbol generator CreateSymbols Message-ID: Adding support for record classes in the historical data for ct.sym. This includes a few changes not strictly needed for the change: -updating and moving tests into test/langtools, so that it is easier to run them. -fixing Record attribute reading in javac's ClassReader (used for tests, but seems like the proper thing to do anyway). -fixing the -Xprint annotation processor to print record component annotations. Changes to jdk.jdeps' classfile library are needed so that the ct.sym creation works. ------------- Commit messages: - Update CreateSymbolsTest.java - Cleaning TODO. - Cleaning TODO. - Adding test verifying javac can read Record attribute properly. - Cleanup. - JDK-8256950: Add record attribute support to symbol generator CreateSymbols Changes: https://git.openjdk.java.net/jdk/pull/1480/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1480&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8256950 Stats: 1910 lines in 12 files changed: 1147 ins; 745 del; 18 mod Patch: https://git.openjdk.java.net/jdk/pull/1480.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1480/head:pull/1480 PR: https://git.openjdk.java.net/jdk/pull/1480 From github.com+43995067+hollowman6 at openjdk.java.net Fri Nov 27 16:10:55 2020 From: github.com+43995067+hollowman6 at openjdk.java.net (Hollow Man) Date: Fri, 27 Nov 2020 16:10:55 GMT Subject: RFR: 8257208: Fix typo in doc/building.md In-Reply-To: References: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> Message-ID: On Fri, 27 Nov 2020 14:22:01 GMT, Magnus Ihse Bursie wrote: >> overriden -> overridden > > Marked as reviewed by ihse (Reviewer). @magicus Thanks for reviewing! Now could you please sponsor this change? Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/1365 From github.com+43995067+hollowman6 at openjdk.java.net Fri Nov 27 19:34:55 2020 From: github.com+43995067+hollowman6 at openjdk.java.net (Hollow Man) Date: Fri, 27 Nov 2020 19:34:55 GMT Subject: Integrated: 8257208: Fix typo in doc/building.md In-Reply-To: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> References: <4EaLgwmFApEOLYa-OWtoTGfwA_Sa52n4V1j3Uvz947w=.25abbeb2-5902-442f-b453-17b0286cd3c2@github.com> Message-ID: On Sat, 21 Nov 2020 14:28:23 GMT, Hollow Man wrote: > overriden -> overridden This pull request has now been integrated. Changeset: f6bfbb2b Author: Hollow Man Committer: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/f6bfbb2b Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod 8257208: Fix typo in doc/building.md Reviewed-by: ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1365 From darcy at openjdk.java.net Fri Nov 27 21:34:54 2020 From: darcy at openjdk.java.net (Joe Darcy) Date: Fri, 27 Nov 2020 21:34:54 GMT Subject: Integrated: 8253753 Enable default constructor warning in client modules In-Reply-To: References: Message-ID: On Tue, 24 Nov 2020 19:10:04 GMT, Joe Darcy wrote: > With the default constructors warnings in java.desktop and jdk.accessibility now fixed, the warning should be enabled in the build for those modules. This pull request has now been integrated. Changeset: 65137ff0 Author: Joe Darcy URL: https://git.openjdk.java.net/jdk/commit/65137ff0 Stats: 5 lines in 1 file changed: 0 ins; 5 del; 0 mod 8253753: Enable default constructor warning in client modules Reviewed-by: prr, serb, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1420 From burban at openjdk.java.net Fri Nov 27 22:32:04 2020 From: burban at openjdk.java.net (Bernhard Urban-Forster) Date: Fri, 27 Nov 2020 22:32:04 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow [v2] In-Reply-To: References: Message-ID: <76U-cTva3pgbGHXgDEyDh-NvTwYVhfK7tdSnb4Vv_J4=.be2f9bfa-9ab0-4140-b63e-5e72d35a4e41@github.com> > This adds the cross-compiled build only, as no Windows+Arm64 machines are available on GitHub Action that we could use to run the tests. > > Due to cross-compilation a build JDK is required. Initially I added EA builds to be downloaded from https://jdk.java.net/16/ and used for that, but then I saw how @shipiliv attempted it for the linux cross-compilation builds in https://github.com/openjdk/jdk/pull/1147. That is using the JDK image produced by the x64 variant. This however add more stress to the "critical path", as now two more jobs depend on the x64 build first. > > Let's see how it works out in the long-run. A Windows+AArch64 build takes 40-50min. Bernhard Urban-Forster has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: - todo note for caching devkit - remove release build - move windows_aarch64_build next to other builds - Merge remote-tracking branch 'upstream/master' into 8256657-win-arm64-gh-submit-workflow - remove fixpath.exe workaround - 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow This adds the cross-compiled build only, as no Windows+Arm64 machines are available on GitHub Action that we could use to run the tests. Due to cross-compilation a build JDK is required. Initially I added EA builds to be downloaded from https://jdk.java.net/16/ and used for that, but then I saw how @shipiliv attempted it for the linux cross-compilation builds in https://github.com/openjdk/jdk/pull/1147. That is using the JDK image produced by the x64 variant. This however add more stress to the "critical path", as now two more jobs depend on the x64 build first. Let's see how it works out in the long-run. A Windows+AArch64 build takes 40-50min. ------------- Changes: https://git.openjdk.java.net/jdk/pull/1379/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1379&range=01 Stats: 155 lines in 1 file changed: 153 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1379.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1379/head:pull/1379 PR: https://git.openjdk.java.net/jdk/pull/1379 From chegar at openjdk.java.net Mon Nov 30 12:20:59 2020 From: chegar at openjdk.java.net (Chris Hegarty) Date: Mon, 30 Nov 2020 12:20:59 GMT Subject: RFR: JDK-8256950: Add record attribute support to symbol generator CreateSymbols In-Reply-To: References: Message-ID: On Fri, 27 Nov 2020 13:21:15 GMT, Jan Lahoda wrote: > Adding support for record classes in the historical data for ct.sym. This includes a few changes not strictly needed for the change: > -updating and moving tests into test/langtools, so that it is easier to run them. > -fixing Record attribute reading in javac's ClassReader (used for tests, but seems like the proper thing to do anyway). > -fixing the -Xprint annotation processor to print record component annotations. > > Changes to jdk.jdeps' classfile library are needed so that the ct.sym creation works. make/langtools/src/classes/build/tools/symbolgenerator/CreateSymbols.java line 965: > 963: new NestMembers_attribute(attributeString, nestMembers)); > 964: } > 965: if (header.recordComponents != null && !header.recordComponents.isEmpty()) { I am not sure of the exact logic here, but it is perfectly fine for a record attribute to contain zero components, and for the class to still be considered a "record class". But maybe that is not all that significant here? I just want to call it out so that it is considered. ------------- PR: https://git.openjdk.java.net/jdk/pull/1480 From jlahoda at openjdk.java.net Mon Nov 30 14:21:16 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Mon, 30 Nov 2020 14:21:16 GMT Subject: RFR: JDK-8256950: Add record attribute support to symbol generator CreateSymbols [v2] In-Reply-To: References: Message-ID: <0IU-_p6RHP0YTRd5q17XiSKjstg63aZBK4GcvliVvTM=.0caf6919-4eb3-4c13-8447-a953e1185153@github.com> > Adding support for record classes in the historical data for ct.sym. This includes a few changes not strictly needed for the change: > -updating and moving tests into test/langtools, so that it is easier to run them. > -fixing Record attribute reading in javac's ClassReader (used for tests, but seems like the proper thing to do anyway). > -fixing the -Xprint annotation processor to print record component annotations. > > Changes to jdk.jdeps' classfile library are needed so that the ct.sym creation works. Jan Lahoda has updated the pull request incrementally with two additional commits since the last revision: - Merge branch 'JDK-8256950' of https://github.com/lahodaj/jdk into JDK-8256950 - CreateSymbols should support records with no components. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1480/files - new: https://git.openjdk.java.net/jdk/pull/1480/files/c3696e72..7ce38709 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1480&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1480&range=00-01 Stats: 54 lines in 2 files changed: 51 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1480.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1480/head:pull/1480 PR: https://git.openjdk.java.net/jdk/pull/1480 From jlahoda at openjdk.java.net Mon Nov 30 14:27:13 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Mon, 30 Nov 2020 14:27:13 GMT Subject: RFR: JDK-8256950: Add record attribute support to symbol generator CreateSymbols [v3] In-Reply-To: References: Message-ID: On Mon, 30 Nov 2020 12:17:46 GMT, Chris Hegarty wrote: >> Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixing tests on Windows - normalizing line endings. > > make/langtools/src/classes/build/tools/symbolgenerator/CreateSymbols.java line 965: > >> 963: new NestMembers_attribute(attributeString, nestMembers)); >> 964: } >> 965: if (header.recordComponents != null && !header.recordComponents.isEmpty()) { > > I am not sure of the exact logic here, but it is perfectly fine for a record attribute to contain zero components, and for the class to still be considered a "record class". But maybe that is not all that significant here? I just want to call it out so that it is considered. Ah, right - fixed in: https://github.com/openjdk/jdk/pull/1480/commits/e1ec2b7ff49e1307a348ad58b12e5ed39ebe7224 Thanks for the comment! ------------- PR: https://git.openjdk.java.net/jdk/pull/1480 From jlahoda at openjdk.java.net Mon Nov 30 14:27:11 2020 From: jlahoda at openjdk.java.net (Jan Lahoda) Date: Mon, 30 Nov 2020 14:27:11 GMT Subject: RFR: JDK-8256950: Add record attribute support to symbol generator CreateSymbols [v3] In-Reply-To: References: Message-ID: > Adding support for record classes in the historical data for ct.sym. This includes a few changes not strictly needed for the change: > -updating and moving tests into test/langtools, so that it is easier to run them. > -fixing Record attribute reading in javac's ClassReader (used for tests, but seems like the proper thing to do anyway). > -fixing the -Xprint annotation processor to print record component annotations. > > Changes to jdk.jdeps' classfile library are needed so that the ct.sym creation works. Jan Lahoda has updated the pull request incrementally with one additional commit since the last revision: Fixing tests on Windows - normalizing line endings. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1480/files - new: https://git.openjdk.java.net/jdk/pull/1480/files/7ce38709..a3f79aba Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1480&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1480&range=01-02 Stats: 6 lines in 2 files changed: 3 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/1480.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1480/head:pull/1480 PR: https://git.openjdk.java.net/jdk/pull/1480 From erikj at openjdk.java.net Mon Nov 30 15:35:56 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 30 Nov 2020 15:35:56 GMT Subject: RFR: 8256541: Sort out what version of awk is used in the build system In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 20:58:38 GMT, Magnus Ihse Bursie wrote: > For historical reasons, there exists a variety of different implementations of awk: awk (the original implementation), gawk (the GNU version), nawk (new awk, iirc) and the lesser known mawk. > > Things are complicated by the fact that the original awk is seldom used, but instead gawk or nawk is typically symlinked to be named "awk". > > In terms of functionality there are very few differences. The original awk is most limited, while nawk and gawk is mostly replaceable. > > So the conditions for this is somewhat messy, but we manage impressively to mess it up even further. :-) > > We set up the following definitions: > `BASIC_REQUIRE_PROGS(NAWK, [nawk gawk awk])` > `BASIC_REQUIRE_SPECIAL(AWK, [AC_PROG_AWK])` > and `AC_PROG_AWK`, according to the documentation, "[c]heck for gawk, mawk, nawk, and awk, in that order". > > So, if you have nawk and awk (but no other) installed, both NAWK and AWK will be set to nawk. If you have only awk, both will be set to awk. The difference is if you have gawk installed, then NAWK will be nawk and AWK will be gawk. > > As an example, on my mac, I only have the original awk, so both AWK and NAWK will be awk. > > On my ubuntu box, things are even more confused. I have: > $ ls -l /usr/bin/*awk > lrwxrwxrwx 1 root root 21 Feb 6 10:36 awk -> /etc/alternatives/awk* > -rwxr-xr-x 1 root root 658072 Feb 11 2018 gawk* > -rwxr-xr-x 1 root root 3189 Feb 11 2018 igawk* > -rwxr-xr-x 1 root root 125416 Apr 3 2018 mawk* > lrwxrwxrwx 1 root root 22 Feb 6 10:37 nawk -> /etc/alternatives/nawk* > > $ ls -l /etc/alternatives/*awk > lrwxrwxrwx 1 root root 13 Feb 10 10:56 /etc/alternatives/awk -> /usr/bin/gawk* > lrwxrwxrwx 1 root root 13 Feb 10 10:56 /etc/alternatives/nawk -> /usr/bin/gawk* > > So awk, nawk and gawk all executes the same binary, i.e. gawk. Only mawk is different. So on that machine, AWK would be gawk and NAWK would be nawk, but both will execute gawk. > > I propose that we remove NAWK, and only use AWK, but we should stop using AC_PROG_AWK and define it in an order that is transparent to us. I recommend [gawk nawk awk], since on Linux systems nawk (as we've seen) is likely to be gawk under disguise anyway, so it's better to be clear about that. > > This reasoning assumes that the awk scripts we write are portable enough to be executed by any awk. If we run into any problem with this, we might have to restrict the variation of awks we support. > > To make this work properly, I also needed to get rid of the awk launched by fixpath in CompileCommand. (This only worked. since AWK was not evolved to a full path by `AC_PROG_AWK`, but was only `awk`(or whatever). Otherwise this could not work with fixpath, so it was very much a hack to begin with... Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1470 From erikj at openjdk.java.net Mon Nov 30 15:46:58 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 30 Nov 2020 15:46:58 GMT Subject: RFR: 8256657: Add cross-compiled build for Windows+Arm64 to submit workflow [v2] In-Reply-To: References: Message-ID: On Mon, 23 Nov 2020 15:19:18 GMT, Bernhard Urban-Forster wrote: >> I think you need to wait for #1350, and then reconcile this patch with that refactoring. > > Thanks for all the comments. > >> - 40-50 min builds seem to be excessive for just the hotspot build, do youknow what exactly takes that long? Is this for release and debug each or both combined? > > It's for each of them. Installing a specific version of MSVC and creating the devkit take each ~10min. Thinking about it, there is a opportunity to cache the devkit and do the MSVC installer invocation only if no cached devkit is available. > > Here is an example run if you want to have a closer look: > https://github.com/lewurm/openjdk/runs/1434118318?check_suite_focus=true > >> - Is it worth the cycles to build both release *and* debug? How probable is > it that a non-win-aarch-dev will break one but not the other? I presume > developers on the windows aarch project will have tested the build locally > before pushing. > > Fair, I'll remove one of them (as suggested by Aleksey in another comment, I'll keep the debug one). Generating a devkit every time we build seems like a pretty big waste of time. The OpenJDK build is expected to work without devkits, using an installed Visual Studio. Is this not possible with the Windows aarch64 build? If not, then that really should be fixed. ------------- PR: https://git.openjdk.java.net/jdk/pull/1379 From ihse at openjdk.java.net Mon Nov 30 16:37:55 2020 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 30 Nov 2020 16:37:55 GMT Subject: Integrated: 8256541: Sort out what version of awk is used in the build system In-Reply-To: References: Message-ID: On Thu, 26 Nov 2020 20:58:38 GMT, Magnus Ihse Bursie wrote: > For historical reasons, there exists a variety of different implementations of awk: awk (the original implementation), gawk (the GNU version), nawk (new awk, iirc) and the lesser known mawk. > > Things are complicated by the fact that the original awk is seldom used, but instead gawk or nawk is typically symlinked to be named "awk". > > In terms of functionality there are very few differences. The original awk is most limited, while nawk and gawk is mostly replaceable. > > So the conditions for this is somewhat messy, but we manage impressively to mess it up even further. :-) > > We set up the following definitions: > `BASIC_REQUIRE_PROGS(NAWK, [nawk gawk awk])` > `BASIC_REQUIRE_SPECIAL(AWK, [AC_PROG_AWK])` > and `AC_PROG_AWK`, according to the documentation, "[c]heck for gawk, mawk, nawk, and awk, in that order". > > So, if you have nawk and awk (but no other) installed, both NAWK and AWK will be set to nawk. If you have only awk, both will be set to awk. The difference is if you have gawk installed, then NAWK will be nawk and AWK will be gawk. > > As an example, on my mac, I only have the original awk, so both AWK and NAWK will be awk. > > On my ubuntu box, things are even more confused. I have: > $ ls -l /usr/bin/*awk > lrwxrwxrwx 1 root root 21 Feb 6 10:36 awk -> /etc/alternatives/awk* > -rwxr-xr-x 1 root root 658072 Feb 11 2018 gawk* > -rwxr-xr-x 1 root root 3189 Feb 11 2018 igawk* > -rwxr-xr-x 1 root root 125416 Apr 3 2018 mawk* > lrwxrwxrwx 1 root root 22 Feb 6 10:37 nawk -> /etc/alternatives/nawk* > > $ ls -l /etc/alternatives/*awk > lrwxrwxrwx 1 root root 13 Feb 10 10:56 /etc/alternatives/awk -> /usr/bin/gawk* > lrwxrwxrwx 1 root root 13 Feb 10 10:56 /etc/alternatives/nawk -> /usr/bin/gawk* > > So awk, nawk and gawk all executes the same binary, i.e. gawk. Only mawk is different. So on that machine, AWK would be gawk and NAWK would be nawk, but both will execute gawk. > > I propose that we remove NAWK, and only use AWK, but we should stop using AC_PROG_AWK and define it in an order that is transparent to us. I recommend [gawk nawk awk], since on Linux systems nawk (as we've seen) is likely to be gawk under disguise anyway, so it's better to be clear about that. > > This reasoning assumes that the awk scripts we write are portable enough to be executed by any awk. If we run into any problem with this, we might have to restrict the variation of awks we support. > > To make this work properly, I also needed to get rid of the awk launched by fixpath in CompileCommand. (This only worked. since AWK was not evolved to a full path by `AC_PROG_AWK`, but was only `awk`(or whatever). Otherwise this could not work with fixpath, so it was very much a hack to begin with... This pull request has now been integrated. Changeset: a3e1980c Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/a3e1980c Stats: 35 lines in 15 files changed: 4 ins; 9 del; 22 mod 8256541: Sort out what version of awk is used in the build system Reviewed-by: erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/1470 From erikj at openjdk.java.net Mon Nov 30 17:43:57 2020 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 30 Nov 2020 17:43:57 GMT Subject: Integrated: JDK-8256810: Incremental rebuild broken on Macosx In-Reply-To: References: Message-ID: On Sat, 21 Nov 2020 00:12:30 GMT, Erik Joelsson wrote: > After fixing JDK-8256751, I noticed that the fix-deps-file macro isn't actually doing the right thing at all. It seems clang on Macosx is inconsistent with outputting relative or absolute paths in the deps files, so some files end up with double WORKSPACE_ROOT in the paths. It's also not handling the trailing slash after WORKSPACE_ROOT correctly, so some files are missing the slash between WORKSPACE_ROOT and the original relative path. > > This time I have checked the output carefully. This pull request has now been integrated. Changeset: 4c86e46d Author: Erik Joelsson URL: https://git.openjdk.java.net/jdk/commit/4c86e46d Stats: 8 lines in 1 file changed: 5 ins; 0 del; 3 mod 8256810: Incremental rebuild broken on Macosx Reviewed-by: mikael, ihse ------------- PR: https://git.openjdk.java.net/jdk/pull/1360