October 2017 - core-libs-dev

parallel IO anomaly with windows REFS formatted drive
by Michael Skells 30 Oct '17

30 Oct '17

Hi It seems to me that some NIO based file creates don't scale work well with REFS formatted drives REFS is an alternative (to NTFS) drive format for Windows 8 and above When working with NTFS on my test system, as we use more threads the performance improves so some degree, (1000 files creates take 5-600 ms with one thread and 2-300 ms with 4 threads) when using REFS the single threaded times are roughly the same for 1 and 2 threads, but get much worse for 3 and 4 threads (1200 - 1600 ms) this showdown is only when using FileChanel.open and synchronous IO, for the AsynchronousFileChannel it seems roughly the to scale as expected CSV attached for the test results, and some test code. The test code is in scala, and I can convert if needed, but it just really 3 calls, to open write and close the files My test system is an I7 4 core window 10 pro with Norton installed, but an exclusion for the specified drives, and indexing disabled. All writes are to an M.2 SSD with write-though cache Some background I am working on the back end code generation of the scala compiler which generates many thousand small .class files, and part of that is to tune the IO. For windows where the IO cost is significant, so I had a few test app to simulate and measure the relative performance of the different alternatives Any ideas for a bulk file creation alternative welcome. I know we could write to a JAR but that requires other re-tooling Regards Mike

2 1

JDK10 RFR of JDK-8185841:Values from getFirstDayOfWeek() are inconsistent with CLDR
by Rachna Goel 27 Oct '17

27 Oct '17

Hi, Kindly review the fix to JDK-8185841. Bug : https://bugs.openjdk.java.net/browse/JDK-8185841. patch : http://cr.openjdk.java.net/~rgoel/JDK-8185841/webrev.08/ Fix is to relocate this region dependent data. New Bundle will be generated when Locale id has non empty script and country code and it contains region dependent data. e.g for "az_Latn_AZ" locale, CalendarData_az_AZ.java Bundle will be generated and the data currently in CalendarData_az_Latn_AZ.java should be relocated into it. -- Thanks, Rachna

2 1

[Proposal/Question] Provide mechanism to monitor thread states more efficiently
by Christoph Dreis 27 Oct '17

27 Oct '17

Hey, I don't know if this is the correct mailing list for this specific topic. Please let me know in case it isn't. I'm currently trying to improve the live-monitoring of an application and would like to get a simple overview of how many threads are in state RUNNABLE, BLOCKED and so on. For this I need to get all threads in the first place to iterate on. I know of two ways how to do that programmatically: 1) ThreadMXBean.dumpAllThreads() which gives me an array of ThreadInfo objects on which I could call getThreadState() 2) Thread.getAllStackTraces().keySet() which gives me a Set of Thread objects on which I could call getState() Either 1) or 2) both dump the threads, which seems like a lot of overhead for this task and therefore presumably aren't supposed to be executed every X seconds. Using reflection to make the private native Thread.getThreads() available - if that's even possible(?) - doesn't seem correct either. Is there an alternative for this? If not, I'd like to suggest the addition of two in my opinion more lightweight alternative APIs in Thread and/or ThreadMXBean (ThreadImpl). Here is what I'm thinking of in a rough and untested state: public static Map<Thread.State, Integer> getAllThreadStateCounts() { // Get a snapshot of the list of all threads Thread[] threads = getThreads(); Map<Thread.State, Integer> m = new EnumMap<>(Thread.State.class); for (int i = 0; i < threads.length; i++) { m.merge(threads[i].getState(), 1, Integer::sum); } return m; } public static int getThreadCountByState(Thread.State state) { // Get a snapshot of the list of all threads Thread[] threads = getThreads(); int counter = 0; for (int i = 0; i < threads.length; i++) { if (state == threads[i].getState()) { counter++; } } return counter; } Is this something that could be worthwhile? Let me know what you think. Cheers, Christoph

3 5

Re: RFR: 8188858: Caching latestUserDefinedLoader() results in ObjectInputStream.readObject()
by Kazunori Ogata 27 Oct '17

27 Oct '17

Hi Alan, Thank you for your review and comments. I updated webrev based on your comments: http://cr.openjdk.java.net/~horii/8188858/webrev.05/ Alan Bateman <Alan.Bateman(a)oracle.com> wrote on 2017/10/26 19:31:53: > I've read through webrev.03 and webrev.04 and they seem correct. Are > there are more comments on the implementation? I'd like to get someone > in the security area to review this before it is sponsored, the reason > is that original patches could be abused and this is security-sensitive > area. I added/revised comments, though I'm not sure it is sufficient. It is helpful if you could point out the changes that merely have insufficient comments. > One comment, or suggestion, is to find a better name for the > cachedLoader field, esp. as its value may be a Thread or a CachedLoader > object. If CachedLoader were renamed to ReaderContext and cachedLoader > changed to something like currentReader then it might be a bit clearer. > I'm sure there are better names, the important thing is that future > maintainers can quickly understand the code. Agreed. I renamed CachedLoader to ReaderContext, and also renamed other variables to make it easier to identify they are related to ReaderContext. Regards, Ogata

1 0

tzdata2017c is out
by Martin Buchholz 27 Oct '17

27 Oct '17

tzdata2017c came out today, and the elves at Google are working hard to incorporate changes in hours or days instead of weeks or quarters. One thing we noticed is that one of the java.time tck tests was broken by tzdata rewrite of history. Here's the fix we're applying (s/1879/1892/g): @@ -941,21 +941,21 @@ } public void test_Apia_jumpForwardOverInternationalDateLine_P12_to_M12() { - // transition occurred at 1879-07-04T00:00+12:33:04 + // transition occurred at 1892-07-04T00:00+12:33:04 ZoneRules test = pacificApia(); - Instant instantBefore = LocalDate.of(1879, 7, 2).atStartOfDay(ZoneOffset.UTC).toInstant(); + Instant instantBefore = LocalDate.of(1892, 7, 2).atStartOfDay(ZoneOffset.UTC).toInstant(); ZoneOffsetTransition trans = test.nextTransition(instantBefore); - assertEquals(trans.getDateTimeBefore(), LocalDateTime.of(1879, 7, 5, 0, 0)); - assertEquals(trans.getDateTimeAfter(), LocalDateTime.of(1879, 7, 4, 0, 0)); + assertEquals(trans.getDateTimeBefore(), LocalDateTime.of(1892, 7, 5, 0, 0)); + assertEquals(trans.getDateTimeAfter(), LocalDateTime.of(1892, 7, 4, 0, 0)); assertEquals(trans.isGap(), false); assertEquals(trans.isOverlap(), true); assertEquals(trans.isValidOffset(ZoneOffset.ofHoursMinutesSeconds(+12, 33, 4)), true); assertEquals(trans.isValidOffset(ZoneOffset.ofHoursMinutesSeconds(-11, -26, -56)), true); assertEquals(trans.getDuration(), Duration.ofHours(-24)); - assertEquals(trans.getInstant(), LocalDateTime.of(1879, 7, 4, 0, 0).toInstant(ZoneOffset.ofHoursMinutesSeconds(-11, -26, -56))); + assertEquals(trans.getInstant(), LocalDateTime.of(1892, 7, 4, 0, 0).toInstant(ZoneOffset.ofHoursMinutesSeconds(-11, -26, -56))); - ZonedDateTime zdt = ZonedDateTime.of(1879, 7, 4, 23, 0, 0, 0, ZoneId.of("Pacific/Apia")); - assertEquals(zdt.plusHours(2).toLocalDateTime(), LocalDateTime.of(1879, 7, 4, 1, 0, 0)); + ZonedDateTime zdt = ZonedDateTime.of(1892, 7, 4, 23, 0, 0, 0, ZoneId.of("Pacific/Apia")); + assertEquals(zdt.plusHours(2).toLocalDateTime(), LocalDateTime.of(1892, 7, 4, 1, 0, 0)); } //-------------------------------------------------------------------------

2 1

JDK 10 RFR of JDK-8189952: New methods on String: chars() and codePoints() should be marked since 9
by joe darcy 27 Oct '17

27 Oct '17

Hello, The initial changeset which introduced these the methods String.chars() and String.codePoints(), JDK-8071477 , had the expected @since tags for the time (@since 1.9), but subsequent changes to String seem to have removed them. Please review the patch below to add them back. Thanks, -Joe --- a/src/java.base/share/classes/java/lang/String.java Thu Oct 26 17:06:52 2017 -0700 +++ b/src/java.base/share/classes/java/lang/String.java Thu Oct 26 17:24:42 2017 -0700 @@ -2664,6 +2664,7 @@ * point</a> is passed through uninterpreted. * * @return an IntStream of char values from this sequence + * @since 9 */ @Override public IntStream chars() { @@ -2683,6 +2684,7 @@ * {@code int} values which are then passed to the stream. * * @return an IntStream of Unicode code points from this sequence + * @since 9 */ @Override public IntStream codePoints() {

3 2

Module naming for logging implementations
by Stephen Colebourne 26 Oct '17

26 Oct '17

I've spent some time discussing module names for logging implementations recently: https://github.com/jodastephen/jpms-module-names/wiki/Logging-APIs https://issues.apache.org/jira/browse/LOG4J2-2056 https://jira.qos.ch/browse/SLF4J-407?jql=text%20~%20%22jpms%22 Most logging projects are split in two - an API and an Implementation - where the recommended solution going forwards is to use ServiceLoader to find the implementations. A few old projects don't have this split, and have API and Implementation together (eg Commons-Logging). Everyone agrees that the module name for the API must always be the same. ie. if the SLF4J team provides a module that simulates the Commons-Logging then it must have the same module name as Commons-Logging. However, there are two choices for the implementation jars. Option 1: All modules that implement a particular logging API must have the same module name eg. every module that implements "org.slf4j" (the API) must be named "org.slf4j.impl" Option 2: The module name of the implementation can be whatever name makes sense. For most service providers, option 2 is obvious, however for logging it is generally the case that only one implementation should be present. If all the jar files that implement a specific logging API had the same module name (option 1) then the module system could ensure that only one was loaded. This is a particular concern as it is not uncommon for a jar file in Maven Central to depend on a specific implementation of logging when it should only be depending on the API. I'm leaning towards option 2, as it is simpler and does not require all implementations to have the same module name (which would be difficult to require). Any other considerations I'm missing? Stephen

1 0

Re: RFR: 8188858: Caching latestUserDefinedLoader() results in ObjectInputStream.readObject()
by Kazunori Ogata 26 Oct '17

26 Oct '17

Hi David, Yes I think the code works correctly even if cachedLoader is not volatile. When a thread see a non-volatile value, it is either a Thread or CachedLoader object and the thread can check if it owns the cache. If it does, the thread uses the cache and cleans it up on return. If two or more threads see null, each of them tries to grab the cache by putting its Thread object, but one of them will win. Then, the won thread use the cache and cleans it up on return. I update webrev by adding a few lines of comment as: 2442 // This field is left non-volatile although there is a benign race here. 2443 // The thread that see a non-null value can always check if the cache is for 2444 // the thread, and such thread always cleans up the cache on return. Comments for revising the text (and fixing my poor English) are welcome. Webrev: http://cr.openjdk.java.net/~horii/8188858/webrev.04/ Regards, Ogata From: David Holmes <david.holmes(a)oracle.com> To: Peter Levart <peter.levart(a)gmail.com>, Kazunori Ogata <OGATAK(a)jp.ibm.com> Cc: core-libs-dev(a)openjdk.java.net Date: 2017/10/24 15:49 Subject: Re: RFR: 8188858: Caching latestUserDefinedLoader() results in ObjectInputStream.readObject() Hi Peter, Ogata, Are you saying this is correct even if cachedLoader is not volatile? If so then that needs to be clearly documented. Cheers, David On 24/10/2017 3:11 PM, Peter Levart wrote: > Hi, > > I think that what Ogata has in webrev.03 is correct and the reasoning > could be as follows: > > - each thread writes to field 'cachedLoader' only from its own set of > values that are distinct from the sets of values that may be written by > any other thread, except null value. > - each thread reads field 'cachedLoader' and can verfy that it has read > a value that belongs to the set of its own written values, or null value > (in other words, a thread can verify that it has read a value that was > written by self or null value). > - reads and writes of own set of non-null values always appear in > program order since they are performed by same thread > - a sequence of writes of own set of non-null values performed by some > thread begins after this thread 1st observes a null value read from the > field and ends before this thread finally writes null value back to the > field. The last write performed by some thread is therefore always a > write of null value. > > No matter how writes performed by a mixture of threads finally hit the > actual field, since each thread that writes to it issues its final write > of null value, the value that eventually ends in the field is null value. > > Does this make sense? > > Regards, Peter > > On 10/23/17 22:47, Peter Levart wrote: >> Hi Ogata, >> >> Sorry for late reply. You are absolutely right. Good catch! I missed >> this scenario. The criteria for placing the mark (current Thread) on a >> cachedLoader must include the check that validates previous value for >> later restoration which uses the same criteria. Only in such case will >> one thread never restore something that has not been placed by it. And >> this guarantees that consumed OIS will never retain a reference to any >> ClassLoader or Thread. Your webrev.03 looks good to me. >> >> Regards, Peter >> >> On 10/17/17 13:48, Kazunori Ogata wrote: >>> Hi Peter, >>> >>> Thank you for your comments and the fix. It's a good idea to mark >>> cachedLoader with the Thread object. >>> >>> >>> I think we need to check the marking thread of cachedLoader before >>> updating it. Otherwise, there is a scenario that can leak a >>> CachedLoader >>> object: >>> >>> //1. Thread-A enters readObject() and then call resolveClass() >>> outerCL-A <- null >>> cachedLoader <- Thread-A >>> cachedLoader <- CachedLoader-A >>> >>> //2. Thread-B enters readObject() and then call resolveClass() >>> outerCL-B <- CachedLoader-A >>> cachedLoader <- Thread-B >>> cachedLoader <- CachedLoader-B1 >>> >>> //3. Thread-B returns from readObject() >>> cachedLoader is unchanged because outerCL.thread == Thread-A >>> >>> //4. Thread-B enters readObject() again and then call resolveClass() >>> outerCL-B <- CachedLoader-B1 >>> cachedLoader <- Thread-B >>> cachedLoader <- CachedLoader-B2 >>> >>> //5. Thread-A returns from readObject() >>> cachedLoader <- null >>> >>> //6. Thread-B returns from readObject() >>> cachedLoader <- CachedLoader-B1 // Because outerCL-B.thread is Thread-B >>> >>> >>> By adding checking before updating the mark, Thread-B won't update >>> cachedLoader, or it only saves null when race occurs (and always >>> restores >>> to null on exit). >>> >>> >>> Here is the updated webrev: >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__cr.openjdk.java.net_-7E… >>> >>> I also made minor changes to reduce the number of invocation of the JNI >>> method Thread.currentThread(). >>> >>> >>> Regards, >>> Ogata >>> >>> >>> >>> From: Peter Levart<peter.levart(a)gmail.com> >>> To: Kazunori Ogata<OGATAK(a)jp.ibm.com>, Alan Bateman >>> <Alan.Bateman(a)oracle.com> >>> Cc:core-libs-dev@openjdk.java.net >>> Date: 2017/10/16 19:58 >>> Subject: Re: RFR: 8188858: Caching latestUserDefinedLoader() >>> results in ObjectInputStream.readObject() >>> >>> >>> >>> Hi Ogata, >>> >>> I found a problem in my last suggestion. See below... >>> >>> On 10/16/2017 11:36 AM, Peter Levart wrote: >>>> On 10/16/2017 11:02 AM, Peter Levart wrote: >>>>> For example: >>>>> - let public readObject() / readUnshared() entry and exit points just >>>>> clear the cached loader (set it to null). >>>> An alternative would be for entry point to save and clear the cached >>>> loader while exit point would restore / clear it if it is from correct >>>> thread / when the call was not nested. Like the following: >>>> >>>> public Object readObject() { >>>> CachedLoader outerCL = cachedLoader; >>>> cachedLoader = null; >>>> try { >>>> ... >>>> } finally { >>>> if (outerCL == null || outerCL.thread == >>>> Thread.currentThread()) { >>>> // restore/clear cached loader when nested/outer call ends >>>> cachedLoader = outerCL; >>>> } >>>> } >>>> } >>>> >>>> with resolveClass() fragment repeated here for comparison: >>>> >>>> CachedLoader cl = cachedLoader; >>>> Thread curThread = Thread.currentThread(); >>>> ClassLoader loader; >>>> if (cl == null) { >>>> loader = latestUserDefinedLoader(); >>>> cachedLoader = new CachedLoader(loader, curThread); >>>> } else if (cl.thread == curThread) { >>>> loader = cl.loader; >>>> } else { >>>> // multi threaded use >>>> loader = latestUserDefinedLoader(); >>>> } >>>> >>>> // and then... >>>> return Class.forName(name, false, loader); >>>> >>>> >>>> There are all sorts of races possible when called concurrently from >>>> multiple threads, but the worst consequence is that the loader is not >>>> cached. I also think that even in the presence of races, the >>>> cachedLoader is eventually cleared when all calls to OIS complete. I >>>> couldn't think of a situation where such cached loader would remain >>>> hanging off the completed OIS because of races. >>>> >>>> Well, there is one such situation but for a different reason. For >>>> example, if an OIS subclass is constructed solely to override >>>> resolveClass method to make it accessible to custom code (for example, >>>> make it public and call super.resolveClass()) in order to provide a >>>> utility for resolving classes with the default OIS semantics, but such >>>> OIS instance is never used for deserialization itself >>>> (readObject()/readUnshared() is never called). >>>> >>>> To solve this problem, resolveClass() logic, including lazy caching, >>>> should be moved to a private method (resolveClass0()) with protected >>>> resolveClass() treated like public readObject()/readUnshared() with >>>> before/after treatment of cached loader around delegation to >>>> resolveClass0(). All OIS internal uses of resolveClass() should then >>>> be redirected to resolveClass0(). >>> Oops, this would not work for subclasses that override resolveClass() >>> with custom logic. Hm... >>> >>> The correct and optimal solution is a little bit more involved, I think. >>> Here's what I think should work (did not run any tests yet): >>> >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__cr.openjdk.java.net_-7E… >>> >>> >>> >>> >>> Regards, Peter >>> >>> >>> >>> >> >

4 3

Re: [BUG PROPOSAL]: C++ code that calls JNI_CreateJavaVM can be exited by java
by Adam Farley8 26 Oct '17

26 Oct '17

Thanks David :) Please let me know what the bug number is when you have time. Everyone else: Any volunteers to champion this? I'm on IRC 9-5 from monday-friday if anyone wants to discuss it. Best Regards Adam Farley P.S. Here's the summary including David's addition, minus the chevrons: 1) Exit(0) (during VM startup) is a big bug because it imitates successful completion of external cpp code accessing JNI methods. 2) One solution for (1) is to specify a new return code for JNI. 3) The supplied code (diff) generates, facilitates, and handles that return code for the exit(0) scenario: -agentlib:jdwp=help 4) The exit(0) problem (in general) is worth fixing, however we may choose not to support the use of this new code in the jdwp example case. 5) The supplied test confirms that the supplied code works (run via unzip, and then bash TestStart.sh <path to jdk home dir that contains bin dir>). 6) To implement this new return code, plus the code that handles it, we would need to follow the CSR process to modify the JNI spec. 7) To implement the jdwp scenario fix, if we choose to support it at all, we would also need to use the CSR process the JVM TI spec. 8) To address all of the worst instances of exit(0), we would need to search for exit(0) and raise a bug for each significant one (or group). 9) To solve (8) in one bug would be a lot of work, arguably too much work for a single bug. 10) If the new return code is identified as the appropriate solution to this problem, we will need to agree on the right name and return code value. Also, here's a shortlist of the main questions I recall being raised here, plus answers people have given. A) What are the potential solutions for the exit(0) problem? i) New JNI Return Code. ii) Remove the info-only options, at least via the JNI, and return a standard error code if they are used. iii) Remove the info-only options, at least via the JNI, and filter them out if they are used. iv) Retain existing behaviour, and document a need for the user to filter out help options before starting the VM via the JNI. B) What are the criteria for the "Best" solution? i) It must prevent "exit(0)" calls. ii) It must be proven to work. iii) It should require minimal (none, ideally) behaviour change from the java.exe user. iv) It should allow the external cpp code accessing the JNI to complete normally, without being prematurely terminated. C) Which solutions meet the (B) criteria? i) Both the "new return code" and the "remove info-only options" solutions meet the (B) criteria. D) Is it right to have any "Do X and then exit." arguments at all, and (if no) what would be the alternatives? i) Given the VM is a loadable shared library the answer should be No. We should not have any "Do X and then exit" arguments. The alternatives would be a mix of: - added Dcmds to "Do X" - added launcher smarts to recognize options that need a Dcmd and "then exit". Note from David: I find the notion of "help" options problematic for a shared library. But I don't have a good answer for how else to document things. That all said I think this ship has sailed and we're unlikely to want to invest the time and effort in trying to clean this up this way. From: David Holmes <david.holmes(a)oracle.com> To: Adam Farley8 <adam.farley(a)uk.ibm.com> Cc: Alan Bateman <Alan.Bateman(a)oracle.com>, core-libs-dev(a)openjdk.java.net, hotspot-runtime-dev(a)openjdk.java.net, thomas.stuefe(a)gmail.com Date: 25/10/2017 13:53 Subject: Re: [BUG PROPOSAL]: C++ code that calls JNI_CreateJavaVM can be exited by java Hi Adam, On 18/10/2017 9:17 PM, Adam Farley8 wrote: > Hi All, > > Updated summary, based on the responses: I think this is a good summary. Thanks. I can file a bug for this, but it will take some work to see how to fit this into the existing specifications and file CSR requests for those changes. This won't make 18.3 (or whatever it gets called). You will need a "champion" to help flesh this out in full and work with you on those CSR requests. I can't volunteer to do that at this time. > 1) Exit(0) (during VM startup) is a big bug because it imitates > successful completion of external cpp code accessing JNI methods. > 2) One solution for (1) is to specify a new return code for JNI. > 3) The supplied code (diff) generates, facilitates, and handles that > return code for the exit(0) scenario: -agentlib:jdwp=help > 4) The exit(0) problem (in general) is worth fixing, however we may > choose not to support the use of this new code in the jdwp example case. > 5) The supplied test confirms that the supplied code works (run via > unzip, and then bash TestStart.sh <path to jdk home dir that contains > bin dir>). > 6) To implement this new return code, plus the code that handles it, we > would need to follow the CSR process to modify the JNI spec. > 7) To implement the jdwp scenario fix, if we choose to support it at > all, we would also need to use the CSR process the JVM TI spec. > 8) To address all of the worst instances of exit(0), we would need to > search for exit(0) and raise a bug for each significant one (or group). > 9) To solve (8) in one bug would be a lot of work, arguably too much > work for a single bug. > 10) If the new return code is identified as the appropriate solution to > this problem, we will need to agree on the right name and return code > value. > > Also, here's a shortlist of the main questions I recall being raised > here, plus answers people have given. > > A) What are the potential solutions for the exit(0) problem? > i) New JNI Return Code. > ii) Remove the info-only options, at least via the JNI, and return > a standard error code if they are used. > iii) Remove the info-only options, at least via the JNI, and filter > them out if they are used. > iv) Retain existing behaviour, and document a need for the user to > filter out help options before starting the VM via the JNI. > > B) What are the criteria for the "Best" solution? > i) It must prevent "exit(0)" calls. > ii) It must be proven to work. > iii) It should require minimal (none, ideally) behaviour change > from the java.exe user. > iv) It should allow the external cpp code accessing the JNI to > complete normally, without being prematurely terminated. > > C) Which solutions meet the (B) criteria? > i) Both the "new return code" and the "remove info-only options" > solutions meet the (B) criteria. > > D) Is it right to have any "Do X and then exit." arguments at all, and > (if no) what would be the alternatives? > i) ? Given the VM is a loadable shared library the answer should be No. We should not have any "Do X and then exit" arguments. The alternatives would be a mix of: - added Dcmds to "Do X" - added launcher smarts to recognize options that need a Dcmd and "then exit". I find the notion of "help" options problematic for a shared library. But I don't have a good answer for how else to document things. That all said I think this ship has sailed and we're unlikely to want to invest the time and effort in trying to clean this up this way. Thanks, David > Best Regards > > Adam Farley > > P.S. As per Alan and David's emails, the exit(#) references have been > removed entirely, as discussing them alongside the original exit(0) > problem risks scope creep. > > This bug, if raised, should only cover the exit(0) cases. I believe we > have consensus here. > > > > From: David Holmes <david.holmes(a)oracle.com> > To: Adam Farley8 <adam.farley(a)uk.ibm.com>, Alan Bateman > <Alan.Bateman(a)oracle.com>, core-libs-dev(a)openjdk.java.net, > hotspot-runtime-dev(a)openjdk.java.net, thomas.stuefe(a)gmail.com > Date: 13/10/2017 14:31 > Subject: Re: [BUG PROPOSAL]: C++ code that calls JNI_CreateJavaVM can be > exited by java > ------------------------------------------------------------------------ > > > > Hi Adam, > > On 13/10/2017 10:16 PM, Adam Farley8 wrote: > > Hi All, > > > > Here's a summary of the email below (which is intended, partly, as a > > summary of the emails before it). > > > > Let me know if you agree/disagree with any of these points. > > > > 1) Exit(#) during vm startup is a bug because it should return a code > > regardless of the state of the VM. > > Yes it's a bug but not one that is likely to be addressed in any > foreseeable timeframe. There are simply too many "exit on error" paths. > If we were to start using C++ exceptions within the VM that might > provide a way to quickly get back to the CreateJavaVM routine where we > could return an error code - but that is itself a major project that has > barely even been discussed AFAIK. (Compiler folk have talked about it > because compiler paths are fairly self-contained - though that was > before Graal and AOT.) > > > 2) Exit(0) is an *especially* big bug because it imitates successful > > completion of external cpp code accessing JNI methods. > > 3) One solution is to specify a new return code for JNI. > > A solution for 2) yes. > > > 4) The supplied code (diff) generates, facilitates, and handles that > > return code for the exit(0) scenario: -agentlib:jdwp=help > > 5) The supplied test confirms that the supplied code works (run via > > unzip, and then bash TestStart.sh <path to jdk home dir that contains > > bin dir>). > > 6) To implement this new return code, plus the code that handles it, we > > would need to follow the CSR process. > > 7) To implement the fix for the scenario used as an example of the new > > return code's use, we would need to modify the JVM TI spec. > > Yes you have demonstrated a potential solution for the agent case. The > question is, is it the right solution? Is it a worthwhile solution? (As > I've said I'd prefer not to have any "do something then exit" VM > arguments.) And can we make it fit into the existing specs without > contorting things too much. > > I still think it easier/preferable for whatever loads the VM to filter > out the VM args that trigger this behaviour. I mean if you pass > -Xshare:dump you don't have any right to expect a functioning VM after > JNI_CreateJavaVM returns - at least I don't think so. Just don't do it. > Yes it is an imperfection in the invocation API, but life isn't perfect. > > > 8) To address all of the worst instances of exit(#), we would need to > > search for exit(#) and raise a bug for each significant one (or group). > > 9) To solve (8) in one bug would be a lot of work, arguably too much > > work for a single bug. > > This is simply impractical. You may be able to pick off a few > low-hanging cases, but that won't really make any practical difference. > > > 10) If the new return code is chosen as the appropriate solution to this > > problem, we may need to choose a better name for the return code. > > > > Is this a fair assessment of the current state of the debate? > > It's a fair summary of your position and proposal. > > Cheers, > David > > > I'm on IRC every weekday from 9am-5pm (4pm fridays) BST (GMT+1) if > > anyone wants to discuss this in real-time on the openjdk channel. > > > > Best Regards > > > > Adam Farley > > > > > > > > -- Previous Email -- > > > > Hi David, Alan, > > > > You are right in that the changes to HotSpot would be nontrivial. > > I see a number of places in (e.g.) arguments.cpp that seem to > > exit in the same manner as Xlog (such as -Xinternalversion). > > > > I would advise ploughing through the CSR process to alter the > > JNI spec, and simultaneously identify some key paths that can > > be raised as bugs. That way, when people have time to address > > these issues, the mechanism to handle a silent exit is already > > in place. > > > > The JDWP fix can be raised separately as one of these bugs, if > > it would make things simpler. > > > > As for the name, JNI_SILENT_EXIT is a placeholder, and can be > > readily changed. Do you have any suggestions? > > > > Lastly, in an ideal world, the VM initialisation should never exit(#). It > > should return a return code that tells the caller something, pass or > > fail, messy or tidy. That way, if someone is using the JNI as part of > > something bigger (like a database or a web server), one of these > > scenarios is just a bug, rather than a world-ender like exit(#). > > > > And now for the individual messages. :) > > > > David: Having help data returned by the launcher seems like a > > good way to avoid exit(0) calls, but I'm not sure how we'd prevent > > a JNI-caller using those options. Ultimately, to be sure, we'd have > > to remove the logic for those options, centralise the data to better > > enable launcher access, and add some logic in there so it can find > > any other help data (e.g. from the jdwp agent library). I feel this would > > be a bigger task than adding the new return code and changing the > > vm, plus it wouldn't provide for any non-help scenarios where the > > vm wants to shut down without error during initialisation. > > > > Alan: I should mention that the silent exit solution is already in > > use in the OpenJ9 VM. Not all of the exit paths have been > > resolved, but many have. > > > > The code is open and can be found here: > < https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_eclipse_ope… > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_eclipse_ope… > > > > And though the silent exit code is disabled for the time being, it > > can be re-enabled by entering this class: > > > > runtime/vm/jvminit.c > > > > and altering line 2343 ( ctrl-f for exit(1) if it's not there). > > > > I won't paste the full code here in case people are concerned > > about contamination, but I would assert that this code (and the > > associated vm files) prove that the concept is possible. > > > > Note that that code should not be enabled until after we've > > integrated the code that can handle a silent exit. > > > > Best Regards > > > > Adam Farley > > > > P.S. Thank you both for your efforts on this. :) > > > > > > > > From: David Holmes <david.holmes(a)oracle.com> > > To: Alan Bateman <Alan.Bateman(a)oracle.com>, Adam Farley8 > > <adam.farley(a)uk.ibm.com>, core-libs-dev(a)openjdk.java.net, > > hotspot-runtime-dev(a)openjdk.java.net, thomas.stuefe(a)gmail.com > > Date: 15/09/2017 12:03 > > Subject: Re: [BUG PROPOSAL]: C++ code that calls JNI_CreateJavaVM can be > > exited by java > > ------------------------------------------------------------------------ > > > > > > > > > > > > On 15/09/2017 8:17 PM, Alan Bateman wrote: > > > On 15/09/2017 02:47, David Holmes wrote: > > >> Hi Adam, > > >> > > >> I am still very much torn over this one. I think the idea of > > >> print-and-exit flags for a potentially hosted library like the JVM is > > >> just wrong - we should never have done that, but we did. Fixing that > > >> by moving the flags to the launcher is far from trivial**. Endorsing > > >> and encouraging these sorts of flag by adding JNI support seems to be > > >> sending the wrong message. > > >> > > >> ** I can envisage a "help xxx" Dcmd that can read back the info from > > >> the VM. The launcher can send the Dcmd, print the output and > exit. The > > >> launcher would not need to know what the xxx values mean, but would > > >> have to intercept the existing ones. > > >> > > >> Another option is just to be aware of these flags (are there more > than > > >> jdwp and Xlog?) and deal with them specially in your custom > launcher - > > >> either filter them out and ignore them, or else launch the VM in its > > >> own process to respond to them. > > >> > > >> Any changes to the JNI specification need to go through the CSR > process. > > > Yes, it would require an update to the JNI spec, also a change to the > > > JVM TI spec where Agent_OnLoad returning a non-0 value is specified to > > > terminates the VM. The name and value needs discussion too, esp. > as the > > > JNI spec uses negative values for failure. > > > > > > In any case, I'm also torn over this one as it's a corner case that is > > > only interesting for custom launchers that load agents with > options that > > > print usage messages. It wouldn't be hard to have the Agent_OnLoad > > > specify a printf hook that the agent could use for output although > there > > > are complications with agents such as JDWP that also announce their > > > transport end point. Beyond that there is still the issue of the > custom > > > launcher that would need to know to destroy the VM without > reporting an > > > error. > > > > > > So what happened to the more meaty part to this which is fixing the > > > various cases in HotSpot that terminate the process during > > > initialization? I would expect some progress could be made on those > > > cases while trying to decide whether to rev the JNI and JVM TI > specs to > > > cover the help case. > > > > Trying to eliminate the vm_exit_during_initialization paths in hotspot > > is a huge undertaking IMHO. > > > > David > > > > > > > > -Alan > > > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > PO6 3AU > > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

RFR: 8183901 : Fix broken links to "Package Sealing" in the JAR spec
by Brent Christian 25 Oct '17

25 Oct '17

Hi, Please review my fix of a few broken links to the "Package Sealing" section of the Jar Spec[1]. http://cr.openjdk.java.net/~bchristi/8183901/ Thanks, -Brent 1. https://docs.oracle.com/javase/9/docs/specs/jar/jar.html#package-sealing

3 2