From david.holmes at oracle.com Sun Jul 1 21:48:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Jul 2018 07:48:10 +1000 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: References: Message-ID: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> Hi Volker, This doesn't really address any of the concerns I had with the original proposal - it just moves the "field" from the Java side to the VM side. There is still a massive amount of Java code execution in relation to this - which itself may encounter secondary exceptions. It's very hard to tell if you will leave things in a suitable state if such exceptions arise. My position remains that the primary place to deal with the initialization error is when initialization occurs and the error happens. Subsequent attempted uses of the erroneous class may benefit from some additional information about the nature of the original exceptions, but I don't think full stacktraces are necessary or desirable (and I do believe they will confuse most users given the lack of continuity in the stack frames and that they may have happened in a different thread!). That aside ... There appears to a race on constructing the Hashtable. At least it was not obvious to me where a lock may be held during that process. I can't determine that clearing backtrace in removeNativeBacktrace is correct with respect to the overall protocol within Throwable for dealing with backtrace and stackTrace. I have to wonder why nothing in Throwable clears the backtrace today ? I'm not clear why you record the ExceptionInInitializerError wrapper instead of the actual exception that occurred? Throwable states: + * This method is currently only called from the VM for instances of + * ExceptionInInitializerError which are stored for later chaining into a + * NoClassDefFoundError in order to prevent keeping classes from the native + * backtrace alive. + */ but IIUC it will also be called for instances of Error that occurred which do not get wrapped in EIIE. Regards, David ------ On 30/06/2018 12:53 AM, Volker Simonis wrote: > Hi, > > can I please have a review for the following change which saves > ExceptionInInitializerError thrown during class initialization and > chains them as cause into potential NoClassDefFoundErrors for the same > class. We are using this features since years in our commercial SAP > JVM and it proved extremely useful for detecting and fixing errors > especially in big deployments. > > This is a follow-up on a discussion previously started by Goetz [1]. > His first proposal (which is close to our current, internal > implementation) inserted an additional field into java.lang.Class > objects to save potential ExceptionInInitializerErrors. This was > considered to much overhead in the initial discussion [1]. > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8203826.v2/ > https://bugs.openjdk.java.net/browse/JDK-8203826 > > So in this change, I've completely re-implemented the feature by using > a java.lang.Hashtable which is attached to the ClassLoaderData object. > The Hashtable is lazily created when the first > ExceptionInInitializerError is thrown and maps the Class which > triggered the ExceptionInInitializerError during the execution of its > static initializer to the corresponding ExceptionInInitializerError. > > If the same class will be accessed once again, this will directly lead > to a plain NoClassDefFoundError (as per the JVMS, 5.5 Initialization) > because the static initializer won't be executed a second time. Until > now, this NoClassDefFoundError wasn't linked in any way to the root > cause of the problem (i.e. the first ExceptionInInitializerError > together with the chained exception that happened during the execution > of the static initializer). With this change, the NoClassDefFoundError > will now chain the initial ExceptionInInitializerError as cause, > making it much easier to detect the problem which lead to the > NoClassDefFoundError. > > Following is an example from the new JTreg tests which comes which > this change to demonstrate the feature. Until know, a typical stack > trace from a NoClassDefFoundError looked as follows: > > java.lang.NoClassDefFoundError: Could not initialize class > NoClassDefFound$ClassWithFailedInitializer > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:291) > at NoClassDefFound.main(NoClassDefFound.java:38) > > With this change, the same stack trace now looks as follows: > > java.lang.NoClassDefFoundError: Could not initialize class > NoClassDefFound$ClassWithFailedInitializer > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:315) > at NoClassDefFound.main(NoClassDefFound.java:38) > Caused by: java.lang.ExceptionInInitializerError > at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) > at java.base/java.lang.Class.newInstance(Class.java:584) > at NoClassDefFound$ClassWithFailedInitializer.(NoClassDefFound.java:20) > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:315) > at NoClassDefFound.main(NoClassDefFound.java:30) > Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 2 out of > bounds for length 1 > at NoClassDefFound$A.(NoClassDefFound.java:9) > ... 9 more > > As you can see, the reason for the NoClassDefFoundError when accessing > the class 'NoClassDefFound$ClassWithFailedInitializer' is actually not > even in the class or its static initializer itself, but in the class > 'NoClassDefFound$A' which is a base class of > 'NoClassDefFound$ClassWithFailedInitializer'. This is not easily > detectible from the old, plain NoClassDefFoundError. > > As I wrote, the only overhead we have with the new implementation is > an additional OopHandle field per ClassLoaderData which I think is > acceptable. The Hashtable object itself is only created lazily, after > the first occurrence of an ExceptionInInitializerError in the > corresponding class loader. The whole Hashtable creation and > storing/quering of ExceptionInInitializerErrors in > ClassLoaderData::record_init_exception()/ClassLoaderData::query_init_exception() > is optional in the sense that any errors/exceptions occurring during > the execution of these functions are ignored and cleared. > > Finally, we also take care to recursively convert all native > backtraces in the stored ExceptionInInitializerErrors (and their > suppressed and chained exceptions) into symbolic stack traces in order > to avoid holding references to classes and prevent their unloading. > This is implemented in the new private, static method > java.lang.Throwable::removeNativeBacktrace() which is called for each > ExceptionInInitializerError before it is stored in the Hashtable. > > Thank you and best regards, > Volker > > [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-June/028310.html > From erik.gahlin at oracle.com Mon Jul 2 08:49:14 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 2 Jul 2018 10:49:14 +0200 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> <5B350AE5.3080401@oracle.com> Message-ID: <4F777EA1-6402-4725-B4E8-6DCB61DD4E19@oracle.com> > On 29 Jun 2018, at 17:34, Se?n Coffey wrote: > > I've introduced a new test helper class in the jdk/test/lib/jfr directory to help with the dual test nature of the new tests. It's helped alot with test code duplication. > My thinking was to put things like the certificates in a separate file, i.e TestCertificates, and then have a logging test and a JFR test reuse it. One rationale for adding logging was to use it if JFR is not present. By putting the tests together, it becomes impossible to compile and test logging without having JFR. > Looked at use of @DataAmount(DataAmount.BITS) also. Not sure if it's fits. The output is displayed in units like "KiB" - not the normal when examining key lengths used in X509Certificates. i.e a 2048 bit key gets displayed as "2 KiB" - I'd prefer to keep the 2048 display version. We should not let the JMC GUI decide how units are specified. There will be other GUIs and this is the first event that uses bits, so I don?t think it is formatted that way because it was considered superior. Erik > > new webrev at: http://cr.openjdk.java.net/~coffeys/webrev.8148188.v4/webrev/ > > Regards, > Sean. > > On 28/06/18 17:59, Se?n Coffey wrote: >> Comments inline. >> >> >> On 28/06/2018 17:20, Erik Gahlin wrote: >>> It's sufficient if an event object escapes to another method (regardless if JFR is enabled or not). >>> >>> Some more feedback: >>> >>> Rename event jdk.CertChain to jdk.CertificateChain >>> Rename event jdk.X509Cert to jdk.X509Certificate >>> Rename field certChain to certificateChain. >>> Rename field serialNum to serialNumber >> all above done. >>> Rename field algId to AlgorithmicId or AlgorithmicID >> maybe "algorithm" works here also ? >>> Rename @Label("Ciphersuite") to @Label("Cipher Suite") >>> Rename @Label("Cert Chain") to @Label("Certificate Chain") >>> Rename @Label("Property Name") to "Name" or "Key" if that makes sense in the context? >>> Rename @Label("Property Value") to "Value". >>> Put events in a subcategory, i.e @Category({"Java Development Kit", "Security"}) >> done. >>> Make classes final. >> done. I had thought that the JFR mechanism required non-final classes. >>> What is the unit of the key in X509Certificate event? Bits? If that is the case, use @DataAmount(DataAmount.BITS) >> Yes - it's essentially the bit length of the keys used. Let me look into that annotation usage. >>> @Label("Serial numbers forming chain of trust") should not be a sentence. How about "Serial Numbers"? >>> >>> I think the tests are hard to read when two things are tested at the same time. It is also likely that if a test fail due to logging issues, it will be assigned to JFR because of the test name, even thought the issues is not JFR related. >> I think we're always going to have some ownership issues when tests serve a dual purpose. I still think it makes sense to keep the test logic in one place. Any failures in these tests will most likely be owned by security team. (moving the tests to security directory is also an option) >>> >>> If you want to reuse code between tests, I would put it in testlibrary. >> Let me check if there's any common patterns that could be added to the testlibary. >> >> Thanks, >> Sean. >>> >>> Erik >>> >>>> Thanks for the update Erik. By default I'm proposing that the new JFR Events and Logger be disabled. As a result the event class shouldn't escape. If performance metrics highlight an issue, we should revisit. >>>> >>>> regards, >>>> Sean. >>>> >>>> >>>> On 27/06/2018 20:57, Erik Gahlin wrote: >>>>> On 2018-06-27 21:14, Se?n Coffey wrote: >>>>>> >>>>>> >>>>>> On 27/06/2018 19:57, Xuelei Fan wrote: >>>>>>> Hi Sean, >>>>>>> >>>>>>> I may reply in several replies. >>>>>>> >>>>>>> PKIXMasterCertPathValidator.java >>>>>>> -------------------------------- >>>>>>> + CertChainEvent cce = new CertChainEvent(); >>>>>>> + if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>>>>>> + String c = reversedCertList.stream() >>>>>>> + .map(x -> x.getSerialNumber().toString(16)) >>>>>>> + .collect(Collectors.joining(", ")); >>>>>>> + EventHelper.commitCertChainEvent(cce, c); >>>>>>> + } >>>>>>> >>>>>>> No matter the event or the JFR mechanism is enabled or not, the above code will create a new instance. Is the return value of cce.isEnabled() dynamically changed or static? >>>>>> This is a requirement from the JFR framework. All their event.isEnabled calls are instance methods and follow a similar pattern. The JFR team assure me that the JIT can optimize away such calls. >>>>> >>>>> The JIT will most likely not be able to optimize if the event class escapes. >>>>> >>>>> From a JFR perspective, this would be the preferred layout: >>>>> >>>>> EventA eventA= new EventA(); >>>>> eventA.value = this.value; >>>>> eventA.commit(); >>>>> >>>>> and then do logging separately: >>>>> >>>>> System.Logger.log(DEBUG, this.value) >>>>> >>>>> With this layout, the JIT will remove the allocation and dead store. >>>>> >>>>> If it is expensive to gather the data for the event, like the CertChainEvent above, the following pattern should be used. >>>>> >>>>> EventB eventB= new EventB (); >>>>> if (eventB.shouldCommit()) { >>>>> eventB.value = calculateValue(); >>>>> eventB .commit(); >>>>> } >>>>> >>>>> If JFR is not enabled, shouldCommit returns false and the JIT should be able to remove the allocation. This will be best from a performance point of view, and also in my opinion from a maintenance and readability perspective. Others may disagree. >>>>> >>>>> Erik >>>>> >>>>>>> >>>>>>> Is there a need to support both logging and JFR? I'm new to record events. I did not get the point to use both. >>>>>> I was initially hoping to concentrate on just JFR events but I got strong feedback to also consider use of Logger (esp. in cases where the jdk.jfr module is not available) >>>>>> >>>>>> regards, >>>>>> Sean. >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Xuelei >>>>>>> >>>>>>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>>>>>> Erik, >>>>>>>> >>>>>>>> I rebased the patch with TLS v1.3 integration today. I hadn't realized how much the handshaker code had changed. Hopefully, I'll get a review from security dev team on that front. >>>>>>>> >>>>>>>> Regards the JFR semantics, I believe the edits should match majority of requests . I still have a preference for the test infra design for these new logger/JFR tests used in version 1 of webrev. I think it makes sense to keep the test functionality together - no sense in separating them out into different components IMO. Also, some of the edits to the JFR testing were made to test for the new dual log/record functionality. I might catch up with you tomorrow to see what the best arrangement would be. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>>>>>> >>>>>>>> regards, >>>>>>>> Sean. >>>>>>>> >>>>>>>> >>>>>>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>>>>>> Many thanks for the review comments Erik. Replies inline. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>>>>>> Hi Sean, >>>>>>>>>> >>>>>>>>>> Some of the changes in the webrev belongs to JDK-8203629 and should be removed for clarity. >>>>>>>>>> >>>>>>>>>> Some initial comments: >>>>>>>>>> >>>>>>>>>> default.jfc, profile.jfr: >>>>>>>>>> The events should not have control="enable-exceptions". The purpose of the control attribute is so to provide parameterized configuration of events for JMC. As it is now, the security events will be enabled when a user turns on the exception events. >>>>>>>>> Makes sense. I'll remove that parameter. >>>>>>>>>> >>>>>>>>>> X509CertEvent: >>>>>>>>>> You should use milliseconds since epoch to represent a date instead of a string value, i.e. >>>>>>>>>> >>>>>>>>>> @Label("Valid From") >>>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>>> public long validFrom; >>>>>>>>>> >>>>>>>>>> @Label("Valid Until") >>>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>>> public long validUntil; >>>>>>>>>> >>>>>>>>> The CertificateValidity class operates on Date Object values. I'll work with the Date.getTime() method then (and update the Logger formatting) >>>>>>>>>> This following annotation adds little value >>>>>>>>>> >>>>>>>>>> @Description("Details of Security Property") >>>>>>>>>> >>>>>>>>>> I would either remove the annotation, or provide information that helps a user understand the event. For instance, what is X509, and what kind of certificates are we talking about? >>>>>>>>> Yes - that looks like the wrong Description. I'll review all of these fields. >>>>>>>>>> >>>>>>>>>> @Category("Java Application") >>>>>>>>>> >>>>>>>>>> I'm a bit worried that we will pollute the "Java Application" namespace if we add lots of JDK events in that category. We may want to add a new top level category "Java Development Kit", analogous to the "Java Virtual Machine" category, and put all security related events in subcategory, perhaps called "Security". >>>>>>>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>>>>>>> >>>>>>>>>> @Label("X509Cert") >>>>>>>>>> >>>>>>>>>> The label should be human readable name, with spaces, title cased etc. I would recommend "X.509 Certificate". In general, avoid abbreviations like "certs" and instead use labels such as "Certificate Chain". The label should be short and not a full sentence. >>>>>>>>>> >>>>>>>>>> For details see, >>>>>>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>>>>>> >>>>>>>>>> I think it would be good to separate testing of JFR and logging into different files / tests. I would prefer that the test name is the same as the event prefixed with "Test", i.e TestX509CertificateEvent, as this is the pattern used by other JFR tests. >>>>>>>>>> >>>>>>>>> I'll take a look at that pattern again. I've separated out the current tests into an (a) outer test to analyze the logger output and (b) the inner test which checks for JFR correctness. I did include extra logic to make sure that the EventHelper configuration was working correctly. "Events.assertField" looks very helpful. Thanks for the pointer. >>>>>>>>> >>>>>>>>> Let me take on board the suggestions below and get a new webrev out on Tuesday. >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> Sean. >>>>>>>>> >>>>>>>>>> I reworked one of the tests to how I like to see it: >>>>>>>>>> >>>>>>>>>> /* >>>>>>>>>> * @test >>>>>>>>>> * @key jfr >>>>>>>>>> * @library /test/lib >>>>>>>>>> * @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>>>>>>>> */ >>>>>>>>>> public class TestX509CertificateEvent { >>>>>>>>>> >>>>>>>>>> private static final String CERTIFICATE_1 = "..."; >>>>>>>>>> private static final String CERTIFICATE_2 = "..."; >>>>>>>>>> >>>>>>>>>> public static void main(String... args) throws CertificateException { >>>>>>>>>> >>>>>>>>>> Recording r = new Recording(); >>>>>>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>>>>>> r.start(); >>>>>>>>>> >>>>>>>>>> CertificateFactory cf = CertificateFactory.getInstance("X.509"); >>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>>> >>>>>>>>>> // Make sure only one event per certificate >>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>>> >>>>>>>>>> r.stop(); >>>>>>>>>> >>>>>>>>>> List events = Events.fromRecording(r); >>>>>>>>>> Asserts.assertEquals(events.size(), 2, "Expected two X.509 Certificate events"); >>>>>>>>>> >>>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>>> "RSA", 2048); >>>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>>> "RSA", 2048); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> private static void assertEvent(List events, String certID, String algId, String subject, >>>>>>>>>> String issuer, String keyType, int length) throws Exception { >>>>>>>>>> >>>>>>>>>> for (RecordedEvent e : events) { >>>>>>>>>> if (e.getString("serialNumber").equals(certID)) { >>>>>>>>>> Events.assertField(e, "algId").equal(algId); >>>>>>>>>> ... >>>>>>>>>> return; >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> System.out.println(events); >>>>>>>>>> throw new Exception("Could not find event with Cert ID: " + certID); >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> The reworked example uses the Events.assertField method, which will give context if the assertion fails. Keeping the test simple, means it can be analyzed quickly if it fails in testing. There is no new test framework to learn, or methods to search for, and it is usually not hard to determine if the failure is product, test or infrastructure related, and what component (team) should be assigned the issue. The use of EventNames.X509Certificate means all occurrences of the event can be tracked done in an IDE using find by reference. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Erik >>>>>>>>>> >>>>>>>>>>> Following on from the recent JDK-8203629 code review, I'd like to propose enhancements on how we can record events in security libs. The introduction of the JFR libraries can give us much better ways of examining JDK actions. For the initial phase, I'm looking to enhance some key security library events in JDK 11 so that they can be either recorded to JFR, logged to a traditional logger, or both. >>>>>>>>>>> >>>>>>>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>>>>>> >>>>>>>>>>> securityProp_2.png gives an example of how the JFR recording can be queried to quickly locate events of interest (in this case, code setting the jdk.tls.* Security properties). I still need to clean up the TLSEvents testcase to improve test coverage and hope to do that in coming days. >>>>>>>>>>> >>>>>>>>>>> JBS record : >>>>>>>>>>> * https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>>>>>> >>>>>>>>>>> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >> > From Sergey.Bylokhov at oracle.com Mon Jul 2 13:36:57 2018 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Mon, 2 Jul 2018 06:36:57 -0700 Subject: [11] Review Request: 8205537 Drop of sun.applet package Message-ID: <34205c13-3f3f-cb20-0ebe-68f9598f6a3a@oracle.com> Hello. Please review the fix for jdk11. Bug: https://bugs.openjdk.java.net/browse/JDK-8205537 Webrev: http://cr.openjdk.java.net/~serb/8205537/webrev.00/ sun.applet is an internal package contained some code related to implementation of applets and appletviewer. Some of its classes were dropped already: JDK-8204454, JDK-8203308. Now there are only 3 classes, all related to the applet security and we can drop them as well. In the fix this package was removed, and the tests were updated to not use it. I am not sure how "ClassnameCharTest.java" is useful after applets removal, but since this test used subclass of the AppletClassLoader, I have copied some code from the removed class to the test. -- Best regards, Sergey. From philip.race at oracle.com Mon Jul 2 16:10:53 2018 From: philip.race at oracle.com (Phil Race) Date: Mon, 2 Jul 2018 09:10:53 -0700 Subject: [11] Review Request: 8205537 Drop of sun.applet package In-Reply-To: <34205c13-3f3f-cb20-0ebe-68f9598f6a3a@oracle.com> References: <34205c13-3f3f-cb20-0ebe-68f9598f6a3a@oracle.com> Message-ID: <39dea342-1fad-121a-2fbb-d67ddd5a2ce0@oracle.com> Sergey, I think at this point, this is not a P3 for 11. So I won't approve it for 11, but will consider it for 12. -phil. On 7/2/2018 6:36 AM, Sergey Bylokhov wrote: > Hello. > Please review the fix for jdk11. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8205537 > Webrev: http://cr.openjdk.java.net/~serb/8205537/webrev.00/ > > sun.applet is an internal package contained some code related to > implementation of applets and appletviewer. Some of its classes were > dropped already: JDK-8204454, JDK-8203308. Now there are only 3 > classes, all related to the applet security and we can drop them as well. > > In the fix this package was removed, and the tests were updated to not > use it. I am not sure how "ClassnameCharTest.java" is useful after > applets removal, but since this test used subclass of the > AppletClassLoader, I have copied some code from the removed class to > the test. > > From Sergey.Bylokhov at oracle.com Mon Jul 2 16:24:52 2018 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Mon, 2 Jul 2018 09:24:52 -0700 Subject: [11] Review Request: 8205537 Drop of sun.applet package In-Reply-To: <39dea342-1fad-121a-2fbb-d67ddd5a2ce0@oracle.com> References: <34205c13-3f3f-cb20-0ebe-68f9598f6a3a@oracle.com> <39dea342-1fad-121a-2fbb-d67ddd5a2ce0@oracle.com> Message-ID: <0242de8e-5de3-bc1e-2cd0-9fa4da3a0bbf@oracle.com> I think it is a good thing to drop this api completely in the same release as JDK-8201446. This is not a p4 and so it can be be fixed and integrated in jdk11 if the fix is ready. On 02/07/2018 09:10, Phil Race wrote: > Sergey, > > I think at this point, this is not a P3 for 11. > So I won't approve it for 11, but will consider it for 12. > > -phil. > > On 7/2/2018 6:36 AM, Sergey Bylokhov wrote: >> Hello. >> Please review the fix for jdk11. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8205537 >> Webrev: http://cr.openjdk.java.net/~serb/8205537/webrev.00/ >> >> sun.applet is an internal package contained some code related to >> implementation of applets and appletviewer. Some of its classes were >> dropped already: JDK-8204454, JDK-8203308. Now there are only 3 >> classes, all related to the applet security and we can drop them as well. >> >> In the fix this package was removed, and the tests were updated to not >> use it. I am not sure how "ClassnameCharTest.java" is useful after >> applets removal, but since this test used subclass of the >> AppletClassLoader, I have copied some code from the removed class to >> the test. >> >> > -- Best regards, Sergey. From philip.race at oracle.com Mon Jul 2 16:30:44 2018 From: philip.race at oracle.com (Phil Race) Date: Mon, 2 Jul 2018 09:30:44 -0700 Subject: [11] Review Request: 8205537 Drop of sun.applet package In-Reply-To: <0242de8e-5de3-bc1e-2cd0-9fa4da3a0bbf@oracle.com> References: <34205c13-3f3f-cb20-0ebe-68f9598f6a3a@oracle.com> <39dea342-1fad-121a-2fbb-d67ddd5a2ce0@oracle.com> <0242de8e-5de3-bc1e-2cd0-9fa4da3a0bbf@oracle.com> Message-ID: It is not necessary and I reject that it is a P3. Let's leave it for 12. -phil. On 07/02/2018 09:24 AM, Sergey Bylokhov wrote: > I think it is a good thing to drop this api completely in the same > release as JDK-8201446. This is not a p4 and so it can be be fixed and > integrated in jdk11 if the fix is ready. > > On 02/07/2018 09:10, Phil Race wrote: >> Sergey, >> >> I think at this point, this is not a P3 for 11. >> So I won't approve it for 11, but will consider it for 12. >> >> -phil. >> >> On 7/2/2018 6:36 AM, Sergey Bylokhov wrote: >>> Hello. >>> Please review the fix for jdk11. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8205537 >>> Webrev: http://cr.openjdk.java.net/~serb/8205537/webrev.00/ >>> >>> sun.applet is an internal package contained some code related to >>> implementation of applets and appletviewer. Some of its classes were >>> dropped already: JDK-8204454, JDK-8203308. Now there are only 3 >>> classes, all related to the applet security and we can drop them as >>> well. >>> >>> In the fix this package was removed, and the tests were updated to >>> not use it. I am not sure how "ClassnameCharTest.java" is useful >>> after applets removal, but since this test used subclass of the >>> AppletClassLoader, I have copied some code from the removed class to >>> the test. >>> >>> >> > > From Sergey.Bylokhov at oracle.com Mon Jul 2 17:16:21 2018 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Mon, 2 Jul 2018 10:16:21 -0700 Subject: [11] Review Request: 8205537 Drop of sun.applet package In-Reply-To: References: <34205c13-3f3f-cb20-0ebe-68f9598f6a3a@oracle.com> <39dea342-1fad-121a-2fbb-d67ddd5a2ce0@oracle.com> <0242de8e-5de3-bc1e-2cd0-9fa4da3a0bbf@oracle.com> Message-ID: <0e94e06a-4465-5e54-6ada-de1c960b2ffd@oracle.com> I can argue that this is not p2 as related JDK-8201446, but it is definitely not p4. On 02/07/2018 09:30, Phil Race wrote: > It is not necessary and I reject that it is a P3. Let's leave it for 12. > > -phil. > > On 07/02/2018 09:24 AM, Sergey Bylokhov wrote: >> I think it is a good thing to drop this api completely in the same >> release as JDK-8201446. This is not a p4 and so it can be be fixed and >> integrated in jdk11 if the fix is ready. >> >> On 02/07/2018 09:10, Phil Race wrote: >>> Sergey, >>> >>> I think at this point, this is not a P3 for 11. >>> So I won't approve it for 11, but will consider it for 12. >>> >>> -phil. >>> >>> On 7/2/2018 6:36 AM, Sergey Bylokhov wrote: >>>> Hello. >>>> Please review the fix for jdk11. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8205537 >>>> Webrev: http://cr.openjdk.java.net/~serb/8205537/webrev.00/ >>>> >>>> sun.applet is an internal package contained some code related to >>>> implementation of applets and appletviewer. Some of its classes were >>>> dropped already: JDK-8204454, JDK-8203308. Now there are only 3 >>>> classes, all related to the applet security and we can drop them as >>>> well. >>>> >>>> In the fix this package was removed, and the tests were updated to >>>> not use it. I am not sure how "ClassnameCharTest.java" is useful >>>> after applets removal, but since this test used subclass of the >>>> AppletClassLoader, I have copied some code from the removed class to >>>> the test. >>>> >>>> >>> >> >> > -- Best regards, Sergey. From huizhe.wang at oracle.com Mon Jul 2 17:44:12 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 02 Jul 2018 10:44:12 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8204329: Java API doc for XMLStreamReader.next() needs to be clarified for the exception thrown when hasNext() method returns false Message-ID: <5B3A646C.90701@oracle.com> Hi, Please review a patch that clarifies that XMLStreamReader.next() throws NoSuchElementException when hasNext() is false by removing the confusing note. JBS: https://bugs.openjdk.java.net/browse/JDK-8204329 CSR: https://bugs.openjdk.java.net/browse/JDK-8206084 webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8204329/webrev/ Thanks, Joe From naoto.sato at oracle.com Mon Jul 2 17:52:26 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 2 Jul 2018 10:52:26 -0700 Subject: [12] RFR: 8206120: Add test cases for lenient Japanese era parsing Message-ID: <1139b79a-e600-a1dc-f03f-928203ed10e3@oracle.com> Hello, Please review the changes to the following issue: https://bugs.openjdk.java.net/browse/JDK-8206120 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8206120/webrev.00/ This is a test case only fix to ensure the new Japanese era implementation introduced with JDK-8202088 accepts lenient era/year, such as "Heisei 32." Naoto From lance.andersen at oracle.com Mon Jul 2 19:07:17 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 2 Jul 2018 15:07:17 -0400 Subject: RFR(JDK12/JAXP/java.xml) 8204329: Java API doc for XMLStreamReader.next() needs to be clarified for the exception thrown when hasNext() method returns false In-Reply-To: <5B3A646C.90701@oracle.com> References: <5B3A646C.90701@oracle.com> Message-ID: +1 > On Jul 2, 2018, at 1:44 PM, Joe Wang wrote: > > Hi, > > Please review a patch that clarifies that XMLStreamReader.next() throws NoSuchElementException when hasNext() is false by removing the confusing note. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8204329 > CSR: https://bugs.openjdk.java.net/browse/JDK-8206084 > webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8204329/webrev/ > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Mon Jul 2 19:21:12 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 2 Jul 2018 15:21:12 -0400 Subject: RFR(JDK12/JAXP/java.xml) 8204329: Java API doc for XMLStreamReader.next() needs to be clarified for the exception thrown when hasNext() method returns false In-Reply-To: References: <5B3A646C.90701@oracle.com> Message-ID: <1596a842-2363-9942-7f7b-0e77aa1825ca@Oracle.com> +1 On 7/2/2018 3:07 PM, Lance Andersen wrote: > +1 > >> On Jul 2, 2018, at 1:44 PM, Joe Wang wrote: >> >> Hi, >> >> Please review a patch that clarifies that XMLStreamReader.next() throws NoSuchElementException when hasNext() is false by removing the confusing note. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8204329 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8206084 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8204329/webrev/ >> >> Thanks, >> Joe > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From huizhe.wang at oracle.com Mon Jul 2 20:55:44 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 02 Jul 2018 13:55:44 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8204329: Java API doc for XMLStreamReader.next() needs to be clarified for the exception thrown when hasNext() method returns false In-Reply-To: <1596a842-2363-9942-7f7b-0e77aa1825ca@Oracle.com> References: <5B3A646C.90701@oracle.com> <1596a842-2363-9942-7f7b-0e77aa1825ca@Oracle.com> Message-ID: <5B3A9150.5040309@oracle.com> Thanks Roger, Lance! Pushed. Best, Joe On 7/2/18, 12:21 PM, Roger Riggs wrote: > +1 > > On 7/2/2018 3:07 PM, Lance Andersen wrote: >> +1 >> >>> On Jul 2, 2018, at 1:44 PM, Joe Wang wrote: >>> >>> Hi, >>> >>> Please review a patch that clarifies that XMLStreamReader.next() >>> throws NoSuchElementException when hasNext() is false by removing >>> the confusing note. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8204329 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8206084 >>> webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8204329/webrev/ >>> >>> Thanks, >>> Joe >> >> >> >> Lance >> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > From awilkinson at pivotal.io Tue Jul 3 08:28:00 2018 From: awilkinson at pivotal.io (Andy Wilkinson) Date: Tue, 3 Jul 2018 09:28:00 +0100 Subject: ClassLoader.getResources() only returns locations in a patch when using the module path and --patch-module Message-ID: I recently raised a JDK issue [1] which, when triaged, was closed as not an issue. I think is incorrect and was advised by Dalibor Topic [2] to email this list so here I am. In a nutshell, it appears that when using the module path and --patch-module, a call to ClassLoader.getResources() will only return resources locations that are within the patch and ignores those that are in the original module. There's a Git repository [3] that should reproduce the problem when you run with ./mvnw test using Java 9 or later. The problem appears when using Maven as Surefire launches the test VM with the main classes on the module path and the test classes patched into the module. As a result, tests are able to load classes from both the module and the patch but getResources() only returns locations within the patch. As far as I can tell, Surefire is configuring the JVM correctly when it launches it so this appears to be a bug in the JDK. Thanks, Andy [1] https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8206162 [2] https://twitter.com/robilad/status/1013850133589610496 [3] https://github.com/wilkinsona/module-path-scanning From daniel.fuchs at oracle.com Tue Jul 3 08:31:19 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 3 Jul 2018 10:31:19 +0200 Subject: RFR(JDK12/JAXP/java.xml) 8204329: Java API doc for XMLStreamReader.next() needs to be clarified for the exception thrown when hasNext() method returns false In-Reply-To: <5B3A9150.5040309@oracle.com> References: <5B3A646C.90701@oracle.com> <1596a842-2363-9942-7f7b-0e77aa1825ca@Oracle.com> <5B3A9150.5040309@oracle.com> Message-ID: <1a7d3223-a1eb-8989-ea8e-5dc73dd3fa00@oracle.com> On 02/07/2018 22:55, Joe Wang wrote: > Thanks Roger, Lance!? Pushed. Oh - well - I was wondering whether there should be a test for StreamReaderDelegate as well - but maybe there's already one? https://docs.oracle.com/javase/10/docs/api/javax/xml/stream/util/StreamReaderDelegate.html best regards, -- daniel > JBS: https://bugs.openjdk.java.net/browse/JDK-8204329 > CSR: https://bugs.openjdk.java.net/browse/JDK-8206084 > webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8204329/webrev > > Best, > Joe From daniel.fuchs at oracle.com Tue Jul 3 08:46:13 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 3 Jul 2018 10:46:13 +0200 Subject: [JDK 11] RFR 8187069: The case auto failed with the "java.lang.ClassNotFoundException: IPv6NameserverPlatformParsingTest" exception In-Reply-To: <410AD2FB-CF0E-41FA-AFE5-C2537FF42E19@oracle.com> References: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> <28421129-efb5-312e-4473-9c219f6d8b0d@oracle.com> <410AD2FB-CF0E-41FA-AFE5-C2537FF42E19@oracle.com> Message-ID: Hi Chris, Looks good to me as well. best regards, -- daniel On 29/06/2018 07:48, Chris Yin wrote: > Thank you, Vyom > > Regards, > Chris > >> On 29 Jun 2018, at 11:13 AM, vyom tewari wrote: >> >> looks good to me. >> >> Vyom >> >> On Friday 29 June 2018 06:22 AM, Chris Yin wrote: >>> Hi, Vyom >>> >>> Sure, fixed the tag order as you suggested, thanks >>> >>> New changes: >>> >>> diff -r 1308189b0848 test/jdk/com/sun/jndi/dns/Test6991580.java >>> --- a/test/jdk/com/sun/jndi/dns/Test6991580.java Thu Jun 28 17:45:59 2018 -0700 >>> +++ b/test/jdk/com/sun/jndi/dns/Test6991580.java Fri Jun 29 08:48:05 2018 +0800 >>> @@ -33,10 +33,11 @@ >>> /* >>> * @test >>> * @bug 6991580 8080108 8133035 >>> - * @requires os.family != "windows" >>> * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException >>> * @modules java.desktop >>> * jdk.naming.dns/com.sun.jndi.dns >>> + * @requires os.family != "windows" >>> + * @build IPv6NameserverPlatformParsingTest >>> * @run main/manual Test6991580 >>> */ >>> >>> Regards, >>> Chris From Alan.Bateman at oracle.com Tue Jul 3 09:53:23 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 3 Jul 2018 10:53:23 +0100 Subject: ClassLoader.getResources() only returns locations in a patch when using the module path and --patch-module In-Reply-To: References: Message-ID: <22b8d34f-b297-f9fd-716d-75f9534cc1ed@oracle.com> On 03/07/2018 09:28, Andy Wilkinson wrote: > I recently raised a JDK issue [1] which, when triaged, was closed as not an > issue. I think is incorrect and was advised by Dalibor Topic [2] to email > this list so here I am. > > In a nutshell, it appears that when using the module path and > --patch-module, a call to ClassLoader.getResources() will only return > resources locations that are within the patch and ignores those that are in > the original module. It's not a bug. If you patch a module to override a resource in that module then the getResourceXXX methods will locate the resource in the patch. If it helps, the reason the output of the Maven project in the bug report is confusing because the Maven Surefire plugin is run with the classes directory on the module path and the test-classes directory on the class path. The same resource is found in both so this is because two resources are located. In addition, the test is using the name of a directory so it's highly implementation specific if it will be found anyway (the test patches an exploded module with an exploded patch and names that locate resources can be located with both layouts). -Alan From rachna.goel at oracle.com Tue Jul 3 09:58:35 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Tue, 3 Jul 2018 15:28:35 +0530 Subject: [11] Review Request: 8204603: Short week days, NaN value and timezone name are inconsistent between CLDR and Java in zh_CN, zh_TW locales. Message-ID: <551e9c56-6ef0-8e16-197f-63595698a640@oracle.com> Hi, Kindly review fix to JDK-8204603. Bug: https://bugs.openjdk.java.net/browse/JDK-8204603 Fix: http://cr.openjdk.java.net/~rgoel/JDK-8204603/webrev.04/ This is a regression caused by JDK-8179071, where locale data is getting incorrectly retrieved (not following languageAliases of CLDR) for some locales. Handling of languageAliases for CLDR provider will solve the issue. -- Thanks, Rachna From naoto.sato at oracle.com Tue Jul 3 16:15:28 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 3 Jul 2018 09:15:28 -0700 Subject: [11] Review Request: 8204603: Short week days, NaN value and timezone name are inconsistent between CLDR and Java in zh_CN, zh_TW locales. In-Reply-To: <551e9c56-6ef0-8e16-197f-63595698a640@oracle.com> References: <551e9c56-6ef0-8e16-197f-63595698a640@oracle.com> Message-ID: Looks good to me. Naoto On 7/3/18 2:58 AM, Rachna Goel wrote: > Hi, > > Kindly review fix to JDK-8204603. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204603 > > Fix: http://cr.openjdk.java.net/~rgoel/JDK-8204603/webrev.04/ > > This is a regression caused by JDK-8179071, where locale data is getting > incorrectly retrieved (not following languageAliases of CLDR) for some > locales. Handling of languageAliases for CLDR provider will solve the > issue. > From paul.sandoz at oracle.com Tue Jul 3 16:53:08 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 3 Jul 2018 09:53:08 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: Looks good. Where do you propose to place the test in the OpenJDK repo? Paul. > On Jun 30, 2018, at 10:29 AM, Martin Buchholz wrote: > > Hi Ivan, > > Thanks for finding this bug - I'll take the blame. > > Forgive my pushiness - I'd like to do a friendly takeover of this. As > penance I ported the similar WhiteBox test from ConcurrentHashMap to > ArrayDeque. I would elide the comment in the default constructor. And we > can delete our noreg label. > > Index: src/main/java/util/ArrayDeque.java > =================================================================== > RCS file: > /export/home/jsr166/jsr166/jsr166/src/main/java/util/ArrayDeque.java,v > retrieving revision 1.133 > diff -u -r1.133 ArrayDeque.java > --- src/main/java/util/ArrayDeque.java 24 Feb 2018 22:04:18 -0000 1.133 > +++ src/main/java/util/ArrayDeque.java 30 Jun 2018 17:07:46 -0000 > @@ -181,7 +181,7 @@ > * sufficient to hold 16 elements. > */ > public ArrayDeque() { > - elements = new Object[16]; > + elements = new Object[16 + 1]; > } > > /** > Index: src/test/jtreg/util/ArrayDeque/WhiteBox.java > =================================================================== > RCS file: src/test/jtreg/util/ArrayDeque/WhiteBox.java > diff -N src/test/jtreg/util/ArrayDeque/WhiteBox.java > --- /dev/null 1 Jan 1970 00:00:00 -0000 > +++ src/test/jtreg/util/ArrayDeque/WhiteBox.java 30 Jun 2018 17:07:46 -0000 > @@ -0,0 +1,125 @@ > +/* > + * Written by Martin Buchholz with assistance from members of JCP > + * JSR-166 Expert Group and released to the public domain, as > + * explained at http://creativecommons.org/publicdomain/zero/1.0/ > + */ > + > +/* > + * @test > + * @modules java.base/java.util:open > + * @run testng WhiteBox > + * @summary White box tests of implementation details > + */ > + > +import static org.testng.Assert.*; > +import org.testng.annotations.Test; > + > +import java.io.ByteArrayInputStream; > +import java.io.ByteArrayOutputStream; > +import java.io.ObjectInputStream; > +import java.io.ObjectOutputStream; > +import java.lang.invoke.MethodHandles; > +import java.lang.invoke.VarHandle; > +import java.util.ArrayDeque; > +import java.util.concurrent.ThreadLocalRandom; > + > + at Test > +public class WhiteBox { > + final ThreadLocalRandom rnd = ThreadLocalRandom.current(); > + final VarHandle ELEMENTS, HEAD, TAIL; > + > + WhiteBox() throws ReflectiveOperationException { > + Class klazz = ArrayDeque.class; > + MethodHandles.Lookup lookup > + = MethodHandles.privateLookupIn(klazz, MethodHandles.lookup()); > + ELEMENTS = lookup.findVarHandle(klazz, "elements", Object[].class); > + HEAD = lookup.findVarHandle(klazz, "head", int.class); > + TAIL = lookup.findVarHandle(klazz, "tail", int.class); > + } > + > + Object[] elements(ArrayDeque d) { return (Object[]) ELEMENTS.get(d); } > + int head(ArrayDeque d) { return (int) HEAD.get(d); } > + int tail(ArrayDeque d) { return (int) TAIL.get(d); } > + > + void checkCapacity(ArrayDeque d, int capacity) { > + assertTrue(d.isEmpty()); > + assertEquals(0, head(d)); > + assertEquals(0, tail(d)); > + assertInvariants(d); > + Object[] initialElements = elements(d); > + > + assertInvariants(d); > + for (int i = capacity; i--> 0; ) { > + d.add(rnd.nextInt(42)); > + assertSame(elements(d), initialElements); > + assertInvariants(d); > + } > + d.add(rnd.nextInt(42)); > + assertNotSame(elements(d), initialElements); > + } > + > + @Test > + public void defaultConstructor() { > + checkCapacity(new ArrayDeque(), 16); > + } > + > + @Test > + public void shouldNotResizeWhenInitialCapacityProvided() { > + int initialCapacity = rnd.nextInt(1, 20); > + checkCapacity(new ArrayDeque(initialCapacity), initialCapacity); > + } > + > + byte[] serialBytes(Object o) { > + try { > + ByteArrayOutputStream bos = new ByteArrayOutputStream(); > + ObjectOutputStream oos = new ObjectOutputStream(bos); > + oos.writeObject(o); > + oos.flush(); > + oos.close(); > + return bos.toByteArray(); > + } catch (Exception fail) { > + throw new AssertionError(fail); > + } > + } > + > + @SuppressWarnings("unchecked") > + T serialClone(T o) { > + try { > + ObjectInputStream ois = new ObjectInputStream > + (new ByteArrayInputStream(serialBytes(o))); > + T clone = (T) ois.readObject(); > + assertNotSame(o, clone); > + assertSame(o.getClass(), clone.getClass()); > + return clone; > + } catch (Exception fail) { > + throw new AssertionError(fail); > + } > + } > + > + @Test > + public void testSerialization() { > + ArrayDeque[] ds = { new ArrayDeque(), new > ArrayDeque(rnd.nextInt(20)) }; > + for (ArrayDeque d : ds) { > + if (rnd.nextBoolean()) d.add(99); > + ArrayDeque clone = serialClone(d); > + assertInvariants(clone); > + assertNotSame(elements(d), elements(clone)); > + assertEquals(d, clone); > + } > + } > + > + /** Checks conditions which should always be true. */ > + void assertInvariants(ArrayDeque d) { > + final Object[] elements = elements(d); > + final int head = head(d); > + final int tail = tail(d); > + final int capacity = elements.length; > + assertTrue(0 <= head && head < capacity); > + assertTrue(0 <= tail && tail < capacity); > + assertTrue(capacity > 0); > + assertTrue(d.size() < capacity); > + assertTrue(head == tail || elements[head] != null); > + assertNull(elements[tail]); > + assertTrue(head == tail || elements[Math.floorMod(tail - 1, > capacity)] != null); > + } > +} > > > On Fri, Jun 29, 2018 at 7:21 PM, Ivan Gerasimov > wrote: > >> Hello! >> >> The spec states that an ArrayDeque created with the default constructor >> should be able to hold 16 elements. >> >> https://docs.oracle.com/javase/10/docs/api/java/util/ArrayDe >> que.html#%3Cinit%3E() >> """ >> Constructs an empty array deque with an initial capacity sufficient to >> hold 16 elements. >> """ >> >> In the constructor an array of size 16 is created: >> elements = new Object[16]; >> >> However, one element must always be null: >> /** >> * The array in which the elements of the deque are stored. >> * All array cells not holding deque elements are always null. >> * The array always has at least one null slot (at tail). >> */ >> transient Object[] elements; >> >> which leaves us space for only 15 elements, contrarily to what the spec >> promises. >> >> >> Would you please help review a trivial fix? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8206123 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8206123/00/webrev/ >> >> Thanks in advance! >> >> -- >> With kind regards, >> Ivan Gerasimov >> >> From martinrb at google.com Tue Jul 3 17:42:22 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 3 Jul 2018 10:42:22 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: On Tue, Jul 3, 2018 at 9:53 AM, Paul Sandoz wrote: > Looks good. Where do you propose to place the test in the OpenJDK repo? > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html I hadn't gotten around to a RFR - the test is slightly modified from my initial post. From paul.sandoz at oracle.com Tue Jul 3 18:04:53 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 3 Jul 2018 11:04:53 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: > On Jul 3, 2018, at 10:42 AM, Martin Buchholz wrote: > > > On Tue, Jul 3, 2018 at 9:53 AM, Paul Sandoz > wrote: > Looks good. Where do you propose to place the test in the OpenJDK repo? > > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > > I hadn't gotten around to a RFR - the test is slightly modified from my initial post. Ok, i see more thorough assertions in the assertInvariants method. Thanks, Paul. From martinrb at google.com Tue Jul 3 19:15:44 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 3 Jul 2018 12:15:44 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: OK, this thread is officially upgraded to a RFR. http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ArrayDeque-capacity/index.html On Tue, Jul 3, 2018 at 11:04 AM, Paul Sandoz wrote: > > > On Jul 3, 2018, at 10:42 AM, Martin Buchholz wrote: > > > On Tue, Jul 3, 2018 at 9:53 AM, Paul Sandoz > wrote: > >> Looks good. Where do you propose to place the test in the OpenJDK repo? >> > > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166- > integration/overview.html > > > I hadn't gotten around to a RFR - the test is slightly modified from my > initial post. > > > Ok, i see more thorough assertions in the assertInvariants method. > > Thanks, > Paul. > > From Roger.Riggs at Oracle.com Tue Jul 3 21:26:45 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 3 Jul 2018 17:26:45 -0400 Subject: [12] RFR: 8206120: Add test cases for lenient Japanese era parsing In-Reply-To: <1139b79a-e600-a1dc-f03f-928203ed10e3@oracle.com> References: <1139b79a-e600-a1dc-f03f-928203ed10e3@oracle.com> Message-ID: <9e1a73d3-8335-c6ae-1c44-0bdf824c621d@Oracle.com> Hi Naoto, Looks fine,? Thanks for adding the test. Roger On 7/2/2018 1:52 PM, Naoto Sato wrote: > Hello, > > Please review the changes to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8206120 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8206120/webrev.00/ > > This is a test case only fix to ensure the new Japanese era > implementation introduced with JDK-8202088 accepts lenient era/year, > such as "Heisei 32." > > Naoto From huizhe.wang at oracle.com Tue Jul 3 22:57:38 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 03 Jul 2018 15:57:38 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8204329: Java API doc for XMLStreamReader.next() needs to be clarified for the exception thrown when hasNext() method returns false In-Reply-To: <1a7d3223-a1eb-8989-ea8e-5dc73dd3fa00@oracle.com> References: <5B3A646C.90701@oracle.com> <1596a842-2363-9942-7f7b-0e77aa1825ca@Oracle.com> <5B3A9150.5040309@oracle.com> <1a7d3223-a1eb-8989-ea8e-5dc73dd3fa00@oracle.com> Message-ID: <5B3BFF62.7040405@oracle.com> On 7/3/18, 1:31 AM, Daniel Fuchs wrote: > On 02/07/2018 22:55, Joe Wang wrote: >> Thanks Roger, Lance! Pushed. > > Oh - well - I was wondering whether there should be a test for > StreamReaderDelegate as well - but maybe there's already one? The APIs should have been consistent. Unfortunately though, they are not. StreamReaderDelegate.next() had the note but did not specify a @throw statement. Arguably, it should be fixed. However, that would be an incompatible change. Since the Delegate interface serves just as a bridge, I'll leave that to the user to consider what's a proper behavior, and therefore leave it out of this patch. Best, Joe > > https://docs.oracle.com/javase/10/docs/api/javax/xml/stream/util/StreamReaderDelegate.html > > > best regards, > > -- daniel > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8204329 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8206084 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8204329/webrev > >> Best, >> Joe From xu.y.yin at oracle.com Wed Jul 4 00:47:44 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Wed, 4 Jul 2018 08:47:44 +0800 Subject: [JDK 11] RFR 8187069: The case auto failed with the "java.lang.ClassNotFoundException: IPv6NameserverPlatformParsingTest" exception In-Reply-To: References: <836D8991-E27C-4929-96FC-19444A48E422@oracle.com> <28421129-efb5-312e-4473-9c219f6d8b0d@oracle.com> <410AD2FB-CF0E-41FA-AFE5-C2537FF42E19@oracle.com> Message-ID: <7F088DC9-D755-416E-BEE6-3DFF233759E7@oracle.com> Thank you, Daniel Regards, Chris > On 3 Jul 2018, at 4:46 PM, Daniel Fuchs wrote: > > Hi Chris, > > Looks good to me as well. > > best regards, > > -- daniel > > On 29/06/2018 07:48, Chris Yin wrote: >> Thank you, Vyom >> Regards, >> Chris >>> On 29 Jun 2018, at 11:13 AM, vyom tewari wrote: >>> >>> looks good to me. >>> >>> Vyom >>> >>> On Friday 29 June 2018 06:22 AM, Chris Yin wrote: >>>> Hi, Vyom >>>> >>>> Sure, fixed the tag order as you suggested, thanks >>>> >>>> New changes: >>>> >>>> diff -r 1308189b0848 test/jdk/com/sun/jndi/dns/Test6991580.java >>>> --- a/test/jdk/com/sun/jndi/dns/Test6991580.java Thu Jun 28 17:45:59 2018 -0700 >>>> +++ b/test/jdk/com/sun/jndi/dns/Test6991580.java Fri Jun 29 08:48:05 2018 +0800 >>>> @@ -33,10 +33,11 @@ >>>> /* >>>> * @test >>>> * @bug 6991580 8080108 8133035 >>>> - * @requires os.family != "windows" >>>> * @summary IPv6 Nameservers in resolv.conf throws NumberFormatException >>>> * @modules java.desktop >>>> * jdk.naming.dns/com.sun.jndi.dns >>>> + * @requires os.family != "windows" >>>> + * @build IPv6NameserverPlatformParsingTest >>>> * @run main/manual Test6991580 >>>> */ >>>> >>>> Regards, >>>> Chris From ivan.gerasimov at oracle.com Wed Jul 4 04:31:20 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 4 Jul 2018 00:31:20 -0400 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: <442e0f6a-c94c-b215-788d-b0ef0d543234@oracle.com> Hi Martin! Why did you exclude the case of zero initial capacity? 96 int initialCapacity = rnd.nextInt(1, 20); I think it may make sense to have "static final EMPTY_DEQUE = ArrayDeque(0);" somewhere, and we may want to make sure it doesn't allocate more memory than needed? It's perfectly fine to have this test: 149 assertTrue((head == tail) ^ (elements[head] != null)); though I would find it a bit easier to read if it were written as assertEquals(head == tail, elements[head] == null); It's a matter of style, of course. With kind regards, Ivan On 7/3/18 3:15 PM, Martin Buchholz wrote: > OK, this thread is officially upgraded to a RFR. > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ArrayDeque-capacity/index.html > > > On Tue, Jul 3, 2018 at 11:04 AM, Paul Sandoz > wrote: > > > >> On Jul 3, 2018, at 10:42 AM, Martin Buchholz > > wrote: >> >> >> On Tue, Jul 3, 2018 at 9:53 AM, Paul Sandoz >> > wrote: >> >> Looks good. Where do you propose to place the test in the >> OpenJDK repo? >> >> >> >> http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html >> >> >> I hadn't gotten around to a RFR - the test is slightly modified >> from my initial post. > > Ok, i see more thorough assertions in the assertInvariants method. > > Thanks, > Paul. > > -- With kind regards, Ivan Gerasimov From martinrb at google.com Wed Jul 4 06:40:32 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 3 Jul 2018 23:40:32 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: <442e0f6a-c94c-b215-788d-b0ef0d543234@oracle.com> References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> <442e0f6a-c94c-b215-788d-b0ef0d543234@oracle.com> Message-ID: On Tue, Jul 3, 2018 at 9:31 PM, Ivan Gerasimov wrote: > Hi Martin! > > Why did you exclude the case of zero initial capacity? > 96 int initialCapacity = rnd.nextInt(1, 20); > > Well spotted - I was just being lazy - ArrayDeque doesn't handle the case of zero specially. public void shouldNotResizeWhenInitialCapacityProvided() { - int initialCapacity = rnd.nextInt(1, 20); + int initialCapacity = rnd.nextInt(20); > I think it may make sense to have "static final EMPTY_DEQUE = > ArrayDeque(0);" somewhere, and we may want to make sure it doesn't allocate > more memory than needed? > > It's perfectly fine to have this test: > 149 assertTrue((head == tail) ^ (elements[head] != null)); > > I read ^ as "exactly one of these is true" OTOH maybe I'm a serial abuser of the ^ operator - I've used it 7 times in a single expression! From goetz.lindenmaier at sap.com Wed Jul 4 08:21:02 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 4 Jul 2018 08:21:02 +0000 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> Message-ID: <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> Hi, Volker, thanks for improving on my original change and implementing David's and Karen's proposals. David, I think the change addresses a row of your concerns. > proposal - it just moves the "field" from the Java side to the VM side. No, the space overhead in case of successful initialization is reduced from O(classes) to O(classloaders). There is only runtime overhead if there was an error, and that should be acceptable. > dealing with backtrace and stackTrace. I have to wonder why nothing in > Throwable clears the backtrace today ? Maybe the concern about the backTraces is pointless and the conversion to stackTraces should be dropped. As you say, it's done nowhere else, and other backTraces should cause similar issues. > I'm not clear why you record the ExceptionInInitializerError wrapper > instead of the actual exception that occurred? Keeping the ExceptionInInitializerError is helpful for people to understand that this has happened during initialization and not directly where the NCDFE is thrown. They will understand that this might have happened in another thread. Best regards, Goetz. > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of David Holmes > Sent: Sonntag, 1. Juli 2018 23:48 > To: Volker Simonis ; hotspot-runtime- > dev at openjdk.java.net runtime ; > Java Core Libs > Subject: Re: RFR(M): 8203826: Chain class initialization exceptions into later > NoClassDefFoundErrors > > Hi Volker, > > This doesn't really address any of the concerns I had with the original > proposal - it just moves the "field" from the Java side to the VM side. > There is still a massive amount of Java code execution in relation to > this - which itself may encounter secondary exceptions. It's very hard > to tell if you will leave things in a suitable state if such exceptions > arise. > > My position remains that the primary place to deal with the > initialization error is when initialization occurs and the error > happens. Subsequent attempted uses of the erroneous class may benefit > from some additional information about the nature of the original > exceptions, but I don't think full stacktraces are necessary or > desirable (and I do believe they will confuse most users given the lack > of continuity in the stack frames and that they may have happened in a > different thread!). > > That aside ... > > There appears to a race on constructing the Hashtable. At least it was > not obvious to me where a lock may be held during that process. > > I can't determine that clearing backtrace in removeNativeBacktrace is > correct with respect to the overall protocol within Throwable for > dealing with backtrace and stackTrace. I have to wonder why nothing in > Throwable clears the backtrace today ? > > I'm not clear why you record the ExceptionInInitializerError wrapper > instead of the actual exception that occurred? > > Throwable states: > > + * This method is currently only called from the VM for instances of > + * ExceptionInInitializerError which are stored for later chaining > into a > + * NoClassDefFoundError in order to prevent keeping classes from > the native > + * backtrace alive. > + */ > > but IIUC it will also be called for instances of Error that occurred > which do not get wrapped in EIIE. > > > Regards, > David > ------ > > > On 30/06/2018 12:53 AM, Volker Simonis wrote: > > Hi, > > > > can I please have a review for the following change which saves > > ExceptionInInitializerError thrown during class initialization and > > chains them as cause into potential NoClassDefFoundErrors for the same > > class. We are using this features since years in our commercial SAP > > JVM and it proved extremely useful for detecting and fixing errors > > especially in big deployments. > > > > This is a follow-up on a discussion previously started by Goetz [1]. > > His first proposal (which is close to our current, internal > > implementation) inserted an additional field into java.lang.Class > > objects to save potential ExceptionInInitializerErrors. This was > > considered to much overhead in the initial discussion [1]. > > > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8203826.v2/ > > https://bugs.openjdk.java.net/browse/JDK-8203826 > > > > So in this change, I've completely re-implemented the feature by using > > a java.lang.Hashtable which is attached to the ClassLoaderData object. > > The Hashtable is lazily created when the first > > ExceptionInInitializerError is thrown and maps the Class which > > triggered the ExceptionInInitializerError during the execution of its > > static initializer to the corresponding ExceptionInInitializerError. > > > > If the same class will be accessed once again, this will directly lead > > to a plain NoClassDefFoundError (as per the JVMS, 5.5 Initialization) > > because the static initializer won't be executed a second time. Until > > now, this NoClassDefFoundError wasn't linked in any way to the root > > cause of the problem (i.e. the first ExceptionInInitializerError > > together with the chained exception that happened during the execution > > of the static initializer). With this change, the NoClassDefFoundError > > will now chain the initial ExceptionInInitializerError as cause, > > making it much easier to detect the problem which lead to the > > NoClassDefFoundError. > > > > Following is an example from the new JTreg tests which comes which > > this change to demonstrate the feature. Until know, a typical stack > > trace from a NoClassDefFoundError looked as follows: > > > > java.lang.NoClassDefFoundError: Could not initialize class > > NoClassDefFound$ClassWithFailedInitializer > > at java.base/java.lang.Class.forName0(Native Method) > > at java.base/java.lang.Class.forName(Class.java:291) > > at NoClassDefFound.main(NoClassDefFound.java:38) > > > > With this change, the same stack trace now looks as follows: > > > > java.lang.NoClassDefFoundError: Could not initialize class > > NoClassDefFound$ClassWithFailedInitializer > > at java.base/java.lang.Class.forName0(Native Method) > > at java.base/java.lang.Class.forName(Class.java:315) > > at NoClassDefFound.main(NoClassDefFound.java:38) > > Caused by: java.lang.ExceptionInInitializerError > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0( > Native > > Method) > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance( > NativeConstructorAccessorImpl.java:62) > > at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstan > ce(DelegatingConstructorAccessorImpl.java:45) > > at > java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) > > at java.base/java.lang.Class.newInstance(Class.java:584) > > at > NoClassDefFound$ClassWithFailedInitializer.(NoClassDefFound.java:2 > 0) > > at java.base/java.lang.Class.forName0(Native Method) > > at java.base/java.lang.Class.forName(Class.java:315) > > at NoClassDefFound.main(NoClassDefFound.java:30) > > Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 2 out of > > bounds for length 1 > > at NoClassDefFound$A.(NoClassDefFound.java:9) > > ... 9 more > > > > As you can see, the reason for the NoClassDefFoundError when accessing > > the class 'NoClassDefFound$ClassWithFailedInitializer' is actually not > > even in the class or its static initializer itself, but in the class > > 'NoClassDefFound$A' which is a base class of > > 'NoClassDefFound$ClassWithFailedInitializer'. This is not easily > > detectible from the old, plain NoClassDefFoundError. > > > > As I wrote, the only overhead we have with the new implementation is > > an additional OopHandle field per ClassLoaderData which I think is > > acceptable. The Hashtable object itself is only created lazily, after > > the first occurrence of an ExceptionInInitializerError in the > > corresponding class loader. The whole Hashtable creation and > > storing/quering of ExceptionInInitializerErrors in > > > ClassLoaderData::record_init_exception()/ClassLoaderData::query_init_exce > ption() > > is optional in the sense that any errors/exceptions occurring during > > the execution of these functions are ignored and cleared. > > > > Finally, we also take care to recursively convert all native > > backtraces in the stored ExceptionInInitializerErrors (and their > > suppressed and chained exceptions) into symbolic stack traces in order > > to avoid holding references to classes and prevent their unloading. > > This is implemented in the new private, static method > > java.lang.Throwable::removeNativeBacktrace() which is called for each > > ExceptionInInitializerError before it is stored in the Hashtable. > > > > Thank you and best regards, > > Volker > > > > [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018- > June/028310.html > > From orionllmain at gmail.com Wed Jul 4 09:42:05 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Wed, 4 Jul 2018 16:42:05 +0700 Subject: Useless null check in HashMap.merge() Message-ID: I noticed dead code in java.util.HashMap.merge(): public V merge(K key, V value, BiFunction remappingFunction) { if (value == null) throw new NullPointerException(); ... if (value != null) { *// Condition ' value != null' is always true* if (t != null) t.putTreeVal(this, tab, hash, key, value); else { tab[i] = newNode(hash, key, value, first); if (binCount >= TREEIFY_THRESHOLD - 1) treeifyBin(tab, hash); } ++modCount; ++size; afterNodeInsertion(true); } return value; } The code in the if branch will never be executed because `value` was previously checked at the beginning of the method. Is this a mistake? From peter.levart at gmail.com Wed Jul 4 10:39:20 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 4 Jul 2018 12:39:20 +0200 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> Message-ID: <593a5eb1-b268-44d5-c9e8-90e082f967f9@gmail.com> Hi Volker, It occurred to me that getting rid of backtrace-s of cause(s)/suppressed exception(s) might not be enough to prevent ClassLoader leaks... On 07/04/2018 10:21 AM, Lindenmaier, Goetz wrote: >> dealing with backtrace and stackTrace. I have to wonder why nothing in >> Throwable clears the backtrace today ? > Maybe the concern about the backTraces is pointless and the > conversion to stackTraces should be dropped. > As you say, it's done nowhere else, and other backTraces should > cause similar issues. > Exception objects are typically not retained for longer periods. They are normally caught, dumped to log and let gone. This change retains exception(s) so that they are reachable from a ClassLoader that loaded the failed class. It could happen that the chain of cause(s)/suppressed exception(s) of some ExceptionInInitializerError is an exception object of a class that is loaded by some child ClassLoader of the ClassLoader that loaded the failed class. Such child ClassLoader would have leaked. The solution would be to replace the chain of cause(s)/suppressed exception(s) with a chain of replacement exception objects like this one (this would also take care of backtraces of original exceptions as it would not retain the original exceptions at all): /** ?* A {@link RuntimeException} that acts as a substitute for the original exception ?* (checked or unchecked) and mimics the original exception in every aspect except it's type. ?*/ public class ExceptionSubstitute extends RuntimeException { ??? private static final long serialVersionUID = 1; ??? private String originalExceptionClassName, localizedMessage; ??? public ExceptionSubstitute(Throwable originalException) { ??????? super(originalException.getMessage()); ??????? this.originalExceptionClassName = originalException.getClass().getName(); ??????? this.localizedMessage = originalException.getLocalizedMessage(); ??????? // substitute originalException's cause ??????? Throwable cause = originalException.getCause(); ??????? initCause(cause == null ? null : new ExceptionSubstitute(cause)); ??????? // substitute originalException's suppressed exceptions if any ??????? for (Throwable suppressed : originalException.getSuppressed()) { ??????????? addSuppressed(new ExceptionSubstitute(suppressed)); ??????? } ??????? // inherit stack trace elements from originalException ??????? setStackTrace(originalException.getStackTrace()); ??? } ??? @Override ??? public Throwable fillInStackTrace() { ??????? // don't need our backtrace - will inherit stack trace elements from originalException ??????? return this; ??? } ??? @Override ??? public String getLocalizedMessage() { ??????? return localizedMessage; ??? } ??? /** ???? * @return the class name of the original exception for which this exception is a substitute ???? */ ??? public String getOriginalExceptionClassName() { ??????? return originalExceptionClassName; ??? } ??? /** ???? * Emulate toString() method as if called upon originalException ???? */ ??? @Override ??? public String toString() { ??????? String message = getLocalizedMessage(); ??????? return (message != null) ?????????????? ? (getOriginalExceptionClassName() + ": " + message) ?????????????? : getOriginalExceptionClassName(); ??? } } Regards, Peter From srinivas.dama at oracle.com Wed Jul 4 11:25:16 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Wed, 4 Jul 2018 04:25:16 -0700 (PDT) Subject: [11]RFR: 8198819: tools/jimage/JImageExtractTest.java, fails intermittently at testExtract (macos) Message-ID: Hi, Please review webrev: http://cr.openjdk.java.net/~sdama/8198819/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8198819 Modified test to work only on modules extracted using jimage instead of considering already existing files in current directory and treating them as bad Modules. Regards, Srinivas From james.laskey at oracle.com Wed Jul 4 11:59:52 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 4 Jul 2018 08:59:52 -0300 Subject: [11]RFR: 8198819: tools/jimage/JImageExtractTest.java, fails intermittently at testExtract (macos) In-Reply-To: References: Message-ID: <5EC0795B-1626-4A8E-9A82-3F05F392FA5B@oracle.com> +1 > On Jul 4, 2018, at 8:25 AM, Srinivas Dama wrote: > > Hi, > > Please review > webrev: http://cr.openjdk.java.net/~sdama/8198819/webrev.01/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8198819 > > Modified test to work only on modules extracted using jimage instead of considering already existing files > in current directory and treating them as bad Modules. > > Regards, > Srinivas From enasser at in.ibm.com Wed Jul 4 12:41:59 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Wed, 4 Jul 2018 18:11:59 +0530 Subject: Adding new IBM extended charsets Message-ID: Hello, Am starting this mail thread to discuss about adding new IBM extended charsets. The questions is whether we need to add the new extended charsets to jdk.charsets or to a new separate charset provider/module like jdk.ibmcharsets. This discussion is in continuation of the suggestion from Alan Bateman in the mail chain - http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html. Am copying his inputs from that mail thread to start the discussion: "I think we should start a discussion here about moving some or all of the IBM charsets to their own service provider module. I realize the AIX port might want to include some of them in its build of java.base but they aren't interesting to include in java.base, or even jdk.charsets, on most platform" First, let me clarify whether IBM charsets are applicable only to IBM platforms like AIX or applicable to other platforms as well. All IBM charsets are applicable to any platforms including Linux and windows if those platforms needs to communicate with an application or database in IBM platforms like AIX. That is the reason, we traditionally add them to the jdk.charsets. However, we agree with Alan that those IBM charsets are not required if the JDK is not communicating to any applications/databases on IBM platforms. Hence, it makes sense to consider a separate charset provider / module for IBM charsets and use build parameters to decide whether to generate the new charset provider or not for any platforms. Let me list out all the possible options I can think of for adding new extended charsets so that we can discuss and decide which is the best option. 1) Continue to add new extended charsets to jdk.charsets. The advantage with this approach is that no need to add new charset provider and all extended charsets are placed in one module. Also, any extended charset is applicable to any platform if they need to communicate with application/database in different platforms. The disadvantage is that the number of charsets in jdk.charsets keep increasing and blot its size. Also, many of those charsets may not be used in the lifetime of the JDK unless it is communicating with application/databases of those platforms. 2) Create a new charset provider and module (say jdk.ibmcharsets) for all IBM charsets and include the new module in JDK on a need basis. The advantage with this approach is that the foot print of jdk.charsets can be reduced and can include the new module only if it is required. The disadvantage is that a new charset provider needs to be created. Also, extended charsets will be located in two different modules and many a times both the modules are required. 3) Remove all extended charsets from JDK (keep only default charsets) and use the extended charsets from third party like ICU4J. I believe this option might be discussed in the past and there might be valid reason not to pursue this option. Am still listing it to ensure that we have considered this option as well. The advantage with this approach is that we can avoid maintaining the same charsets by two different open source communities. The disadvantage with this option is that the release cycle of the two communities may be different and we may need to maintain the level ourselves for LTS releases as we may not want to change the specification in a service stream. Please share your thoughts on your preferred option and list out any other options which I missed out. Thank you for your time. Regards, Nasser Ebrahim From swpalmer at gmail.com Wed Jul 4 12:55:55 2018 From: swpalmer at gmail.com (Scott Palmer) Date: Wed, 4 Jul 2018 08:55:55 -0400 Subject: Useless null check in HashMap.merge() In-Reply-To: References: Message-ID: <5213AD37-A1F5-41F8-8A9F-2989AC911202@gmail.com> On Jul 4, 2018, at 5:42 AM, Zheka Kozlov wrote: > > I noticed dead code in java.util.HashMap.merge(): > > public V merge(K key, V value, > BiFunction > remappingFunction) { > if (value == null) > throw new NullPointerException(); > > ... > > if (value != null) { *// Condition ' value != null' is always true* > if (t != null) > t.putTreeVal(this, tab, hash, key, value); > else { > tab[i] = newNode(hash, key, value, first); > if (binCount >= TREEIFY_THRESHOLD - 1) > treeifyBin(tab, hash); > } > ++modCount; > ++size; > afterNodeInsertion(true); > } > return value; > } > > The code in the if branch will never be executed because `value` was > previously checked at the beginning of the method. > > Is this a mistake? You mean it will ALWAYS be executed. Yes, it looks to me like the ?if? is useless. From orionllmain at gmail.com Wed Jul 4 13:22:13 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Wed, 4 Jul 2018 20:22:13 +0700 Subject: Useless null check in HashMap.merge() In-Reply-To: <5213AD37-A1F5-41F8-8A9F-2989AC911202@gmail.com> References: <5213AD37-A1F5-41F8-8A9F-2989AC911202@gmail.com> Message-ID: Oh yes, you are right. Then this is not dead code, just a useless null check. 2018-07-04 19:55 GMT+07:00 Scott Palmer : > On Jul 4, 2018, at 5:42 AM, Zheka Kozlov wrote: > > > > I noticed dead code in java.util.HashMap.merge(): > > > > public V merge(K key, V value, > > BiFunction > > remappingFunction) { > > if (value == null) > > throw new NullPointerException(); > > > > ... > > > > if (value != null) { *// Condition ' value != null' is always true* > > if (t != null) > > t.putTreeVal(this, tab, hash, key, value); > > else { > > tab[i] = newNode(hash, key, value, first); > > if (binCount >= TREEIFY_THRESHOLD - 1) > > treeifyBin(tab, hash); > > } > > ++modCount; > > ++size; > > afterNodeInsertion(true); > > } > > return value; > > } > > > > The code in the if branch will never be executed because `value` was > > previously checked at the beginning of the method. > > > > Is this a mistake? > > You mean it will ALWAYS be executed. Yes, it looks to me like the ?if? is > useless. > > > From fweimer at redhat.com Wed Jul 4 13:41:09 2018 From: fweimer at redhat.com (Florian Weimer) Date: Wed, 4 Jul 2018 15:41:09 +0200 Subject: Adding new IBM extended charsets In-Reply-To: References: Message-ID: <36548af8-d5fd-204f-d08e-fc60ffbc09a9@redhat.com> On 07/04/2018 02:41 PM, Nasser Ebrahim wrote: > Please share your thoughts on your preferred option and list out any other > options which I missed out. Thank you for your time. Could you use the platform iconv implementation instead? That would avoid shipping the tables in the JDK. Thanks, Florian From james.laskey at oracle.com Wed Jul 4 13:47:24 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 4 Jul 2018 10:47:24 -0300 Subject: Useless null check in HashMap.merge() In-Reply-To: References: <5213AD37-A1F5-41F8-8A9F-2989AC911202@gmail.com> Message-ID: <72AA2B0E-FFC0-4894-93F5-ED18BECABCBB@oracle.com> Just as a note, it is reasonable to make value ?final? so that value can not be nulled out between the test and its use. Cheers, ? Jim > On Jul 4, 2018, at 10:22 AM, Zheka Kozlov wrote: > > Oh yes, you are right. Then this is not dead code, just a useless null > check. > > 2018-07-04 19:55 GMT+07:00 Scott Palmer : > >> On Jul 4, 2018, at 5:42 AM, Zheka Kozlov wrote: >>> >>> I noticed dead code in java.util.HashMap.merge(): >>> >>> public V merge(K key, V value, >>> BiFunction >>> remappingFunction) { >>> if (value == null) >>> throw new NullPointerException(); >>> >>> ... >>> >>> if (value != null) { *// Condition ' value != null' is always true* >>> if (t != null) >>> t.putTreeVal(this, tab, hash, key, value); >>> else { >>> tab[i] = newNode(hash, key, value, first); >>> if (binCount >= TREEIFY_THRESHOLD - 1) >>> treeifyBin(tab, hash); >>> } >>> ++modCount; >>> ++size; >>> afterNodeInsertion(true); >>> } >>> return value; >>> } >>> >>> The code in the if branch will never be executed because `value` was >>> previously checked at the beginning of the method. >>> >>> Is this a mistake? >> >> You mean it will ALWAYS be executed. Yes, it looks to me like the ?if? is >> useless. >> >> >> From dl at cs.oswego.edu Wed Jul 4 15:35:44 2018 From: dl at cs.oswego.edu (Doug Lea) Date: Wed, 4 Jul 2018 11:35:44 -0400 Subject: Useless null check in HashMap.merge() In-Reply-To: References: <5213AD37-A1F5-41F8-8A9F-2989AC911202@gmail.com> Message-ID: <0dbe0a4a-612d-71e5-33b8-2bafd4d53ca8@cs.oswego.edu> On 07/04/2018 09:22 AM, Zheka Kozlov wrote: > Oh yes, you are right. Then this is not dead code, just a useless null > check. My vague recollection is that the check was there to ensure that the block was identical to those in the other methods when considering ways to factor it out (none of which worked out, so it still sprawls). It could be removed. -Doug > > 2018-07-04 19:55 GMT+07:00 Scott Palmer : > >> On Jul 4, 2018, at 5:42 AM, Zheka Kozlov wrote: >>> >>> I noticed dead code in java.util.HashMap.merge(): >>> >>> public V merge(K key, V value, >>> BiFunction >>> remappingFunction) { >>> if (value == null) >>> throw new NullPointerException(); >>> >>> ... >>> >>> if (value != null) { *// Condition ' value != null' is always true* >>> if (t != null) >>> t.putTreeVal(this, tab, hash, key, value); >>> else { >>> tab[i] = newNode(hash, key, value, first); >>> if (binCount >= TREEIFY_THRESHOLD - 1) >>> treeifyBin(tab, hash); >>> } >>> ++modCount; >>> ++size; >>> afterNodeInsertion(true); >>> } >>> return value; >>> } >>> >>> The code in the if branch will never be executed because `value` was >>> previously checked at the beginning of the method. >>> >>> Is this a mistake? >> >> You mean it will ALWAYS be executed. Yes, it looks to me like the ?if? is >> useless. >> >> >> > From david.holmes at oracle.com Wed Jul 4 23:01:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 5 Jul 2018 09:01:34 +1000 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> Message-ID: On 4/07/2018 6:21 PM, Lindenmaier, Goetz wrote: > Hi, > > Volker, thanks for improving on my original change and > implementing David's and Karen's proposals. > > David, I think the change addresses a row of your concerns. > >> proposal - it just moves the "field" from the Java side to the VM side. > No, the space overhead in case of successful initialization is > reduced from O(classes) to O(classloaders). I don't think space usage was one of my concerns. > There is only runtime overhead if there was an error, and > that should be acceptable. > >> dealing with backtrace and stackTrace. I have to wonder why nothing in >> Throwable clears the backtrace today ? > Maybe the concern about the backTraces is pointless and the > conversion to stackTraces should be dropped. > As you say, it's done nowhere else, and other backTraces should > cause similar issues. My comment on the clearing of Throwable.backtrace was purely in relation to the internal state protocol of Throwable, not about the conversion process as such. The conversion process is needed, as previously discussed, to avoid class/classloader leaks. Exceptions are normally thread-contained so there is limited scope for creating class/classloader leaks. However as soon as you allow this original exception to be stored and potentially later accessed in a different thread then the leak is possible - it might even be possible to hit some kind of loader constraint error if a same named type can be loaded by different classloaders used by the different threads. Add some extra information about the type of the original exception by all means, but not the stacktrace (unless you've stringified it). As I say the place the stack is of interest in when the original exception is encountered. If something is swallowing this then logging is the tool to expose it IMHO. >> I'm not clear why you record the ExceptionInInitializerError wrapper >> instead of the actual exception that occurred? > Keeping the ExceptionInInitializerError is helpful for people > to understand that this has happened during initialization and not directly > where the NCDFE is thrown. They will understand that this might have > happened in another thread. The NCDFE message already states something like "cannot initialize class because prior initialization attempt failed", which I think makes it quite clear. I dispute "they will understand this might have happened in another thread". Cheers, David > Best regards, > Goetz. > > > > > > >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of David Holmes >> Sent: Sonntag, 1. Juli 2018 23:48 >> To: Volker Simonis ; hotspot-runtime- >> dev at openjdk.java.net runtime ; >> Java Core Libs >> Subject: Re: RFR(M): 8203826: Chain class initialization exceptions into later >> NoClassDefFoundErrors >> >> Hi Volker, >> >> This doesn't really address any of the concerns I had with the original >> proposal - it just moves the "field" from the Java side to the VM side. >> There is still a massive amount of Java code execution in relation to >> this - which itself may encounter secondary exceptions. It's very hard >> to tell if you will leave things in a suitable state if such exceptions >> arise. >> >> My position remains that the primary place to deal with the >> initialization error is when initialization occurs and the error >> happens. Subsequent attempted uses of the erroneous class may benefit >> from some additional information about the nature of the original >> exceptions, but I don't think full stacktraces are necessary or >> desirable (and I do believe they will confuse most users given the lack >> of continuity in the stack frames and that they may have happened in a >> different thread!). >> >> That aside ... >> >> There appears to a race on constructing the Hashtable. At least it was >> not obvious to me where a lock may be held during that process. >> >> I can't determine that clearing backtrace in removeNativeBacktrace is >> correct with respect to the overall protocol within Throwable for >> dealing with backtrace and stackTrace. I have to wonder why nothing in >> Throwable clears the backtrace today ? >> >> I'm not clear why you record the ExceptionInInitializerError wrapper >> instead of the actual exception that occurred? >> >> Throwable states: >> >> + * This method is currently only called from the VM for instances of >> + * ExceptionInInitializerError which are stored for later chaining >> into a >> + * NoClassDefFoundError in order to prevent keeping classes from >> the native >> + * backtrace alive. >> + */ >> >> but IIUC it will also be called for instances of Error that occurred >> which do not get wrapped in EIIE. >> >> >> Regards, >> David >> ------ >> >> >> On 30/06/2018 12:53 AM, Volker Simonis wrote: >>> Hi, >>> >>> can I please have a review for the following change which saves >>> ExceptionInInitializerError thrown during class initialization and >>> chains them as cause into potential NoClassDefFoundErrors for the same >>> class. We are using this features since years in our commercial SAP >>> JVM and it proved extremely useful for detecting and fixing errors >>> especially in big deployments. >>> >>> This is a follow-up on a discussion previously started by Goetz [1]. >>> His first proposal (which is close to our current, internal >>> implementation) inserted an additional field into java.lang.Class >>> objects to save potential ExceptionInInitializerErrors. This was >>> considered to much overhead in the initial discussion [1]. >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/2018/8203826.v2/ >>> https://bugs.openjdk.java.net/browse/JDK-8203826 >>> >>> So in this change, I've completely re-implemented the feature by using >>> a java.lang.Hashtable which is attached to the ClassLoaderData object. >>> The Hashtable is lazily created when the first >>> ExceptionInInitializerError is thrown and maps the Class which >>> triggered the ExceptionInInitializerError during the execution of its >>> static initializer to the corresponding ExceptionInInitializerError. >>> >>> If the same class will be accessed once again, this will directly lead >>> to a plain NoClassDefFoundError (as per the JVMS, 5.5 Initialization) >>> because the static initializer won't be executed a second time. Until >>> now, this NoClassDefFoundError wasn't linked in any way to the root >>> cause of the problem (i.e. the first ExceptionInInitializerError >>> together with the chained exception that happened during the execution >>> of the static initializer). With this change, the NoClassDefFoundError >>> will now chain the initial ExceptionInInitializerError as cause, >>> making it much easier to detect the problem which lead to the >>> NoClassDefFoundError. >>> >>> Following is an example from the new JTreg tests which comes which >>> this change to demonstrate the feature. Until know, a typical stack >>> trace from a NoClassDefFoundError looked as follows: >>> >>> java.lang.NoClassDefFoundError: Could not initialize class >>> NoClassDefFound$ClassWithFailedInitializer >>> at java.base/java.lang.Class.forName0(Native Method) >>> at java.base/java.lang.Class.forName(Class.java:291) >>> at NoClassDefFound.main(NoClassDefFound.java:38) >>> >>> With this change, the same stack trace now looks as follows: >>> >>> java.lang.NoClassDefFoundError: Could not initialize class >>> NoClassDefFound$ClassWithFailedInitializer >>> at java.base/java.lang.Class.forName0(Native Method) >>> at java.base/java.lang.Class.forName(Class.java:315) >>> at NoClassDefFound.main(NoClassDefFound.java:38) >>> Caused by: java.lang.ExceptionInInitializerError >>> at >> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0( >> Native >>> Method) >>> at >> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance( >> NativeConstructorAccessorImpl.java:62) >>> at >> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstan >> ce(DelegatingConstructorAccessorImpl.java:45) >>> at >> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) >>> at java.base/java.lang.Class.newInstance(Class.java:584) >>> at >> NoClassDefFound$ClassWithFailedInitializer.(NoClassDefFound.java:2 >> 0) >>> at java.base/java.lang.Class.forName0(Native Method) >>> at java.base/java.lang.Class.forName(Class.java:315) >>> at NoClassDefFound.main(NoClassDefFound.java:30) >>> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 2 out of >>> bounds for length 1 >>> at NoClassDefFound$A.(NoClassDefFound.java:9) >>> ... 9 more >>> >>> As you can see, the reason for the NoClassDefFoundError when accessing >>> the class 'NoClassDefFound$ClassWithFailedInitializer' is actually not >>> even in the class or its static initializer itself, but in the class >>> 'NoClassDefFound$A' which is a base class of >>> 'NoClassDefFound$ClassWithFailedInitializer'. This is not easily >>> detectible from the old, plain NoClassDefFoundError. >>> >>> As I wrote, the only overhead we have with the new implementation is >>> an additional OopHandle field per ClassLoaderData which I think is >>> acceptable. The Hashtable object itself is only created lazily, after >>> the first occurrence of an ExceptionInInitializerError in the >>> corresponding class loader. The whole Hashtable creation and >>> storing/quering of ExceptionInInitializerErrors in >>> >> ClassLoaderData::record_init_exception()/ClassLoaderData::query_init_exce >> ption() >>> is optional in the sense that any errors/exceptions occurring during >>> the execution of these functions are ignored and cleared. >>> >>> Finally, we also take care to recursively convert all native >>> backtraces in the stored ExceptionInInitializerErrors (and their >>> suppressed and chained exceptions) into symbolic stack traces in order >>> to avoid holding references to classes and prevent their unloading. >>> This is implemented in the new private, static method >>> java.lang.Throwable::removeNativeBacktrace() which is called for each >>> ExceptionInInitializerError before it is stored in the Hashtable. >>> >>> Thank you and best regards, >>> Volker >>> >>> [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018- >> June/028310.html >>> From martinrb at google.com Thu Jul 5 15:26:05 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 5 Jul 2018 08:26:05 -0700 Subject: Useless null check in HashMap.merge() In-Reply-To: References: Message-ID: The null check will be removed as part of the next jsr166 integration. http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html On Wed, Jul 4, 2018 at 2:42 AM, Zheka Kozlov wrote: > I noticed dead code in java.util.HashMap.merge(): > > public V merge(K key, V value, > BiFunction > remappingFunction) { > if (value == null) > throw new NullPointerException(); > > ... > > if (value != null) { *// Condition ' value != null' is always true* > if (t != null) > t.putTreeVal(this, tab, hash, key, value); > else { > tab[i] = newNode(hash, key, value, first); > if (binCount >= TREEIFY_THRESHOLD - 1) > treeifyBin(tab, hash); > } > ++modCount; > ++size; > afterNodeInsertion(true); > } > return value; > } > > The code in the if branch will never be executed because `value` was > previously checked at the beginning of the method. > > Is this a mistake? > From xueming.shen at oracle.com Thu Jul 5 16:43:52 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 05 Jul 2018 09:43:52 -0700 Subject: Adding new IBM extended charsets In-Reply-To: References: Message-ID: <5B3E4AC8.3070307@oracle.com> On 7/4/18, 5:41 AM, Nasser Ebrahim wrote: > Hello, > > Am starting this mail thread to discuss about adding new IBM extended > charsets. The questions is whether we need to add the new extended > charsets to jdk.charsets or to a new separate charset provider/module like > jdk.ibmcharsets. This discussion is in continuation of the suggestion from > Alan Bateman in the mail chain - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html. > > > Am copying his inputs from that mail thread to start the discussion: > "I think we should start a discussion here about moving some or all of the > IBM charsets to their own service provider module. I realize the AIX port > might want to include some of them in its build of java.base but they > aren't interesting to include in java.base, or even jdk.charsets, on most > platform" > > First, let me clarify whether IBM charsets are applicable only to IBM > platforms like AIX or applicable to other platforms as well. All IBM > charsets are applicable to any platforms including Linux and windows if > those platforms needs to communicate with an application or database in > IBM platforms like AIX. That is the reason, we traditionally add them to > the jdk.charsets. However, we agree with Alan that those IBM charsets are > not required if the JDK is not communicating to any applications/databases > on IBM platforms. Hence, it makes sense to consider a separate charset > provider / module for IBM charsets and use build parameters to decide > whether to generate the new charset provider or not for any platforms. > > Let me list out all the possible options I can think of for adding new > extended charsets so that we can discuss and decide which is the best > option. > > 1) Continue to add new extended charsets to jdk.charsets. > The advantage with this approach is that no need to add new charset > provider and all extended charsets are placed in one module. Also, any > extended charset is applicable to any platform if they need to communicate > with application/database in different platforms. The disadvantage is that > the number of charsets in jdk.charsets keep increasing and blot its size. > Also, many of those charsets may not be used in the lifetime of the JDK > unless it is communicating with application/databases of those platforms. > > 2) Create a new charset provider and module (say jdk.ibmcharsets) for all > IBM charsets and include the new module in JDK on a need basis. > The advantage with this approach is that the foot print of jdk.charsets > can be reduced and can include the new module only if it is required. The > disadvantage is that a new charset provider needs to be created. Also, > extended charsets will be located in two different modules and many a > times both the modules are required. > > 3) Remove all extended charsets from JDK (keep only default charsets) and > use the extended charsets from third party like ICU4J. > I believe this option might be discussed in the past and there might be > valid reason not to pursue this option. Am still listing it to ensure that > we have considered this option as well. The advantage with this approach > is that we can avoid maintaining the same charsets by two different open > source communities. The disadvantage with this option is that the release > cycle of the two communities may be different and we may need to maintain > the level ourselves for LTS releases as we may not want to change the > specification in a service stream. > > Please share your thoughts on your preferred option and list out any other > options which I missed out. Thank you for your time. > > Hi Nasser, I would assume the best option might be (3), in which only keeps those ibm charsets that might be used/configured to be the default charset for vm startup on IBM platform, and leaves the rest to icu4j charsets provider. There are hundreds of IBM specific charsets, and with various versions that probably are not going to be used by most of the Java developers in most normal use scenarios. I would assume it's NOT in our best interest to implement, test and support/maintain them in openjdk project/repo. Since the jar is maven repo already, any request for the support of those charsets can be pointed to there accordingly. I did not know there is such package/jar in the maven. I did check the icu4j years ago and there was no nio provider interface provided back then. For those to be kept in openjdk repo, they could/should be separated from the extended charsets and kept into a separate package/module as well, and probably are only configured to build into the binaries for IBM platforms, and maintained by the AIX port. Thanks, Sherman From srinivas.dama at oracle.com Thu Jul 5 17:42:29 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 5 Jul 2018 10:42:29 -0700 (PDT) Subject: [11] RFR: 8198405: JImageExtractTest.java & JImageListTest.java failed in Windows. Message-ID: Hi, Please review Webrev: http://cr.openjdk.java.net/~sdama/8198405/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8198405 Regards, Srinivas From james.laskey at oracle.com Thu Jul 5 17:44:10 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 5 Jul 2018 14:44:10 -0300 Subject: [11] RFR: 8198405: JImageExtractTest.java & JImageListTest.java failed in Windows. In-Reply-To: References: Message-ID: +1 > On Jul 5, 2018, at 2:42 PM, Srinivas Dama wrote: > > Hi, > > Please review > Webrev: http://cr.openjdk.java.net/~sdama/8198405/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8198405 > > Regards, > Srinivas From huizhe.wang at oracle.com Thu Jul 5 19:29:50 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 05 Jul 2018 12:29:50 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8206164: forgot to "throw" TransformerConfigurationException Message-ID: <5B3E71AE.9030707@oracle.com> Hi, Please review a quick fix for a missing throw in what supposed to be a throw statement. JBS: https://bugs.openjdk.java.net/browse/JDK-8206164 webrev: http://cr.openjdk.java.net/~joehw/jdk12/8206164/webrev/ Thanks, Joe From lance.andersen at oracle.com Thu Jul 5 20:01:39 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 5 Jul 2018 16:01:39 -0400 Subject: RFR(JDK12/JAXP/java.xml) 8206164: forgot to "throw" TransformerConfigurationException In-Reply-To: <5B3E71AE.9030707@oracle.com> References: <5B3E71AE.9030707@oracle.com> Message-ID: <97D6EA70-C452-44EB-A3C4-189DC1AFFEBD@oracle.com> Hi Joe, The change looks fine as does the newly added test > On Jul 5, 2018, at 3:29 PM, Joe Wang wrote: > > Hi, > > Please review a quick fix for a missing throw in what supposed to be a throw statement. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8206164 > webrev: http://cr.openjdk.java.net/~joehw/jdk12/8206164/webrev/ > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From naoto.sato at oracle.com Thu Jul 5 20:16:21 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 5 Jul 2018 13:16:21 -0700 Subject: [12] RFR: 8206350: java/util/Locale/bcp47u/SystemPropertyTests.java failed on Mac 10.13 with zh_CN and zh_TW locales. Message-ID: <7db8f039-1e5d-9fa3-2068-ed28611345c5@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8206350 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8206350/webrev.00/ The fix is to set the script property explicitly empty so that the test works on platforms that provide the default locales with scripts involved (e.g., zh-Hans-CN). Naoto From Roger.Riggs at Oracle.com Thu Jul 5 20:39:47 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 5 Jul 2018 16:39:47 -0400 Subject: [12] RFR: 8206350: java/util/Locale/bcp47u/SystemPropertyTests.java failed on Mac 10.13 with zh_CN and zh_TW locales. In-Reply-To: <7db8f039-1e5d-9fa3-2068-ed28611345c5@oracle.com> References: <7db8f039-1e5d-9fa3-2068-ed28611345c5@oracle.com> Message-ID: <3701a17c-4770-cf8c-08f4-bf749554784a@Oracle.com> Hi Naoto, Looks fine Roger On 7/5/2018 4:16 PM, Naoto Sato wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8206350 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8206350/webrev.00/ > > The fix is to set the script property explicitly empty so that the > test works on platforms that provide the default locales with scripts > involved (e.g., zh-Hans-CN). > > Naoto > From david.holmes at oracle.com Thu Jul 5 21:51:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Jul 2018 07:51:49 +1000 Subject: hg: jdk/jdk11: 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: <201807051633.w65GXpxV017200@aojmv0008.oracle.com> References: <201807051633.w65GXpxV017200@aojmv0008.oracle.com> Message-ID: <21e6cfe2-3498-3068-8028-0fecd4dda15c@oracle.com> Volker, This change breaks our testing. Your edit to TestIBMBugs.java contains non-ascii characters in the comments, resulting in javac failure: /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: error: unmappable character (0xC2) for encoding US-ASCII // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e ^ /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: error: unmappable character (0xA5) for encoding US-ASCII // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e ^ /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: error: unmappable character (0xC2) for encoding US-ASCII // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e ^ /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: error: unmappable character (0xA5) for encoding US-ASCII // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e ^ 4 errors result: Failed. Compilation failed: Compilation failed David ----- On 6/07/2018 2:33 AM, volker.simonis at gmail.com wrote: > Changeset: faf1cd52a5b7 > Author: simonis > Date: 2018-06-12 13:00 +0530 > URL: http://hg.openjdk.java.net/jdk/jdk11/rev/faf1cd52a5b7 > > 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 > Reviewed-by: simonis, stuefe > Contributed-by: bhamaram at in.ibm.com > > ! src/java.base/unix/native/libjava/java_props_md.c > ! test/jdk/sun/nio/cs/TestIBMBugs.java > From joe.darcy at oracle.com Thu Jul 5 23:40:00 2018 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 5 Jul 2018 16:40:00 -0700 Subject: RFR 8206123 : ArrayDeque created with default constructor can only hold 15 elements In-Reply-To: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> References: <2e77872a-79ce-488a-fee3-d761998154a2@oracle.com> Message-ID: Hello, Note that another approach would be to change the spec to match the long-standing implementation. That is the the default resolution patch in situations like this, but in this case I don't think it matters very much how the matter is resolved. Cheers, -Joe On 6/29/2018 7:21 PM, Ivan Gerasimov wrote: > Hello! > > The spec states that an ArrayDeque created with the default > constructor should be able to hold 16 elements. > > https://docs.oracle.com/javase/10/docs/api/java/util/ArrayDeque.html#%3Cinit%3E() > > """ > Constructs an empty array deque with an initial capacity sufficient to > hold 16 elements. > """ > > In the constructor an array of size 16 is created: > ??????? elements = new Object[16]; > > However, one element must always be null: > ??? /** > ???? * The array in which the elements of the deque are stored. > ???? * All array cells not holding deque elements are always null. > ???? * The array always has at least one null slot (at tail). > ???? */ > ??? transient Object[] elements; > > which leaves us space for only 15 elements, contrarily to what the > spec promises. > > > Would you please help review a trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8206123 > WEBREV: http://cr.openjdk.java.net/~igerasim/8206123/00/webrev/ > > Thanks in advance! > From ioi.lam at oracle.com Fri Jul 6 00:45:29 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 5 Jul 2018 17:45:29 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> Message-ID: <64c6255c-7e5f-7688-6848-018504f479be@oracle.com> Hi Jiangli, Thank you so much for working on this. I think it's great that we can get the start-up improvement by archiving the ModuleDescriptor. I just have some coding style comments regarding heapShared.cpp. This file contains the code for coping objects and relocating pointers. By its nature, this kind of code is usually complicated, so I think we should try to make it as easy to understand as possible. [1] HeapShared::walk_from_field_and_archiving: ??? This name is not grammatically correct. How about HeapShared::archive_reachable_objects_from_static_field [2] How about changing the parameter field_offset -> static_field_offset ??? When I first read the code I was confused whether it's talking ??? about static or instance fields. Usually, "field" ??? implies instance field, so it's better to specifically ??? say "static field". [3] This code would fail if "f" is already archived. ??? 473?? // get the archived copy of the field referenced object ??? 474?? oop af = MetaspaceShared::archive_heap_object(f, THREAD); ??? 475?? WalkOopAndArchiveClosure walker(1, subgraph_info, f, af); ??? 476?? f->oop_iterate(&walker); [4] There's duplicated code between walk_from_field_and_archiving and ? ? WalkOopAndArchiveClosure::do_oop_work ??? 403?? assert(relocated_k == MetaspaceShared::get_relocated_klass(orig_k), ? ? 404????????? "must be the relocated Klass in the shared space"); ??? 405?? _subgraph_info->add_subgraph_object_klass(orig_k, relocated_k); ??? - vs - ? ? 484?? assert(relocated_k == MetaspaceShared::get_relocated_klass(orig_k), ? ? 485????????? "must be the relocated Klass in the shared space"); ? ? 486?? subgraph_info->add_subgraph_object_klass(orig_k, relocated_k); [5] This code? is also duplicated: ? ? 375?? RawAccess::oop_store(new_p, archived); ? ? 376?? log.print("--- archived copy existing, store archived " PTR_FORMAT " in " PTR_FORMAT, ? ? 377???????????? p2i(archived), p2i(new_p)); ??? - vs - ? ? 395? RawAccess::oop_store(new_p, archived); ??? 396? log.print("=== store archived " PTR_FORMAT " in " PTR_FORMAT, ??? 397??????????? p2i(archived), p2i(new_p)); [6] This code, even though it's correct, is hard to understand -- ? ? why are we calculating the distance between the two objects? ? ? 368? size_t delta = pointer_delta((HeapWord*)_archived_referencing_obj, ? ? 369 (HeapWord*)_orig_referencing_obj); ? ? 370? T* new_p = (T*)((HeapWord*)p + delta); ??? I thin it would be easier to understand if we change the order of the ? ? two arithmetic operations: ??? // new_p is the address of the same field inside _archived_referencing_obj. ??? size_t field_offset_in_bytes = pointer_delta(p, _orig_referencing_obj, 1); ??? T* new_p = (T*)(address(_orig_referencing_obj) + field_offset_in_bytes); [7] I have a hard time understand this log: ??? 376?? log.print("--- archived copy existing, store archived " PTR_FORMAT " in " PTR_FORMAT, ??? 377???????????? p2i(archived), p2i(new_p)); ??? How about this? ??? log.print("--- updated embedded pointer @[" PTR_FORMAT "] => " PTR_FORMAT, ????????????? p2i(new_p), p2i(archived)); For your consideration, I've incorporated my comments above into heapShared.cpp. I've not tested it so it most likely won't build :-( http://cr.openjdk.java.net/~iklam/misc/heapShared.old.cpp? [your version] http://cr.openjdk.java.net/~iklam/misc/heapShared.new.cpp? [my version] Please take a look and see if you like it. Thanks - Ioi On 6/28/18 4:15 PM, Jiangli Zhou wrote: > This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). > > The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. > > The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. > > webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 > > Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. > > Following are the details of system module archiving, which are duplicated in above bug report. > --------------------------------------------------------------------------------------------------------------------------- > Support archiving system module graph when the initial module is unnamed module from -cp currently. > > Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. > > Dump time system module object archiving > ================================= > At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. > > private static SystemModules archivedSystemModules; > private static ModuleFinder archivedSystemModuleFinder; > private static String archivedMainModule; > > The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. > > 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. > 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. > 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. > 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. > 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. > > Runtime initialization from archived system module objects > ============================================ > VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. > > If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. > > In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. > > Thanks, > Jiangli > > From david.holmes at oracle.com Fri Jul 6 01:00:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Jul 2018 11:00:09 +1000 Subject: hg: jdk/jdk11: 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: <21e6cfe2-3498-3068-8028-0fecd4dda15c@oracle.com> References: <201807051633.w65GXpxV017200@aojmv0008.oracle.com> <21e6cfe2-3498-3068-8028-0fecd4dda15c@oracle.com> Message-ID: <138fc4fa-afca-6dbb-a3f6-b9edc16f8a35@oracle.com> Filed: https://bugs.openjdk.java.net/browse/JDK-8206436 David On 6/07/2018 7:51 AM, David Holmes wrote: > edit to TestIBMBugs.java contains non-ascii characters in the comments, > resulting in javac failure: > > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: > error: unmappable character (0xC2) for encoding US-ASCII > ??????? // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e > ?????????????? ^ > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: > error: unmappable character (0xA5) for encoding US-ASCII > ??????? // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e > ??????????????? ^ > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: > error: unmappable character (0xC2) for encoding US-ASCII > ??????? // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e > ?????????????? ^ > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: > error: unmappable character (0xA5) for encoding US-ASCII > ??????? // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e > ??????????????? ^ > 4 errors > result: Failed. Compilation failed: Compilation failed From david.holmes at oracle.com Fri Jul 6 01:25:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Jul 2018 11:25:50 +1000 Subject: [11] RFR: 8198405: JImageExtractTest.java & JImageListTest.java failed in Windows. In-Reply-To: References: Message-ID: <33b4c2a6-9473-1c05-f8c6-28548b33ac45@oracle.com> The test still fails in JDK 11 CI tier 2 after this patch was applied. See link in bug report. David On 6/07/2018 3:44 AM, Jim Laskey wrote: > +1 > >> On Jul 5, 2018, at 2:42 PM, Srinivas Dama wrote: >> >> Hi, >> >> Please review >> Webrev: http://cr.openjdk.java.net/~sdama/8198405/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8198405 >> >> Regards, >> Srinivas > From jiangli.zhou at Oracle.COM Fri Jul 6 02:38:38 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 5 Jul 2018 19:38:38 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <64c6255c-7e5f-7688-6848-018504f479be@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> <64c6255c-7e5f-7688-6848-018504f479be@oracle.com> Message-ID: Hi Ioi, Thanks for the review! > On Jul 5, 2018, at 5:45 PM, Ioi Lam wrote: > > Hi Jiangli, > > Thank you so much for working on this. I think it's great that we can get the > start-up improvement by archiving the ModuleDescriptor. > > I just have some coding style comments regarding heapShared.cpp. This file > contains the code for coping objects and relocating pointers. By its nature, > this kind of code is usually complicated, so I think we should try to make > it as easy to understand as possible. > > > [1] HeapShared::walk_from_field_and_archiving: > > This name is not grammatically correct. How about > HeapShared::archive_reachable_objects_from_static_field Sounds good. > > [2] How about changing the parameter field_offset -> static_field_offset > When I first read the code I was confused whether it's talking > about static or instance fields. Usually, "field" > implies instance field, so it's better to specifically > say "static field?. Ok. > > [3] This code would fail if "f" is already archived. > > 473 // get the archived copy of the field referenced object > 474 oop af = MetaspaceShared::archive_heap_object(f, THREAD); > 475 WalkOopAndArchiveClosure walker(1, subgraph_info, f, af); > 476 f->oop_iterate(&walker); Hmmm, it?s possible we might encounter an archived object during reference walking & archiving in future cases. I?ll add a check. > > [4] There's duplicated code between walk_from_field_and_archiving and > WalkOopAndArchiveClosure::do_oop_work > > 403 assert(relocated_k == MetaspaceShared::get_relocated_klass(orig_k), > 404 "must be the relocated Klass in the shared space"); > 405 _subgraph_info->add_subgraph_object_klass(orig_k, relocated_k); > > - vs - > > 484 assert(relocated_k == MetaspaceShared::get_relocated_klass(orig_k), > 485 "must be the relocated Klass in the shared space"); > 486 subgraph_info->add_subgraph_object_klass(orig_k, relocated_k); I?ll move the assert into add_subgraph_object_klass(). > > [5] This code is also duplicated: > > 375 RawAccess::oop_store(new_p, archived); > 376 log.print("--- archived copy existing, store archived " PTR_FORMAT " in " PTR_FORMAT, > 377 p2i(archived), p2i(new_p)); > > - vs - > > 395 RawAccess::oop_store(new_p, archived); > 396 log.print("=== store archived " PTR_FORMAT " in " PTR_FORMAT, > 397 p2i(archived), p2i(new_p)); The first case is for existing archived copy and the second is for newly archived. The different logging messages are helpful for debugging. Not sure if using a function to encapsulate the store & log worth it in this case. Any suggestion? > > [6] This code, even though it's correct, is hard to understand -- > why are we calculating the distance between the two objects? > > 368 size_t delta = pointer_delta((HeapWord*)_archived_referencing_obj, > 369 (HeapWord*)_orig_referencing_obj); > 370 T* new_p = (T*)((HeapWord*)p + delta); > > I thin it would be easier to understand if we change the order of the > two arithmetic operations: > > // new_p is the address of the same field inside _archived_referencing_obj. > size_t field_offset_in_bytes = pointer_delta(p, _orig_referencing_obj, 1); > T* new_p = (T*)(address(_orig_referencing_obj) + field_offset_in_bytes); I think this works too. I?ll change as you suggested. > > [7] I have a hard time understand this log: > > 376 log.print("--- archived copy existing, store archived " PTR_FORMAT " in " PTR_FORMAT, > 377 p2i(archived), p2i(new_p)); > > How about this? > > log.print("--- updated embedded pointer @[" PTR_FORMAT "] => " PTR_FORMAT, > p2i(new_p), p2i(archived)); It is for the case where there is an existing copy of the archived object. Maybe ?found existing archived copy? would help? > > > For your consideration, I've incorporated my comments above into heapShared.cpp. > I've not tested it so it most likely won't build :-( > > > http://cr.openjdk.java.net/~iklam/misc/heapShared.old.cpp [your version] > http://cr.openjdk.java.net/~iklam/misc/heapShared.new.cpp [my version] > > Please take a look and see if you like it. Thanks a lot! I?ll take a look and incorporate your suggestions. Thanks again! Jiangli > > Thanks > - Ioi > > On 6/28/18 4:15 PM, Jiangli Zhou wrote: >> This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). >> >> The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. >> >> The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. >> >> webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 >> >> Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. >> >> Following are the details of system module archiving, which are duplicated in above bug report. >> --------------------------------------------------------------------------------------------------------------------------- >> Support archiving system module graph when the initial module is unnamed module from -cp currently. >> >> Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. >> >> Dump time system module object archiving >> ================================= >> At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. >> >> private static SystemModules archivedSystemModules; >> private static ModuleFinder archivedSystemModuleFinder; >> private static String archivedMainModule; >> >> The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. >> >> 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. >> 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. >> 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. >> 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. >> 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. >> >> Runtime initialization from archived system module objects >> ============================================ >> VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. >> >> If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. >> >> In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. >> >> Thanks, >> Jiangli >> >> > From joe.darcy at oracle.com Fri Jul 6 03:37:29 2018 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 5 Jul 2018 20:37:29 -0700 Subject: JDK 12 RFR of JDK-8206440: Remove javac -source/-target 6 from jdk regression tests Message-ID: <0340ec5c-6121-0c9c-4f46-2381cebc4ed9@oracle.com> Hello, With javac support for -source/-target 6 and 1.6 planned to be removed later in JDK 12 (JDK-8028563), please review the removal of usage of these options in the non-manual jdk regression tests: ??? http://cr.openjdk.java.net/~darcy/8206440.0/ Patch below. Thanks, -Joe --- old/test/jdk/java/lang/reflect/OldenCompilingWithDefaults.java 2018-07-05 20:34:07.084000000 -0700 +++ new/test/jdk/java/lang/reflect/OldenCompilingWithDefaults.java 2018-07-05 20:34:06.936000000 -0700 @@ -25,7 +25,6 @@ ? * @test ? * @bug 8009267 ? * @summary Verify uses of isAnnotationPresent compile under older source versions - * @compile -source 1.6 -target 1.6 OldenCompilingWithDefaults.java ? * @compile -source 1.7 -target 1.7 OldenCompilingWithDefaults.java ? * @compile OldenCompilingWithDefaults.java ? */ From bhamaram at in.ibm.com Fri Jul 6 05:10:51 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Fri, 6 Jul 2018 05:10:51 +0000 Subject: hg: jdk/jdk11: 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: <138fc4fa-afca-6dbb-a3f6-b9edc16f8a35@oracle.com> References: <138fc4fa-afca-6dbb-a3f6-b9edc16f8a35@oracle.com>, <201807051633.w65GXpxV017200@aojmv0008.oracle.com> <21e6cfe2-3498-3068-8028-0fecd4dda15c@oracle.com> Message-ID: Hi, Sorry for the trouble. It had got compiled in my build environment, might be due to encoding is different. Please check if following patch make sense. diff -r 5cb2d4a8b8d4 test/jdk/sun/nio/cs/TestIBMBugs.java --- a/test/jdk/sun/nio/cs/TestIBMBugs.java Tue Jun 12 13:00:50 2018 +0530 +++ b/test/jdk/sun/nio/cs/TestIBMBugs.java Fri Jul 06 10:24:56 2018 +0530 @@ -174,7 +174,7 @@ } private static void bug8202329() throws Exception { - String original = "\\\u007E\u00A5\u203E"; // "b"; \\ [backslash][tilde][yen][overscore] + String original = "\\\u007E\u00A5\u203E"; // [backslash][tilde][yen][overscore] byte[] expectedBytes; // bytes after conversion String expectedStringfromBytes; // String constructed from bytes @@ -187,7 +187,7 @@ // Test IBM943, where \ and ~ are encoded to unmappable i.e., 0x3f - // and ? andoverscore are encoded to 0x5c and 0x7e + // [yen] and [overscore] are encoded to 0x5c and 0x7e charset = Charset.forName("IBM943"); expectedBytes = new byte[] {0x3f, 0x3f, 0x5c, 0x7e}; expectedStringfromBytes = "??\u00A5\u203E"; @@ -204,7 +204,7 @@ // Test IBM943C, where \ and ~ are encoded to 0x5c and 0x7e - // and ? an overscore are encoded to 0x5c and 0x7e + // [yen] and [overscore] are encoded to 0x5c and 0x7e charset = Charset.forName("IBM943C"); expectedBytes = new byte[] {0x5c, 0x7e, 0x5c, 0x7e}; expectedStringfromBytes = "\\~\\~"; Thanks, Bhaktavatsal Reddy -----"core-libs-dev" wrote: ----- To: volker.simonis at gmail.com From: David Holmes Sent by: "core-libs-dev" Date: 07/06/2018 06:31AM Cc: core-libs-dev Libs Subject: Re: hg: jdk/jdk11: 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 Filed: https://bugs.openjdk.java.net/browse/JDK-8206436 David On 6/07/2018 7:51 AM, David Holmes wrote: > edit to TestIBMBugs.java contains non-ascii characters in the comments, > resulting in javac failure: > > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: > error: unmappable character (0xC2) for encoding US-ASCII > // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e > ^ > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: > error: unmappable character (0xA5) for encoding US-ASCII > // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e > ^ > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: > error: unmappable character (0xC2) for encoding US-ASCII > // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e > ^ > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: > error: unmappable character (0xA5) for encoding US-ASCII > // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e > ^ > 4 errors > result: Failed. Compilation failed: Compilation failed From volker.simonis at gmail.com Fri Jul 6 05:30:21 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 6 Jul 2018 07:30:21 +0200 Subject: hg: jdk/jdk11: 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: <138fc4fa-afca-6dbb-a3f6-b9edc16f8a35@oracle.com> References: <201807051633.w65GXpxV017200@aojmv0008.oracle.com> <21e6cfe2-3498-3068-8028-0fecd4dda15c@oracle.com> <138fc4fa-afca-6dbb-a3f6-b9edc16f8a35@oracle.com> Message-ID: Thanks for opening the bug. I?ll fix it in an hour when I?m in the office. Regards, Volker David Holmes schrieb am Fr. 6. Juli 2018 um 03:00: > Filed: https://bugs.openjdk.java.net/browse/JDK-8206436 > > David > > On 6/07/2018 7:51 AM, David Holmes wrote: > > edit to TestIBMBugs.java contains non-ascii characters in the comments, > > resulting in javac failure: > > > > > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: > > > error: unmappable character (0xC2) for encoding US-ASCII > > // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e > > ^ > > > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: > > > error: unmappable character (0xA5) for encoding US-ASCII > > // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e > > ^ > > > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: > > > error: unmappable character (0xC2) for encoding US-ASCII > > // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e > > ^ > > > /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: > > > error: unmappable character (0xA5) for encoding US-ASCII > > // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e > > ^ > > 4 errors > > result: Failed. Compilation failed: Compilation failed > From david.holmes at oracle.com Fri Jul 6 05:47:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Jul 2018 15:47:15 +1000 Subject: hg: jdk/jdk11: 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: References: <138fc4fa-afca-6dbb-a3f6-b9edc16f8a35@oracle.com> <201807051633.w65GXpxV017200@aojmv0008.oracle.com> <21e6cfe2-3498-3068-8028-0fecd4dda15c@oracle.com> Message-ID: <309bf51d-21c4-27bc-eca3-78622e3e5045@oracle.com> Hi, On 6/07/2018 3:10 PM, Bhaktavatsal R Maram wrote: > Hi, > > Sorry for the trouble. It had got compiled in my build environment, might be due to encoding is different. Yes likely the encoding. > Please check if following patch make sense. Yes looks fine. Only the latter two are needed to fix the problem, but the first cleanup seems reasonable too. Thanks, David ----- > > diff -r 5cb2d4a8b8d4 test/jdk/sun/nio/cs/TestIBMBugs.java > --- a/test/jdk/sun/nio/cs/TestIBMBugs.java Tue Jun 12 13:00:50 2018 +0530 > +++ b/test/jdk/sun/nio/cs/TestIBMBugs.java Fri Jul 06 10:24:56 2018 +0530 > @@ -174,7 +174,7 @@ > } > > private static void bug8202329() throws Exception { > - String original = "\\\u007E\u00A5\u203E"; // "b"; \\ [backslash][tilde][yen][overscore] > + String original = "\\\u007E\u00A5\u203E"; // [backslash][tilde][yen][overscore] > byte[] expectedBytes; // bytes after conversion > String expectedStringfromBytes; // String constructed from bytes > > @@ -187,7 +187,7 @@ > > > // Test IBM943, where \ and ~ are encoded to unmappable i.e., 0x3f > - // and ? andoverscore are encoded to 0x5c and 0x7e > + // [yen] and [overscore] are encoded to 0x5c and 0x7e > charset = Charset.forName("IBM943"); > expectedBytes = new byte[] {0x3f, 0x3f, 0x5c, 0x7e}; > expectedStringfromBytes = "??\u00A5\u203E"; > @@ -204,7 +204,7 @@ > > > // Test IBM943C, where \ and ~ are encoded to 0x5c and 0x7e > - // and ? an overscore are encoded to 0x5c and 0x7e > + // [yen] and [overscore] are encoded to 0x5c and 0x7e > charset = Charset.forName("IBM943C"); > expectedBytes = new byte[] {0x5c, 0x7e, 0x5c, 0x7e}; > expectedStringfromBytes = "\\~\\~"; > > > Thanks, > Bhaktavatsal Reddy > > > -----"core-libs-dev" wrote: ----- > To: volker.simonis at gmail.com > From: David Holmes > Sent by: "core-libs-dev" > Date: 07/06/2018 06:31AM > Cc: core-libs-dev Libs > Subject: Re: hg: jdk/jdk11: 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 > > Filed: https://bugs.openjdk.java.net/browse/JDK-8206436 > > David > > On 6/07/2018 7:51 AM, David Holmes wrote: >> edit to TestIBMBugs.java contains non-ascii characters in the comments, >> resulting in javac failure: >> >> /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: >> error: unmappable character (0xC2) for encoding US-ASCII >> // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e >> ^ >> /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:191: >> error: unmappable character (0xA5) for encoding US-ASCII >> // and \ufffd\ufffd andoverscore are encoded to 0x5c and 0x7e >> ^ >> /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: >> error: unmappable character (0xC2) for encoding US-ASCII >> // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e >> ^ >> /scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk11-jdk.1010/src.full/open/test/jdk/sun/nio/cs/TestIBMBugs.java:208: >> error: unmappable character (0xA5) for encoding US-ASCII >> // and \ufffd\ufffd an overscore are encoded to 0x5c and 0x7e >> ^ >> 4 errors >> result: Failed. Compilation failed: Compilation failed > > From volker.simonis at gmail.com Fri Jul 6 06:56:35 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 6 Jul 2018 08:56:35 +0200 Subject: [11] RFR(XXS): 8206436: sun/nio/cs/TestIBMBugs.java no longer compiles Message-ID: Hi, can I please have a review for this trivial test fix? http://cr.openjdk.java.net/~simonis/webrevs/2018/8206436/ https://bugs.openjdk.java.net/browse/JDK-8206436 The problem is that the test contains some non-ASCII characters in comments which makes the compilation fail for non-Unicode locales. Thank you and best regards, Volker From mikael.vidstedt at oracle.com Fri Jul 6 07:03:04 2018 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Fri, 6 Jul 2018 00:03:04 -0700 Subject: [11] RFR(XXS): 8206436: sun/nio/cs/TestIBMBugs.java no longer compiles In-Reply-To: References: Message-ID: Fix the typo while at it? + // and [yen] an [overscore] are encoded to 0x5c and 0x7e an -> and Cheers, Mikael > On Jul 5, 2018, at 11:56 PM, Volker Simonis wrote: > > Hi, > > can I please have a review for this trivial test fix? > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8206436/ > https://bugs.openjdk.java.net/browse/JDK-8206436 > > The problem is that the test contains some non-ASCII characters in > comments which makes the compilation fail for non-Unicode locales. > > Thank you and best regards, > Volker From thomas.stuefe at gmail.com Fri Jul 6 07:05:01 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 6 Jul 2018 09:05:01 +0200 Subject: [11] RFR(XXS): 8206436: sun/nio/cs/TestIBMBugs.java no longer compiles In-Reply-To: References: Message-ID: Hi Volker, looks good and trivial. I did not test-compile it, but I trust you did that already. Best Regards, Thomas On Fri, Jul 6, 2018 at 8:56 AM, Volker Simonis wrote: > Hi, > > can I please have a review for this trivial test fix? > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8206436/ > https://bugs.openjdk.java.net/browse/JDK-8206436 > > The problem is that the test contains some non-ASCII characters in > comments which makes the compilation fail for non-Unicode locales. > > Thank you and best regards, > Volker From Alan.Bateman at oracle.com Fri Jul 6 07:06:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 6 Jul 2018 08:06:39 +0100 Subject: [11] RFR(XXS): 8206436: sun/nio/cs/TestIBMBugs.java no longer compiles In-Reply-To: References: Message-ID: <5f86d965-b53e-45f7-aafa-a813740b4dcd@oracle.com> On 06/07/2018 08:03, Mikael Vidstedt wrote: > Fix the typo while at it? > > + // and [yen] an [overscore] are encoded to 0x5c and 0x7e > > an -> and > > I agree, otherwise looks okay to me. -Alan From Alan.Bateman at oracle.com Fri Jul 6 07:10:08 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 6 Jul 2018 08:10:08 +0100 Subject: JDK 12 RFR of JDK-8206440: Remove javac -source/-target 6 from jdk regression tests In-Reply-To: <0340ec5c-6121-0c9c-4f46-2381cebc4ed9@oracle.com> References: <0340ec5c-6121-0c9c-4f46-2381cebc4ed9@oracle.com> Message-ID: On 06/07/2018 04:37, joe darcy wrote: > Hello, > > With javac support for -source/-target 6 and 1.6 planned to be removed > later in JDK 12 (JDK-8028563), please review the removal of usage of > these options in the non-manual jdk regression tests: This looks okay. From volker.simonis at gmail.com Fri Jul 6 07:25:37 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 6 Jul 2018 09:25:37 +0200 Subject: [11] RFR(XXS): 8206436: sun/nio/cs/TestIBMBugs.java no longer compiles In-Reply-To: <5f86d965-b53e-45f7-aafa-a813740b4dcd@oracle.com> References: <5f86d965-b53e-45f7-aafa-a813740b4dcd@oracle.com> Message-ID: Hi Mikael, Thomas, Alan, thanks a lot for the quick reviews. I've just fixed the typo and pushed the fix. Sorry for the trouble, Volker On Fri, Jul 6, 2018 at 9:06 AM, Alan Bateman wrote: > On 06/07/2018 08:03, Mikael Vidstedt wrote: >> >> Fix the typo while at it? >> >> + // and [yen] an [overscore] are encoded to 0x5c and 0x7e >> >> an -> and >> >> > I agree, otherwise looks okay to me. > > -Alan From srinivas.dama at oracle.com Fri Jul 6 08:03:04 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Fri, 6 Jul 2018 01:03:04 -0700 (PDT) Subject: [11] RFR: 8198405: JImageExtractTest.java & JImageListTest.java failed in Windows. Message-ID: Hi, This patch fixes JImageExtractTest.java but nothing done specific to JImageListTest.java as I couldn't reproduce this failure earlier. So I just removed JImageListTest.java from ProblemList.txt. Now I am reverting this change and raised a new bug specific to this failure[JDK-8206445]. Please review this trivial fix to revert ProblemList.txt webrev: http://cr.openjdk.java.net/~sdama/8206450/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8206450 Regards, Srinivas ----- Original Message ----- From: david.holmes at oracle.com To: james.laskey at oracle.com, srinivas.dama at oracle.com Cc: core-libs-dev at openjdk.java.net Sent: Friday, 6 July, 2018 6:56:00 AM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [11] RFR: 8198405: JImageExtractTest.java & JImageListTest.java failed in Windows. The test still fails in JDK 11 CI tier 2 after this patch was applied. See link in bug report. David On 6/07/2018 3:44 AM, Jim Laskey wrote: > +1 > >> On Jul 5, 2018, at 2:42 PM, Srinivas Dama wrote: >> >> Hi, >> >> Please review >> Webrev: http://cr.openjdk.java.net/~sdama/8198405/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8198405 >> >> Regards, >> Srinivas > From david.holmes at oracle.com Fri Jul 6 08:05:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Jul 2018 18:05:09 +1000 Subject: [11] RFR: 8198405: JImageExtractTest.java & JImageListTest.java failed in Windows. In-Reply-To: References: Message-ID: <1e0fbdb5-3efc-4d98-ce36-3482032197c4@oracle.com> On 6/07/2018 6:03 PM, Srinivas Dama wrote: > Hi, > > This patch fixes JImageExtractTest.java but nothing done specific to JImageListTest.java as > I couldn't reproduce this failure earlier. > So I just removed JImageListTest.java from ProblemList.txt. > > Now I am reverting this change and raised a new bug specific to this failure[JDK-8206445]. > > Please review this trivial fix to revert ProblemList.txt > webrev: http://cr.openjdk.java.net/~sdama/8206450/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8206450 That looks okay. Thanks, David > Regards, > Srinivas > ----- Original Message ----- > From: david.holmes at oracle.com > To: james.laskey at oracle.com, srinivas.dama at oracle.com > Cc: core-libs-dev at openjdk.java.net > Sent: Friday, 6 July, 2018 6:56:00 AM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: Re: [11] RFR: 8198405: JImageExtractTest.java & JImageListTest.java failed in Windows. > > The test still fails in JDK 11 CI tier 2 after this patch was applied. > See link in bug report. > > David > > On 6/07/2018 3:44 AM, Jim Laskey wrote: >> +1 >> >>> On Jul 5, 2018, at 2:42 PM, Srinivas Dama wrote: >>> >>> Hi, >>> >>> Please review >>> Webrev: http://cr.openjdk.java.net/~sdama/8198405/webrev.00/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8198405 >>> >>> Regards, >>> Srinivas >> From takiguc at linux.vnet.ibm.com Fri Jul 6 09:23:20 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 06 Jul 2018 18:23:20 +0900 Subject: [11] RFR(XXS): 8206436: sun/nio/cs/TestIBMBugs.java no longer compiles In-Reply-To: References: Message-ID: <9ffd92b61a1497d28728591208bdfe1e@linux.vnet.ibm.com> Hello. I'm not reviewer, but I tested your fix with some locales (non-UTF8 locales) on AIX platform. $ LANG=C javac TestIBMBugs.java $ LANG=ko_KR javac TestIBMBugs.java $ LANG=zh_CN javac TestIBMBugs.java Also testcase worked fine as expected. Thanks, Ichiroh Takiguchi IBM Japan, Ltd. On 2018-07-06 15:56, Volker Simonis wrote: > Hi, > > can I please have a review for this trivial test fix? > > http://cr.openjdk.java.net/~simonis/webrevs/2018/8206436/ > https://bugs.openjdk.java.net/browse/JDK-8206436 > > The problem is that the test contains some non-ASCII characters in > comments which makes the compilation fail for non-Unicode locales. > > Thank you and best regards, > Volker From ramanand.patil at oracle.com Fri Jul 6 09:34:58 2018 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Fri, 6 Jul 2018 02:34:58 -0700 (PDT) Subject: RFR[jdk8u-dev]: 8134124: sun/security/tools/jarsigner/warnings.sh fails when using Hindi locale Message-ID: Hi all, Please review this trivial test fix: Bug: https://bugs.openjdk.java.net/browse/JDK-8134124 Webrev: http://cr.openjdk.java.net/~rpatil/8134124/webrev.00/ The test is made locale independent now. Regards, Ramanand. From enasser at in.ibm.com Fri Jul 6 12:23:50 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Fri, 6 Jul 2018 12:23:50 +0000 Subject: Adding new IBM extended charsets In-Reply-To: <36548af8-d5fd-204f-d08e-fc60ffbc09a9@redhat.com> References: <36548af8-d5fd-204f-d08e-fc60ffbc09a9@redhat.com> Message-ID: Hi Florian, Thank you for your response. iconv is platform dependent and not good for the platform agnostic nature of Java. Also, many charsets in Java are not available across platforms. I believe Java decided to have its own charsets due to those reasons so that it can work seamlessly on any supported platforms. Thank you, Nasser Ebrahim From: Florian Weimer To: Nasser Ebrahim , Java Core Libs Date: 07/04/2018 07:11 PM Subject: Re: Adding new IBM extended charsets On 07/04/2018 02:41 PM, Nasser Ebrahim wrote: > Please share your thoughts on your preferred option and list out any other > options which I missed out. Thank you for your time. Could you use the platform iconv implementation instead? That would avoid shipping the tables in the JDK. Thanks, Florian From matthias.baesken at sap.com Fri Jul 6 12:44:10 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 6 Jul 2018 12:44:10 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> Message-ID: <036f88e423b743bbb68c73e2a59486d4@sap.com> Hi Alan ,so it looks like JDK-8204233 added a switch (system property) to enable the enhanced socket IOException messages . That would be an option as well for 8205525 . 8205525 adds the jar file name and the line number info to the exception message . In case that only the jar file name would be considered sensitive , I would prefer to just output the line number (and omit the system property ). What do you think ? Best regards, Matthias > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Montag, 25. Juni 2018 16:52 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > On 25/06/2018 15:29, Baesken, Matthias wrote: > > Hi, do you consider both the file name and line number as sensitive ? > > > >> There was a similar discussion on net-dev recently related to > >> leaking host names in exceptions. Something similar may be needed here > >> > > Do you know the outcome of this discussion ? > > > All the details are in JDK-8204233 and the associated CSR. > > -Alan From enasser at in.ibm.com Fri Jul 6 13:56:10 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Fri, 6 Jul 2018 13:56:10 +0000 Subject: Adding new IBM extended charsets In-Reply-To: <5B3E4AC8.3070307@oracle.com> References: <5B3E4AC8.3070307@oracle.com> Message-ID: Hi Sherman, Thank you for your valuable inputs. I would like to clarify some of your points. > I would assume the best option might be (3), in which only keeps those > ibm charsets that might be used/configured to be the default charset for > vm startup on IBM platform, and leaves the rest to icu4j charsets provider. I understood you preferred option is 3 [Remove all extended charsets from JDK (keep only default charsets) and use the extended charsets from third party like ICU4J]. Just to confirm, so you meant we need to keep only the standard charsets in the JDK and remove all the extended charsets from JDK and use them from ICU4J OR you meant apply that only for the new extended charsets. I think it is better to keep the consistency - either take all extended charsets from ICU4J or maintain all extended charsets with JDK. Keeping some extended charsets within JDK and use ICU4J for other extended charsets may confuse the Java user. > There are hundreds of IBM specific charsets, and with various versions that > probably are not going to be used by most of the Java developers in most > normal use scenarios. I would assume it's NOT in our best interest to > implement, > test and support/maintain them in openjdk project/repo. As I explained in the previous note, the IBM charsets are not just applicable to IBM Platforms. It can be applicable to any platforms if Java has to access an application or database on IBM platforms. For example, if a Java application on Linux or Windows requires EBCDIC code pages for JDBC to access DB2 on z/OS. Precisely, there are 75 charsets from IBM JDK which are currently missing in openjdk. Within the 75 missing charsets, 2 charsets are default charsets on AIX platform. Out of the remaining 73 charsets, some of them are default charsets for z/OS platform. We would like to contribute those 75 charsets to openJDK. > For those to be kept in openjdk repo, they could/should be separated from > the extended charsets and kept into a separate package/module as well, and > probably are only configured to build into the binaries for IBM > platforms, and > maintained by the AIX port. Yes. I agree it makes sense to keep all IBM charsets as a separate charset provider / module and include them on a need basis to reduce the footprint. If everybody agrees, we can start working towards creating a separate charset provider/module for IBM charsets in JDK. Please advise. Thank you, Nasser Ebrahim From: Xueming Shen To: core-libs-dev at openjdk.java.net Date: 07/05/2018 10:14 PM Subject: Re: Adding new IBM extended charsets Sent by: "core-libs-dev" On 7/4/18, 5:41 AM, Nasser Ebrahim wrote: > Hello, > > Am starting this mail thread to discuss about adding new IBM extended > charsets. The questions is whether we need to add the new extended > charsets to jdk.charsets or to a new separate charset provider/module like > jdk.ibmcharsets. This discussion is in continuation of the suggestion from > Alan Bateman in the mail chain - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html . > > > Am copying his inputs from that mail thread to start the discussion: > "I think we should start a discussion here about moving some or all of the > IBM charsets to their own service provider module. I realize the AIX port > might want to include some of them in its build of java.base but they > aren't interesting to include in java.base, or even jdk.charsets, on most > platform" > > First, let me clarify whether IBM charsets are applicable only to IBM > platforms like AIX or applicable to other platforms as well. All IBM > charsets are applicable to any platforms including Linux and windows if > those platforms needs to communicate with an application or database in > IBM platforms like AIX. That is the reason, we traditionally add them to > the jdk.charsets. However, we agree with Alan that those IBM charsets are > not required if the JDK is not communicating to any applications/databases > on IBM platforms. Hence, it makes sense to consider a separate charset > provider / module for IBM charsets and use build parameters to decide > whether to generate the new charset provider or not for any platforms. > > Let me list out all the possible options I can think of for adding new > extended charsets so that we can discuss and decide which is the best > option. > > 1) Continue to add new extended charsets to jdk.charsets. > The advantage with this approach is that no need to add new charset > provider and all extended charsets are placed in one module. Also, any > extended charset is applicable to any platform if they need to communicate > with application/database in different platforms. The disadvantage is that > the number of charsets in jdk.charsets keep increasing and blot its size. > Also, many of those charsets may not be used in the lifetime of the JDK > unless it is communicating with application/databases of those platforms. > > 2) Create a new charset provider and module (say jdk.ibmcharsets) for all > IBM charsets and include the new module in JDK on a need basis. > The advantage with this approach is that the foot print of jdk.charsets > can be reduced and can include the new module only if it is required. The > disadvantage is that a new charset provider needs to be created. Also, > extended charsets will be located in two different modules and many a > times both the modules are required. > > 3) Remove all extended charsets from JDK (keep only default charsets) and > use the extended charsets from third party like ICU4J. > I believe this option might be discussed in the past and there might be > valid reason not to pursue this option. Am still listing it to ensure that > we have considered this option as well. The advantage with this approach > is that we can avoid maintaining the same charsets by two different open > source communities. The disadvantage with this option is that the release > cycle of the two communities may be different and we may need to maintain > the level ourselves for LTS releases as we may not want to change the > specification in a service stream. > > Please share your thoughts on your preferred option and list out any other > options which I missed out. Thank you for your time. > > Hi Nasser, I would assume the best option might be (3), in which only keeps those ibm charsets that might be used/configured to be the default charset for vm startup on IBM platform, and leaves the rest to icu4j charsets provider. There are hundreds of IBM specific charsets, and with various versions that probably are not going to be used by most of the Java developers in most normal use scenarios. I would assume it's NOT in our best interest to implement, test and support/maintain them in openjdk project/repo. Since the jar is maven repo already, any request for the support of those charsets can be pointed to there accordingly. I did not know there is such package/jar in the maven. I did check the icu4j years ago and there was no nio provider interface provided back then. For those to be kept in openjdk repo, they could/should be separated from the extended charsets and kept into a separate package/module as well, and probably are only configured to build into the binaries for IBM platforms, and maintained by the AIX port. Thanks, Sherman From peter.levart at gmail.com Fri Jul 6 16:10:19 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 6 Jul 2018 18:10:19 +0200 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> Message-ID: Hi, On 07/05/2018 01:01 AM, David Holmes wrote: > I dispute "they will understand this might have happened in another > thread". What if the stack trace was like the following... Before patch: 1st attempt [ForkJoinPool.commonPool-worker-3]: java.lang.ExceptionInInitializerError ??????? at ClinitFailure.lambda$main$0(ClinitFailure.java:20) ??????? at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) ??????? at java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) ??????? at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ??????? at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ??????? at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ??????? at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ??????? at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) Caused by: java.lang.RuntimeException: Can't get it! ??????? at ClinitFailure$Faulty.(ClinitFailure.java:12) ??????? ... 8 more Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0 ??????? at ClinitFailure$Faulty.(ClinitFailure.java:10) ??????? ... 8 more 2nd attempt [ForkJoinPool.commonPool-worker-5]: java.lang.NoClassDefFoundError: Could not initialize class ClinitFailure$Faulty ??????? at ClinitFailure.lambda$main$1(ClinitFailure.java:28) ??????? at java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) ??????? at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) ??????? at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ??????? at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ??????? at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ??????? at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ??????? at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) After patch: 1st attempt [ForkJoinPool.commonPool-worker-3]: java.lang.ExceptionInInitializerError ??????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) ??????? at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) ??????? at java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) ??????? at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ??????? at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ??????? at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ??????? at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ??????? at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) Caused by: java.lang.RuntimeException: Can't get it! ??????? at ClinitFailure$Faulty.(ClinitFailure.java:10) ??????? ... 8 more Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0 ??????? at ClinitFailure$Faulty.(ClinitFailure.java:8) ??????? ... 8 more 2nd attempt [ForkJoinPool.commonPool-worker-5]: java.lang.NoClassDefFoundError: Could not initialize class ClinitFailure$Faulty ??????? at java.base/java.lang.ClassLoader.throwReinitException(ClassLoader.java:3062) ??????? at ClinitFailure.lambda$main$1(ClinitFailure.java:25) ??????? at java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) ??????? at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) ??????? at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ??????? at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ??????? at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ??????? at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ??????? at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) Caused by: java.lang.ExceptionInInitializerError: 11 ms ago in thread ForkJoinPool.commonPool-worker-3 ??????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) ??????? at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) ??????? at java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) ??????? ... 5 more Caused by: java.lang.RuntimeException: Can't get it! ??????? at ClinitFailure$Faulty.(ClinitFailure.java:10) ??????? ... 8 more Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0 ??????? at ClinitFailure$Faulty.(ClinitFailure.java:8) ??????? ... 8 more This is what gets printed by the sample program: public class ClinitFailure { ??? static class Faulty { ??????? static { ??????????? try { ??????????????? int i = (new int[0])[1]; ??????????? } catch (Exception e) { ??????????????? throw new RuntimeException("Can't get it!", e); ??????????? } ??????? } ??? } ??? public static void main(String[] args) throws Exception { ??????? CompletableFuture.runAsync(() -> { ??????????? try { ??????????????? new Faulty(); ??????????? } catch (Throwable e) { ??????????????? System.out.printf("\n1st attempt [%s]:\n\n", Thread.currentThread().getName()); ??????????????? e.printStackTrace(System.out); ??????????? } ??????? }).thenRunAsync(() -> { ??????????? try { ??????????????? new Faulty(); ??????????? } catch (Throwable e) { ??????????????? System.out.printf("\n2nd attempt [%s]:\n\n", Thread.currentThread().getName()); ??????????????? e.printStackTrace(System.out); ??????????? } ??????? }).join(); ??? } } When the following patch is applied: http://cr.openjdk.java.net/~plevart/jdk-dev/8203826_NoClassDefFoundError.cause/webrev.01/ I took Volker's patch and modified it a bit: - The logic to construct and throw NoClassDefFoundError and to record initial exception is in java now. It uses ClassLoaderValue internal API to save the chains of exception(s) for faulty classes. It is easier to do such logic in Java and less error prone. - The chain of original exception(s) is replaced with substitutes that mimic .toString() and .printStackTrace() methods of original chain, but don't reference any classes outside bootstrap class loader - The replacement chain of original exceptions adds a custom message insert into the top exception as a hint to the user: ??? ??? java.lang.ExceptionInInitializerError: 11 ms ago in thread ForkJoinPool.commonPool-worker-3 So, what do you think of this one? Regards, Peter From calvin.cheung at oracle.com Fri Jul 6 16:15:39 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 06 Jul 2018 09:15:39 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> Message-ID: <5B3F95AB.7060702@oracle.com> Hi Jiangli, Thanks for this start-up improvement. The changes look good overall. I've the following minor comments. 1) make/hotspot/symbols/symbols-unix 134 JVM_InitializeFromArchive If you want the symbols to be in alphabetical order, the above should be moved after JVM_InitStackTraceElementArray. 2) metaspaceShared.cpp 1927 oop MetaspaceShared::materialize_archived_object(oop obj) { 1928 if (obj != NULL) { 1929 return G1CollectedHeap::heap()->materialize_archived_object(obj); 1930 } 1931 return NULL; 1932 } Instead of two return statements, how about replacing lines 1928 - 1931 with the following? return (obj != NULL) ? G1CollectedHeap::heap()->materialize_archived_object(obj) : NULL; 3) ArchivedModuleComboTest.java 55 Path moduleDir = Files.createTempDirectory(userDir, "mods"); I don't see anything got placed under the "mods" dir, is it by design? For the "dump with --module-path" cases, there seems to be a missing test case with "--show-module-resolution" (similar to Test case 2). 4) CheckArchivedModuleApp.java 53 if (expectArchived && wb.isShared(md)) { 54 System.out.println(name + " is archived. Expected."); 55 } else if (!expectArchived && !wb.isShared(md)) { 56 System.out.println(name + " is not archived. Expected."); 57 } else if (expectArchived) { 58 throw new RuntimeException( 59 "FAILED. " + name + " is not archived. Expect archived."); 60 } else { 61 throw new RuntimeException( 62 "FAILED. " + name + " is archived. Expect not archived."); 63 } I'd suggest the following so that the code is easier to understand: if (expectArchived) { if (wb.isShared(md)) { System.out.println(name + " is archived. Expected."); } else { throw new RuntimeException( "FAILED. " + name + " is not archived. Expect archived."); } } else { if (!wb.isShared(md)) { System.out.println(name + " is not archived. Expected."); } else { throw new RuntimeException( "FAILED. " + name + " is archived. Expect not archived."); } } 5) ArchivedModuleWithCustomImageTest.java 178 private static void printCommand(String opts[]) { 179 StringBuilder cmdLine = new StringBuilder(); 180 for (String cmd : opts) 181 cmdLine.append(cmd).append(' '); 182 System.out.println("Command line: [" + cmdLine.toString() + "]"); 183 } Consider putting the above method in ProcessTools.java so that ProcessTools.createJavaProcessBuilder() and the above test can call it and avoiding duplicate code. A separate follow-up bug to address this is fine. 6) PrintSystemModulesApp.java I don't think it is being used? thanks, Calvin On 6/28/18, 4:15 PM, Jiangli Zhou wrote: > This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). > > The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. > > The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. > > webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 > > Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. > > Following are the details of system module archiving, which are duplicated in above bug report. > --------------------------------------------------------------------------------------------------------------------------- > Support archiving system module graph when the initial module is unnamed module from -cp currently. > > Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. > > Dump time system module object archiving > ================================= > At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. > > private static SystemModules archivedSystemModules; > private static ModuleFinder archivedSystemModuleFinder; > private static String archivedMainModule; > > The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. > > 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. > 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. > 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. > 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. > 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. > > Runtime initialization from archived system module objects > ============================================ > VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. > > If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. > > In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. > > Thanks, > Jiangli > > From james.laskey at oracle.com Fri Jul 6 17:16:14 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 6 Jul 2018 14:16:14 -0300 Subject: RFR: CSR - JDK-8200435 String::align, String::indent (Preview) Message-ID: <058F895B-B27D-407D-88E1-B2EA49E4D601@oracle.com> csr: https://bugs.openjdk.java.net/browse/JDK-8200435 From Roger.Riggs at Oracle.com Fri Jul 6 17:31:20 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 6 Jul 2018 13:31:20 -0400 Subject: RFR: CSR - JDK-8200435 String::align, String::indent (Preview) In-Reply-To: <058F895B-B27D-407D-88E1-B2EA49E4D601@oracle.com> References: <058F895B-B27D-407D-88E1-B2EA49E4D601@oracle.com> Message-ID: Hi Jim, The phrase: "All Character#isWhitespace(int) white space characters, including tab, are treated as a single space." Can be mis-interpreted to mean collectively they are replaced by a single space. Perhaps replace "All" with "Each". Regards, Roger On 7/6/2018 1:16 PM, Jim Laskey wrote: > csr: https://bugs.openjdk.java.net/browse/JDK-8200435 From james.laskey at oracle.com Fri Jul 6 17:43:53 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 6 Jul 2018 14:43:53 -0300 Subject: RFR: CSR - JDK-8200435 String::align, String::indent (Preview) In-Reply-To: References: <058F895B-B27D-407D-88E1-B2EA49E4D601@oracle.com> Message-ID: Changed. Thank you. > On Jul 6, 2018, at 2:31 PM, Roger Riggs wrote: > > Hi Jim, > > The phrase: > > "All Character#isWhitespace(int) white space characters, including tab, are treated as a single space." > > Can be mis-interpreted to mean collectively they are replaced by a single space. > > Perhaps replace "All" with "Each". > > Regards, Roger > > On 7/6/2018 1:16 PM, Jim Laskey wrote: >> csr: https://bugs.openjdk.java.net/browse/JDK-8200435 > From martinrb at google.com Fri Jul 6 18:40:37 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 6 Jul 2018 11:40:37 -0700 Subject: RFR: jsr166 integration 2018-07 Message-ID: http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html Some smallish changes; all of them already seen review elsewhere. 8206123: ArrayDeque created with default constructor can only hold 15 elements http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ArrayDeque-capacity/index.html https://bugs.openjdk.java.net/browse/JDK-8206123 8205576: forkjoin/FJExceptionTableLeak.java fails "AssertionError: failed to satisfy condition" http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/FJExceptionTableLeak/index.html https://bugs.openjdk.java.net/browse/JDK-8205576 8205620: Miscellaneous changes imported from jsr166 CVS 2018-07 http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html https://bugs.openjdk.java.net/browse/JDK-8205620 From Roger.Riggs at Oracle.com Fri Jul 6 19:08:36 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 6 Jul 2018 15:08:36 -0400 Subject: RFR: 8206495 [testbug] Redundant System.setProperty(null) tests Message-ID: <544b5c98-d3a2-64d6-19a3-6ef1be91dde6@Oracle.com> Please review the removal of a duplicate test for System.setProperty(null). Webrev: ?? http://cr.openjdk.java.net/~rriggs/webrev-dup-prop-null/ Thanks, Roger From mandy.chung at oracle.com Fri Jul 6 19:15:13 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 6 Jul 2018 12:15:13 -0700 Subject: RFR: 8206495 [testbug] Redundant System.setProperty(null) tests In-Reply-To: <544b5c98-d3a2-64d6-19a3-6ef1be91dde6@Oracle.com> References: <544b5c98-d3a2-64d6-19a3-6ef1be91dde6@Oracle.com> Message-ID: +1 Mandy On 7/6/18 12:08 PM, Roger Riggs wrote: > Please review the removal of a duplicate test for System.setProperty(null). > > Webrev: > ?? http://cr.openjdk.java.net/~rriggs/webrev-dup-prop-null/ > > Thanks, Roger > From lance.andersen at oracle.com Fri Jul 6 19:20:05 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 6 Jul 2018 15:20:05 -0400 Subject: RFR: 8206495 [testbug] Redundant System.setProperty(null) tests In-Reply-To: <544b5c98-d3a2-64d6-19a3-6ef1be91dde6@Oracle.com> References: <544b5c98-d3a2-64d6-19a3-6ef1be91dde6@Oracle.com> Message-ID: <082480D3-9D0C-4EA2-8A1C-39A360CB5EC0@oracle.com> Hi Roger, Looks fine. > On Jul 6, 2018, at 3:08 PM, Roger Riggs wrote: > > Please review the removal of a duplicate test for System.setProperty(null). > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-dup-prop-null/ > > Thanks, Roger > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From jiangli.zhou at Oracle.COM Fri Jul 6 19:34:59 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Fri, 6 Jul 2018 12:34:59 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <5B3F95AB.7060702@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> <5B3F95AB.7060702@oracle.com> Message-ID: Hi Calvin, Thanks for the review! Here is the updated webrevs that address the feedbacks from you and Ioi: http://cr.openjdk.java.net/~jiangli/8202035/webrev_inc.01/ Full webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev_full.01/ > On Jul 6, 2018, at 9:15 AM, Calvin Cheung wrote: > > Hi Jiangli, > > Thanks for this start-up improvement. The changes look good overall. I've the following minor comments. > > 1) make/hotspot/symbols/symbols-unix > > 134 JVM_InitializeFromArchive > > If you want the symbols to be in alphabetical order, the above should be moved after JVM_InitStackTraceElementArray. Fixed. > > 2) metaspaceShared.cpp > > 1927 oop MetaspaceShared::materialize_archived_object(oop obj) { > 1928 if (obj != NULL) { > 1929 return G1CollectedHeap::heap()->materialize_archived_object(obj); > 1930 } > 1931 return NULL; > 1932 } > > Instead of two return statements, how about replacing lines 1928 - 1931 with the following? > > return (obj != NULL) ? G1CollectedHeap::heap()->materialize_archived_object(obj) : NULL; The original format probably is slightly easier to read, so I left it unchanged. Hope that?s okay with you. > > 3) ArchivedModuleComboTest.java > > 55 Path moduleDir = Files.createTempDirectory(userDir, "mods"); > > I don't see anything got placed under the "mods" dir, is it by design? Yes. > > For the "dump with --module-path" cases, there seems to be a missing test case with "--show-module-resolution" (similar to Test case 2). When --module-path is specified at dump time, system module graph is not archived currently. There is no need for additional test case with --show-module-resolution in this case since all module objects are created as normal. > > > 4) CheckArchivedModuleApp.java > > 53 if (expectArchived && wb.isShared(md)) { > 54 System.out.println(name + " is archived. Expected."); > 55 } else if (!expectArchived && !wb.isShared(md)) { > 56 System.out.println(name + " is not archived. Expected."); > 57 } else if (expectArchived) { > 58 throw new RuntimeException( > 59 "FAILED. " + name + " is not archived. Expect archived."); > 60 } else { > 61 throw new RuntimeException( > 62 "FAILED. " + name + " is archived. Expect not archived."); > 63 } > > I'd suggest the following so that the code is easier to understand: > > if (expectArchived) { > if (wb.isShared(md)) { > System.out.println(name + " is archived. Expected."); > } else { > throw new RuntimeException( > "FAILED. " + name + " is not archived. Expect archived."); > } > } else { > if (!wb.isShared(md)) { > System.out.println(name + " is not archived. Expected."); > } else { > throw new RuntimeException( > "FAILED. " + name + " is archived. Expect not archived."); > } > } Reformatted as suggested. > > 5) ArchivedModuleWithCustomImageTest.java > > 178 private static void printCommand(String opts[]) { > 179 StringBuilder cmdLine = new StringBuilder(); > 180 for (String cmd : opts) > 181 cmdLine.append(cmd).append(' '); > 182 System.out.println("Command line: [" + cmdLine.toString() + "]"); > 183 } > > Consider putting the above method in ProcessTools.java so that ProcessTools.createJavaProcessBuilder() and the above test can call it and avoiding duplicate code. > A separate follow-up bug to address this is fine. That sounds good to me. We might need some reformatting for consolidation. I will file a follow-up RFE. > > 6) PrintSystemModulesApp.java > > I don't think it is being used? It?s used by ArchivedModuleCompareTest.java. Looks like it was missing from the earlier webrev. Thanks for catching that. The file is included in the updated webrev. Thanks! Jiangli > > thanks, > Calvin > > On 6/28/18, 4:15 PM, Jiangli Zhou wrote: >> This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). >> >> The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. >> >> The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. >> >> webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 >> >> Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. >> >> Following are the details of system module archiving, which are duplicated in above bug report. >> --------------------------------------------------------------------------------------------------------------------------- >> Support archiving system module graph when the initial module is unnamed module from -cp currently. >> >> Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. >> >> Dump time system module object archiving >> ================================= >> At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. >> >> private static SystemModules archivedSystemModules; >> private static ModuleFinder archivedSystemModuleFinder; >> private static String archivedMainModule; >> >> The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. >> >> 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. >> 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. >> 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. >> 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. >> 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. >> >> Runtime initialization from archived system module objects >> ============================================ >> VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. >> >> If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. >> >> In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. >> >> Thanks, >> Jiangli >> >> From kevin.rushforth at oracle.com Fri Jul 6 20:14:29 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Fri, 6 Jul 2018 13:14:29 -0700 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> Message-ID: An initial prototype of the jpackager tool has been pushed to a new 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is interested in taking a look, you can clone it as follows: ??? hg clone http://hg.openjdk.java.net/jdk/sandbox ??? cd ./sandbox ??? hg update JDK-8200758-branch I plan to reply to the feedback already provided, and update the JEP [2] next week some time, but in the mean time if you have additional questions or comments, feel free to reply. -- Kevin [1] http://hg.openjdk.java.net/jdk/sandbox/shortlog/JDK-8200758-branch [2] https://bugs.openjdk.java.net/browse/JDK-8200758 On 6/27/2018 3:30 PM, Kevin Rushforth wrote: > We're aiming to get this into JDK 12 early enough so that an EA build > would be available around the time JDK 11 ships. That will allow you > to take a jlinked image with JDK 11 and package it up using (the EA) > jpackager. > > We will create a development branch in the JDK sandbox [1] some time > in the next week or so so you can follow the development. > > Also, thank you to those who have provided feedback. I'll reply to > feedback soon and then incorporate it into an updated JEP. > > -- Kevin From ioi.lam at oracle.com Fri Jul 6 20:36:05 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 6 Jul 2018 13:36:05 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> <5B3F95AB.7060702@oracle.com> Message-ID: Hi Jiangli, The VM changes look good to me. For the tests: I think we need a comment here saying that "mods" is intentionally empty, and also an explanation why it's not necessary to actually fill with actual modules? Thanks - Ioi >> 3) ArchivedModuleComboTest.java >> >> 55 Path moduleDir = Files.createTempDirectory(userDir, "mods"); >> >> I don't see anything got placed under the "mods" dir, is it by design? > Yes. > On 7/6/18 12:34 PM, Jiangli Zhou wrote: > Hi Calvin, > > Thanks for the review! Here is the updated webrevs that address the feedbacks from you and Ioi: > > http://cr.openjdk.java.net/~jiangli/8202035/webrev_inc.01/ > > Full webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev_full.01/ > >> On Jul 6, 2018, at 9:15 AM, Calvin Cheung wrote: >> >> Hi Jiangli, >> >> Thanks for this start-up improvement. The changes look good overall. I've the following minor comments. >> >> 1) make/hotspot/symbols/symbols-unix >> >> 134 JVM_InitializeFromArchive >> >> If you want the symbols to be in alphabetical order, the above should be moved after JVM_InitStackTraceElementArray. > Fixed. > >> 2) metaspaceShared.cpp >> >> 1927 oop MetaspaceShared::materialize_archived_object(oop obj) { >> 1928 if (obj != NULL) { >> 1929 return G1CollectedHeap::heap()->materialize_archived_object(obj); >> 1930 } >> 1931 return NULL; >> 1932 } >> >> Instead of two return statements, how about replacing lines 1928 - 1931 with the following? >> >> return (obj != NULL) ? G1CollectedHeap::heap()->materialize_archived_object(obj) : NULL; > The original format probably is slightly easier to read, so I left it unchanged. Hope that?s okay with you. > >> 3) ArchivedModuleComboTest.java >> >> 55 Path moduleDir = Files.createTempDirectory(userDir, "mods"); >> >> I don't see anything got placed under the "mods" dir, is it by design? > Yes. > >> For the "dump with --module-path" cases, there seems to be a missing test case with "--show-module-resolution" (similar to Test case 2). > When --module-path is specified at dump time, system module graph is not archived currently. There is no need for additional test case with --show-module-resolution in this case since all module objects are created as normal. > >> >> 4) CheckArchivedModuleApp.java >> >> 53 if (expectArchived && wb.isShared(md)) { >> 54 System.out.println(name + " is archived. Expected."); >> 55 } else if (!expectArchived && !wb.isShared(md)) { >> 56 System.out.println(name + " is not archived. Expected."); >> 57 } else if (expectArchived) { >> 58 throw new RuntimeException( >> 59 "FAILED. " + name + " is not archived. Expect archived."); >> 60 } else { >> 61 throw new RuntimeException( >> 62 "FAILED. " + name + " is archived. Expect not archived."); >> 63 } >> >> I'd suggest the following so that the code is easier to understand: >> >> if (expectArchived) { >> if (wb.isShared(md)) { >> System.out.println(name + " is archived. Expected."); >> } else { >> throw new RuntimeException( >> "FAILED. " + name + " is not archived. Expect archived."); >> } >> } else { >> if (!wb.isShared(md)) { >> System.out.println(name + " is not archived. Expected."); >> } else { >> throw new RuntimeException( >> "FAILED. " + name + " is archived. Expect not archived."); >> } >> } > Reformatted as suggested. > >> 5) ArchivedModuleWithCustomImageTest.java >> >> 178 private static void printCommand(String opts[]) { >> 179 StringBuilder cmdLine = new StringBuilder(); >> 180 for (String cmd : opts) >> 181 cmdLine.append(cmd).append(' '); >> 182 System.out.println("Command line: [" + cmdLine.toString() + "]"); >> 183 } >> >> Consider putting the above method in ProcessTools.java so that ProcessTools.createJavaProcessBuilder() and the above test can call it and avoiding duplicate code. >> A separate follow-up bug to address this is fine. > That sounds good to me. We might need some reformatting for consolidation. I will file a follow-up RFE. > >> 6) PrintSystemModulesApp.java >> >> I don't think it is being used? > It?s used by ArchivedModuleCompareTest.java. Looks like it was missing from the earlier webrev. Thanks for catching that. The file is included in the updated webrev. > > Thanks! > Jiangli > >> thanks, >> Calvin >> >> On 6/28/18, 4:15 PM, Jiangli Zhou wrote: >>> This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). >>> >>> The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. >>> >>> The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. >>> >>> webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 >>> >>> Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. >>> >>> Following are the details of system module archiving, which are duplicated in above bug report. >>> --------------------------------------------------------------------------------------------------------------------------- >>> Support archiving system module graph when the initial module is unnamed module from -cp currently. >>> >>> Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. >>> >>> Dump time system module object archiving >>> ================================= >>> At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. >>> >>> private static SystemModules archivedSystemModules; >>> private static ModuleFinder archivedSystemModuleFinder; >>> private static String archivedMainModule; >>> >>> The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. >>> >>> 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. >>> 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. >>> 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. >>> 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. >>> 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. >>> >>> Runtime initialization from archived system module objects >>> ============================================ >>> VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. >>> >>> If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. >>> >>> In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. >>> >>> Thanks, >>> Jiangli >>> >>> From mandy.chung at oracle.com Fri Jul 6 20:40:03 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 6 Jul 2018 13:40:03 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> Message-ID: <9aafedcf-4abf-77fd-e455-823c9e11c8b0@oracle.com> Hi Jiangli, On 6/28/18 4:15 PM, Jiangli Zhou wrote:> webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 Good work. I'm glad to see a pretty good startup improvement. I reviewed java.base change that looks good. Mandy From jiangli.zhou at Oracle.COM Fri Jul 6 20:40:24 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Fri, 6 Jul 2018 13:40:24 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> <5B3F95AB.7060702@oracle.com> Message-ID: > On Jul 6, 2018, at 1:36 PM, Ioi Lam wrote: > > Hi Jiangli, > > The VM changes look good to me. Thanks! > > For the tests: I think we need a comment here saying that "mods" is intentionally empty, and also an explanation why it's not necessary to actually fill with actual modules? Will do. Thanks, Jiangli > > Thanks > > - Ioi > > >>> 3) ArchivedModuleComboTest.java >>> >>> 55 Path moduleDir = Files.createTempDirectory(userDir, "mods"); >>> >>> I don't see anything got placed under the "mods" dir, is it by design? >> Yes. >> > > > On 7/6/18 12:34 PM, Jiangli Zhou wrote: >> Hi Calvin, >> >> Thanks for the review! Here is the updated webrevs that address the feedbacks from you and Ioi: >> >> http://cr.openjdk.java.net/~jiangli/8202035/webrev_inc.01/ >> >> Full webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev_full.01/ >> >>> On Jul 6, 2018, at 9:15 AM, Calvin Cheung wrote: >>> >>> Hi Jiangli, >>> >>> Thanks for this start-up improvement. The changes look good overall. I've the following minor comments. >>> >>> 1) make/hotspot/symbols/symbols-unix >>> >>> 134 JVM_InitializeFromArchive >>> >>> If you want the symbols to be in alphabetical order, the above should be moved after JVM_InitStackTraceElementArray. >> Fixed. >> >>> 2) metaspaceShared.cpp >>> >>> 1927 oop MetaspaceShared::materialize_archived_object(oop obj) { >>> 1928 if (obj != NULL) { >>> 1929 return G1CollectedHeap::heap()->materialize_archived_object(obj); >>> 1930 } >>> 1931 return NULL; >>> 1932 } >>> >>> Instead of two return statements, how about replacing lines 1928 - 1931 with the following? >>> >>> return (obj != NULL) ? G1CollectedHeap::heap()->materialize_archived_object(obj) : NULL; >> The original format probably is slightly easier to read, so I left it unchanged. Hope that?s okay with you. >> >>> 3) ArchivedModuleComboTest.java >>> >>> 55 Path moduleDir = Files.createTempDirectory(userDir, "mods"); >>> >>> I don't see anything got placed under the "mods" dir, is it by design? >> Yes. >> >>> For the "dump with --module-path" cases, there seems to be a missing test case with "--show-module-resolution" (similar to Test case 2). >> When --module-path is specified at dump time, system module graph is not archived currently. There is no need for additional test case with --show-module-resolution in this case since all module objects are created as normal. >> >>> >>> 4) CheckArchivedModuleApp.java >>> >>> 53 if (expectArchived && wb.isShared(md)) { >>> 54 System.out.println(name + " is archived. Expected."); >>> 55 } else if (!expectArchived && !wb.isShared(md)) { >>> 56 System.out.println(name + " is not archived. Expected."); >>> 57 } else if (expectArchived) { >>> 58 throw new RuntimeException( >>> 59 "FAILED. " + name + " is not archived. Expect archived."); >>> 60 } else { >>> 61 throw new RuntimeException( >>> 62 "FAILED. " + name + " is archived. Expect not archived."); >>> 63 } >>> >>> I'd suggest the following so that the code is easier to understand: >>> >>> if (expectArchived) { >>> if (wb.isShared(md)) { >>> System.out.println(name + " is archived. Expected."); >>> } else { >>> throw new RuntimeException( >>> "FAILED. " + name + " is not archived. Expect archived."); >>> } >>> } else { >>> if (!wb.isShared(md)) { >>> System.out.println(name + " is not archived. Expected."); >>> } else { >>> throw new RuntimeException( >>> "FAILED. " + name + " is archived. Expect not archived."); >>> } >>> } >> Reformatted as suggested. >> >>> 5) ArchivedModuleWithCustomImageTest.java >>> >>> 178 private static void printCommand(String opts[]) { >>> 179 StringBuilder cmdLine = new StringBuilder(); >>> 180 for (String cmd : opts) >>> 181 cmdLine.append(cmd).append(' '); >>> 182 System.out.println("Command line: [" + cmdLine.toString() + "]"); >>> 183 } >>> >>> Consider putting the above method in ProcessTools.java so that ProcessTools.createJavaProcessBuilder() and the above test can call it and avoiding duplicate code. >>> A separate follow-up bug to address this is fine. >> That sounds good to me. We might need some reformatting for consolidation. I will file a follow-up RFE. >> >>> 6) PrintSystemModulesApp.java >>> >>> I don't think it is being used? >> It?s used by ArchivedModuleCompareTest.java. Looks like it was missing from the earlier webrev. Thanks for catching that. The file is included in the updated webrev. >> >> Thanks! >> Jiangli >> >>> thanks, >>> Calvin >>> >>> On 6/28/18, 4:15 PM, Jiangli Zhou wrote: >>>> This is a follow-up RFE of JDK-8201650 (Move iteration order randomization of unmodifiable Set and Map to iterators), which was resolved to allow Set/Map objects being archived at CDS dump time (thanks Claes and Stuart Marks). In the current RFE, it archives the set of system ModuleReference and ModuleDescriptor objects (including their referenced objects) in 'open' archive heap region at CDS dump time. It allows reusing of the objects and bypassing the process of creating the system ModuleDescriptors and ModuleReferences at runtime for startup improvement. My preliminary measurements on linux-x64 showed ~5% startup improvement when running HelloWorld from -cp using archived module objects at runtime (without extra tuning). >>>> >>>> The library changes in the following webrev are contributed by Alan Bateman. Thanks Alan and Mandy for discussions and help. Thanks Karen, Lois and Ioi for discussion and suggestions on initialization ordering. >>>> >>>> The majority of the module object archiving code are in heapShared.hpp and heapShared.cpp. Thanks Coleen for pre-review and Eric Caspole for helping performance tests. >>>> >>>> webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 >>>> >>>> Tested using tier1 - tier6 via mach5 including all new test cases added in the webrev. >>>> >>>> Following are the details of system module archiving, which are duplicated in above bug report. >>>> --------------------------------------------------------------------------------------------------------------------------- >>>> Support archiving system module graph when the initial module is unnamed module from -cp currently. >>>> >>>> Support G1 GC, 64-bit (non-Windows). Requires UseCompressedOops and UseCompressedClassPointers. >>>> >>>> Dump time system module object archiving >>>> ================================= >>>> At dump time, the following fields in ArchivedModuleGraph are set to record the system module information created by ModuleBootstrap for archiving. >>>> >>>> private static SystemModules archivedSystemModules; >>>> private static ModuleFinder archivedSystemModuleFinder; >>>> private static String archivedMainModule; >>>> >>>> The archiving process starts from a given static field in ArchivedModuleGraph class instance (java mirror object). The process archives the complete network of java heap objects that are reachable directly or indirectly from the starting object by following references. >>>> >>>> 1. Starts from a given static field within the Class instance (java mirror). If the static field is a refererence field and points to a non-null java object, proceed to the next step. The static field and it's value is recorded and stored outside the archived mirror. >>>> 2. Archives the referenced java object. If an archived copy of the current object already exists, updates the pointer in the archived copy of the referencing object to point to the current archived object. Otherwise, proceed to the next step. >>>> 3. Follows all references within the current java object and recursively archive the sub-graph of objects starting from each reference encountered within the object. >>>> 4. Updates the pointer in the archived copy of referecing object to point to the current archived object. >>>> 5. The Klass of the current java object is added to a list of Klasses for loading and initializing before any object in the archived graph can be accessed at runtime. >>>> >>>> Runtime initialization from archived system module objects >>>> ============================================ >>>> VM.initializeFromArchive() is called from ArchivedModuleGraph's static initializer to initialize from the archived module information. Klasses in the recorded list are loaded, linked and initialized. The static fields in ArchivedModuleGraph class instance are initialized using the archived field values. After initialization, the archived system module objects can be used directly. >>>> >>>> If the archived java heap data is not successfully mapped at runtime, or there is an error during VM.initializeFromArchive(), then all static fields in ArchivedModuleGraph are not initialized. In that case, system ModuleDescriptor and ModuleReference objects are created as normal. >>>> >>>> In non-CDS mode, VM.initializeFromArchive() returns immediately with minimum added overhead for normal execution. >>>> >>>> Thanks, >>>> Jiangli >>>> >>>> > From jiangli.zhou at oracle.com Fri Jul 6 20:41:30 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 6 Jul 2018 13:41:30 -0700 Subject: RFR(L): 8202035: Archive the set of ModuleDescriptor and ModuleReference objects for system modules In-Reply-To: <9aafedcf-4abf-77fd-e455-823c9e11c8b0@oracle.com> References: <386DA770-8E9D-43A0-87CE-0E380977F884@oracle.com> <9aafedcf-4abf-77fd-e455-823c9e11c8b0@oracle.com> Message-ID: <39F0EBB2-3721-4A08-9661-955E4B5E6920@oracle.com> Thanks a lot for reviewing, Mandy! Jiangli > On Jul 6, 2018, at 1:40 PM, mandy chung wrote: > > Hi Jiangli, > > On 6/28/18 4:15 PM, Jiangli Zhou wrote:> webrev: http://cr.openjdk.java.net/~jiangli/8202035/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8202035?filter=14921 > > Good work. I'm glad to see a pretty good startup improvement. > > I reviewed java.base change that looks good. > > Mandy From forax at univ-mlv.fr Fri Jul 6 22:07:54 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 7 Jul 2018 00:07:54 +0200 (CEST) Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> Message-ID: <1973155195.1899675.1530914874386.JavaMail.zimbra@u-pem.fr> I've just taking a look at the patch, i don't see how this can be integrated soon, the code is consistently inconsistent as one of my colleague would say, even the coding conventions are not respected. i believe that's it's because the code have been written first in Java 6 an without refactoring was moved to use Java 7, 8, 9, 10 and 11. The I/O code still using java.io.File for some parts, no try-with-resources so most of the try/finally are not safe, a lot of code like new BufferedWriter(new FileWriter(file)) instead of Files.newBufferedWriter, etc. The code should use the package java.nio.file, and not the old java.io, most of the code try to manage the exception right were they appear instead of propagating them so there are too many try/catch, a lot of catch are ignored which is a code smell, some codes use the internal logger (jdk.packager.internal.Log), but a lot of codes doesn't, for the collection code, there is a lot of copy of data small structures (which suggest that published collections are not immutable), there are dubious @SuppressWarnings("unchecked"), some or them are due to the fact that the code use Class as a type token instead of using lambdas, Stream are not used when they should (to avoid multiple copy between data structures) and streams that need to be closed are not (the result of Files.list by example), there are usual "don't do that in Java" like a loop using an integer index to traverse a List without knowing if it's a random access list or not, there is a lot of nullchecks instead of using Optional, a lot of code initialize local variables to null which is a code smell (and a side effect of having a lot of try/catch but not only), constructors should not do work, just initialization, use static factory method instead (so you will not have to debug half constructed objects), the code uses BigIntegers to parse a bundle version, just in case, the code uses an AtomicReference as a box that a lambda can mutate, instead of wrapping the exception into a runtime and unwrapping it at call site, The code of jdk.packager.internal.IOUtils should be updated to use methods of the JDK 11 and methods like readFully should be replaced by the JDK's one. listOfPathToString and setOfStringToString are just WTF, like in getRedistributableModules(), where the variable stream is an Optional, A class like Platform should be used everywhere to do platform specific stuff, a lot of code still use String matching (the version parsing should use System.Version). All the argument parsing should be delegated to JOpt (the one integrated with the JDK), so it will be consistent with the other JDK tools, There is a UnsupportedPlatformException but a Platform can be UNKOWN ?? I will stop here, my point is that there is a lot of cleaning that should appear before the code is integrated into the JDK and given the size of the code, i wonder if it's not better to start an OpenJDK project for it and iterate on the code before trying to include it in the JDK. For me, the code is too big to use the jdk/sandbox. regards, R?mi ----- Mail original ----- > De: "Kevin Rushforth" > ?: "core-libs-dev" > Cc: "Alexey Semenyuk" , "Andy Herrick" > Envoy?: Vendredi 6 Juillet 2018 22:14:29 > Objet: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] > An initial prototype of the jpackager tool has been pushed to a new > 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is > interested in taking a look, you can clone it as follows: > > ??? hg clone http://hg.openjdk.java.net/jdk/sandbox > ??? cd ./sandbox > ??? hg update JDK-8200758-branch > > I plan to reply to the feedback already provided, and update the JEP [2] > next week some time, but in the mean time if you have additional > questions or comments, feel free to reply. > > -- Kevin > > [1] http://hg.openjdk.java.net/jdk/sandbox/shortlog/JDK-8200758-branch > [2] https://bugs.openjdk.java.net/browse/JDK-8200758 > > > On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >> We're aiming to get this into JDK 12 early enough so that an EA build >> would be available around the time JDK 11 ships. That will allow you >> to take a jlinked image with JDK 11 and package it up using (the EA) >> jpackager. >> >> We will create a development branch in the JDK sandbox [1] some time >> in the next week or so so you can follow the development. >> >> Also, thank you to those who have provided feedback. I'll reply to >> feedback soon and then incorporate it into an updated JEP. >> > > -- Kevin From xueming.shen at oracle.com Fri Jul 6 23:46:09 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 06 Jul 2018 16:46:09 -0700 Subject: RFR JDK-8206389: JarEntry.setCreation/LastAccessTime without setLastModifiedTime causes Invalid CEN header Message-ID: <5B3FFF41.200@oracle.com> Hi Please help review the changeset for JDK-8206389 issue: https://bugs.openjdk.java.net/browse/JDK-8206389 webrev: http://cr.openjdk.java.net/~sherman/8206389/webrev Cause: ZipOutputStream.writeCEN().writeCEN() incorrectly calculate the length of bytes needed for the "unix timestamps" when the "last modified time" is NOT set. The info-zip spec specifies that The central-header extra field contains the modification time only, or no timestamp at all. TSize is used to flag its presence or absence. But note: So in this case, when "creation time", "last access time" are set but the "last modified time" is not. only the "tag", "size" and "flag" should be output to the extra field, no real "mtime" timestamp, so the total size of the bytes needed is 5, not 9. Thanks, Sherman From martinrb at google.com Sat Jul 7 00:43:40 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 6 Jul 2018 17:43:40 -0700 Subject: RFR JDK-8206389: JarEntry.setCreation/LastAccessTime without setLastModifiedTime causes Invalid CEN header In-Reply-To: <5B3FFF41.200@oracle.com> References: <5B3FFF41.200@oracle.com> Message-ID: I would use different timestamps for the 4 possible times used in this test, if only to make it clearer which value comes from where. + // ze.setLastModifiedTime(now);+ ze.setTime(now.toMillis()); setTime only sets the DOS time? Which only has a granularity of 2 seconds? If so, how do you get back the same value you put in if the current second is odd? Or am I misunderstanding the test? On Fri, Jul 6, 2018 at 4:46 PM, Xueming Shen wrote: > Hi > > Please help review the changeset for JDK-8206389 > > issue: https://bugs.openjdk.java.net/browse/JDK-8206389 > webrev: http://cr.openjdk.java.net/~sherman/8206389/webrev > > Cause: ZipOutputStream.writeCEN().writeCEN() incorrectly calculate the > length > of bytes needed for the "unix timestamps" when the "last modified time" is > NOT set. The info-zip spec specifies that > > The central-header extra field contains the modification time > only, > or no timestamp at all. TSize is used to flag its presence or > absence. But note: > > So in this case, when "creation time", "last access time" are set but the > "last modified time" is not. only the "tag", "size" and "flag" should be > output to > the extra field, no real "mtime" timestamp, so the total size of the bytes > needed > is 5, not 9. > > Thanks, > Sherman > > From xueming.shen at oracle.com Sat Jul 7 03:50:14 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 06 Jul 2018 20:50:14 -0700 Subject: RFR JDK-8206389: JarEntry.setCreation/LastAccessTime without setLastModifiedTime causes Invalid CEN header In-Reply-To: References: <5B3FFF41.200@oracle.com> Message-ID: <5B403876.9030909@oracle.com> On 7/6/18, 5:43 PM, Martin Buchholz wrote: > I would use different timestamps for the 4 possible times used in this > test, if only to make it clearer which value comes from where. webrev has been updated accordingly. > > + // ze.setLastModifiedTime(now); > + ze.setTime(now.toMillis()); > > setTime only sets the DOS time? Which only has a granularity of 2 > seconds? If so, how do you get back the same value you put in if the > current second is odd? Or am I misunderstanding the test? > no misunderstanding, good catch. The test does fail when hits the odd second. added a special "check" version for the 2 second granularity set/getTime(). Thanks, Sherman From jai.forums2013 at gmail.com Sat Jul 7 13:35:11 2018 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Sat, 7 Jul 2018 19:05:11 +0530 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <036f88e423b743bbb68c73e2a59486d4@sap.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> Message-ID: <27629b9f-144b-9d32-578c-d3fd41b70f65@gmail.com> Hi Matthias, I am not a reviewer and neither do I have enough knowledge about whether jar/file _names_ are considered security sensitive. However, the patch that's proposed for this change, prints the file _path_ (and not just the name). That I believe is security sensitive. -Jaikiran On 06/07/18 6:14 PM, Baesken, Matthias wrote: > Hi Alan ,so it looks like JDK-8204233 added a switch (system property) > to enable the enhanced socket IOException messages . That would be an > option as well for 8205525 . 8205525 adds the jar file name and the > line number info to the exception message . In case that only the jar > file name would be considered sensitive , I would prefer to just > output the line number (and omit the system property ). What do you > think ? Best regards, Matthias >> -----Original Message----- From: Alan Bateman >> [mailto:Alan.Bateman at oracle.com] Sent: Montag, 25. Juni 2018 16:52 >> To: Baesken, Matthias ; core-libs- >> dev at openjdk.java.net Cc: Lindenmaier, Goetz >> Subject: Re: [RFR] 8205525 : Improve >> exception messages during manifest parsing of jar archives On >> 25/06/2018 15:29, Baesken, Matthias wrote: >>> Hi, do you consider both the file name and line number as sensitive ? >>>> There was a similar discussion on net-dev recently related to >>>> leaking host names in exceptions. Something similar may be needed here >>> Do you know the outcome of this discussion ? >> All the details are in JDK-8204233 and the associated CSR. -Alan From kevin.rushforth at oracle.com Sat Jul 7 13:47:01 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Sat, 7 Jul 2018 06:47:01 -0700 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: <1973155195.1899675.1530914874386.JavaMail.zimbra@u-pem.fr> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <1973155195.1899675.1530914874386.JavaMail.zimbra@u-pem.fr> Message-ID: <142986b8-e2d7-3744-469d-e0209da290e6@oracle.com> Hi Remy, Thank you for taking a look. Yes, the javapackager code that forms the basis for the jpackager prototype was initially developed on older JDKs and evolved from there. I'm sure the improvements you suggest are all good ones, and it doesn't seem like it would be too hard to address the most important of them, especially the try-with-resources or anything else that would affect the robustness of the tool. As long as we do address the robustness issues, I think it is more important to get the feature set right, and make sure that the public interfaces -- the command line options and ToolProvider interface -- are clean. I don't see the need to rewrite the tool or take an extra couple of months to modernize all of the implementation to use JDK 11 APIs everywhere. Also, I don't agree that jpackager is too large for jdk/sandbox or that it needs it own project. The jdk/sandbox is perfect for new modules / new tools that don't impact other parts of the JDK. -- Kevin On 7/6/2018 3:07 PM, Remi Forax wrote: > I've just taking a look at the patch, > i don't see how this can be integrated soon, the code is consistently inconsistent as one of my colleague would say, even the coding conventions are not respected. > i believe that's it's because the code have been written first in Java 6 an without refactoring was moved to use Java 7, 8, 9, 10 and 11. > > The I/O code still using java.io.File for some parts, no try-with-resources so most of the try/finally are not safe, > a lot of code like new BufferedWriter(new FileWriter(file)) instead of Files.newBufferedWriter, etc. The code should use the package java.nio.file, and not the old java.io, > most of the code try to manage the exception right were they appear instead of propagating them so there are too many try/catch, > a lot of catch are ignored which is a code smell, some codes use the internal logger (jdk.packager.internal.Log), but a lot of codes doesn't, > for the collection code, there is a lot of copy of data small structures (which suggest that published collections are not immutable), > there are dubious @SuppressWarnings("unchecked"), some or them are due to the fact that the code use Class as a type token instead of using lambdas, > Stream are not used when they should (to avoid multiple copy between data structures) and streams that need to be closed are not (the result of Files.list by example), > there are usual "don't do that in Java" like a loop using an integer index to traverse a List without knowing if it's a random access list or not, > there is a lot of nullchecks instead of using Optional, > a lot of code initialize local variables to null which is a code smell (and a side effect of having a lot of try/catch but not only), > constructors should not do work, just initialization, use static factory method instead (so you will not have to debug half constructed objects), > the code uses BigIntegers to parse a bundle version, just in case, > the code uses an AtomicReference as a box that a lambda can mutate, instead of wrapping the exception into a runtime and unwrapping it at call site, > The code of jdk.packager.internal.IOUtils should be updated to use methods of the JDK 11 and methods like readFully should be replaced by the JDK's one. > listOfPathToString and setOfStringToString are just WTF, like in getRedistributableModules(), where the variable stream is an Optional, > A class like Platform should be used everywhere to do platform specific stuff, a lot of code still use String matching (the version parsing should use System.Version). > All the argument parsing should be delegated to JOpt (the one integrated with the JDK), so it will be consistent with the other JDK tools, > There is a UnsupportedPlatformException but a Platform can be UNKOWN ?? > > I will stop here, my point is that there is a lot of cleaning that should appear before the code is integrated into the JDK and given the size of the code, i wonder if it's not better to start an OpenJDK project for it and iterate on the code before trying to include it in the JDK. For me, the code is too big to use the jdk/sandbox. > > regards, > R?mi > > ----- Mail original ----- >> De: "Kevin Rushforth" >> ?: "core-libs-dev" >> Cc: "Alexey Semenyuk" , "Andy Herrick" >> Envoy?: Vendredi 6 Juillet 2018 22:14:29 >> Objet: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] >> An initial prototype of the jpackager tool has been pushed to a new >> 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is >> interested in taking a look, you can clone it as follows: >> >> ??? hg clone http://hg.openjdk.java.net/jdk/sandbox >> ??? cd ./sandbox >> ??? hg update JDK-8200758-branch >> >> I plan to reply to the feedback already provided, and update the JEP [2] >> next week some time, but in the mean time if you have additional >> questions or comments, feel free to reply. >> >> -- Kevin >> >> [1] http://hg.openjdk.java.net/jdk/sandbox/shortlog/JDK-8200758-branch >> [2] https://bugs.openjdk.java.net/browse/JDK-8200758 >> >> >> On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >>> We're aiming to get this into JDK 12 early enough so that an EA build >>> would be available around the time JDK 11 ships. That will allow you >>> to take a jlinked image with JDK 11 and package it up using (the EA) >>> jpackager. >>> >>> We will create a development branch in the JDK sandbox [1] some time >>> in the next week or so so you can follow the development. >>> >>> Also, thank you to those who have provided feedback. I'll reply to >>> feedback soon and then incorporate it into an updated JEP. >>> >>> -- Kevin From forax at univ-mlv.fr Sat Jul 7 14:10:22 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Sat, 7 Jul 2018 16:10:22 +0200 (CEST) Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: <142986b8-e2d7-3744-469d-e0209da290e6@oracle.com> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <1973155195.1899675.1530914874386.JavaMail.zimbra@u-pem.fr> <142986b8-e2d7-3744-469d-e0209da290e6@oracle.com> Message-ID: <1883394348.1955480.1530972622847.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Kevin Rushforth" > ?: "Remi Forax" > Cc: "core-libs-dev" , "Alexey Semenyuk" , "Andy Herrick" > > Envoy?: Samedi 7 Juillet 2018 15:47:01 > Objet: Re: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] > Hi Remy, > > Thank you for taking a look. > > Yes, the javapackager code that forms the basis for the jpackager > prototype was initially developed on older JDKs and evolved from there. > I'm sure the improvements you suggest are all good ones, and it doesn't > seem like it would be too hard to address the most important of them, > especially the try-with-resources or anything else that would affect the > robustness of the tool. As long as we do address the robustness issues, > I think it is more important to get the feature set right, and make sure > that the public interfaces -- the command line options and ToolProvider > interface -- are clean. I don't see the need to rewrite the tool or take > an extra couple of months to modernize all of the implementation to use > JDK 11 APIs everywhere. > > Also, I don't agree that jpackager is too large for jdk/sandbox or that > it needs it own project. The jdk/sandbox is perfect for new modules / > new tools that don't impact other parts of the JDK. > > -- Kevin Hi Kevin, like you, i don't think that a full rewrite is necessary, as you said having the right public 'interfaces' is enough, but reducing the size the duplicated code (with the JDK and internally) is as important in my opinion. For the first point, it means that jpackager should use jopt for the argument parsing (to be fully compatible with the GNU style of options). For the second point, it means to change a lot of code that may break because it's less mechanical than introducing try-with-resources. regards, R?mi > > > On 7/6/2018 3:07 PM, Remi Forax wrote: >> I've just taking a look at the patch, >> i don't see how this can be integrated soon, the code is consistently >> inconsistent as one of my colleague would say, even the coding conventions are >> not respected. >> i believe that's it's because the code have been written first in Java 6 an >> without refactoring was moved to use Java 7, 8, 9, 10 and 11. >> >> The I/O code still using java.io.File for some parts, no try-with-resources so >> most of the try/finally are not safe, >> a lot of code like new BufferedWriter(new FileWriter(file)) instead of >> Files.newBufferedWriter, etc. The code should use the package java.nio.file, >> and not the old java.io, >> most of the code try to manage the exception right were they appear instead of >> propagating them so there are too many try/catch, >> a lot of catch are ignored which is a code smell, some codes use the internal >> logger (jdk.packager.internal.Log), but a lot of codes doesn't, >> for the collection code, there is a lot of copy of data small structures (which >> suggest that published collections are not immutable), >> there are dubious @SuppressWarnings("unchecked"), some or them are due to the >> fact that the code use Class as a type token instead of using lambdas, >> Stream are not used when they should (to avoid multiple copy between data >> structures) and streams that need to be closed are not (the result of >> Files.list by example), >> there are usual "don't do that in Java" like a loop using an integer index to >> traverse a List without knowing if it's a random access list or not, >> there is a lot of nullchecks instead of using Optional, >> a lot of code initialize local variables to null which is a code smell (and a >> side effect of having a lot of try/catch but not only), >> constructors should not do work, just initialization, use static factory method >> instead (so you will not have to debug half constructed objects), >> the code uses BigIntegers to parse a bundle version, just in case, >> the code uses an AtomicReference as a box that a lambda can mutate, instead of >> wrapping the exception into a runtime and unwrapping it at call site, >> The code of jdk.packager.internal.IOUtils should be updated to use methods of >> the JDK 11 and methods like readFully should be replaced by the JDK's one. >> listOfPathToString and setOfStringToString are just WTF, like in >> getRedistributableModules(), where the variable stream is an Optional, >> A class like Platform should be used everywhere to do platform specific stuff, a >> lot of code still use String matching (the version parsing should use >> System.Version). >> All the argument parsing should be delegated to JOpt (the one integrated with >> the JDK), so it will be consistent with the other JDK tools, >> There is a UnsupportedPlatformException but a Platform can be UNKOWN ?? >> >> I will stop here, my point is that there is a lot of cleaning that should appear >> before the code is integrated into the JDK and given the size of the code, i >> wonder if it's not better to start an OpenJDK project for it and iterate on the >> code before trying to include it in the JDK. For me, the code is too big to use >> the jdk/sandbox. >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "Kevin Rushforth" >>> ?: "core-libs-dev" >>> Cc: "Alexey Semenyuk" , "Andy Herrick" >>> >>> Envoy?: Vendredi 6 Juillet 2018 22:14:29 >>> Objet: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: >>> JDK-8200758: Packaging Tool] >>> An initial prototype of the jpackager tool has been pushed to a new >>> 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is >>> interested in taking a look, you can clone it as follows: >>> >>> ??? hg clone http://hg.openjdk.java.net/jdk/sandbox >>> ??? cd ./sandbox >>> ??? hg update JDK-8200758-branch >>> >>> I plan to reply to the feedback already provided, and update the JEP [2] >>> next week some time, but in the mean time if you have additional >>> questions or comments, feel free to reply. >>> >>> -- Kevin >>> >>> [1] http://hg.openjdk.java.net/jdk/sandbox/shortlog/JDK-8200758-branch >>> [2] https://bugs.openjdk.java.net/browse/JDK-8200758 >>> >>> >>> On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >>>> We're aiming to get this into JDK 12 early enough so that an EA build >>>> would be available around the time JDK 11 ships. That will allow you >>>> to take a jlinked image with JDK 11 and package it up using (the EA) >>>> jpackager. >>>> >>>> We will create a development branch in the JDK sandbox [1] some time >>>> in the next week or so so you can follow the development. >>>> >>>> Also, thank you to those who have provided feedback. I'll reply to >>>> feedback soon and then incorporate it into an updated JEP. >>>> > >>> -- Kevin From martinrb at google.com Sun Jul 8 00:12:08 2018 From: martinrb at google.com (Martin Buchholz) Date: Sat, 7 Jul 2018 17:12:08 -0700 Subject: RFR JDK-8206389: JarEntry.setCreation/LastAccessTime without setLastModifiedTime causes Invalid CEN header In-Reply-To: <5B403876.9030909@oracle.com> References: <5B3FFF41.200@oracle.com> <5B403876.9030909@oracle.com> Message-ID: OK, looks good! I would add something to check() to make sure that e.g. atime == null iff ze.getLastAccessTime() == null The zip time stuff is surprisingly messy. On Fri, Jul 6, 2018 at 8:50 PM, Xueming Shen wrote: > On 7/6/18, 5:43 PM, Martin Buchholz wrote: > > I would use different timestamps for the 4 possible times used in this > test, if only to make it clearer which value comes from where. > > > webrev has been updated accordingly. > > > + // ze.setLastModifiedTime(now);+ ze.setTime(now.toMillis()); > > > setTime only sets the DOS time? Which only has a granularity of 2 > seconds? If so, how do you get back the same value you put in if the > current second is odd? Or am I misunderstanding the test? > > > no misunderstanding, good catch. The test does fail when hits the odd > second. > added a special "check" version for the 2 second granularity set/getTime(). > > Thanks, > Sherman > > > From Alan.Bateman at oracle.com Sun Jul 8 07:20:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 8 Jul 2018 08:20:22 +0100 Subject: RFR JDK-8206389: JarEntry.setCreation/LastAccessTime without setLastModifiedTime causes Invalid CEN header In-Reply-To: <5B3FFF41.200@oracle.com> References: <5B3FFF41.200@oracle.com> Message-ID: <035c8d27-82e5-5125-0249-8204f58239f8@oracle.com> On 07/07/2018 00:46, Xueming Shen wrote: > Hi > > Please help review the changeset for JDK-8206389 > > issue: https://bugs.openjdk.java.net/browse/JDK-8206389 > webrev: http://cr.openjdk.java.net/~sherman/8206389/webrev > > Cause: ZipOutputStream.writeCEN().writeCEN() incorrectly calculate the > length > of bytes needed for the? "unix timestamps" when the "last modified > time" is > NOT set. The updated webrev looks okay to me, just a minor nit that "elem +=5" is missing a space. A suggestion to avoid repeating the comment is to bump elem by 5, then += 4 when there is a mtime. -Alan From Alan.Bateman at oracle.com Sun Jul 8 07:36:12 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 8 Jul 2018 08:36:12 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <036f88e423b743bbb68c73e2a59486d4@sap.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> Message-ID: On 06/07/2018 13:44, Baesken, Matthias wrote: > Hi Alan ,so it looks like JDK-8204233 added a switch (system property) to enable the enhanced socket IOException messages . > > That would be an option as well for 8205525 . Yes, it's documented in conf/security/java.security and something equivalent could be done here. The giveaway in your original patch is that it needed a privileged block to create the exception message. > > 8205525 adds the jar file name and the line number info to the exception message . > > In case that only the jar file name would be considered sensitive , I would prefer to just output the line number (and omit the system property ). > That should be okay (I can't think of any concerns). -Alan From Alan.Bateman at oracle.com Sun Jul 8 19:55:18 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 8 Jul 2018 20:55:18 +0100 Subject: Adding new IBM extended charsets In-Reply-To: References: <5B3E4AC8.3070307@oracle.com> Message-ID: <4cba7633-33b7-f29a-fb34-05a06b224057@oracle.com> On 06/07/2018 14:56, Nasser Ebrahim wrote: > : > I understood you preferred option is 3 [Remove all extended charsets from > JDK (keep only default charsets) and use the extended charsets from third > party like ICU4J]. Just to confirm, so you meant we need to keep only the > standard charsets in the JDK and remove all the extended charsets from JDK > and use them from ICU4J OR you meant apply that only for the new extended > charsets. I think it is better to keep the consistency - either take all > extended charsets from ICU4J or maintain all extended charsets with JDK. > Keeping some extended charsets within JDK and use ICU4J for other extended > charsets may confuse the Java user. I think the suggestion in Sherman's mail is to drop the 70 or so IBM charsets from jdk.charsets. This will reduce the size of jdk.charsets and eliminate the need to maintain these charsets (at least on non-AIX builds). If developers need these charsets, say when connecting to database on an IBM system, then they can deploy the ICU4J provider on the class path or module path. I don't think the suggestion impacts the 11 IBM charsets in java.base on non-AIX builds or the non-IBM charsets in jdk.charsets. They may be opportunities to drop some of these but that can be looked at separately. Also I don't think the suggestion impacts the additional 12 IBM charsets that are included in the AIX build of java.base at this time. From the review threads, it seems there are supported locales on AIX that map to these charsets so this is why they are in java.base. -Alan. From david.holmes at oracle.com Mon Jul 9 01:37:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 9 Jul 2018 11:37:12 +1000 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> Message-ID: <6bccaebc-9a3d-ef01-a8bf-58613fcfdd9c@oracle.com> Hi Peter, On 7/07/2018 2:10 AM, Peter Levart wrote: > Hi, > > On 07/05/2018 01:01 AM, David Holmes wrote: >> I dispute "they will understand this might have happened in another >> thread". > > What if the stack trace was like the following... Yes your suggestion makes it much clearer. But ... my whole objection here is doing all this extraneous execution of Java code in response to the initial exception. The more Java code we execute the more likely we will hit secondary exceptions and the greater the possibility of unintended interactions that might lead back to the class that can't be initialized. I just don't think this level of effort is warranted. Cheers, David ----- > Before patch: > > 1st attempt [ForkJoinPool.commonPool-worker-3]: > > java.lang.ExceptionInInitializerError > ??????? at ClinitFailure.lambda$main$0(ClinitFailure.java:20) > ??????? at > java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) > > ??????? at > java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) > > ??????? at > java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) > ??????? at > java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) > > ??????? at > java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) > ??????? at > java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) > > ??????? at > java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) > > Caused by: java.lang.RuntimeException: Can't get it! > ??????? at ClinitFailure$Faulty.(ClinitFailure.java:12) > ??????? ... 8 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of > bounds for length 0 > ??????? at ClinitFailure$Faulty.(ClinitFailure.java:10) > ??????? ... 8 more > > 2nd attempt [ForkJoinPool.commonPool-worker-5]: > > java.lang.NoClassDefFoundError: Could not initialize class > ClinitFailure$Faulty > ??????? at ClinitFailure.lambda$main$1(ClinitFailure.java:28) > ??????? at > java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) > > ??????? at > java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) > > ??????? at > java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) > ??????? at > java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) > > ??????? at > java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) > ??????? at > java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) > > ??????? at > java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) > > > > After patch: > > 1st attempt [ForkJoinPool.commonPool-worker-3]: > > java.lang.ExceptionInInitializerError > ??????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) > ??????? at > java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) > > ??????? at > java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) > > ??????? at > java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) > ??????? at > java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) > > ??????? at > java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) > ??????? at > java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) > > ??????? at > java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) > > Caused by: java.lang.RuntimeException: Can't get it! > ??????? at ClinitFailure$Faulty.(ClinitFailure.java:10) > ??????? ... 8 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of > bounds for length 0 > ??????? at ClinitFailure$Faulty.(ClinitFailure.java:8) > ??????? ... 8 more > > 2nd attempt [ForkJoinPool.commonPool-worker-5]: > > java.lang.NoClassDefFoundError: Could not initialize class > ClinitFailure$Faulty > ??????? at > java.base/java.lang.ClassLoader.throwReinitException(ClassLoader.java:3062) > ??????? at ClinitFailure.lambda$main$1(ClinitFailure.java:25) > ??????? at > java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) > > ??????? at > java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) > > ??????? at > java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) > ??????? at > java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) > > ??????? at > java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) > ??????? at > java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) > > ??????? at > java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) > > Caused by: java.lang.ExceptionInInitializerError: 11 ms ago in thread > ForkJoinPool.commonPool-worker-3 > ??????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) > ??????? at > java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) > > ??????? at > java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) > > ??????? ... 5 more > Caused by: java.lang.RuntimeException: Can't get it! > ??????? at ClinitFailure$Faulty.(ClinitFailure.java:10) > ??????? ... 8 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of > bounds for length 0 > ??????? at ClinitFailure$Faulty.(ClinitFailure.java:8) > ??????? ... 8 more > > > > This is what gets printed by the sample program: > > public class ClinitFailure { > > ??? static class Faulty { > ??????? static { > ??????????? try { > ??????????????? int i = (new int[0])[1]; > ??????????? } catch (Exception e) { > ??????????????? throw new RuntimeException("Can't get it!", e); > ??????????? } > ??????? } > ??? } > > ??? public static void main(String[] args) throws Exception { > ??????? CompletableFuture.runAsync(() -> { > ??????????? try { > ??????????????? new Faulty(); > ??????????? } catch (Throwable e) { > ??????????????? System.out.printf("\n1st attempt [%s]:\n\n", > Thread.currentThread().getName()); > ??????????????? e.printStackTrace(System.out); > ??????????? } > ??????? }).thenRunAsync(() -> { > ??????????? try { > ??????????????? new Faulty(); > ??????????? } catch (Throwable e) { > ??????????????? System.out.printf("\n2nd attempt [%s]:\n\n", > Thread.currentThread().getName()); > ??????????????? e.printStackTrace(System.out); > ??????????? } > ??????? }).join(); > ??? } > } > > > When the following patch is applied: > > http://cr.openjdk.java.net/~plevart/jdk-dev/8203826_NoClassDefFoundError.cause/webrev.01/ > > > > I took Volker's patch and modified it a bit: > > - The logic to construct and throw NoClassDefFoundError and to record > initial exception is in java now. It uses ClassLoaderValue > internal API to save the chains of exception(s) for faulty classes. It > is easier to do such logic in Java and less error prone. > - The chain of original exception(s) is replaced with > substitutes that mimic .toString() and .printStackTrace() methods of > original chain, but don't reference any classes outside bootstrap class > loader > - The replacement chain of original exceptions adds a custom message > insert into the top exception as a hint to the user: > > ??? ??? java.lang.ExceptionInInitializerError: 11 ms ago in thread > ForkJoinPool.commonPool-worker-3 > > > So, what do you think of this one? > > Regards, Peter > > From weijun.wang at oracle.com Mon Jul 9 01:53:52 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 9 Jul 2018 09:53:52 +0800 Subject: RFR[jdk8u-dev]: 8134124: sun/security/tools/jarsigner/warnings.sh fails when using Hindi locale In-Reply-To: References: Message-ID: Changes looks fine. One day we can modify it to a Java program using SecurityTools::jarsigner. Thanks Max > On Jul 6, 2018, at 5:34 PM, Ramanand Patil wrote: > > Hi all, > Please review this trivial test fix: > Bug: https://bugs.openjdk.java.net/browse/JDK-8134124 > Webrev: http://cr.openjdk.java.net/~rpatil/8134124/webrev.00/ > > The test is made locale independent now. > > Regards, > Ramanand. From peter.levart at gmail.com Mon Jul 9 07:22:30 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 9 Jul 2018 09:22:30 +0200 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: <6bccaebc-9a3d-ef01-a8bf-58613fcfdd9c@oracle.com> References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> <6bccaebc-9a3d-ef01-a8bf-58613fcfdd9c@oracle.com> Message-ID: Hi David, On 07/09/2018 03:37 AM, David Holmes wrote: > Hi Peter, > > On 7/07/2018 2:10 AM, Peter Levart wrote: >> Hi, >> >> On 07/05/2018 01:01 AM, David Holmes wrote: >>> I dispute "they will understand this might have happened in another >>> thread". >> >> What if the stack trace was like the following... > > Yes your suggestion makes it much clearer. > > But ... my whole objection here is doing all this extraneous execution > of Java code in response to the initial exception. The more Java code > we execute the more likely we will hit secondary exceptions and the > greater the possibility of unintended interactions that might lead > back to the class that can't be initialized. I just don't think this > level of effort is warranted. I I agree that more classes are involved, but they are all JDK classes and their number is constant. Meaning, if they are OK and don't fail when initializing, there's no danger of unintended interactions that would be caused by initialization errors in other classes. And even if those additional needed classes had problems in their initialization, I think that the consequences would be under control. Let's see what additional classes are needed when the presented patch is used as opposed to classes needed in current logic: In step 7, when super class/interface initialization fails and in steps 10/11 when the class initialization fails, we record the error thrown (record_init_exception). In addition to previously needed classes we also need: - ClassLoader, - ClassLoader$InitExceptionSubst (with dependencies: RuntimeException, Exception), - ClassLoaderValue (with dependencies: AbstractClassLoaderValue, AbstractClassLoaderValue$Sub, AbstractClassLoaderValue$Memoizer, ConcurrentHashMap + deps) When we throw NoClassDefFoundError, we don't need any other additional classes that wouldn't already be needed originally. So I can see that when above additional classes had problems initializing themselves, there would be errors thrown from their usage when recording initial initialization exception of some unrelated class, but such errors would be ignored (step 7): ?979???????? // Record the exception thrown from super class/interface initialization so that ?980???????? // it can be chained into potential later NoClassDefFoundErrors. ?981 class_loader_data()->record_init_exception(java_mirror_handle(), e, THREAD); ?982???????? // Locks object, set state, and notify all waiting threads ?983 set_initialization_state_and_notify(initialization_error, THREAD); ?984 *CLEAR_PENDING_EXCEPTION*; steps 10/11: 1037?????? // Record the exception that originally caused to fail so 1038?????? // it can be chained into potential later NoClassDefFoundErrors. 1039 class_loader_data()->record_init_exception(java_mirror_handle(), e, THREAD); 1040?????? // Locks object, set state, and notify all waiting threads 1041?????? set_initialization_state_and_notify(initialization_error, THREAD); 1042 *CLEAR_PENDING_EXCEPTION*; It might be that those ignored exceptions would cause later use of those additional classes to throw NoClassDefFoundError instead of ExceptionInInitializerError (depending on whether it was an initial attempt to initialize those additional classes or not), but I can't see any other undesirable consequence. Do you? I'll try to provoke initialization errors in those additional classes to see what happens. Will get back when I have results of the experiment... Regards, Peter P.S. Executing java code as part of VM logic plays well in Jigsaw for example. If there is an acceptable fallback in case of java logic failure, everything seems to be OK. > Cheers, > David > ----- > >> Before patch: >> >> 1st attempt [ForkJoinPool.commonPool-worker-3]: >> >> java.lang.ExceptionInInitializerError >> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:20) >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >> >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >> >> Caused by: java.lang.RuntimeException: Can't get it! >> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:12) >> ???????? ... 8 more >> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >> bounds for length 0 >> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >> ???????? ... 8 more >> >> 2nd attempt [ForkJoinPool.commonPool-worker-5]: >> >> java.lang.NoClassDefFoundError: Could not initialize class >> ClinitFailure$Faulty >> ???????? at ClinitFailure.lambda$main$1(ClinitFailure.java:28) >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) >> >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >> >> >> >> After patch: >> >> 1st attempt [ForkJoinPool.commonPool-worker-3]: >> >> java.lang.ExceptionInInitializerError >> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >> >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >> >> Caused by: java.lang.RuntimeException: Can't get it! >> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >> ???????? ... 8 more >> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >> bounds for length 0 >> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:8) >> ???????? ... 8 more >> >> 2nd attempt [ForkJoinPool.commonPool-worker-5]: >> >> java.lang.NoClassDefFoundError: Could not initialize class >> ClinitFailure$Faulty >> ???????? at >> java.base/java.lang.ClassLoader.throwReinitException(ClassLoader.java:3062) >> ???????? at ClinitFailure.lambda$main$1(ClinitFailure.java:25) >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) >> >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >> ???????? at >> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >> >> ???????? at >> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >> >> Caused by: java.lang.ExceptionInInitializerError: 11 ms ago in thread >> ForkJoinPool.commonPool-worker-3 >> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >> >> ???????? at >> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >> >> ???????? ... 5 more >> Caused by: java.lang.RuntimeException: Can't get it! >> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >> ???????? ... 8 more >> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >> bounds for length 0 >> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:8) >> ???????? ... 8 more >> >> >> >> This is what gets printed by the sample program: >> >> public class ClinitFailure { >> >> ???? static class Faulty { >> ???????? static { >> ???????????? try { >> ???????????????? int i = (new int[0])[1]; >> ???????????? } catch (Exception e) { >> ???????????????? throw new RuntimeException("Can't get it!", e); >> ???????????? } >> ???????? } >> ???? } >> >> ???? public static void main(String[] args) throws Exception { >> ???????? CompletableFuture.runAsync(() -> { >> ???????????? try { >> ???????????????? new Faulty(); >> ???????????? } catch (Throwable e) { >> ???????????????? System.out.printf("\n1st attempt [%s]:\n\n", >> Thread.currentThread().getName()); >> ???????????????? e.printStackTrace(System.out); >> ???????????? } >> ???????? }).thenRunAsync(() -> { >> ???????????? try { >> ???????????????? new Faulty(); >> ???????????? } catch (Throwable e) { >> ???????????????? System.out.printf("\n2nd attempt [%s]:\n\n", >> Thread.currentThread().getName()); >> ???????????????? e.printStackTrace(System.out); >> ???????????? } >> ???????? }).join(); >> ???? } >> } >> >> >> When the following patch is applied: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/8203826_NoClassDefFoundError.cause/webrev.01/ >> >> >> >> I took Volker's patch and modified it a bit: >> >> - The logic to construct and throw NoClassDefFoundError and to record >> initial exception is in java now. It uses ClassLoaderValue >> internal API to save the chains of exception(s) for faulty classes. >> It is easier to do such logic in Java and less error prone. >> - The chain of original exception(s) is replaced with >> substitutes that mimic .toString() and .printStackTrace() methods of >> original chain, but don't reference any classes outside bootstrap >> class loader >> - The replacement chain of original exceptions adds a custom message >> insert into the top exception as a hint to the user: >> >> ???? ??? java.lang.ExceptionInInitializerError: 11 ms ago in thread >> ForkJoinPool.commonPool-worker-3 >> >> >> So, what do you think of this one? >> >> Regards, Peter >> >> From david.holmes at oracle.com Mon Jul 9 07:33:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 9 Jul 2018 17:33:07 +1000 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> <6bccaebc-9a3d-ef01-a8bf-58613fcfdd9c@oracle.com> Message-ID: On 9/07/2018 5:22 PM, Peter Levart wrote: > Hi David, > > On 07/09/2018 03:37 AM, David Holmes wrote: >> Hi Peter, >> >> On 7/07/2018 2:10 AM, Peter Levart wrote: >>> Hi, >>> >>> On 07/05/2018 01:01 AM, David Holmes wrote: >>>> I dispute "they will understand this might have happened in another >>>> thread". >>> >>> What if the stack trace was like the following... >> >> Yes your suggestion makes it much clearer. >> >> But ... my whole objection here is doing all this extraneous execution >> of Java code in response to the initial exception. The more Java code >> we execute the more likely we will hit secondary exceptions and the >> greater the possibility of unintended interactions that might lead >> back to the class that can't be initialized. I just don't think this >> level of effort is warranted. I > > I agree that more classes are involved, but they are all JDK classes and > their number is constant. Meaning, if they are OK and don't fail when > initializing, there's no danger of unintended interactions that would be > caused by initialization errors in other classes. And even if those > additional needed classes had problems in their initialization, I think > that the consequences would be under control. That's not what I mean. I'm not concerned about circular initialization failures due to failing to initialize the classes used in this "hook". I'm concerned about the overall amount of Java code execution that this involves, which may trigger other exceptions (e.g. OOME) and which may incur additional logging or event generation that may in turn interact in some way with the original class being initialized. I just think this is complete overkill for addressing the perceived problem. David ----- > > Let's see what additional classes are needed when the presented patch is > used as opposed to classes needed in current logic: > > In step 7, when super class/interface initialization fails and in steps > 10/11 when the class initialization fails, we record the error thrown > (record_init_exception). In addition to previously needed classes we > also need: > - ClassLoader, > - ClassLoader$InitExceptionSubst (with dependencies: RuntimeException, > Exception), > - ClassLoaderValue (with dependencies: AbstractClassLoaderValue, > AbstractClassLoaderValue$Sub, AbstractClassLoaderValue$Memoizer, > ConcurrentHashMap + deps) > > When we throw NoClassDefFoundError, we don't need any other additional > classes that wouldn't already be needed originally. > > So I can see that when above additional classes had problems > initializing themselves, there would be errors thrown from their usage > when recording initial initialization exception of some unrelated class, > but such errors would be ignored (step 7): > > ?979???????? // Record the exception thrown from super class/interface > initialization so that > ?980???????? // it can be chained into potential later > NoClassDefFoundErrors. > ?981 class_loader_data()->record_init_exception(java_mirror_handle(), > e, THREAD); > ?982???????? // Locks object, set state, and notify all waiting threads > ?983 set_initialization_state_and_notify(initialization_error, THREAD); > ?984 *CLEAR_PENDING_EXCEPTION*; > > steps 10/11: > > 1037?????? // Record the exception that originally caused to > fail so > 1038?????? // it can be chained into potential later NoClassDefFoundErrors. > 1039 class_loader_data()->record_init_exception(java_mirror_handle(), e, > THREAD); > 1040?????? // Locks object, set state, and notify all waiting threads > 1041?????? set_initialization_state_and_notify(initialization_error, > THREAD); > 1042 *CLEAR_PENDING_EXCEPTION*; > > > It might be that those ignored exceptions would cause later use of those > additional classes to throw NoClassDefFoundError instead of > ExceptionInInitializerError (depending on whether it was an initial > attempt to initialize those additional classes or not), but I can't see > any other undesirable consequence. Do you? > > I'll try to provoke initialization errors in those additional classes to > see what happens. Will get back when I have results of the experiment... > > Regards, Peter > > > P.S. > > Executing java code as part of VM logic plays well in Jigsaw for > example. If there is an acceptable fallback in case of java logic > failure, everything seems to be OK. > >> Cheers, >> David >> ----- >> >>> Before patch: >>> >>> 1st attempt [ForkJoinPool.commonPool-worker-3]: >>> >>> java.lang.ExceptionInInitializerError >>> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:20) >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >>> >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>> >>> Caused by: java.lang.RuntimeException: Can't get it! >>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:12) >>> ???????? ... 8 more >>> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >>> bounds for length 0 >>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >>> ???????? ... 8 more >>> >>> 2nd attempt [ForkJoinPool.commonPool-worker-5]: >>> >>> java.lang.NoClassDefFoundError: Could not initialize class >>> ClinitFailure$Faulty >>> ???????? at ClinitFailure.lambda$main$1(ClinitFailure.java:28) >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) >>> >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>> >>> >>> >>> After patch: >>> >>> 1st attempt [ForkJoinPool.commonPool-worker-3]: >>> >>> java.lang.ExceptionInInitializerError >>> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >>> >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>> >>> Caused by: java.lang.RuntimeException: Can't get it! >>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >>> ???????? ... 8 more >>> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >>> bounds for length 0 >>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:8) >>> ???????? ... 8 more >>> >>> 2nd attempt [ForkJoinPool.commonPool-worker-5]: >>> >>> java.lang.NoClassDefFoundError: Could not initialize class >>> ClinitFailure$Faulty >>> ???????? at >>> java.base/java.lang.ClassLoader.throwReinitException(ClassLoader.java:3062) >>> ???????? at ClinitFailure.lambda$main$1(ClinitFailure.java:25) >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) >>> >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>> >>> ???????? at >>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>> >>> Caused by: java.lang.ExceptionInInitializerError: 11 ms ago in thread >>> ForkJoinPool.commonPool-worker-3 >>> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >>> >>> ???????? at >>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >>> >>> ???????? ... 5 more >>> Caused by: java.lang.RuntimeException: Can't get it! >>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >>> ???????? ... 8 more >>> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >>> bounds for length 0 >>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:8) >>> ???????? ... 8 more >>> >>> >>> >>> This is what gets printed by the sample program: >>> >>> public class ClinitFailure { >>> >>> ???? static class Faulty { >>> ???????? static { >>> ???????????? try { >>> ???????????????? int i = (new int[0])[1]; >>> ???????????? } catch (Exception e) { >>> ???????????????? throw new RuntimeException("Can't get it!", e); >>> ???????????? } >>> ???????? } >>> ???? } >>> >>> ???? public static void main(String[] args) throws Exception { >>> ???????? CompletableFuture.runAsync(() -> { >>> ???????????? try { >>> ???????????????? new Faulty(); >>> ???????????? } catch (Throwable e) { >>> ???????????????? System.out.printf("\n1st attempt [%s]:\n\n", >>> Thread.currentThread().getName()); >>> ???????????????? e.printStackTrace(System.out); >>> ???????????? } >>> ???????? }).thenRunAsync(() -> { >>> ???????????? try { >>> ???????????????? new Faulty(); >>> ???????????? } catch (Throwable e) { >>> ???????????????? System.out.printf("\n2nd attempt [%s]:\n\n", >>> Thread.currentThread().getName()); >>> ???????????????? e.printStackTrace(System.out); >>> ???????????? } >>> ???????? }).join(); >>> ???? } >>> } >>> >>> >>> When the following patch is applied: >>> >>> http://cr.openjdk.java.net/~plevart/jdk-dev/8203826_NoClassDefFoundError.cause/webrev.01/ >>> >>> >>> >>> I took Volker's patch and modified it a bit: >>> >>> - The logic to construct and throw NoClassDefFoundError and to record >>> initial exception is in java now. It uses ClassLoaderValue >>> internal API to save the chains of exception(s) for faulty classes. >>> It is easier to do such logic in Java and less error prone. >>> - The chain of original exception(s) is replaced with >>> substitutes that mimic .toString() and .printStackTrace() methods of >>> original chain, but don't reference any classes outside bootstrap >>> class loader >>> - The replacement chain of original exceptions adds a custom message >>> insert into the top exception as a hint to the user: >>> >>> ???? ??? java.lang.ExceptionInInitializerError: 11 ms ago in thread >>> ForkJoinPool.commonPool-worker-3 >>> >>> >>> So, what do you think of this one? >>> >>> Regards, Peter >>> >>> > From peter.levart at gmail.com Mon Jul 9 07:51:41 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 9 Jul 2018 09:51:41 +0200 Subject: RFR(M): 8203826: Chain class initialization exceptions into later NoClassDefFoundErrors In-Reply-To: References: <06ed3db2-e98c-014b-564a-6080dec06837@oracle.com> <75e66ebc9ebe475d8c8fbcdba4722138@sap.com> <6bccaebc-9a3d-ef01-a8bf-58613fcfdd9c@oracle.com> Message-ID: <55f3a8df-3a9e-84e1-f729-ec5a0d119360@gmail.com> Hi David, On 07/09/2018 09:33 AM, David Holmes wrote: > On 9/07/2018 5:22 PM, Peter Levart wrote: >> Hi David, >> >> On 07/09/2018 03:37 AM, David Holmes wrote: >>> Hi Peter, >>> >>> On 7/07/2018 2:10 AM, Peter Levart wrote: >>>> Hi, >>>> >>>> On 07/05/2018 01:01 AM, David Holmes wrote: >>>>> I dispute "they will understand this might have happened in >>>>> another thread". >>>> >>>> What if the stack trace was like the following... >>> >>> Yes your suggestion makes it much clearer. >>> >>> But ... my whole objection here is doing all this extraneous >>> execution of Java code in response to the initial exception. The >>> more Java code we execute the more likely we will hit secondary >>> exceptions and the greater the possibility of unintended >>> interactions that might lead back to the class that can't be >>> initialized. I just don't think this level of effort is warranted. I >> >> I agree that more classes are involved, but they are all JDK classes >> and their number is constant. Meaning, if they are OK and don't fail >> when initializing, there's no danger of unintended interactions that >> would be caused by initialization errors in other classes. And even >> if those additional needed classes had problems in their >> initialization, I think that the consequences would be under control. > > That's not what I mean. I'm not concerned about circular > initialization failures due to failing to initialize the classes used > in this "hook". I'm concerned about the overall amount of Java code > execution that this involves, which may trigger other exceptions (e.g. > OOME) and which may incur additional logging or event generation that > may in turn interact in some way with the original class being > initialized. > I still can't see what you see. If java code that records initial initialization error throws OOME, it will be ignored and initial exception will not be recorded. Later NoClassDefFoundError would not contain the nice chain of causes, but that's the only undesirable consequence. The code doesn't log OOME, it simply ignores it. If those additional exceptions cause any events on the VM level and those events interact in some way with the original class being initialized, this interaction will fail, but such interaction with original class would fail even if it was caused by anything else, because it is the original class that had problems initializing itself in the 1st place. > I just think this is complete overkill for addressing the perceived > problem. Perhaps. But it would be nice to have. Regards, Peter > > David > ----- > >> >> Let's see what additional classes are needed when the presented patch >> is used as opposed to classes needed in current logic: >> >> In step 7, when super class/interface initialization fails and in >> steps 10/11 when the class initialization fails, we record the error >> thrown (record_init_exception). In addition to previously needed >> classes we also need: >> - ClassLoader, >> - ClassLoader$InitExceptionSubst (with dependencies: >> RuntimeException, Exception), >> - ClassLoaderValue (with dependencies: AbstractClassLoaderValue, >> AbstractClassLoaderValue$Sub, AbstractClassLoaderValue$Memoizer, >> ConcurrentHashMap + deps) >> >> When we throw NoClassDefFoundError, we don't need any other >> additional classes that wouldn't already be needed originally. >> >> So I can see that when above additional classes had problems >> initializing themselves, there would be errors thrown from their >> usage when recording initial initialization exception of some >> unrelated class, but such errors would be ignored (step 7): >> >> ??979???????? // Record the exception thrown from super >> class/interface initialization so that >> ??980???????? // it can be chained into potential later >> NoClassDefFoundErrors. >> ??981 >> class_loader_data()->record_init_exception(java_mirror_handle(), e, >> THREAD); >> ??982???????? // Locks object, set state, and notify all waiting threads >> ??983 set_initialization_state_and_notify(initialization_error, THREAD); >> ??984 *CLEAR_PENDING_EXCEPTION*; >> >> steps 10/11: >> >> 1037?????? // Record the exception that originally caused to >> fail so >> 1038?????? // it can be chained into potential later >> NoClassDefFoundErrors. >> 1039 class_loader_data()->record_init_exception(java_mirror_handle(), >> e, THREAD); >> 1040?????? // Locks object, set state, and notify all waiting threads >> 1041 set_initialization_state_and_notify(initialization_error, THREAD); >> 1042 *CLEAR_PENDING_EXCEPTION*; >> >> >> It might be that those ignored exceptions would cause later use of >> those additional classes to throw NoClassDefFoundError instead of >> ExceptionInInitializerError (depending on whether it was an initial >> attempt to initialize those additional classes or not), but I can't >> see any other undesirable consequence. Do you? >> >> I'll try to provoke initialization errors in those additional classes >> to see what happens. Will get back when I have results of the >> experiment... >> >> Regards, Peter >> >> >> P.S. >> >> Executing java code as part of VM logic plays well in Jigsaw for >> example. If there is an acceptable fallback in case of java logic >> failure, everything seems to be OK. >> >>> Cheers, >>> David >>> ----- >>> >>>> Before patch: >>>> >>>> 1st attempt [ForkJoinPool.commonPool-worker-3]: >>>> >>>> java.lang.ExceptionInInitializerError >>>> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:20) >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>>> >>>> Caused by: java.lang.RuntimeException: Can't get it! >>>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:12) >>>> ???????? ... 8 more >>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >>>> bounds for length 0 >>>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >>>> ???????? ... 8 more >>>> >>>> 2nd attempt [ForkJoinPool.commonPool-worker-5]: >>>> >>>> java.lang.NoClassDefFoundError: Could not initialize class >>>> ClinitFailure$Faulty >>>> ???????? at ClinitFailure.lambda$main$1(ClinitFailure.java:28) >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>>> >>>> >>>> >>>> After patch: >>>> >>>> 1st attempt [ForkJoinPool.commonPool-worker-3]: >>>> >>>> java.lang.ExceptionInInitializerError >>>> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>>> >>>> Caused by: java.lang.RuntimeException: Can't get it! >>>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >>>> ???????? ... 8 more >>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >>>> bounds for length 0 >>>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:8) >>>> ???????? ... 8 more >>>> >>>> 2nd attempt [ForkJoinPool.commonPool-worker-5]: >>>> >>>> java.lang.NoClassDefFoundError: Could not initialize class >>>> ClinitFailure$Faulty >>>> ???????? at >>>> java.base/java.lang.ClassLoader.throwReinitException(ClassLoader.java:3062) >>>> ???????? at ClinitFailure.lambda$main$1(ClinitFailure.java:25) >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) >>>> >>>> Caused by: java.lang.ExceptionInInitializerError: 11 ms ago in >>>> thread ForkJoinPool.commonPool-worker-3 >>>> ???????? at ClinitFailure.lambda$main$0(ClinitFailure.java:18) >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) >>>> >>>> ???????? at >>>> java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1728) >>>> >>>> ???????? ... 5 more >>>> Caused by: java.lang.RuntimeException: Can't get it! >>>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:10) >>>> ???????? ... 8 more >>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of >>>> bounds for length 0 >>>> ???????? at ClinitFailure$Faulty.(ClinitFailure.java:8) >>>> ???????? ... 8 more >>>> >>>> >>>> >>>> This is what gets printed by the sample program: >>>> >>>> public class ClinitFailure { >>>> >>>> ???? static class Faulty { >>>> ???????? static { >>>> ???????????? try { >>>> ???????????????? int i = (new int[0])[1]; >>>> ???????????? } catch (Exception e) { >>>> ???????????????? throw new RuntimeException("Can't get it!", e); >>>> ???????????? } >>>> ???????? } >>>> ???? } >>>> >>>> ???? public static void main(String[] args) throws Exception { >>>> ???????? CompletableFuture.runAsync(() -> { >>>> ???????????? try { >>>> ???????????????? new Faulty(); >>>> ???????????? } catch (Throwable e) { >>>> ???????????????? System.out.printf("\n1st attempt [%s]:\n\n", >>>> Thread.currentThread().getName()); >>>> ???????????????? e.printStackTrace(System.out); >>>> ???????????? } >>>> ???????? }).thenRunAsync(() -> { >>>> ???????????? try { >>>> ???????????????? new Faulty(); >>>> ???????????? } catch (Throwable e) { >>>> ???????????????? System.out.printf("\n2nd attempt [%s]:\n\n", >>>> Thread.currentThread().getName()); >>>> ???????????????? e.printStackTrace(System.out); >>>> ???????????? } >>>> ???????? }).join(); >>>> ???? } >>>> } >>>> >>>> >>>> When the following patch is applied: >>>> >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/8203826_NoClassDefFoundError.cause/webrev.01/ >>>> >>>> >>>> >>>> I took Volker's patch and modified it a bit: >>>> >>>> - The logic to construct and throw NoClassDefFoundError and to >>>> record initial exception is in java now. It uses >>>> ClassLoaderValue internal API to save the chains of exception(s) >>>> for faulty classes. It is easier to do such logic in Java and less >>>> error prone. >>>> - The chain of original exception(s) is replaced with >>>> substitutes that mimic .toString() and .printStackTrace() methods >>>> of original chain, but don't reference any classes outside >>>> bootstrap class loader >>>> - The replacement chain of original exceptions adds a custom >>>> message insert into the top exception as a hint to the user: >>>> >>>> ???? ??? java.lang.ExceptionInInitializerError: 11 ms ago in thread >>>> ForkJoinPool.commonPool-worker-3 >>>> >>>> >>>> So, what do you think of this one? >>>> >>>> Regards, Peter >>>> >>>> >> From srinivas.dama at oracle.com Mon Jul 9 10:38:21 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 9 Jul 2018 03:38:21 -0700 (PDT) Subject: [11] RFR: 8206445: JImageListTest.java failed in Windows Message-ID: <2adf9614-a915-49e6-b406-a90ce1795fc2@default> Hi, Please review webrev: http://cr.openjdk.java.net/~sdama/8206445/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8206445 Regards, Srinivas From Alan.Bateman at oracle.com Mon Jul 9 10:54:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 9 Jul 2018 11:54:11 +0100 Subject: [11] RFR: 8206445: JImageListTest.java failed in Windows In-Reply-To: <2adf9614-a915-49e6-b406-a90ce1795fc2@default> References: <2adf9614-a915-49e6-b406-a90ce1795fc2@default> Message-ID: <7f14e796-76a3-7761-a07f-c7ff11bf6890@oracle.com> On 09/07/2018 11:38, Srinivas Dama wrote: > Hi, > > Please review > webrev: http://cr.openjdk.java.net/~sdama/8206445/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8206445 > testListEmptyFile looks okay but you can reduce the code if if you use Path.of("."). testListNotAnImage can use Path.of(".") too. Also it can use Files.writeString which will avoid the test trying to list the contents of the image while the file is open. Will you make sure to test this on all platforms before pushing? These tests have been on and off the exclude list several times in the last week and it would be good to check that this one is stable before it runs again. -Alan From sean.coffey at oracle.com Mon Jul 9 11:13:15 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 9 Jul 2018 12:13:15 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <4F777EA1-6402-4725-B4E8-6DCB61DD4E19@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> <5B350AE5.3080401@oracle.com> <4F777EA1-6402-4725-B4E8-6DCB61DD4E19@oracle.com> Message-ID: As per request from Erik, I separated the tests out into individual ones to test the JFR and Logger functionality. I introduced a new separate test for the CertificateChainEvent event also. Originally this was wrapped into the TLSHandshakeEvent test. Thanks to Erik for extra refactoring and modifications carried out to clean up the code up further. http://cr.openjdk.java.net/~coffeys/webrev.8148188.v5/webrev/ This enhancement has a dependency on JDK-8203629 Regards, Sean. On 02/07/18 09:49, Erik Gahlin wrote: > >> On 29 Jun 2018, at 17:34, Se?n Coffey wrote: >> >> I've introduced a new test helper class in the jdk/test/lib/jfr directory to help with the dual test nature of the new tests. It's helped alot with test code duplication. >> > My thinking was to put things like the certificates in a separate file, i.e TestCertificates, and then have a logging test and a JFR test reuse it. > > One rationale for adding logging was to use it if JFR is not present. By putting the tests together, it becomes impossible to compile and test logging without having JFR. > >> Looked at use of @DataAmount(DataAmount.BITS) also. Not sure if it's fits. The output is displayed in units like "KiB" - not the normal when examining key lengths used in X509Certificates. i.e a 2048 bit key gets displayed as "2 KiB" - I'd prefer to keep the 2048 display version. > We should not let the JMC GUI decide how units are specified. There will be other GUIs and this is the first event that uses bits, so I don?t think it is formatted that way because it was considered superior. > > Erik > >> new webrev at: http://cr.openjdk.java.net/~coffeys/webrev.8148188.v4/webrev/ >> >> Regards, >> Sean. >> >> On 28/06/18 17:59, Se?n Coffey wrote: >>> Comments inline. >>> >>> >>> On 28/06/2018 17:20, Erik Gahlin wrote: >>>> It's sufficient if an event object escapes to another method (regardless if JFR is enabled or not). >>>> >>>> Some more feedback: >>>> >>>> Rename event jdk.CertChain to jdk.CertificateChain >>>> Rename event jdk.X509Cert to jdk.X509Certificate >>>> Rename field certChain to certificateChain. >>>> Rename field serialNum to serialNumber >>> all above done. >>>> Rename field algId to AlgorithmicId or AlgorithmicID >>> maybe "algorithm" works here also ? >>>> Rename @Label("Ciphersuite") to @Label("Cipher Suite") >>>> Rename @Label("Cert Chain") to @Label("Certificate Chain") >>>> Rename @Label("Property Name") to "Name" or "Key" if that makes sense in the context? >>>> Rename @Label("Property Value") to "Value". >>>> Put events in a subcategory, i.e @Category({"Java Development Kit", "Security"}) >>> done. >>>> Make classes final. >>> done. I had thought that the JFR mechanism required non-final classes. >>>> What is the unit of the key in X509Certificate event? Bits? If that is the case, use @DataAmount(DataAmount.BITS) >>> Yes - it's essentially the bit length of the keys used. Let me look into that annotation usage. >>>> @Label("Serial numbers forming chain of trust") should not be a sentence. How about "Serial Numbers"? >>>> >>>> I think the tests are hard to read when two things are tested at the same time. It is also likely that if a test fail due to logging issues, it will be assigned to JFR because of the test name, even thought the issues is not JFR related. >>> I think we're always going to have some ownership issues when tests serve a dual purpose. I still think it makes sense to keep the test logic in one place. Any failures in these tests will most likely be owned by security team. (moving the tests to security directory is also an option) >>>> If you want to reuse code between tests, I would put it in testlibrary. >>> Let me check if there's any common patterns that could be added to the testlibary. >>> >>> Thanks, >>> Sean. >>>> Erik >>>> >>>>> Thanks for the update Erik. By default I'm proposing that the new JFR Events and Logger be disabled. As a result the event class shouldn't escape. If performance metrics highlight an issue, we should revisit. >>>>> >>>>> regards, >>>>> Sean. >>>>> >>>>> >>>>> On 27/06/2018 20:57, Erik Gahlin wrote: >>>>>> On 2018-06-27 21:14, Se?n Coffey wrote: >>>>>>> >>>>>>> On 27/06/2018 19:57, Xuelei Fan wrote: >>>>>>>> Hi Sean, >>>>>>>> >>>>>>>> I may reply in several replies. >>>>>>>> >>>>>>>> PKIXMasterCertPathValidator.java >>>>>>>> -------------------------------- >>>>>>>> + CertChainEvent cce = new CertChainEvent(); >>>>>>>> + if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>>>>>>> + String c = reversedCertList.stream() >>>>>>>> + .map(x -> x.getSerialNumber().toString(16)) >>>>>>>> + .collect(Collectors.joining(", ")); >>>>>>>> + EventHelper.commitCertChainEvent(cce, c); >>>>>>>> + } >>>>>>>> >>>>>>>> No matter the event or the JFR mechanism is enabled or not, the above code will create a new instance. Is the return value of cce.isEnabled() dynamically changed or static? >>>>>>> This is a requirement from the JFR framework. All their event.isEnabled calls are instance methods and follow a similar pattern. The JFR team assure me that the JIT can optimize away such calls. >>>>>> The JIT will most likely not be able to optimize if the event class escapes. >>>>>> >>>>>> From a JFR perspective, this would be the preferred layout: >>>>>> >>>>>> EventA eventA= new EventA(); >>>>>> eventA.value = this.value; >>>>>> eventA.commit(); >>>>>> >>>>>> and then do logging separately: >>>>>> >>>>>> System.Logger.log(DEBUG, this.value) >>>>>> >>>>>> With this layout, the JIT will remove the allocation and dead store. >>>>>> >>>>>> If it is expensive to gather the data for the event, like the CertChainEvent above, the following pattern should be used. >>>>>> >>>>>> EventB eventB= new EventB (); >>>>>> if (eventB.shouldCommit()) { >>>>>> eventB.value = calculateValue(); >>>>>> eventB .commit(); >>>>>> } >>>>>> >>>>>> If JFR is not enabled, shouldCommit returns false and the JIT should be able to remove the allocation. This will be best from a performance point of view, and also in my opinion from a maintenance and readability perspective. Others may disagree. >>>>>> >>>>>> Erik >>>>>> >>>>>>>> Is there a need to support both logging and JFR? I'm new to record events. I did not get the point to use both. >>>>>>> I was initially hoping to concentrate on just JFR events but I got strong feedback to also consider use of Logger (esp. in cases where the jdk.jfr module is not available) >>>>>>> >>>>>>> regards, >>>>>>> Sean. >>>>>>> >>>>>>>> Thanks, >>>>>>>> Xuelei >>>>>>>> >>>>>>>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>>>>>>> Erik, >>>>>>>>> >>>>>>>>> I rebased the patch with TLS v1.3 integration today. I hadn't realized how much the handshaker code had changed. Hopefully, I'll get a review from security dev team on that front. >>>>>>>>> >>>>>>>>> Regards the JFR semantics, I believe the edits should match majority of requests . I still have a preference for the test infra design for these new logger/JFR tests used in version 1 of webrev. I think it makes sense to keep the test functionality together - no sense in separating them out into different components IMO. Also, some of the edits to the JFR testing were made to test for the new dual log/record functionality. I might catch up with you tomorrow to see what the best arrangement would be. >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> Sean. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>>>>>>> Many thanks for the review comments Erik. Replies inline. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>>>>>>> Hi Sean, >>>>>>>>>>> >>>>>>>>>>> Some of the changes in the webrev belongs to JDK-8203629 and should be removed for clarity. >>>>>>>>>>> >>>>>>>>>>> Some initial comments: >>>>>>>>>>> >>>>>>>>>>> default.jfc, profile.jfr: >>>>>>>>>>> The events should not have control="enable-exceptions". The purpose of the control attribute is so to provide parameterized configuration of events for JMC. As it is now, the security events will be enabled when a user turns on the exception events. >>>>>>>>>> Makes sense. I'll remove that parameter. >>>>>>>>>>> X509CertEvent: >>>>>>>>>>> You should use milliseconds since epoch to represent a date instead of a string value, i.e. >>>>>>>>>>> >>>>>>>>>>> @Label("Valid From") >>>>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>>>> public long validFrom; >>>>>>>>>>> >>>>>>>>>>> @Label("Valid Until") >>>>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>>>> public long validUntil; >>>>>>>>>>> >>>>>>>>>> The CertificateValidity class operates on Date Object values. I'll work with the Date.getTime() method then (and update the Logger formatting) >>>>>>>>>>> This following annotation adds little value >>>>>>>>>>> >>>>>>>>>>> @Description("Details of Security Property") >>>>>>>>>>> >>>>>>>>>>> I would either remove the annotation, or provide information that helps a user understand the event. For instance, what is X509, and what kind of certificates are we talking about? >>>>>>>>>> Yes - that looks like the wrong Description. I'll review all of these fields. >>>>>>>>>>> @Category("Java Application") >>>>>>>>>>> >>>>>>>>>>> I'm a bit worried that we will pollute the "Java Application" namespace if we add lots of JDK events in that category. We may want to add a new top level category "Java Development Kit", analogous to the "Java Virtual Machine" category, and put all security related events in subcategory, perhaps called "Security". >>>>>>>>>> Yes - Open to suggestions. "Security" sounds like a good suggestion. >>>>>>>>>>> @Label("X509Cert") >>>>>>>>>>> >>>>>>>>>>> The label should be human readable name, with spaces, title cased etc. I would recommend "X.509 Certificate". In general, avoid abbreviations like "certs" and instead use labels such as "Certificate Chain". The label should be short and not a full sentence. >>>>>>>>>>> >>>>>>>>>>> For details see, >>>>>>>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>>>>>>> >>>>>>>>>>> I think it would be good to separate testing of JFR and logging into different files / tests. I would prefer that the test name is the same as the event prefixed with "Test", i.e TestX509CertificateEvent, as this is the pattern used by other JFR tests. >>>>>>>>>>> >>>>>>>>>> I'll take a look at that pattern again. I've separated out the current tests into an (a) outer test to analyze the logger output and (b) the inner test which checks for JFR correctness. I did include extra logic to make sure that the EventHelper configuration was working correctly. "Events.assertField" looks very helpful. Thanks for the pointer. >>>>>>>>>> >>>>>>>>>> Let me take on board the suggestions below and get a new webrev out on Tuesday. >>>>>>>>>> >>>>>>>>>> regards, >>>>>>>>>> Sean. >>>>>>>>>> >>>>>>>>>>> I reworked one of the tests to how I like to see it: >>>>>>>>>>> >>>>>>>>>>> /* >>>>>>>>>>> * @test >>>>>>>>>>> * @key jfr >>>>>>>>>>> * @library /test/lib >>>>>>>>>>> * @run main/othervm jdk.jfr.event.security.TestX509CertificateEvent >>>>>>>>>>> */ >>>>>>>>>>> public class TestX509CertificateEvent { >>>>>>>>>>> >>>>>>>>>>> private static final String CERTIFICATE_1 = "..."; >>>>>>>>>>> private static final String CERTIFICATE_2 = "..."; >>>>>>>>>>> >>>>>>>>>>> public static void main(String... args) throws CertificateException { >>>>>>>>>>> >>>>>>>>>>> Recording r = new Recording(); >>>>>>>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>>>>>>> r.start(); >>>>>>>>>>> >>>>>>>>>>> CertificateFactory cf = CertificateFactory.getInstance("X.509"); >>>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>>>> >>>>>>>>>>> // Make sure only one event per certificate >>>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>>>> cf.generateCertificate(new ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>>>> >>>>>>>>>>> r.stop(); >>>>>>>>>>> >>>>>>>>>>> List events = Events.fromRecording(r); >>>>>>>>>>> Asserts.assertEquals(events.size(), 2, "Expected two X.509 Certificate events"); >>>>>>>>>>> >>>>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>>>> "RSA", 2048); >>>>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>>>> "RSA", 2048); >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> private static void assertEvent(List events, String certID, String algId, String subject, >>>>>>>>>>> String issuer, String keyType, int length) throws Exception { >>>>>>>>>>> >>>>>>>>>>> for (RecordedEvent e : events) { >>>>>>>>>>> if (e.getString("serialNumber").equals(certID)) { >>>>>>>>>>> Events.assertField(e, "algId").equal(algId); >>>>>>>>>>> ... >>>>>>>>>>> return; >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> System.out.println(events); >>>>>>>>>>> throw new Exception("Could not find event with Cert ID: " + certID); >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> The reworked example uses the Events.assertField method, which will give context if the assertion fails. Keeping the test simple, means it can be analyzed quickly if it fails in testing. There is no new test framework to learn, or methods to search for, and it is usually not hard to determine if the failure is product, test or infrastructure related, and what component (team) should be assigned the issue. The use of EventNames.X509Certificate means all occurrences of the event can be tracked done in an IDE using find by reference. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> Erik >>>>>>>>>>> >>>>>>>>>>>> Following on from the recent JDK-8203629 code review, I'd like to propose enhancements on how we can record events in security libs. The introduction of the JFR libraries can give us much better ways of examining JDK actions. For the initial phase, I'm looking to enhance some key security library events in JDK 11 so that they can be either recorded to JFR, logged to a traditional logger, or both. >>>>>>>>>>>> >>>>>>>>>>>> Examples of how useful JFR recordings could be can be seen here : >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>>>>>>> >>>>>>>>>>>> securityProp_2.png gives an example of how the JFR recording can be queried to quickly locate events of interest (in this case, code setting the jdk.tls.* Security properties). I still need to clean up the TLSEvents testcase to improve test coverage and hope to do that in coming days. >>>>>>>>>>>> >>>>>>>>>>>> JBS record : >>>>>>>>>>>> * https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>>>>>>> >>>>>>>>>>>> webrev : http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>>>>>>> From kevin.rushforth at oracle.com Mon Jul 9 15:25:24 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Mon, 9 Jul 2018 08:25:24 -0700 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: <1883394348.1955480.1530972622847.JavaMail.zimbra@u-pem.fr> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <1973155195.1899675.1530914874386.JavaMail.zimbra@u-pem.fr> <142986b8-e2d7-3744-469d-e0209da290e6@oracle.com> <1883394348.1955480.1530972622847.JavaMail.zimbra@u-pem.fr> Message-ID: <99d5dc13-3722-d423-6614-e68ee44bdb50@oracle.com> > For the first point, it means that jpackager should use jopt for the argument parsing (to be fully compatible with the GNU style of options). > For the second point, it means to change a lot of code that may break because it's less mechanical than introducing try-with-resources. This seems quite a reasonable suggestion. Thanks. -- Kevin On 7/7/2018 7:10 AM, forax at univ-mlv.fr wrote: > > ----- Mail original ----- >> De: "Kevin Rushforth" >> ?: "Remi Forax" >> Cc: "core-libs-dev" , "Alexey Semenyuk" , "Andy Herrick" >> >> Envoy?: Samedi 7 Juillet 2018 15:47:01 >> Objet: Re: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] >> Hi Remy, >> >> Thank you for taking a look. >> >> Yes, the javapackager code that forms the basis for the jpackager >> prototype was initially developed on older JDKs and evolved from there. >> I'm sure the improvements you suggest are all good ones, and it doesn't >> seem like it would be too hard to address the most important of them, >> especially the try-with-resources or anything else that would affect the >> robustness of the tool. As long as we do address the robustness issues, >> I think it is more important to get the feature set right, and make sure >> that the public interfaces -- the command line options and ToolProvider >> interface -- are clean. I don't see the need to rewrite the tool or take >> an extra couple of months to modernize all of the implementation to use >> JDK 11 APIs everywhere. >> >> Also, I don't agree that jpackager is too large for jdk/sandbox or that >> it needs it own project. The jdk/sandbox is perfect for new modules / >> new tools that don't impact other parts of the JDK. >> >> -- Kevin > Hi Kevin, > like you, i don't think that a full rewrite is necessary, as you said having the right public 'interfaces' is enough, > but reducing the size the duplicated code (with the JDK and internally) is as important in my opinion. > > For the first point, it means that jpackager should use jopt for the argument parsing (to be fully compatible with the GNU style of options). > For the second point, it means to change a lot of code that may break because it's less mechanical than introducing try-with-resources. > > regards, > R?mi > >> >> On 7/6/2018 3:07 PM, Remi Forax wrote: >>> I've just taking a look at the patch, >>> i don't see how this can be integrated soon, the code is consistently >>> inconsistent as one of my colleague would say, even the coding conventions are >>> not respected. >>> i believe that's it's because the code have been written first in Java 6 an >>> without refactoring was moved to use Java 7, 8, 9, 10 and 11. >>> >>> The I/O code still using java.io.File for some parts, no try-with-resources so >>> most of the try/finally are not safe, >>> a lot of code like new BufferedWriter(new FileWriter(file)) instead of >>> Files.newBufferedWriter, etc. The code should use the package java.nio.file, >>> and not the old java.io, >>> most of the code try to manage the exception right were they appear instead of >>> propagating them so there are too many try/catch, >>> a lot of catch are ignored which is a code smell, some codes use the internal >>> logger (jdk.packager.internal.Log), but a lot of codes doesn't, >>> for the collection code, there is a lot of copy of data small structures (which >>> suggest that published collections are not immutable), >>> there are dubious @SuppressWarnings("unchecked"), some or them are due to the >>> fact that the code use Class as a type token instead of using lambdas, >>> Stream are not used when they should (to avoid multiple copy between data >>> structures) and streams that need to be closed are not (the result of >>> Files.list by example), >>> there are usual "don't do that in Java" like a loop using an integer index to >>> traverse a List without knowing if it's a random access list or not, >>> there is a lot of nullchecks instead of using Optional, >>> a lot of code initialize local variables to null which is a code smell (and a >>> side effect of having a lot of try/catch but not only), >>> constructors should not do work, just initialization, use static factory method >>> instead (so you will not have to debug half constructed objects), >>> the code uses BigIntegers to parse a bundle version, just in case, >>> the code uses an AtomicReference as a box that a lambda can mutate, instead of >>> wrapping the exception into a runtime and unwrapping it at call site, >>> The code of jdk.packager.internal.IOUtils should be updated to use methods of >>> the JDK 11 and methods like readFully should be replaced by the JDK's one. >>> listOfPathToString and setOfStringToString are just WTF, like in >>> getRedistributableModules(), where the variable stream is an Optional, >>> A class like Platform should be used everywhere to do platform specific stuff, a >>> lot of code still use String matching (the version parsing should use >>> System.Version). >>> All the argument parsing should be delegated to JOpt (the one integrated with >>> the JDK), so it will be consistent with the other JDK tools, >>> There is a UnsupportedPlatformException but a Platform can be UNKOWN ?? >>> >>> I will stop here, my point is that there is a lot of cleaning that should appear >>> before the code is integrated into the JDK and given the size of the code, i >>> wonder if it's not better to start an OpenJDK project for it and iterate on the >>> code before trying to include it in the JDK. For me, the code is too big to use >>> the jdk/sandbox. >>> >>> regards, >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "Kevin Rushforth" >>>> ?: "core-libs-dev" >>>> Cc: "Alexey Semenyuk" , "Andy Herrick" >>>> >>>> Envoy?: Vendredi 6 Juillet 2018 22:14:29 >>>> Objet: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: >>>> JDK-8200758: Packaging Tool] >>>> An initial prototype of the jpackager tool has been pushed to a new >>>> 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is >>>> interested in taking a look, you can clone it as follows: >>>> >>>> ??? hg clone http://hg.openjdk.java.net/jdk/sandbox >>>> ??? cd ./sandbox >>>> ??? hg update JDK-8200758-branch >>>> >>>> I plan to reply to the feedback already provided, and update the JEP [2] >>>> next week some time, but in the mean time if you have additional >>>> questions or comments, feel free to reply. >>>> >>>> -- Kevin >>>> >>>> [1] http://hg.openjdk.java.net/jdk/sandbox/shortlog/JDK-8200758-branch >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8200758 >>>> >>>> >>>> On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >>>>> We're aiming to get this into JDK 12 early enough so that an EA build >>>>> would be available around the time JDK 11 ships. That will allow you >>>>> to take a jlinked image with JDK 11 and package it up using (the EA) >>>>> jpackager. >>>>> >>>>> We will create a development branch in the JDK sandbox [1] some time >>>>> in the next week or so so you can follow the development. >>>>> >>>>> Also, thank you to those who have provided feedback. I'll reply to >>>>> feedback soon and then incorporate it into an updated JEP. >>>>> >>>>> -- Kevin From erik.gahlin at oracle.com Mon Jul 9 16:21:22 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 9 Jul 2018 18:21:22 +0200 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> <5B350AE5.3080401@oracle.com> <4F777EA1-6402-4725-B4E8-6DCB61DD4E19@oracle.com> Message-ID: <5B438B82.6070601@oracle.com> Thanks Sean. Some feedback on the code in the security libraries. - We should use camel case naming convention for variables (not underscore). - Looking at sun/security/ssl/Finished.java, I wonder if it wouldn't be less code and more easy to read, if we would commit the event in a local private function and then use the EventHelper class for common functionality. Instead of: private static void recordEvent(SSLSessionImpl session) { TLSHandshakeEvent hs_event = new TLSHandshakeEvent(); if (hs_event.isEnabled() || EventHelper.loggingSecurity()) { String certIDs = ""; try { certIDs = Stream.of(session.getPeerCertificates()) .filter(c -> c instanceof X509Certificate) .map(c -> (X509Certificate) c) .map(c -> c.getSerialNumber().toString(16)) .collect(Collectors.joining(", ")); } catch (SSLPeerUnverifiedException e) { certIDs = e.getMessage(); // not verified msg } EventHelper.commitTLSHandshakeEvent(hs_event, null, session.getPeerHost(), session.getPeerPort(), session.getCipherSuite(), session.getProtocol(), certIDs); } } ... public static void commitTLSHandshakeEvent(TLSHandshakeEvent event, Instant start, String remoteHost, int remotePort, String cipherSuite, String protocolVersion, String certChain) { if (event != null && event.shouldCommit()) { event.remoteHost = remoteHost; event.remotePort = remotePort; event.cipherSuite = cipherSuite; event.protocolVersion = protocolVersion; event.certificateChain = certChain; event.commit(); } if (loggingSecurity()) { String prepend = getDurationString(start); SECURITY_LOGGER.log(LOG_LEVEL, prepend + " TLSHandshake: {0}:{1,number,#}, {2}, {3}, {4}", remoteHost, remotePort, protocolVersion, cipherSuite, certChain); } } We would do: private static void logHandshake(SSLSessionImpl s) { TLSHandshakeEvent event = new TLSHandshakeEvent(); if (event.shouldCommit()) { event.remoteHost = s.getPeerHost(); event.remotePort = s.getPeerPort(); event.cipherSuite = s.getCipherSuite(); event.protocolVersion = s.getProtocol(); event.certificateChain = EventHelper.certificateChain(s.getPeerCerticates()); event.commit(); } if (EventHelper.isLoggingSecurity()) { EventHelper.SECURITY_LOGGER.log(LOG_LEVEL, " TLSHandshake: {0}:{1,number,#}, {2}, {3}, {4}", s.getPeerHost(), s.getPeerPort(), s.getProtocol(), s.getCipherSuite(), EventHelper.certificateChain(s.getPeerCerticates())); } } ... public static String certificateChain(Certificate[] certificates) { try { return Stream.of(certificates) .filter(c -> c instanceof X509Certificate) .map(c -> (X509Certificate) c) .map(c -> c.getSerialNumber().toString(16)) .collect(Collectors.joining(", ")); } catch (SSLPeerUnverifiedException e) { return e.getMessage(); // not verified msg } } It will also meanless overhead, since only one check is needed for the log and the JIT will be able to remove the allocation. Maybe a similar pattern could be used for the other events as well? Thanks Erik > As per request from Erik, I separated the tests out into individual > ones to test the JFR and Logger functionality. I introduced a new > separate test for the CertificateChainEvent event also. Originally > this was wrapped into the TLSHandshakeEvent test. > > Thanks to Erik for extra refactoring and modifications carried out to > clean up the code up further. > > http://cr.openjdk.java.net/~coffeys/webrev.8148188.v5/webrev/ > > This enhancement has a dependency on JDK-8203629 > > Regards, > Sean. > > On 02/07/18 09:49, Erik Gahlin wrote: >> >>> On 29 Jun 2018, at 17:34, Se?n Coffey wrote: >>> >>> I've introduced a new test helper class in the jdk/test/lib/jfr >>> directory to help with the dual test nature of the new tests. It's >>> helped alot with test code duplication. >>> >> My thinking was to put things like the certificates in a separate >> file, i.e TestCertificates, and then have a logging test and a JFR >> test reuse it. >> >> One rationale for adding logging was to use it if JFR is not present. >> By putting the tests together, it becomes impossible to compile and >> test logging without having JFR. >> >>> Looked at use of @DataAmount(DataAmount.BITS) also. Not sure if it's >>> fits. The output is displayed in units like "KiB" - not the normal >>> when examining key lengths used in X509Certificates. i.e a 2048 bit >>> key gets displayed as "2 KiB" - I'd prefer to keep the 2048 display >>> version. >> We should not let the JMC GUI decide how units are specified. There >> will be other GUIs and this is the first event that uses bits, so I >> don?t think it is formatted that way because it was considered superior. >> >> Erik >> >>> new webrev at: >>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v4/webrev/ >>> >>> Regards, >>> Sean. >>> >>> On 28/06/18 17:59, Se?n Coffey wrote: >>>> Comments inline. >>>> >>>> >>>> On 28/06/2018 17:20, Erik Gahlin wrote: >>>>> It's sufficient if an event object escapes to another method >>>>> (regardless if JFR is enabled or not). >>>>> >>>>> Some more feedback: >>>>> >>>>> Rename event jdk.CertChain to jdk.CertificateChain >>>>> Rename event jdk.X509Cert to jdk.X509Certificate >>>>> Rename field certChain to certificateChain. >>>>> Rename field serialNum to serialNumber >>>> all above done. >>>>> Rename field algId to AlgorithmicId or AlgorithmicID >>>> maybe "algorithm" works here also ? >>>>> Rename @Label("Ciphersuite") to @Label("Cipher Suite") >>>>> Rename @Label("Cert Chain") to @Label("Certificate Chain") >>>>> Rename @Label("Property Name") to "Name" or "Key" if that makes >>>>> sense in the context? >>>>> Rename @Label("Property Value") to "Value". >>>>> Put events in a subcategory, i.e @Category({"Java Development >>>>> Kit", "Security"}) >>>> done. >>>>> Make classes final. >>>> done. I had thought that the JFR mechanism required non-final classes. >>>>> What is the unit of the key in X509Certificate event? Bits? If >>>>> that is the case, use @DataAmount(DataAmount.BITS) >>>> Yes - it's essentially the bit length of the keys used. Let me look >>>> into that annotation usage. >>>>> @Label("Serial numbers forming chain of trust") should not be a >>>>> sentence. How about "Serial Numbers"? >>>>> >>>>> I think the tests are hard to read when two things are tested at >>>>> the same time. It is also likely that if a test fail due to >>>>> logging issues, it will be assigned to JFR because of the test >>>>> name, even thought the issues is not JFR related. >>>> I think we're always going to have some ownership issues when tests >>>> serve a dual purpose. I still think it makes sense to keep the test >>>> logic in one place. Any failures in these tests will most likely be >>>> owned by security team. (moving the tests to security directory is >>>> also an option) >>>>> If you want to reuse code between tests, I would put it in >>>>> testlibrary. >>>> Let me check if there's any common patterns that could be added to >>>> the testlibary. >>>> >>>> Thanks, >>>> Sean. >>>>> Erik >>>>> >>>>>> Thanks for the update Erik. By default I'm proposing that the new >>>>>> JFR Events and Logger be disabled. As a result the event class >>>>>> shouldn't escape. If performance metrics highlight an issue, we >>>>>> should revisit. >>>>>> >>>>>> regards, >>>>>> Sean. >>>>>> >>>>>> >>>>>> On 27/06/2018 20:57, Erik Gahlin wrote: >>>>>>> On 2018-06-27 21:14, Se?n Coffey wrote: >>>>>>>> >>>>>>>> On 27/06/2018 19:57, Xuelei Fan wrote: >>>>>>>>> Hi Sean, >>>>>>>>> >>>>>>>>> I may reply in several replies. >>>>>>>>> >>>>>>>>> PKIXMasterCertPathValidator.java >>>>>>>>> -------------------------------- >>>>>>>>> + CertChainEvent cce = new CertChainEvent(); >>>>>>>>> + if(cce.isEnabled() || EventHelper.loggingSecurity()) { >>>>>>>>> + String c = reversedCertList.stream() >>>>>>>>> + .map(x -> x.getSerialNumber().toString(16)) >>>>>>>>> + .collect(Collectors.joining(", ")); >>>>>>>>> + EventHelper.commitCertChainEvent(cce, c); >>>>>>>>> + } >>>>>>>>> >>>>>>>>> No matter the event or the JFR mechanism is enabled or not, >>>>>>>>> the above code will create a new instance. Is the return >>>>>>>>> value of cce.isEnabled() dynamically changed or static? >>>>>>>> This is a requirement from the JFR framework. All their >>>>>>>> event.isEnabled calls are instance methods and follow a similar >>>>>>>> pattern. The JFR team assure me that the JIT can optimize away >>>>>>>> such calls. >>>>>>> The JIT will most likely not be able to optimize if the event >>>>>>> class escapes. >>>>>>> >>>>>>> From a JFR perspective, this would be the preferred layout: >>>>>>> >>>>>>> EventA eventA= new EventA(); >>>>>>> eventA.value = this.value; >>>>>>> eventA.commit(); >>>>>>> >>>>>>> and then do logging separately: >>>>>>> >>>>>>> System.Logger.log(DEBUG, this.value) >>>>>>> >>>>>>> With this layout, the JIT will remove the allocation and dead >>>>>>> store. >>>>>>> >>>>>>> If it is expensive to gather the data for the event, like the >>>>>>> CertChainEvent above, the following pattern should be used. >>>>>>> >>>>>>> EventB eventB= new EventB (); >>>>>>> if (eventB.shouldCommit()) { >>>>>>> eventB.value = calculateValue(); >>>>>>> eventB .commit(); >>>>>>> } >>>>>>> >>>>>>> If JFR is not enabled, shouldCommit returns false and the JIT >>>>>>> should be able to remove the allocation. This will be best from >>>>>>> a performance point of view, and also in my opinion from a >>>>>>> maintenance and readability perspective. Others may disagree. >>>>>>> >>>>>>> Erik >>>>>>> >>>>>>>>> Is there a need to support both logging and JFR? I'm new to >>>>>>>>> record events. I did not get the point to use both. >>>>>>>> I was initially hoping to concentrate on just JFR events but I >>>>>>>> got strong feedback to also consider use of Logger (esp. in >>>>>>>> cases where the jdk.jfr module is not available) >>>>>>>> >>>>>>>> regards, >>>>>>>> Sean. >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Xuelei >>>>>>>>> >>>>>>>>> On 6/26/2018 3:18 PM, Se?n Coffey wrote: >>>>>>>>>> Erik, >>>>>>>>>> >>>>>>>>>> I rebased the patch with TLS v1.3 integration today. I hadn't >>>>>>>>>> realized how much the handshaker code had changed. Hopefully, >>>>>>>>>> I'll get a review from security dev team on that front. >>>>>>>>>> >>>>>>>>>> Regards the JFR semantics, I believe the edits should match >>>>>>>>>> majority of requests . I still have a preference for the test >>>>>>>>>> infra design for these new logger/JFR tests used in version 1 >>>>>>>>>> of webrev. I think it makes sense to keep the test >>>>>>>>>> functionality together - no sense in separating them out into >>>>>>>>>> different components IMO. Also, some of the edits to the JFR >>>>>>>>>> testing were made to test for the new dual log/record >>>>>>>>>> functionality. I might catch up with you tomorrow to see what >>>>>>>>>> the best arrangement would be. >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v2/webrev/ >>>>>>>>>> >>>>>>>>>> regards, >>>>>>>>>> Sean. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 25/06/2018 21:22, Se?n Coffey wrote: >>>>>>>>>>> Many thanks for the review comments Erik. Replies inline. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 24/06/2018 14:21, Erik Gahlin wrote: >>>>>>>>>>>> Hi Sean, >>>>>>>>>>>> >>>>>>>>>>>> Some of the changes in the webrev belongs to JDK-8203629 >>>>>>>>>>>> and should be removed for clarity. >>>>>>>>>>>> >>>>>>>>>>>> Some initial comments: >>>>>>>>>>>> >>>>>>>>>>>> default.jfc, profile.jfr: >>>>>>>>>>>> The events should not have control="enable-exceptions". The >>>>>>>>>>>> purpose of the control attribute is so to provide >>>>>>>>>>>> parameterized configuration of events for JMC. As it is >>>>>>>>>>>> now, the security events will be enabled when a user turns >>>>>>>>>>>> on the exception events. >>>>>>>>>>> Makes sense. I'll remove that parameter. >>>>>>>>>>>> X509CertEvent: >>>>>>>>>>>> You should use milliseconds since epoch to represent a date >>>>>>>>>>>> instead of a string value, i.e. >>>>>>>>>>>> >>>>>>>>>>>> @Label("Valid From") >>>>>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>>>>> public long validFrom; >>>>>>>>>>>> >>>>>>>>>>>> @Label("Valid Until") >>>>>>>>>>>> @Timestamp(Timestamp.MILLISECONDS_SINCE_EPOCH) >>>>>>>>>>>> public long validUntil; >>>>>>>>>>>> >>>>>>>>>>> The CertificateValidity class operates on Date Object >>>>>>>>>>> values. I'll work with the Date.getTime() method then (and >>>>>>>>>>> update the Logger formatting) >>>>>>>>>>>> This following annotation adds little value >>>>>>>>>>>> >>>>>>>>>>>> @Description("Details of Security Property") >>>>>>>>>>>> >>>>>>>>>>>> I would either remove the annotation, or provide >>>>>>>>>>>> information that helps a user understand the event. For >>>>>>>>>>>> instance, what is X509, and what kind of certificates are >>>>>>>>>>>> we talking about? >>>>>>>>>>> Yes - that looks like the wrong Description. I'll review all >>>>>>>>>>> of these fields. >>>>>>>>>>>> @Category("Java Application") >>>>>>>>>>>> >>>>>>>>>>>> I'm a bit worried that we will pollute the "Java >>>>>>>>>>>> Application" namespace if we add lots of JDK events in that >>>>>>>>>>>> category. We may want to add a new top level category "Java >>>>>>>>>>>> Development Kit", analogous to the "Java Virtual Machine" >>>>>>>>>>>> category, and put all security related events in >>>>>>>>>>>> subcategory, perhaps called "Security". >>>>>>>>>>> Yes - Open to suggestions. "Security" sounds like a good >>>>>>>>>>> suggestion. >>>>>>>>>>>> @Label("X509Cert") >>>>>>>>>>>> >>>>>>>>>>>> The label should be human readable name, with spaces, title >>>>>>>>>>>> cased etc. I would recommend "X.509 Certificate". In >>>>>>>>>>>> general, avoid abbreviations like "certs" and instead use >>>>>>>>>>>> labels such as "Certificate Chain". The label should be >>>>>>>>>>>> short and not a full sentence. >>>>>>>>>>>> >>>>>>>>>>>> For details see, >>>>>>>>>>>> https://docs.oracle.com/javase/10/docs/api/jdk/jfr/Label.html >>>>>>>>>>>> >>>>>>>>>>>> I think it would be good to separate testing of JFR and >>>>>>>>>>>> logging into different files / tests. I would prefer that >>>>>>>>>>>> the test name is the same as the event prefixed with >>>>>>>>>>>> "Test", i.e TestX509CertificateEvent, as this is the >>>>>>>>>>>> pattern used by other JFR tests. >>>>>>>>>>>> >>>>>>>>>>> I'll take a look at that pattern again. I've separated out >>>>>>>>>>> the current tests into an (a) outer test to analyze the >>>>>>>>>>> logger output and (b) the inner test which checks for JFR >>>>>>>>>>> correctness. I did include extra logic to make sure that the >>>>>>>>>>> EventHelper configuration was working correctly. >>>>>>>>>>> "Events.assertField" looks very helpful. Thanks for the >>>>>>>>>>> pointer. >>>>>>>>>>> >>>>>>>>>>> Let me take on board the suggestions below and get a new >>>>>>>>>>> webrev out on Tuesday. >>>>>>>>>>> >>>>>>>>>>> regards, >>>>>>>>>>> Sean. >>>>>>>>>>> >>>>>>>>>>>> I reworked one of the tests to how I like to see it: >>>>>>>>>>>> >>>>>>>>>>>> /* >>>>>>>>>>>> * @test >>>>>>>>>>>> * @key jfr >>>>>>>>>>>> * @library /test/lib >>>>>>>>>>>> * @run main/othervm >>>>>>>>>>>> jdk.jfr.event.security.TestX509CertificateEvent >>>>>>>>>>>> */ >>>>>>>>>>>> public class TestX509CertificateEvent { >>>>>>>>>>>> >>>>>>>>>>>> private static final String CERTIFICATE_1 = "..."; >>>>>>>>>>>> private static final String CERTIFICATE_2 = "..."; >>>>>>>>>>>> >>>>>>>>>>>> public static void main(String... args) throws >>>>>>>>>>>> CertificateException { >>>>>>>>>>>> >>>>>>>>>>>> Recording r = new Recording(); >>>>>>>>>>>> r.enable(EventNames.X509Certificate).withoutStackTrace(); >>>>>>>>>>>> r.start(); >>>>>>>>>>>> >>>>>>>>>>>> CertificateFactory cf = >>>>>>>>>>>> CertificateFactory.getInstance("X.509"); >>>>>>>>>>>> cf.generateCertificate(new >>>>>>>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>>>>> cf.generateCertificate(new >>>>>>>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>>>>> >>>>>>>>>>>> // Make sure only one event per certificate >>>>>>>>>>>> cf.generateCertificate(new >>>>>>>>>>>> ByteArrayInputStream(CERTIFICATE_1.getBytes())); >>>>>>>>>>>> cf.generateCertificate(new >>>>>>>>>>>> ByteArrayInputStream(CERTIFICATE_2.getBytes())); >>>>>>>>>>>> >>>>>>>>>>>> r.stop(); >>>>>>>>>>>> >>>>>>>>>>>> List events = Events.fromRecording(r); >>>>>>>>>>>> Asserts.assertEquals(events.size(), 2, "Expected >>>>>>>>>>>> two X.509 Certificate events"); >>>>>>>>>>>> >>>>>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>>>>> "RSA", 2048); >>>>>>>>>>>> assertEvent(events, "1000", "SHA256withRSA", >>>>>>>>>>>> "CN=SSLCertificate, O=SomeCompany", >>>>>>>>>>>> "CN=Intermediate CA Cert, O=SomeCompany", >>>>>>>>>>>> "RSA", 2048); >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> private static void assertEvent(List >>>>>>>>>>>> events, String certID, String algId, String subject, >>>>>>>>>>>> String issuer, String keyType, int length) >>>>>>>>>>>> throws Exception { >>>>>>>>>>>> >>>>>>>>>>>> for (RecordedEvent e : events) { >>>>>>>>>>>> if (e.getString("serialNumber").equals(certID)) { >>>>>>>>>>>> Events.assertField(e, "algId").equal(algId); >>>>>>>>>>>> ... >>>>>>>>>>>> return; >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> System.out.println(events); >>>>>>>>>>>> throw new Exception("Could not find event with Cert >>>>>>>>>>>> ID: " + certID); >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> The reworked example uses the Events.assertField method, >>>>>>>>>>>> which will give context if the assertion fails. Keeping the >>>>>>>>>>>> test simple, means it can be analyzed quickly if it fails >>>>>>>>>>>> in testing. There is no new test framework to learn, or >>>>>>>>>>>> methods to search for, and it is usually not hard to >>>>>>>>>>>> determine if the failure is product, test or infrastructure >>>>>>>>>>>> related, and what component (team) should be assigned the >>>>>>>>>>>> issue. The use of EventNames.X509Certificate means all >>>>>>>>>>>> occurrences of the event can be tracked done in an IDE >>>>>>>>>>>> using find by reference. >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> Erik >>>>>>>>>>>> >>>>>>>>>>>>> Following on from the recent JDK-8203629 code review, I'd >>>>>>>>>>>>> like to propose enhancements on how we can record events >>>>>>>>>>>>> in security libs. The introduction of the JFR libraries >>>>>>>>>>>>> can give us much better ways of examining JDK actions. For >>>>>>>>>>>>> the initial phase, I'm looking to enhance some key >>>>>>>>>>>>> security library events in JDK 11 so that they can be >>>>>>>>>>>>> either recorded to JFR, logged to a traditional logger, or >>>>>>>>>>>>> both. >>>>>>>>>>>>> >>>>>>>>>>>>> Examples of how useful JFR recordings could be can be seen >>>>>>>>>>>>> here : >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/X509Event_1.png >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_1.png >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/securityProp_2.png >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/event_snaps/TLSEvent_1.png >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> securityProp_2.png gives an example of how the JFR >>>>>>>>>>>>> recording can be queried to quickly locate events of >>>>>>>>>>>>> interest (in this case, code setting the jdk.tls.* >>>>>>>>>>>>> Security properties). I still need to clean up the >>>>>>>>>>>>> TLSEvents testcase to improve test coverage and hope to do >>>>>>>>>>>>> that in coming days. >>>>>>>>>>>>> >>>>>>>>>>>>> JBS record : >>>>>>>>>>>>> * https://bugs.openjdk.java.net/browse/JDK-8148188 >>>>>>>>>>>>> >>>>>>>>>>>>> webrev : >>>>>>>>>>>>> http://cr.openjdk.java.net/~coffeys/webrev.8148188.v1/webrev/ >>>>>>>>>>>>> > From sean.coffey at oracle.com Mon Jul 9 17:01:54 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 9 Jul 2018 18:01:54 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <5B438B82.6070601@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> <5B350AE5.3080401@oracle.com> <4F777EA1-6402-4725-B4E8-6DCB61DD4E19@oracle.com> <5B438B82.6070601@oracle.com> Message-ID: <00051b46-46a7-5c96-f36b-fb2e2fed8a58@oracle.com> Erik, Thanks for reviewing. Comments inline.. On 09/07/18 17:21, Erik Gahlin wrote: > Thanks Sean. > > Some feedback on the code in the security libraries. > > - We should use camel case naming convention for variables (not > underscore). Sure. I see two offending variable names which I'll fix up. > > - Looking at sun/security/ssl/Finished.java, > > I wonder if it wouldn't be less code and more easy to read, if we > would commit the event in a local private function and then use the > EventHelper class for common functionality. I'm fine with your suggested approach. I figured that tucking the recording/logging logic away in a helper class also had benefits but I'll re-factor to your suggested style unless I hear objections. regards, Sean. From martinrb at google.com Mon Jul 9 18:17:33 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 9 Jul 2018 11:17:33 -0700 Subject: RFR: 8206863: A closed JarVerifier.VerifierStream should throw IOException Message-ID: Here's another fix from Tobias and myself: 8206863: A closed JarVerifier.VerifierStream should throw IOException http://cr.openjdk.java.net/~martin/webrevs/jdk/JarVerifier-stream-closed/ https://bugs.openjdk.java.net/browse/JDK-8206863 From alexfoster at hotmail.ca Thu Jul 5 17:04:06 2018 From: alexfoster at hotmail.ca (Alex Foster) Date: Thu, 5 Jul 2018 17:04:06 +0000 Subject: ArrayList.ensureCapacity should expand the default capacity empty array Message-ID: Hi, I think that ensureCapacity should actually ensure the capacity by expanding the default capacity empty array, otherwise you could end up with a situation like this: List> ls = getLists();// Large list of ArrayLists ls.forEach(l -> l.ensureCapacity(10)); //later... ArrayList a = ls.get(0); if(a.size() < 10) a.add(null); // throws OutOfMemoryError when array is allocated I think it's still OK to not allocate the array directly in the no-arg constructor because if users actually care about the capacity they can call the constructor with an initialCapacity argument or ensureCapacity, but maybe a note should be added that it can be allocated lazily? Also it's debatable if ensureCapacity should call grow or just resize the array to exactly the minCapacity argument, which might save space if the exact size needed is known, or maybe to max(minCapacity, DEFAULT_CAPACITY) if elementData == DEFAULTCAPACITY_EMPTY_ELEMENTDATA which would use more space but otherwise ensureCapacity could cause you to end up with lower than default capacity (which might be wanted). diff --git a/src/java.base/share/classes/java/util/ArrayList.java b/src/java.base/share/classes/java/util/ArrayList.java --- a/src/java.base/share/classes/java/util/ArrayList.java +++ b/src/java.base/share/classes/java/util/ArrayList.java @@ -210,11 +210,9 @@ * @param minCapacity the desired minimum capacity */ public void ensureCapacity(int minCapacity) { - if (minCapacity > elementData.length - && !(elementData == DEFAULTCAPACITY_EMPTY_ELEMENTDATA - && minCapacity <= DEFAULT_CAPACITY)) { + if (minCapacity > elementData.length) { modCount++; - grow(minCapacity); + elementData = Arrays.copyOf(elementData, minCapacity); } } I noticed this while writing a similar method for ArrayDeque which could also be added to ArrayList: /** * Attempts to reallocate the backing array so that is has space for as close as * possible to, but not less than, minSize elements unless it already has space * for between minSize and maxSize elements. If the size() of this ArrayDeque is * greater than minSize then size() will be treated as minSize. * *

This method can be used to achieve the same result as {@link ArrayList#trimToSize} * by calling {@code reallocate(0, 0)} or {@link ArrayList#ensureCapacity(n)} by calling * {@code reallocate(n, Integer.MAX_VALUE)}. * * @param minSize the requested minimum capacity * @param maxSize the requested maximum capacity * @return the new (or unchanged) capacity * @throws IllegalArgumentException if minSize > maxSize (size() may be greater than maxSize * without throwing an exception) * @throws OutOfMemoryError if there is not enough memory available to allocate * the array or the array would be larger than an implementation defined limit */ public int reallocate(int minSize, int maxSize){ if(minSize > maxSize) throw new IllegalArgumentException(minSize + " > " + maxSize); int c = elements.length - 1, s = size(); if(c < minSize){ if(minSize + 1 < 0) throw new OutOfMemoryError(); } else { if(c <= maxSize) return c; if(s > minSize){ if(c == (minSize = s)) return c; } } elements = minSize == 0 ? EMPTY : copyElements(new Object[minSize + 1], head, tail); head = 0; tail = s; return minSize; } From orionllmain at gmail.com Mon Jul 9 18:31:02 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Tue, 10 Jul 2018 01:31:02 +0700 Subject: WeakReference with null referent Message-ID: It is possible to create a WeakReference/SoftReference/PhantomReference with a null value in which case the Reference will never be enqueued. This is quite obvious (since null cannot be weakly/softly/phantom reachable). But I think it's worth being mentioned in the JavaDoc. What do you think? From mandy.chung at oracle.com Mon Jul 9 18:49:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 9 Jul 2018 11:49:22 -0700 Subject: WeakReference with null referent In-Reply-To: References: Message-ID: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> On 7/9/18 11:31 AM, Zheka Kozlov wrote: > It is possible to create a WeakReference/SoftReference/PhantomReference > with a null value in which case the Reference will never be enqueued. This > is quite obvious (since null cannot be weakly/softly/phantom reachable). > But I think it's worth being mentioned in the JavaDoc. What do you think? > Alternatively, the constructor should require non-null referent and throws NPE if null. I created https://bugs.openjdk.java.net/browse/JDK-8206933 to track this. Mandy From per.liden at oracle.com Mon Jul 9 19:59:03 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 9 Jul 2018 21:59:03 +0200 Subject: WeakReference with null referent In-Reply-To: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> Message-ID: <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> On 2018-07-09 20:49, mandy chung wrote: > > > On 7/9/18 11:31 AM, Zheka Kozlov wrote: >> It is possible to create a WeakReference/SoftReference/PhantomReference >> with a null value in which case the Reference will never be enqueued. >> This >> is quite obvious (since null cannot be weakly/softly/phantom reachable). >> But I think it's worth being mentioned in the JavaDoc. What do you think? >> > > Alternatively, the constructor should require non-null referent and > throws NPE if null. > > I created https://bugs.openjdk.java.net/browse/JDK-8206933 to track this. It's not completely obvious to me that throwing NPE or otherwise blocking this is the right thing to do. Sure, creating a Reference with a null referent seems pretty useless, but it's also very similar to creating a Reference and immediately calling its clear() method, which is perfectly valid (and equally useless). Are you saying we should block this because we can easily detect this particular case/misuse, as opposed to the immediately-called-clear case? Or is there some other rationale? cheers, Per From roger.riggs at oracle.com Mon Jul 9 21:07:53 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 9 Jul 2018 17:07:53 -0400 Subject: WeakReference with null referent In-Reply-To: <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> Message-ID: Hi, After so many years of accepting null, throwing NPE would an incompatible change and break existing uses. Documenting it as ill advised or useless is would be a useful caution. Regards, Roger On 7/9/18 3:59 PM, Per Liden wrote: > > On 2018-07-09 20:49, mandy chung wrote: >> >> >> On 7/9/18 11:31 AM, Zheka Kozlov wrote: >>> It is possible to create a WeakReference/SoftReference/PhantomReference >>> with a null value in which case the Reference will never be >>> enqueued. This >>> is quite obvious (since null cannot be weakly/softly/phantom >>> reachable). >>> But I think it's worth being mentioned in the JavaDoc. What do you >>> think? >>> >> >> Alternatively, the constructor should require non-null referent and >> throws NPE if null. >> >> I created https://bugs.openjdk.java.net/browse/JDK-8206933 to track >> this. > > It's not completely obvious to me that throwing NPE or otherwise > blocking this is the right thing to do. Sure, creating a Reference > with a null referent seems pretty useless, but it's also very similar > to creating a Reference and immediately calling its clear() method, > which is perfectly valid (and equally useless). > > Are you saying we should block this because we can easily detect this > particular case/misuse, as opposed to the immediately-called-clear > case? Or is there some other rationale? > > cheers, > Per From mandy.chung at oracle.com Mon Jul 9 22:18:10 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 9 Jul 2018 15:18:10 -0700 Subject: WeakReference with null referent In-Reply-To: <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> Message-ID: <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> On 7/9/18 12:59 PM, Per Liden wrote: > > On 2018-07-09 20:49, mandy chung wrote: >> >> >> On 7/9/18 11:31 AM, Zheka Kozlov wrote: >>> It is possible to create a WeakReference/SoftReference/PhantomReference >>> with a null value in which case the Reference will never be enqueued. >>> This >>> is quite obvious (since null cannot be weakly/softly/phantom reachable). >>> But I think it's worth being mentioned in the JavaDoc. What do you >>> think? >>> >> >> Alternatively, the constructor should require non-null referent and >> throws NPE if null. >> >> I created https://bugs.openjdk.java.net/browse/JDK-8206933 to track this. > > It's not completely obvious to me that throwing NPE or otherwise > blocking this is the right thing to do. Sure, creating a Reference with > a null referent seems pretty useless, but it's also very similar to > creating a Reference and immediately calling its clear() method, which > is perfectly valid (and equally useless). > > Are you saying we should block this because we can easily detect this > particular case/misuse, as opposed to the immediately-called-clear case? > Or is there some other rationale? I don't expect any one wants to create a reference with null referent Throwing NPE would be one way to catch if a reference object is created with null referent. I think the compatibility risk is low. I agree that Reference::clear is equally useless. Are you thinking something should be done with Reference::clear? Mandy From martinrb at google.com Tue Jul 10 00:34:44 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 9 Jul 2018 17:34:44 -0700 Subject: ArrayList.ensureCapacity should expand the default capacity empty array In-Reply-To: References: Message-ID: Thanks Alex, I filed https://bugs.openjdk.java.net/browse/JDK-8206945 I've had similar thoughts for years, but none have ended up in the actual code. I haven't seem the approach you used in your reallocate. On Thu, Jul 5, 2018 at 10:04 AM, Alex Foster wrote: > Hi, > > > I think that ensureCapacity should actually ensure the capacity by > expanding the default capacity empty array, otherwise you could end up with > a situation like this: > > List> ls = getLists();// Large list of ArrayLists > ls.forEach(l -> l.ensureCapacity(10)); > //later... > ArrayList a = ls.get(0); > if(a.size() < 10) a.add(null); // throws OutOfMemoryError when > array is allocated > > I think it's still OK to not allocate the array directly in the no-arg > constructor because if users actually care about the capacity they can call > the constructor with an initialCapacity argument or ensureCapacity, but > maybe a note should be added that it can be allocated lazily? > > Also it's debatable if ensureCapacity should call grow or just resize the > array to exactly the minCapacity argument, which might save space if the > exact size needed is known, or maybe to max(minCapacity, DEFAULT_CAPACITY) > if elementData == DEFAULTCAPACITY_EMPTY_ELEMENTDATA which would use more > space but otherwise ensureCapacity could cause you to end up with lower > than default capacity (which might be wanted). > > diff --git a/src/java.base/share/classes/java/util/ArrayList.java > b/src/java.base/share/classes/java/util/ArrayList.java > --- a/src/java.base/share/classes/java/util/ArrayList.java > +++ b/src/java.base/share/classes/java/util/ArrayList.java > @@ -210,11 +210,9 @@ > * @param minCapacity the desired minimum capacity > */ > public void ensureCapacity(int minCapacity) { > - if (minCapacity > elementData.length > - && !(elementData == DEFAULTCAPACITY_EMPTY_ELEMENTDATA > - && minCapacity <= DEFAULT_CAPACITY)) { > + if (minCapacity > elementData.length) { > modCount++; > - grow(minCapacity); > + elementData = Arrays.copyOf(elementData, minCapacity); > } > } > > I noticed this while writing a similar method for ArrayDeque which could > also be added to ArrayList: > > /** > * Attempts to reallocate the backing array so that is has space for > as close as > * possible to, but not less than, minSize elements unless it already > has space > * for between minSize and maxSize elements. If the size() of this > ArrayDeque is > * greater than minSize then size() will be treated as minSize. > * > *

This method can be used to achieve the same result as {@link > ArrayList#trimToSize} > * by calling {@code reallocate(0, 0)} or {@link > ArrayList#ensureCapacity(n)} by calling > * {@code reallocate(n, Integer.MAX_VALUE)}. > * > * @param minSize the requested minimum capacity > * @param maxSize the requested maximum capacity > * @return the new (or unchanged) capacity > * @throws IllegalArgumentException if minSize > maxSize (size() may > be greater than maxSize > * without throwing an exception) > * @throws OutOfMemoryError if there is not enough memory available to > allocate > * the array or the array would be larger than an implementation > defined limit > */ > public int reallocate(int minSize, int maxSize){ > if(minSize > maxSize) throw new IllegalArgumentException(minSize > + " > " + maxSize); > int c = elements.length - 1, s = size(); > if(c < minSize){ > if(minSize + 1 < 0) throw new OutOfMemoryError(); > } else { > if(c <= maxSize) return c; > if(s > minSize){ > if(c == (minSize = s)) return c; > } > } > elements = minSize == 0 ? EMPTY : copyElements(new Object[minSize > + 1], head, tail); > head = 0; > tail = s; > return minSize; > } > > From vladimir.kozlov at oracle.com Tue Jul 10 05:04:50 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 9 Jul 2018 22:04:50 -0700 Subject: [11] RFR(XS) 8206952: java/lang/Class/GetPackageBootLoaderChildLayer.java fails with Graal Message-ID: https://bugs.openjdk.java.net/browse/JDK-8206952 Don't run the test with Graal because --limit-modules flag excludes Graal module: diff -r 1835f9fca157 test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java --- a/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java +++ b/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java @@ -23,6 +23,7 @@ /** * @test + * @requires !vm.graal.enabled * @modules jdk.attach * @run main/othervm --limit-modules jdk.attach -Djdk.attach.allowAttachSelf * GetPackageBootLoaderChildLayer Thanks, Vladimir From mandy.chung at oracle.com Tue Jul 10 05:02:44 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 9 Jul 2018 22:02:44 -0700 Subject: [11] RFR(XS) 8206952: java/lang/Class/GetPackageBootLoaderChildLayer.java fails with Graal In-Reply-To: References: Message-ID: +1 Mandy On 7/9/18 10:04 PM, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8206952 > > Don't run the test with Graal because --limit-modules flag excludes > Graal module: > > diff -r 1835f9fca157 > test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java > --- a/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java > +++ b/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java > @@ -23,6 +23,7 @@ > > ?/** > ? * @test > + * @requires !vm.graal.enabled > ? * @modules jdk.attach > ? * @run main/othervm --limit-modules jdk.attach > -Djdk.attach.allowAttachSelf > ? *??? GetPackageBootLoaderChildLayer > > Thanks, > Vladimir From ekaterina.pavlova at oracle.com Tue Jul 10 05:03:02 2018 From: ekaterina.pavlova at oracle.com (Ekaterina Pavlova) Date: Mon, 9 Jul 2018 22:03:02 -0700 Subject: [11] RFR(XS) 8206952: java/lang/Class/GetPackageBootLoaderChildLayer.java fails with Graal In-Reply-To: References: Message-ID: <47750b57-8acb-ab14-c3b4-b1082f6bf189@oracle.com> Good. Thanks for fixing! -katya On 7/9/18 10:04 PM, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8206952 > > Don't run the test with Graal because --limit-modules flag excludes Graal module: > > diff -r 1835f9fca157 test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java > --- a/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java > +++ b/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java > @@ -23,6 +23,7 @@ > > ?/** > ? * @test > + * @requires !vm.graal.enabled > ? * @modules jdk.attach > ? * @run main/othervm --limit-modules jdk.attach -Djdk.attach.allowAttachSelf > ? *??? GetPackageBootLoaderChildLayer > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Tue Jul 10 05:08:56 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 9 Jul 2018 22:08:56 -0700 Subject: [11] RFR(XS) 8206952: java/lang/Class/GetPackageBootLoaderChildLayer.java fails with Graal In-Reply-To: References: Message-ID: <7c70b14c-e4db-4e60-f421-5bc844747ce6@oracle.com> Thank you, Mandy Vladimir On 7/9/18 10:02 PM, mandy chung wrote: > +1 > > Mandy > > On 7/9/18 10:04 PM, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8206952 >> >> Don't run the test with Graal because --limit-modules flag excludes Graal module: >> >> diff -r 1835f9fca157 test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java >> --- a/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java >> +++ b/test/jdk/java/lang/Class/GetPackageBootLoaderChildLayer.java >> @@ -23,6 +23,7 @@ >> >> ??/** >> ?? * @test >> + * @requires !vm.graal.enabled >> ?? * @modules jdk.attach >> ?? * @run main/othervm --limit-modules jdk.attach -Djdk.attach.allowAttachSelf >> ?? *??? GetPackageBootLoaderChildLayer >> >> Thanks, >> Vladimir From orionllmain at gmail.com Tue Jul 10 05:10:00 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Tue, 10 Jul 2018 12:10:00 +0700 Subject: WeakReference with null referent In-Reply-To: <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> Message-ID: I also created an issue in IDEA bug tracker: https://youtrack.jetbrains.com/issue/IDEA-195298. This at least should be handled by an IDE. From peter.levart at gmail.com Tue Jul 10 06:59:00 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 10 Jul 2018 08:59:00 +0200 Subject: WeakReference with null referent In-Reply-To: <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> Message-ID: <1331e58f-1257-bc5e-9ca1-212b4488e413@gmail.com> Hi Mandy, On 07/10/2018 12:18 AM, mandy chung wrote: > > > On 7/9/18 12:59 PM, Per Liden wrote: >> >> On 2018-07-09 20:49, mandy chung wrote: >>> >>> >>> On 7/9/18 11:31 AM, Zheka Kozlov wrote: >>>> It is possible to create a >>>> WeakReference/SoftReference/PhantomReference >>>> with a null value in which case the Reference will never be >>>> enqueued. This >>>> is quite obvious (since null cannot be weakly/softly/phantom >>>> reachable). >>>> But I think it's worth being mentioned in the JavaDoc. What do you >>>> think? >>>> >>> >>> Alternatively, the constructor should require non-null referent and >>> throws NPE if null. >>> >>> I created https://bugs.openjdk.java.net/browse/JDK-8206933 to track >>> this. >> >> It's not completely obvious to me that throwing NPE or otherwise >> blocking this is the right thing to do. Sure, creating a Reference >> with a null referent seems pretty useless, but it's also very similar >> to creating a Reference and immediately calling its clear() method, >> which is perfectly valid (and equally useless). >> >> Are you saying we should block this because we can easily detect this >> particular case/misuse, as opposed to the immediately-called-clear >> case? Or is there some other rationale? > > I don't expect any one wants to create a reference with null referent > Throwing NPE would be one way to catch if a reference object is created > with null referent.? I think the compatibility risk is low. > I agree that Reference::clear is equally useless.? Are you thinking > something should be done with Reference::clear? > > Mandy There are situations where being able to create a XxxReference without a referent is useful. For example, imagine a doubly-linked list of WeakReference subclasses - a special instance without a referent can be used as the head of such list that never goes away and simplifies insertion/deletion logic, etc... JDK's java.lang.ref.Cleaner API uses such scheme in its implementation (see jdk.internal.ref.CleanerImpl.[Phantom|Weak|Soft]CleanableRef no-arg constructors). Regards, Peter From matthias.baesken at sap.com Tue Jul 10 09:53:12 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 10 Jul 2018 09:53:12 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> Message-ID: Hi Alan, thanks for commenting on this . Jaikiran mentioned that printing just the jar file name and not file with path might be okay : > I am not a reviewer and neither do I have enough knowledge about whether > jar/file _names_ are considered security sensitive. However, the patch > that's proposed for this change, prints the file _path_ (and not just > the name). That I believe is security sensitive. What do you think ? Best regards, Matthias > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Sonntag, 8. Juli 2018 09:36 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > On 06/07/2018 13:44, Baesken, Matthias wrote: > > Hi Alan ,so it looks like JDK-8204233 added a switch (system property) to > enable the enhanced socket IOException messages . > > > > That would be an option as well for 8205525 . > Yes, it's documented in conf/security/java.security and something > equivalent could be done here. The giveaway in your original patch is > that it needed a privileged block to create the exception message. > > > > > 8205525 adds the jar file name and the line number info to the > exception message . > > > > In case that only the jar file name would be considered sensitive , I would > prefer to just output the line number (and omit the system property ). > > > That should be okay (I can't think of any concerns). > > -Alan From norman.maurer at googlemail.com Tue Jul 10 10:06:57 2018 From: norman.maurer at googlemail.com (Norman Maurer) Date: Tue, 10 Jul 2018 11:06:57 +0100 Subject: Consider making TerminatingThreadLocal public Message-ID: <3BAB12F3-CCBC-4286-99C1-B59C4B4635FB@googlemail.com> Hi all, It came to my attention that you introduced TerminatingThreadLocal in openjdk lately as an ?internal class?: http://hg.openjdk.java.net/jdk/jdk/rev/106dc156ce6b I would like to take the chance to ask if you ever considered to make this ?non-internal? as I think it would super helpful for some use cases. To give you some more context let me give you an example of why we would love it in netty. So in netty we sometimes pool ?direct memory? in ThreadLocals [1] and have to do some tricks to ensure this direct memory is really released once a Thread exit. Which I think is also exactly one use-case you have here. Sure we can do this with a WeakReference + ReferenceQueue , Cleaner or finalizer but I feel exposing TerminatingThreadLocal would be a lot more elegant. Another advantage would be that we can be sure that the cleanup action would be called in the Thread that holds the value of the ThreadLocal, which is not really possible with using WeakReference + ReferenceQueue , Cleaner or finalizer. What we currently doing in netty is basically have our own ThreadLocal [1] implementation and out own Thread sub-class [2] that will ensure we call some method on our TheadLocal that gives us the chance to do the cleanup by wrapping the Runnable[3] that is passed in. This works great but we can not always guarantee that our sub-class of Thread is used to access our ThreadLocal implementation, which means the cleanup action will not run in this case. Thanks, Norman [1] https://github.com/netty/netty/blob/netty-4.1.26.Final/common/src/main/java/io/netty/util/concurrent/FastThreadLocal.java [2] https://github.com/netty/netty/blob/netty-4.1.26.Final/common/src/main/java/io/netty/util/concurrent/FastThreadLocalThread.java [3] https://github.com/netty/netty/blob/netty-4.1.26.Final/common/src/main/java/io/netty/util/concurrent/FastThreadLocalRunnable.java From Joerg.Buchberger at pruftechnik.com Tue Jul 10 11:35:44 2018 From: Joerg.Buchberger at pruftechnik.com (Buchberger, Joerg) Date: Tue, 10 Jul 2018 11:35:44 +0000 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> Message-ID: <75106ab90b974292890c71b6d320b56c@DEISMMXS02.pt.local> Hi thanks for the update/info. I had a quick look at the changes. So, here some thoughts... As described in JDK-8200758, and therefore expected, WinExeBundler has been removed in favor of putting focus on WinMsiBundler. (Although, I regret that decision - since my personal experience has been that InnoSetup based WinExeBundler has worked much better than wix based WinMsiBundler for our use cases - I can live with that.) What is much more disturbing: WinServiceBundler has also been actively removed, although that was working fine together with both wix/msi and exe/iss. Why has service wrapping been removed as well, while the command line option for it is kept in place? Is there any chance of service bundler coming back into scope of JDK-8200758 or coming back in at all? Cheers J?rg -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Kevin Rushforth Sent: Freitag, 6. Juli 2018 22:14 To: core-libs-dev at openjdk.java.net Cc: Alexey Semenyuk ; Andy Herrick Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] An initial prototype of the jpackager tool has been pushed to a new 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is interested in taking a look, you can clone it as follows: ??? hg clone https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=jWWENz_KIkmyh-9-kQQvoZ0BwBymwQ-BKx8hG3F5Iy0&e= ??? cd ./sandbox ??? hg update JDK-8200758-branch I plan to reply to the feedback already provided, and update the JEP [2] next week some time, but in the mean time if you have additional questions or comments, feel free to reply. -- Kevin [1] https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox_shortlog_JDK-2D8200758-2Dbranch&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=F-CqPAWlz-Cfb0kae2FBEj4Ncd3ZBVu7BeOVY1AM-cA&e= [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8200758&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=DFIAHtCR1o--KMLuBzurIzx5MDu67NgtUrEdQ22wI9I&e= On 6/27/2018 3:30 PM, Kevin Rushforth wrote: > We're aiming to get this into JDK 12 early enough so that an EA build > would be available around the time JDK 11 ships. That will allow you > to take a jlinked image with JDK 11 and package it up using (the EA) > jpackager. > > We will create a development branch in the JDK sandbox [1] some time > in the next week or so so you can follow the development. > > Also, thank you to those who have provided feedback. I'll reply to > feedback soon and then incorporate it into an updated JEP. > > -- Kevin From sean.coffey at oracle.com Tue Jul 10 12:50:42 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Tue, 10 Jul 2018 13:50:42 +0100 Subject: RFR: 8148188: Enhance the security libraries to record events of interest In-Reply-To: <00051b46-46a7-5c96-f36b-fb2e2fed8a58@oracle.com> References: <822f8eb2-95b6-410d-6504-ce52d9154ed0@oracle.com> <5B2F9ADD.1040809@oracle.com> <03d38c51-cd81-3c91-b4e0-d380fad92725@oracle.com> <02b1a436-a496-f3e2-8e04-a82e841eda81@oracle.com> <1d287e4c-194c-5684-cd65-8d6b5a2f2047@oracle.com> <5B33EC28.6090604@oracle.com> <5B350AE5.3080401@oracle.com> <4F777EA1-6402-4725-B4E8-6DCB61DD4E19@oracle.com> <5B438B82.6070601@oracle.com> <00051b46-46a7-5c96-f36b-fb2e2fed8a58@oracle.com> Message-ID: Erik, After some trial edits, I'm not so sure if moving the event & logger commit code into the class where it's used works too well after all. In the code you suggested, there's an assumption that calls such as EventHelper.certificateChain(..) are low cost. While that might be the case here, it's something we can't always assume. It's called once for the JFR operation and once for the Logger operation. That pattern multiplies itself further in other Events. i.e. set up and assign the variables for a JFR event without knowing whether they'll be needed again for the Logger call. We could work around it by setting up some local variables and re-using them in the Logger code but then, we're back to where we were. The current EventHelper design eliminates this problem and also helps to abstract the recording/logging functionality away from the functionality of the class itself. It also becomes a problem where we record events in multiple different classes (e.g. SecurityPropertyEvent). While we could leave the code in EventHelper for this case, we then have a mixed design pattern. Are you ok with leaving as is for now? A future wish might be one where JFR would handle Logger via its own framework/API in a future JDK release. I've removed the variable names using underscore. Also optimized some variable assignments in X509Impl.commitEvent(..) http://cr.openjdk.java.net/~coffeys/webrev.8148188.v5/webrev/ regards, Sean. On 09/07/2018 18:01, Se?n Coffey wrote: > Erik, > > Thanks for reviewing. Comments inline.. > > On 09/07/18 17:21, Erik Gahlin wrote: >> Thanks Sean. >> >> Some feedback on the code in the security libraries. >> >> - We should use camel case naming convention for variables (not >> underscore). > Sure. I see two offending variable names which I'll fix up. >> >> - Looking at sun/security/ssl/Finished.java, >> >> I wonder if it wouldn't be less code and more easy to read, if we >> would commit the event in a local private function and then use the >> EventHelper class for common functionality. > I'm fine with your suggested approach. I figured that tucking the > recording/logging logic away in a helper class also had benefits but > I'll re-factor to your suggested style unless I hear objections. > > regards, > Sean. > From mandy.chung at oracle.com Tue Jul 10 17:11:05 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 10 Jul 2018 10:11:05 -0700 Subject: WeakReference with null referent In-Reply-To: <1331e58f-1257-bc5e-9ca1-212b4488e413@gmail.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> <1331e58f-1257-bc5e-9ca1-212b4488e413@gmail.com> Message-ID: <8b975afb-621e-9202-3814-474be357b1b5@oracle.com> On 7/9/18 11:59 PM, Peter Levart wrote: > > There are situations where being able to create a XxxReference without a > referent is useful. For example, imagine a doubly-linked list of > WeakReference subclasses - a special instance without a referent can be > used as the head of such list that never goes away and simplifies > insertion/deletion logic, etc... > > JDK's java.lang.ref.Cleaner API uses such scheme in its implementation > (see jdk.internal.ref.CleanerImpl.[Phantom|Weak|Soft]CleanableRef no-arg > constructors). Thanks for pointing this out. It's used as a convenience and it's not hard to handle insertion/deletion logic though. Another alternative may be to create an object as the strongly reachable referent. I wonder if any existing code depends on null referent like the cleaner case and the compatibility risk. Do you know any other example? Mandy From brent.christian at oracle.com Tue Jul 10 21:02:38 2018 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 10 Jul 2018 14:02:38 -0700 Subject: RFR: 8207005 : Disable the file canonicalization cache by default Message-ID: <6018a39f-2ea7-849d-4daf-2830d1773cd5@oracle.com> Hi, Please review this change to disable the file canonicalization cache by default. Bug: https://bugs.openjdk.java.net/browse/JDK-8207005 From the bug report: "The file canonicalization cache was added back in JDK 1.4.2 in order to improve startup time. The cache has long-standing correctness issues (JDK-7066948, for example). For this reason, it has been desired to disable the cache by default, but the increase to startup time was undesirable. Recent JDK releases have removed usages of the cache, in particular from FilePermission (JDK-8164705 in JDK 9). This reduced usage of the canonicalization cache should also reduce the startup effect of disabling the cache by default. Measurements support this. Previous measurements showed the startup effect of disabling the cache on Linux to be 3-6%, depending on the specific benchmark. The same comparison performed last month now shows no startup change on Linux. The file canonicalization cache can still be enabled by setting the "sun.io.useCanonCaches" system property. This is merely a change to the default value." Here is the diff: --- a/src/java.base/share/classes/java/io/FileSystem.java @@ -231,8 +231,8 @@ // Flags for enabling/disabling performance optimizations for file // name canonicalization - static boolean useCanonCaches = true; - static boolean useCanonPrefixCache = true; + static boolean useCanonCaches = false; + static boolean useCanonPrefixCache = false; private static boolean getBooleanProperty(String prop, boolean defaultVal) { -- Thanks, -Brent From paul.sandoz at oracle.com Tue Jul 10 22:43:03 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 10 Jul 2018 15:43:03 -0700 Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods Message-ID: Hi, Please review this fix to MethodHandleProxies.asInterfaceInstance to support default methods: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8206955-mh-func-iface-proxy-default-methods/webrev/ It probably requires a CSR, which i shall do after this review. Thanks, Paul. From mandy.chung at oracle.com Tue Jul 10 23:03:25 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 10 Jul 2018 16:03:25 -0700 Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: References: Message-ID: This patch looks good to me. Mandy On 7/10/18 3:43 PM, Paul Sandoz wrote: > Hi, > > Please review this fix to MethodHandleProxies.asInterfaceInstance to > support default methods: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8206955-mh-func-iface-proxy-default-methods/webrev/ > > It probably requires a CSR, which i shall do after this review. > > Thanks, Paul. > From emcmanus at google.com Tue Jul 10 23:04:54 2018 From: emcmanus at google.com (=?UTF-8?Q?=C3=89amonn_McManus?=) Date: Tue, 10 Jul 2018 16:04:54 -0700 Subject: WeakReference with null referent In-Reply-To: <8b975afb-621e-9202-3814-474be357b1b5@oracle.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> <1331e58f-1257-bc5e-9ca1-212b4488e413@gmail.com> <8b975afb-621e-9202-3814-474be357b1b5@oracle.com> Message-ID: > Do you know any other example? Here's an example from the JDK itself: http://hg.openjdk.java.net/jdk/jdk11/file/2a89e80301b1/src/java.desktop/share/classes/sun/font/Type1Font.java#l101 On Tue, 10 Jul 2018 at 10:11, mandy chung wrote: > > > > On 7/9/18 11:59 PM, Peter Levart wrote: > > > > > There are situations where being able to create a XxxReference without a > > referent is useful. For example, imagine a doubly-linked list of > > WeakReference subclasses - a special instance without a referent can be > > used as the head of such list that never goes away and simplifies > > insertion/deletion logic, etc... > > > > JDK's java.lang.ref.Cleaner API uses such scheme in its implementation > > (see jdk.internal.ref.CleanerImpl.[Phantom|Weak|Soft]CleanableRef no-arg > > constructors). > > Thanks for pointing this out. It's used as a convenience and it's > not hard to handle insertion/deletion logic though. Another > alternative may be to create an object as the strongly reachable > referent. I wonder if any existing code depends on null referent > like the cleaner case and the compatibility risk. Do you know > any other example? > > Mandy From mandy.chung at oracle.com Tue Jul 10 23:26:48 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 10 Jul 2018 16:26:48 -0700 Subject: WeakReference with null referent In-Reply-To: References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> <1331e58f-1257-bc5e-9ca1-212b4488e413@gmail.com> <8b975afb-621e-9202-3814-474be357b1b5@oracle.com> Message-ID: <845cf29b-58ba-09a4-2386-89cfeaea72b7@oracle.com> On 7/10/18 4:04 PM, ?amonn McManus wrote: >> Do you know any other example? > > Here's an example from the JDK itself: > http://hg.openjdk.java.net/jdk/jdk11/file/2a89e80301b1/src/java.desktop/share/classes/sun/font/Type1Font.java#l101 Thanks Eamonn. I also found a few other similar usages in JDK to initialize with an "empty" WeakReference field that may be updated with a weak ref with a referent at runtime so that the code can always assume this field is non-null. Due to the current usage and compatibility risk, not worth to enforce non-null referent. Mandy > On Tue, 10 Jul 2018 at 10:11, mandy chung wrote: >> >> >> >> On 7/9/18 11:59 PM, Peter Levart wrote: >> >>> >>> There are situations where being able to create a XxxReference without a >>> referent is useful. For example, imagine a doubly-linked list of >>> WeakReference subclasses - a special instance without a referent can be >>> used as the head of such list that never goes away and simplifies >>> insertion/deletion logic, etc... >>> >>> JDK's java.lang.ref.Cleaner API uses such scheme in its implementation >>> (see jdk.internal.ref.CleanerImpl.[Phantom|Weak|Soft]CleanableRef no-arg >>> constructors). >> >> Thanks for pointing this out. It's used as a convenience and it's >> not hard to handle insertion/deletion logic though. Another >> alternative may be to create an object as the strongly reachable >> referent. I wonder if any existing code depends on null referent >> like the cleaner case and the compatibility risk. Do you know >> any other example? >> >> Mandy From huizhe.wang at oracle.com Tue Jul 10 23:27:51 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 10 Jul 2018 16:27:51 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8194680: StartElement#getAttributes and getNamespaces refer to incorrect package Message-ID: <5B4540F7.5080107@oracle.com> Hi, Please review a javadoc fix for a couple of incorrect references to the javax.xml.stream package. Attribute and Namespace are both in the javax.xml.stream.events, and so is StartElement where they are referenced. JBS: https://bugs.openjdk.java.net/browse/JDK-8194680 webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8194680/webrev/ Thanks, Joe From jonathan.gibbons at oracle.com Tue Jul 10 23:39:37 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 10 Jul 2018 16:39:37 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8194680: StartElement#getAttributes and getNamespaces refer to incorrect package In-Reply-To: <5B4540F7.5080107@oracle.com> References: <5B4540F7.5080107@oracle.com> Message-ID: <52d2d7da-0d0c-f22c-381d-98d5054a9181@oracle.com> On 7/10/18 4:27 PM, Joe Wang wrote: > Hi, > > Please review a javadoc fix for a couple of incorrect references to > the javax.xml.stream package. Attribute and Namespace are both in the > javax.xml.stream.events, and so is StartElement where they are > referenced. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8194680 > webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8194680/webrev/ > > Thanks, > Joe The following text does not read well: 54 * Attributes are fundamentally unordered and may not be reported 55 * in any order. It would read better if it said: 54 * Attributes are fundamentally unordered and may be reported 55 * in any order. (i.e. delete "not") -- Jon From ivan.gerasimov at oracle.com Wed Jul 11 02:33:50 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 10 Jul 2018 19:33:50 -0700 Subject: RFR 8207016 : [win] Avoid redundant native memory allocation in getFinalPath() Message-ID: Hello! File src/java.base/windows/native/libjava/WinNTFileSystem_md.c The function getFinalPath() is wrapper for GetFinalPathNameByHandleW(), which is used to resolve the symlinks on Windows. If the resulting string begins with a prefix "\\?\" or "\\?\UNC", a copy without the prefix is created and returned. Instead of creating the copy, we can override the result with memmove() to avoid allocating extra native memory. The result of getFinalPath() is short living anyway, so a few unused bytes at the end of the buffer should not be a problem. The advantage is a shorter code and minus one point of possible failure. Would you please help review it? BUG: https://bugs.openjdk.java.net/browse/JDK-8207016 Webrev: http://cr.openjdk.java.net/~igerasim/8207016/00/webrev/ Thanks in advance! -- With kind regards, Ivan Gerasimov From huizhe.wang at oracle.com Wed Jul 11 03:10:16 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 10 Jul 2018 20:10:16 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8194680: StartElement#getAttributes and getNamespaces refer to incorrect package In-Reply-To: <52d2d7da-0d0c-f22c-381d-98d5054a9181@oracle.com> References: <5B4540F7.5080107@oracle.com> <52d2d7da-0d0c-f22c-381d-98d5054a9181@oracle.com> Message-ID: <5B457518.2010108@oracle.com> On 7/10/18, 4:39 PM, Jonathan Gibbons wrote: > > > > On 7/10/18 4:27 PM, Joe Wang wrote: >> Hi, >> >> Please review a javadoc fix for a couple of incorrect references to >> the javax.xml.stream package. Attribute and Namespace are both in the >> javax.xml.stream.events, and so is StartElement where they are >> referenced. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8194680 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8194680/webrev/ >> >> Thanks, >> Joe > The following text does not read well: > 54 * Attributes are fundamentally unordered and may not be reported > 55 * in any order. > > It would read better if it said: > > 54 * Attributes are fundamentally unordered and may be reported > 55 * in any order. > > (i.e. delete "not") Done (deleted "not"). Thanks, Joe > > -- Jon From orionllmain at gmail.com Wed Jul 11 05:02:52 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Wed, 11 Jul 2018 12:02:52 +0700 Subject: WeakReference with null referent In-Reply-To: <845cf29b-58ba-09a4-2386-89cfeaea72b7@oracle.com> References: <57ac6c46-9ac7-c349-2511-0bb10d001768@oracle.com> <2d4057eb-7664-d0cf-5844-bbf976408966@oracle.com> <37cce314-5507-e8df-665d-c5797050fa16@oracle.com> <1331e58f-1257-bc5e-9ca1-212b4488e413@gmail.com> <8b975afb-621e-9202-3814-474be357b1b5@oracle.com> <845cf29b-58ba-09a4-2386-89cfeaea72b7@oracle.com> Message-ID: There are a few valid use cases where `new WeakReference<>(null)` makes some sense. So, we shouldn't forbid nulls. `new WeakReference<>(null, queue)` makes less sense but I bet someone is using this too for some reason. My initial message was not about changing the behavior but only about changing the JavaDoc. I just want to add a clear specification of what happens to references with null referents. 2018-07-11 6:26 GMT+07:00 mandy chung : > > > On 7/10/18 4:04 PM, ?amonn McManus wrote: > >> Do you know any other example? >>> >> >> Here's an example from the JDK itself: >> http://hg.openjdk.java.net/jdk/jdk11/file/2a89e80301b1/src/ >> java.desktop/share/classes/sun/font/Type1Font.java#l101 >> > > Thanks Eamonn. > > I also found a few other similar usages in JDK to initialize > with an "empty" WeakReference field that may be updated with > a weak ref with a referent at runtime so that the code can > always assume this field is non-null. > > Due to the current usage and compatibility risk, not worth > to enforce non-null referent. > > Mandy > > > On Tue, 10 Jul 2018 at 10:11, mandy chung wrote: >> >>> >>> >>> >>> On 7/9/18 11:59 PM, Peter Levart wrote: >>> >>> >>>> There are situations where being able to create a XxxReference without a >>>> referent is useful. For example, imagine a doubly-linked list of >>>> WeakReference subclasses - a special instance without a referent can be >>>> used as the head of such list that never goes away and simplifies >>>> insertion/deletion logic, etc... >>>> >>>> JDK's java.lang.ref.Cleaner API uses such scheme in its implementation >>>> (see jdk.internal.ref.CleanerImpl.[Phantom|Weak|Soft]CleanableRef >>>> no-arg >>>> constructors). >>>> >>> >>> Thanks for pointing this out. It's used as a convenience and it's >>> not hard to handle insertion/deletion logic though. Another >>> alternative may be to create an object as the strongly reachable >>> referent. I wonder if any existing code depends on null referent >>> like the cleaner case and the compatibility risk. Do you know >>> any other example? >>> >>> Mandy >>> >> From Alan.Bateman at oracle.com Wed Jul 11 11:12:26 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Jul 2018 12:12:26 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> Message-ID: <1c5c8c7c-d083-b2ef-a645-c3c3500e8aeb@oracle.com> On 10/07/2018 10:53, Baesken, Matthias wrote: > Hi Alan, thanks for commenting on this . > > Jaikiran mentioned that printing just the jar file name and not file with path might be okay : > >> I am not a reviewer and neither do I have enough knowledge about whether >> jar/file _names_ are considered security sensitive. However, the patch >> that's proposed for this change, prints the file _path_ (and not just >> the name). That I believe is security sensitive. > What do you think ? > In the ZipFile API, the "name" may include path information but if you strip that and include just the file name then it should be okay. A useful way to think about is the information revealed when a HTTP response serves up a tasty stack trace. -Alan. From adam.farley at uk.ibm.com Wed Jul 11 13:27:32 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 11 Jul 2018 14:27:32 +0100 Subject: RFR (Unraised): JDK8 ResourceBundle vulnerable to GC Message-ID: Hi All, -- Summary: When calling "ResourceBundle.getBundle(String, Locale, ClassLoader)" on JDK8, the ClassLoader can get GC'd before we're finished with it. This can result in us getting the wrong result back, like if we asked for "Stuff" with the locale "fr, CA" and got back "Stuff_fr.class" instead of "Stuff_fr_CA.class". -- Likely Cause: Inside the CacheKey object in the getBundleImpl method, we use a soft reference that can result in the ClassLoader object getting GC'd before we're done with it. -- Example Fix: One fix proven to work is to create a static, volatile Object with global scope in ResourceBundle, and compare it ("==" style) to the loader right before it gets returned from getBundleImpl. The result of the compare can be something direct like "throw new Error("Unexpected error.")", as this compare should never return true. -- Only seen on... This defect is only seen on JDK8, on ppcle and zLinux, and hasn't yet been seen using the Hotspot VM (defect found using OpenJ9 VM). Best Regards Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From peter.levart at gmail.com Wed Jul 11 15:15:09 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 11 Jul 2018 17:15:09 +0200 Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: References: Message-ID: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> Hi Paul, The patch looks ok. I hope IMPL_LOOKUP has access to all methods (even if located in package-private interfaces and/or in concealed packages of modules)? Just a thought... Would it be possible to implement this API in terms of LambdaMetafactory ? Regards, Peter On 07/11/2018 12:43 AM, Paul Sandoz wrote: > Hi, > > Please review this fix to MethodHandleProxies.asInterfaceInstance to support default methods: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8206955-mh-func-iface-proxy-default-methods/webrev/ > > It probably requires a CSR, which i shall do after this review. > > Thanks, > Paul. From forax at univ-mlv.fr Wed Jul 11 15:31:09 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 11 Jul 2018 17:31:09 +0200 (CEST) Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> References: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> Message-ID: <1584272678.214335.1531323069357.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Peter Levart" > ?: "Paul Sandoz" , "core-libs-dev" > Envoy?: Mercredi 11 Juillet 2018 17:15:09 > Objet: Re: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods > Hi Paul, > > The patch looks ok. I hope IMPL_LOOKUP has access to all methods (even > if located in package-private interfaces and/or in concealed packages of > modules)? > > Just a thought... Would it be possible to implement this API in terms of > LambdaMetafactory ? > > Regards, Peter Hi Peter, not with the current LambdaMetaFactory, the LambdaMetaFactory only accept some kind of method handles (constant method calls) not all kind of method handles. That said the current implementation of MethodHandleProxies is very raw and not very efficient, we should use the same idea as the lambda meta factory i.e spin an anonymous class and use the mechanism of constant patching offer by unsafe.defineAnonymousClass to inject the method handle into proxy so it will work with any method handle. For each interface, you should cache the bytecode of the anonymous class you want to load and use defineAnonymousClass with the method handle each time asInterfaceInstance is called. cheers, R?mi > > > On 07/11/2018 12:43 AM, Paul Sandoz wrote: >> Hi, >> >> Please review this fix to MethodHandleProxies.asInterfaceInstance to support >> default methods: >> >> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8206955-mh-func-iface-proxy-default-methods/webrev/ >> >> >> It probably requires a CSR, which i shall do after this review. >> >> Thanks, > > Paul. From Alan.Bateman at oracle.com Wed Jul 11 15:52:28 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Jul 2018 16:52:28 +0100 Subject: RFR: 8207005 : Disable the file canonicalization cache by default In-Reply-To: <6018a39f-2ea7-849d-4daf-2830d1773cd5@oracle.com> References: <6018a39f-2ea7-849d-4daf-2830d1773cd5@oracle.com> Message-ID: <499bbe18-18cb-f29a-9122-5013c2c27efa@oracle.com> On 10/07/2018 22:02, Brent Christian wrote: > Hi, > > Please review this change to disable the file canonicalization cache > by default. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8207005 > > From the bug report: > "The file canonicalization cache was added back in JDK 1.4.2 in order > to improve startup time. > > The cache has long-standing correctness issues (JDK-7066948, for > example). For this reason, it has been desired to disable the cache by > default, but the increase to startup time was undesirable. > > Recent JDK releases have removed usages of the cache, in particular > from FilePermission (JDK-8164705 in JDK 9). This reduced usage of the > canonicalization cache should also reduce the startup effect of > disabling the cache by default. Measurements support this. Previous > measurements showed the startup effect of disabling the cache on Linux > to be 3-6%, depending on the specific benchmark. The same comparison > performed last month now shows no startup change on Linux. > > The file canonicalization cache can still be enabled by setting the > "sun.io.useCanonCaches" system property. This is merely a change to > the default value." Happy to see this disabled as it has always been problematic. The patch looks okay although you don't of couse need to set the initial values to false. -Alan From ivan.gerasimov at oracle.com Wed Jul 11 16:15:13 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 11 Jul 2018 09:15:13 -0700 Subject: RFR 8207060 : Memory leak when malloc fails within WITH_UNICODE_STRING block Message-ID: <857818a6-a289-6e02-e576-dd9adb3c3ec0@oracle.com> Hello! File src/java.base/windows/native/libjava/io_util_md.c In the function pathToNTPath(), memory is allocated with malloc() within a block of macros WITH_UNICODE_STRING / END_UNICODE_STRING. In an unlikely event of malloc() failure, the function returns, failing to release the string array acquired with WITH_UNICODE_STRING. Would you please help review the fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207060 WEBREV: http://cr.openjdk.java.net/~igerasim/8207060/00/webrev/ Thanks in advance! -- With kind regards, Ivan Gerasimov From peter.levart at gmail.com Wed Jul 11 16:32:13 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 11 Jul 2018 18:32:13 +0200 Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: <1584272678.214335.1531323069357.JavaMail.zimbra@u-pem.fr> References: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> <1584272678.214335.1531323069357.JavaMail.zimbra@u-pem.fr> Message-ID: Sorry Paul for hijacking the thread, just answering to Remi ... On 07/11/2018 05:31 PM, Remi Forax wrote: > ----- Mail original ----- >> De: "Peter Levart" >> ?: "Paul Sandoz" , "core-libs-dev" >> Envoy?: Mercredi 11 Juillet 2018 17:15:09 >> Objet: Re: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods >> Hi Paul, >> >> The patch looks ok. I hope IMPL_LOOKUP has access to all methods (even >> if located in package-private interfaces and/or in concealed packages of >> modules)? >> >> Just a thought... Would it be possible to implement this API in terms of >> LambdaMetafactory ? >> >> Regards, Peter > Hi Peter, > not with the current LambdaMetaFactory, the LambdaMetaFactory only accept some kind of method handles (constant method calls) not all kind of method handles. > > That said the current implementation of MethodHandleProxies is very raw and not very efficient, we should use the same idea as the lambda meta factory i.e spin an anonymous class and use the mechanism of constant patching offer by unsafe.defineAnonymousClass to inject the method handle into proxy so it will work with any method handle. > > For each interface, you should cache the bytecode of the anonymous class you want to load and use defineAnonymousClass with the method handle each time asInterfaceInstance is called. If the generated class used invokeExact on the method handle, bytecode should be generated specifically for each tuple (interface type, method handle type), as the needed conversions of arguments/return values would be specific for each distinct combination of the two types. ...which would still mean that you would define new anonymous class for each method handle instance, just the bytecodes would be generated once per (interface type, method handle type) combination. The method handle could then be constant-folded in the generated class, but selection of the underlying proxy class would still be governed by the proxy instance which would be invoked via the interface method on the functional interface. If the proxy instance could not be constant-folded (i.e. was not assigned to static final field and used from it), the combined invocation performance would still not be the same as using invokeExact on the constant method handle, would it? So perhaps for this API it is more suitable to: - define the specific proxy class once per (interface type, method handle type) combination (and cache the class itself, not just bytecode) - have that proxy class implement a constructor taking the method handle and assign it to a @Stable instance field - implement the single interface method as parameter/return value conversions around invokeExact on the method handle taken from @Stable instance field If such proxy instance was constant-folded, so would be the @Stable method handle field, right? What do you think of this strategy? Regards, Peter > > cheers, > R?mi > >> >> On 07/11/2018 12:43 AM, Paul Sandoz wrote: >>> Hi, >>> >>> Please review this fix to MethodHandleProxies.asInterfaceInstance to support >>> default methods: >>> >>> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8206955-mh-func-iface-proxy-default-methods/webrev/ >>> >>> >>> It probably requires a CSR, which i shall do after this review. >>> >>> Thanks, >>> Paul. From brian.burkhalter at oracle.com Wed Jul 11 16:33:17 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 11 Jul 2018 09:33:17 -0700 Subject: RFR: 8207005 : Disable the file canonicalization cache by default In-Reply-To: <6018a39f-2ea7-849d-4daf-2830d1773cd5@oracle.com> References: <6018a39f-2ea7-849d-4daf-2830d1773cd5@oracle.com> Message-ID: <30A1F1BB-6077-43AD-9F0C-033185D21F19@oracle.com> +1 Brian On Jul 11, 2018, at 8:52 AM, Alan Bateman wrote: > Happy to see this disabled as it has always been problematic. The patch looks okay although you don't of couse need to set the initial values to false. On Jul 10, 2018, at 2:02 PM, Brent Christian wrote: > --- a/src/java.base/share/classes/java/io/FileSystem.java > @@ -231,8 +231,8 @@ > > // Flags for enabling/disabling performance optimizations for file > // name canonicalization > - static boolean useCanonCaches = true; > - static boolean useCanonPrefixCache = true; > + static boolean useCanonCaches = false; > + static boolean useCanonPrefixCache = false; From brent.christian at oracle.com Wed Jul 11 17:29:35 2018 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 11 Jul 2018 10:29:35 -0700 Subject: RFR: 8207005 : Disable the file canonicalization cache by default In-Reply-To: <499bbe18-18cb-f29a-9122-5013c2c27efa@oracle.com> References: <6018a39f-2ea7-849d-4daf-2830d1773cd5@oracle.com> <499bbe18-18cb-f29a-9122-5013c2c27efa@oracle.com> Message-ID: <8dbc34dc-ae70-b4b5-a8a6-b6dc42616a13@oracle.com> On 07/11/2018 08:52 AM, Alan Bateman wrote: > > Happy to see this disabled as it has always been problematic. The patch > looks okay although you don't of couse need to set the initial values to > false. OK, yeah, I'll omit that. Thanks, Alan and Brian. -Brent From naoto.sato at oracle.com Wed Jul 11 17:42:58 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 11 Jul 2018 10:42:58 -0700 Subject: [11] RFR: 8206980: DateTimeFormatter throws parsing a valid string depending on the locale Message-ID: <9834dcbe-d8e8-2469-0743-c5e07f66c173@oracle.com> Hello, Please review the changes to the following issue: https://bugs.openjdk.java.net/browse/JDK-8206980 The proposed fix is located at: http://cr.openjdk.java.net/~naoto/8206980/webrev.00/ CLDR time zone names in "en" bundle are expected to be pre-populated for performance. This is not the case for locales that have "en-001" (World English) as the parent bundle, such as Locale.CANADA. The fast path determining logic was incorrectly assumed all English locales are using pre-populated bundles. Limiting the assumption to only "en" and "en-US" will solve the problem. Naoto From lance.andersen at oracle.com Wed Jul 11 17:49:22 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 11 Jul 2018 13:49:22 -0400 Subject: RFR(JDK12/JAXP/java.xml) 8194680: StartElement#getAttributes and getNamespaces refer to incorrect package In-Reply-To: <5B457518.2010108@oracle.com> References: <5B4540F7.5080107@oracle.com> <52d2d7da-0d0c-f22c-381d-98d5054a9181@oracle.com> <5B457518.2010108@oracle.com> Message-ID: Hi Joe, The updates like fine > On Jul 10, 2018, at 11:10 PM, Joe Wang wrote: > > > > On 7/10/18, 4:39 PM, Jonathan Gibbons wrote: >> >> >> >> On 7/10/18 4:27 PM, Joe Wang wrote: >>> Hi, >>> >>> Please review a javadoc fix for a couple of incorrect references to the javax.xml.stream package. Attribute and Namespace are both in the javax.xml.stream.events, and so is StartElement where they are referenced. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8194680 >>> webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8194680/webrev/ >>> >>> Thanks, >>> Joe >> The following text does not read well: >> 54 * Attributes are fundamentally unordered and may not be reported >> 55 * in any order. >> >> It would read better if it said: >> >> 54 * Attributes are fundamentally unordered and may be reported >> 55 * in any order. >> >> (i.e. delete "not") > > Done (deleted "not"). > > Thanks, > Joe > >> >> -- Jon Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From paul.sandoz at oracle.com Wed Jul 11 17:53:48 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 11 Jul 2018 10:53:48 -0700 Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> References: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> Message-ID: Hi Peter, > On Jul 11, 2018, at 8:15 AM, Peter Levart wrote: > > Hi Paul, > > The patch looks ok. Thanks. > I hope IMPL_LOOKUP has access to all methods (even if located in package-private interfaces and/or in concealed packages of modules)? > I think it should be ok, because IMPL_LOOKUP is used it does not matter what is passed as the specialCaller since IMPL_LOOK is trusted and a full power lookup will be given for Lookup.in. (FWIW because this code is in j.l.invoke it is currently not possible to instantiate a lookup). Since access control is gated by the invokevirtual to the interface method it should be ok to use the full powerful lookup. > Just a thought... Would it be possible to implement this API in terms of LambdaMetafactory ? > It?s definitely possible to do better spinning an explicit proxy class rather the current proxy class deferring to an invocation handler. My focus with this fix is to make it functional without unduly increasing the cost of when default methods are not invoked, and not focus on the performance right now. There are some subtle differences between functional interfaces and this functionality which was introduced prior to the latter e.g. it's possible to explicitly declare overloaded abstract methods whose return/parameter types are compatible with the target type. Paul. > Regards, Peter > > > On 07/11/2018 12:43 AM, Paul Sandoz wrote: >> Hi, >> >> Please review this fix to MethodHandleProxies.asInterfaceInstance to support default methods: >> >> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8206955-mh-func-iface-proxy-default-methods/webrev/ >> >> It probably requires a CSR, which i shall do after this review. >> >> Thanks, >> Paul. > From john.r.rose at oracle.com Wed Jul 11 19:07:37 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 11 Jul 2018 12:07:37 -0700 Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: References: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> <1584272678.214335.1531323069357.JavaMail.zimbra@u-pem.fr> Message-ID: On Jul 11, 2018, at 9:32 AM, Peter Levart wrote: > > Sorry Paul for hijacking the thread, just answering to Remi ... > > On 07/11/2018 05:31 PM, Remi Forax wrote: >> ----- Mail original ----- >>> De: "Peter Levart" >>> ?: "Paul Sandoz" , "core-libs-dev" >>> Envoy?: Mercredi 11 Juillet 2018 17:15:09 >>> Objet: Re: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods >>> Hi Paul, >>> >>> The patch looks ok. I hope IMPL_LOOKUP has access to all methods (even >>> if located in package-private interfaces and/or in concealed packages of >>> modules)? >>> >>> Just a thought... Would it be possible to implement this API in terms of >>> LambdaMetafactory ? >>> >>> Regards, Peter >> Hi Peter, >> not with the current LambdaMetaFactory, the LambdaMetaFactory only accept some kind of method handles (constant method calls) not all kind of method handles. >> >> That said the current implementation of MethodHandleProxies is very raw and not very efficient, we should use the same idea as the lambda meta factory i.e spin an anonymous class and use the mechanism of constant patching offer by unsafe.defineAnonymousClass to inject the method handle into proxy so it will work with any method handle. >> >> For each interface, you should cache the bytecode of the anonymous class you want to load and use defineAnonymousClass with the method handle each time asInterfaceInstance is called. > > If the generated class used invokeExact on the method handle, bytecode should be generated specifically for each tuple (interface type, method handle type), as the needed conversions of arguments/return values would be specific for each distinct combination of the two types. > > ...which would still mean that you would define new anonymous class for each method handle instance, just the bytecodes would be generated once per (interface type, method handle type) combination. > > The method handle could then be constant-folded in the generated class, but selection of the underlying proxy class would still be governed by the proxy instance which would be invoked via the interface method on the functional interface. If the proxy instance could not be constant-folded (i.e. was not assigned to static final field and used from it), the combined invocation performance would still not be the same as using invokeExact on the constant method handle, would it? > > So perhaps for this API it is more suitable to: > > - define the specific proxy class once per (interface type, method handle type) combination (and cache the class itself, not just bytecode) > - have that proxy class implement a constructor taking the method handle and assign it to a @Stable instance field > - implement the single interface method as parameter/return value conversions around invokeExact on the method handle taken from @Stable instance field > > If such proxy instance was constant-folded, so would be the @Stable method handle field, right? > > What do you think of this strategy? Here's my $0.02: - Performance does not seem to be important yet for MHP; Paul is mainly concerned with functionality. - But such implementation strategies are desirable for performance. - The ClassSpecializer mechanism is a good candidate for managing the tuple species (it was factored from BMH). - I think a reasonable balance would be for a MHP instance to factor into (a) a @Stable MH (or List::of), and (b) a @Stable tuple. Downsides of factoring MHP into MH and tuple sub-objects: - Three heap nodes instead of the minimum single node. - Indirection costs. (Note that constants fold through @Stable links including List::of or stable-array.) Upsides: - Fewer generated classes: One per interface, one per payload tuple, one per MH. - Possible shared code with publicly-visible tuple carrier API (probably needed for pattern "extractors"). More ideas: An alternative is to do a full-custom species for every distinct MHP combination. This might be OK, until people start to use MHP in earnest; then it will cause startup problems IMO. Factoring the MHP nodes makes the thing scale better in terms of static code, at a modest cost in heap flatness. If (long-term if) we want to flatten the MPH nodes, but only for hot-path MHPs (assuming in the long term a wide variety of usage), then we might try Vladimir's MH customization hack, which puts invocation counters on MH transform chains, and upgrades them in-place to use customized LFs if they get hot. Something like this could be done with MHP combos that turn hot. Perhaps existing MHPs that get hot could even be internally converted to delegate by one patched hop to a flattened version of the object, and new MHPs of the same syndrome would just generate the flattened version from the start, sans secret indirection. ? John From forax at univ-mlv.fr Wed Jul 11 20:32:25 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 11 Jul 2018 22:32:25 +0200 (CEST) Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: References: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> <1584272678.214335.1531323069357.JavaMail.zimbra@u-pem.fr> Message-ID: <1171296277.261154.1531341145966.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Peter Levart" > ?: "Remi Forax" > Cc: "Paul Sandoz" , "core-libs-dev" > Envoy?: Mercredi 11 Juillet 2018 18:32:13 > Objet: Re: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods > Sorry Paul for hijacking the thread, just answering to Remi ... > > On 07/11/2018 05:31 PM, Remi Forax wrote: >> ----- Mail original ----- >>> De: "Peter Levart" >>> ?: "Paul Sandoz" , "core-libs-dev" >>> >>> Envoy?: Mercredi 11 Juillet 2018 17:15:09 >>> Objet: Re: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support >>> default methods >>> Hi Paul, >>> >>> The patch looks ok. I hope IMPL_LOOKUP has access to all methods (even >>> if located in package-private interfaces and/or in concealed packages of >>> modules)? >>> >>> Just a thought... Would it be possible to implement this API in terms of >>> LambdaMetafactory ? >>> >>> Regards, Peter >> Hi Peter, >> not with the current LambdaMetaFactory, the LambdaMetaFactory only accept some >> kind of method handles (constant method calls) not all kind of method handles. >> >> That said the current implementation of MethodHandleProxies is very raw and not >> very efficient, we should use the same idea as the lambda meta factory i.e spin >> an anonymous class and use the mechanism of constant patching offer by >> unsafe.defineAnonymousClass to inject the method handle into proxy so it will >> work with any method handle. >> >> For each interface, you should cache the bytecode of the anonymous class you >> want to load and use defineAnonymousClass with the method handle each time >> asInterfaceInstance is called. > > If the generated class used invokeExact on the method handle, bytecode > should be generated specifically for each tuple (interface type, method > handle type), as the needed conversions of arguments/return values would > be specific for each distinct combination of the two types. > > ...which would still mean that you would define new anonymous class for > each method handle instance, just the bytecodes would be generated once > per (interface type, method handle type) combination. I was thinking to still collect the arguments in an array as this is done now, to only have one anonymous class by interface type. > > The method handle could then be constant-folded in the generated class, > but selection of the underlying proxy class would still be governed by > the proxy instance which would be invoked via the interface method on > the functional interface. If the proxy instance could not be > constant-folded (i.e. was not assigned to static final field and used > from it), the combined invocation performance would still not be the > same as using invokeExact on the constant method handle, would it? > > So perhaps for this API it is more suitable to: > > - define the specific proxy class once per (interface type, method > handle type) combination (and cache the class itself, not just bytecode) > - have that proxy class implement a constructor taking the method handle > and assign it to a @Stable instance field > - implement the single interface method as parameter/return value > conversions around invokeExact on the method handle taken from @Stable > instance field > > If such proxy instance was constant-folded, so would be the @Stable > method handle field, right? If you use an anonymous class, you do not need @Stable because final fields of an anonymous class are trusted by the VM. The problem with storing the mh in a field is that proxies are rarely used as constant, so the mh will rarely be fully optimized, that why i've proposed to inject the mh as a constant pool constant. > > What do you think of this strategy? The problem is code sharing vs performance, if you share the proxy class, you end up with a non constant-foldable mh, if you don't share the proxy class, you delay the time to steady state. Currently the lambda metafactory do not do code sharing and it seems the right choice at least for Java programs, hence my proposal to consider the mh as a constant pool constant. > > Regards, Peter regards, R?mi > >> >> cheers, >> R?mi >> >>> >>> On 07/11/2018 12:43 AM, Paul Sandoz wrote: >>>> Hi, >>>> >>>> Please review this fix to MethodHandleProxies.asInterfaceInstance to support >>>> default methods: >>>> >>>> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8206955-mh-func-iface-proxy-default-methods/webrev/ >>>> >>>> >>>> It probably requires a CSR, which i shall do after this review. >>>> >>>> Thanks, > >>> Paul. From forax at univ-mlv.fr Wed Jul 11 20:48:57 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 11 Jul 2018 22:48:57 +0200 (CEST) Subject: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods In-Reply-To: References: <8b57f35f-dad9-9fcd-d155-295fd86109b6@gmail.com> <1584272678.214335.1531323069357.JavaMail.zimbra@u-pem.fr> Message-ID: <1069860948.262409.1531342137733.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "John Rose" > ?: "Peter Levart" > Cc: "Remi Forax" , "core-libs-dev" > Envoy?: Mercredi 11 Juillet 2018 21:07:37 > Objet: Re: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support default methods > On Jul 11, 2018, at 9:32 AM, Peter Levart wrote: >> >> Sorry Paul for hijacking the thread, just answering to Remi ... >> >> On 07/11/2018 05:31 PM, Remi Forax wrote: >>> ----- Mail original ----- >>>> De: "Peter Levart" >>>> ?: "Paul Sandoz" , "core-libs-dev" >>>> >>>> Envoy?: Mercredi 11 Juillet 2018 17:15:09 >>>> Objet: Re: RFR 8206955 MethodHandleProxies.asInterfaceInstance does not support >>>> default methods >>>> Hi Paul, >>>> >>>> The patch looks ok. I hope IMPL_LOOKUP has access to all methods (even >>>> if located in package-private interfaces and/or in concealed packages of >>>> modules)? >>>> >>>> Just a thought... Would it be possible to implement this API in terms of >>>> LambdaMetafactory ? >>>> >>>> Regards, Peter >>> Hi Peter, >>> not with the current LambdaMetaFactory, the LambdaMetaFactory only accept some >>> kind of method handles (constant method calls) not all kind of method handles. >>> >>> That said the current implementation of MethodHandleProxies is very raw and not >>> very efficient, we should use the same idea as the lambda meta factory i.e spin >>> an anonymous class and use the mechanism of constant patching offer by >>> unsafe.defineAnonymousClass to inject the method handle into proxy so it will >>> work with any method handle. >>> >>> For each interface, you should cache the bytecode of the anonymous class you >>> want to load and use defineAnonymousClass with the method handle each time >>> asInterfaceInstance is called. >> >> If the generated class used invokeExact on the method handle, bytecode should be >> generated specifically for each tuple (interface type, method handle type), as >> the needed conversions of arguments/return values would be specific for each >> distinct combination of the two types. >> >> ...which would still mean that you would define new anonymous class for each >> method handle instance, just the bytecodes would be generated once per >> (interface type, method handle type) combination. >> >> The method handle could then be constant-folded in the generated class, but >> selection of the underlying proxy class would still be governed by the proxy >> instance which would be invoked via the interface method on the functional >> interface. If the proxy instance could not be constant-folded (i.e. was not >> assigned to static final field and used from it), the combined invocation >> performance would still not be the same as using invokeExact on the constant >> method handle, would it? >> >> So perhaps for this API it is more suitable to: >> >> - define the specific proxy class once per (interface type, method handle type) >> combination (and cache the class itself, not just bytecode) >> - have that proxy class implement a constructor taking the method handle and >> assign it to a @Stable instance field >> - implement the single interface method as parameter/return value conversions >> around invokeExact on the method handle taken from @Stable instance field >> >> If such proxy instance was constant-folded, so would be the @Stable method >> handle field, right? >> >> What do you think of this strategy? > > > Here's my $0.02: > > - Performance does not seem to be important yet for MHP; Paul is mainly > concerned with functionality. > - But such implementation strategies are desirable for performance. > - The ClassSpecializer mechanism is a good candidate for managing the tuple > species (it was factored from BMH). > - I think a reasonable balance would be for a MHP instance to factor into (a) a > @Stable MH (or List::of), and (b) a @Stable tuple. > > Downsides of factoring MHP into MH and tuple sub-objects: > > - Three heap nodes instead of the minimum single node. > - Indirection costs. (Note that constants fold through @Stable links including > List::of or stable-array.) > > Upsides: > > - Fewer generated classes: One per interface, one per payload tuple, one per > MH. > - Possible shared code with publicly-visible tuple carrier API (probably needed > for pattern "extractors"). yes, but without value types, tuple carrier still do boxing, but only one boxing instead of one by arguments. > > More ideas: > > An alternative is to do a full-custom species for every distinct MHP > combination. This might be OK, > until people start to use MHP in earnest; then it will cause startup problems > IMO. Factoring the > MHP nodes makes the thing scale better in terms of static code, at a modest cost > in heap flatness. > > If (long-term if) we want to flatten the MPH nodes, but only for hot-path MHPs > (assuming in the long > term a wide variety of usage), then we might try Vladimir's MH customization > hack, which puts > invocation counters on MH transform chains, and upgrades them in-place to use > customized LFs > if they get hot. Something like this could be done with MHP combos that turn > hot. Perhaps existing > MHPs that get hot could even be internally converted to delegate by one patched > hop to a flattened > version of the object, and new MHPs of the same syndrome would just generate the > flattened > version from the start, sans secret indirection. What about my beloved fork-and-patch operator ? It takes an instance and a bytecode to bytecode function, when calls this operator calls the function with the bytecode of the class of the instance, get the new bytecode, create a new class from it and then patch the class of the instance to new class. Like with the redefinition, the transformation from old class and the new class should avoid structural transformations like inheritance change, and only allows change of the bytecode of the method. so by default all MHP on the same interface uses the same code, with a counter inside the MPH, when the MPH becomes hot, the fork-and-patch operator is used to provide another class specialized for underlying mh (using constant patching). > > ? John R?mi From Roger.Riggs at Oracle.com Wed Jul 11 21:25:06 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 11 Jul 2018 17:25:06 -0400 Subject: [11] RFR: 8206980: DateTimeFormatter throws parsing a valid string depending on the locale In-Reply-To: <9834dcbe-d8e8-2469-0743-c5e07f66c173@oracle.com> References: <9834dcbe-d8e8-2469-0743-c5e07f66c173@oracle.com> Message-ID: Hi Naoto, Looks fine, Roger On 7/11/2018 1:42 PM, naoto.sato at oracle.com wrote: > Hello, > > Please review the changes to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8206980 > > The proposed fix is located at: > > http://cr.openjdk.java.net/~naoto/8206980/webrev.00/ > > CLDR time zone names in "en" bundle are expected to be pre-populated > for performance. This is not the case for locales that have "en-001" > (World English) as the parent bundle, such as Locale.CANADA. The fast > path determining logic was incorrectly assumed all English locales are > using pre-populated bundles. Limiting the assumption to only "en" and > "en-US" will solve the problem. > > Naoto From kevin.rushforth at oracle.com Wed Jul 11 23:08:56 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 11 Jul 2018 16:08:56 -0700 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: <75106ab90b974292890c71b6d320b56c@DEISMMXS02.pt.local> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <75106ab90b974292890c71b6d320b56c@DEISMMXS02.pt.local> Message-ID: <7c2de54d-793d-77a9-556e-34f5871bdcc8@oracle.com> We will likely be able to deliver the .exe installer (with its dependency on Inno Setup). As for the service bundler, this will be a "nice to have" (a stretch goal) for this version, but isn't on the list of committed features. Alexsei might be able to comment further on how much work it would be to provide it, including documenting and testing it. This might give you, and other interested developers, a sense of how likely this is to make it for this version. -- Kevin On 7/10/2018 4:35 AM, Buchberger, Joerg wrote: > Hi > > thanks for the update/info. I had a quick look at the changes. So, here some thoughts... > > As described in JDK-8200758, and therefore expected, WinExeBundler has been removed in favor of putting focus on WinMsiBundler. > (Although, I regret that decision - since my personal experience has been that InnoSetup based WinExeBundler has worked much better than wix based WinMsiBundler for our use cases - I can live with that.) > > What is much more disturbing: WinServiceBundler has also been actively removed, although that was working fine together with both wix/msi and exe/iss. Why has service wrapping been removed as well, while the command line option for it is kept in place? > > Is there any chance of service bundler coming back into scope of JDK-8200758 or coming back in at all? > > Cheers > J?rg > > > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Kevin Rushforth > Sent: Freitag, 6. Juli 2018 22:14 > To: core-libs-dev at openjdk.java.net > Cc: Alexey Semenyuk ; Andy Herrick > Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] > > An initial prototype of the jpackager tool has been pushed to a new 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is interested in taking a look, you can clone it as follows: > > ??? hg clone https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=jWWENz_KIkmyh-9-kQQvoZ0BwBymwQ-BKx8hG3F5Iy0&e= > ??? cd ./sandbox > ??? hg update JDK-8200758-branch > > I plan to reply to the feedback already provided, and update the JEP [2] next week some time, but in the mean time if you have additional questions or comments, feel free to reply. > > -- Kevin > > [1] https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox_shortlog_JDK-2D8200758-2Dbranch&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=F-CqPAWlz-Cfb0kae2FBEj4Ncd3ZBVu7BeOVY1AM-cA&e= > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8200758&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=DFIAHtCR1o--KMLuBzurIzx5MDu67NgtUrEdQ22wI9I&e= > > > On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >> We're aiming to get this into JDK 12 early enough so that an EA build >> would be available around the time JDK 11 ships. That will allow you >> to take a jlinked image with JDK 11 and package it up using (the EA) >> jpackager. >> >> We will create a development branch in the JDK sandbox [1] some time >> in the next week or so so you can follow the development. >> >> Also, thank you to those who have provided feedback. I'll reply to >> feedback soon and then incorporate it into an updated JEP. >> >> -- Kevin From paul.sandoz at oracle.com Thu Jul 12 00:01:02 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 11 Jul 2018 17:01:02 -0700 Subject: [11] RFR 8207027 Lookup.accessClass fails for an array type in the same package when assertions are enabled Message-ID: <689C8A42-F06D-4820-87C3-3A5C6A7CB1EB@oracle.com> Hi, Please review this fix for 11: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8207027-lookup-access-class-array-type/webrev/ The fix removes an old and now incorrect assert statement. This statement causes MethodHandles.Lookup.accessClass to fail when system assertions are enabled and an array class in the same package as the lookup class is passed as an argument. Usages of MethodHandles.Lookup.accessClass such as by ConstantBootstraps.arrayVarHandle will also fail, which results in a JCK test failure (reported in bug). Thanks, Paul. From swpalmer at gmail.com Thu Jul 12 00:22:37 2018 From: swpalmer at gmail.com (Scott Palmer) Date: Wed, 11 Jul 2018 20:22:37 -0400 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: <7c2de54d-793d-77a9-556e-34f5871bdcc8@oracle.com> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <75106ab90b974292890c71b6d320b56c@DEISMMXS02.pt.local> <7c2de54d-793d-77a9-556e-34f5871bdcc8@oracle.com> Message-ID: Since support for services/daemons was already in javapackager, why does this have to be a stretch goal? Isn?t it mostly already done? I would like to see this in the initial implementation. It is something I?m currently using via javapackager. I?m still trying to figure out the best strategy from bridging the gap in Java 11 where suddenly no packager is available. My company has already decided to skip Java 9 and 10 altogether. Modules were too disruptive when they came in Java 9, which didn?t last long until Java 10. Given that Java 11 unbundled JavaFX, we decided not to bother trying to go to 10 just to have to re-jig everything for 11. Now 11 is missing key tools (javapackager) that we began to rely on, so we keep falling behind, unable to adopt the newer JDK/JRE because the cost is so high. The first time ever in the history of Java where we couldn?t just adapt the new JDK and tweak for minor issues if any. Migrating projects beyond Java 8 is a big pain. It would suck if Java 12 lacks what was available in Java 8. (Please excuse the rant.) Scott > On Jul 11, 2018, at 7:08 PM, Kevin Rushforth wrote: > > We will likely be able to deliver the .exe installer (with its dependency on Inno Setup). > > As for the service bundler, this will be a "nice to have" (a stretch goal) for this version, but isn't on the list of committed features. Alexsei might be able to comment further on how much work it would be to provide it, including documenting and testing it. This might give you, and other interested developers, a sense of how likely this is to make it for this version. > > -- Kevin > > >> On 7/10/2018 4:35 AM, Buchberger, Joerg wrote: >> Hi >> >> thanks for the update/info. I had a quick look at the changes. So, here some thoughts... >> >> As described in JDK-8200758, and therefore expected, WinExeBundler has been removed in favor of putting focus on WinMsiBundler. >> (Although, I regret that decision - since my personal experience has been that InnoSetup based WinExeBundler has worked much better than wix based WinMsiBundler for our use cases - I can live with that.) >> >> What is much more disturbing: WinServiceBundler has also been actively removed, although that was working fine together with both wix/msi and exe/iss. Why has service wrapping been removed as well, while the command line option for it is kept in place? >> >> Is there any chance of service bundler coming back into scope of JDK-8200758 or coming back in at all? >> >> Cheers >> J?rg >> >> >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Kevin Rushforth >> Sent: Freitag, 6. Juli 2018 22:14 >> To: core-libs-dev at openjdk.java.net >> Cc: Alexey Semenyuk ; Andy Herrick >> Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] >> >> An initial prototype of the jpackager tool has been pushed to a new 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is interested in taking a look, you can clone it as follows: >> >> hg clone https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=jWWENz_KIkmyh-9-kQQvoZ0BwBymwQ-BKx8hG3F5Iy0&e= >> cd ./sandbox >> hg update JDK-8200758-branch >> >> I plan to reply to the feedback already provided, and update the JEP [2] next week some time, but in the mean time if you have additional questions or comments, feel free to reply. >> >> -- Kevin >> >> [1] https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox_shortlog_JDK-2D8200758-2Dbranch&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=F-CqPAWlz-Cfb0kae2FBEj4Ncd3ZBVu7BeOVY1AM-cA&e= >> [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8200758&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=DFIAHtCR1o--KMLuBzurIzx5MDu67NgtUrEdQ22wI9I&e= >> >> >>> On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >>> We're aiming to get this into JDK 12 early enough so that an EA build >>> would be available around the time JDK 11 ships. That will allow you >>> to take a jlinked image with JDK 11 and package it up using (the EA) >>> jpackager. >>> >>> We will create a development branch in the JDK sandbox [1] some time >>> in the next week or so so you can follow the development. >>> >>> Also, thank you to those who have provided feedback. I'll reply to >>> feedback soon and then incorporate it into an updated JEP. >>> >>> -- Kevin > From vyom.tewari at oracle.com Thu Jul 12 06:40:29 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Thu, 12 Jul 2018 12:10:29 +0530 Subject: RFR 8207060 : Memory leak when malloc fails within WITH_UNICODE_STRING block In-Reply-To: <857818a6-a289-6e02-e576-dd9adb3c3ec0@oracle.com> References: <857818a6-a289-6e02-e576-dd9adb3c3ec0@oracle.com> Message-ID: <311e5021-e5e1-4a77-b991-6eef8516d4e9@oracle.com> Hi Ivan, Changes looks good to me, nice cleanup. Thanks, Vyom On Wednesday 11 July 2018 09:45 PM, Ivan Gerasimov wrote: > Hello! > > File src/java.base/windows/native/libjava/io_util_md.c > > In the function pathToNTPath(), memory is allocated with malloc() > within a block of macros WITH_UNICODE_STRING / END_UNICODE_STRING. > > In an unlikely event of malloc() failure, the function returns, > failing to release the string array acquired with WITH_UNICODE_STRING. > > Would you please help review the fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207060 > WEBREV: http://cr.openjdk.java.net/~igerasim/8207060/00/webrev/ > > Thanks in advance! > From nishit.jain at oracle.com Thu Jul 12 08:26:23 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Thu, 12 Jul 2018 13:56:23 +0530 Subject: [12] RFR: 8193444: SimpleDateFormat throws ArrayIndexOutOfBoundsException when format contains long sequences of unicode characters Message-ID: <800c0442-b44e-4353-22cd-25bf2a8b5916@oracle.com> Hi, Please review the fix for JDK-8193444 Bug: https://bugs.openjdk.java.net/browse/JDK-8193444 Webrev: http://cr.openjdk.java.net/~nishjain/8193444/webrev.02/ Cause: If a format pattern contains a sequence of over 256 non-ASCII unicode characters, the length field overflows as it is stored as 8-bit. Fix: The handling of long_length (> 254) for non-ASCII unicode characters is missing in the compilation phase of the format pattern, added that in the fix. Regards, Nishit Jain From xu.y.yin at oracle.com Thu Jul 12 08:38:02 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Thu, 12 Jul 2018 16:38:02 +0800 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> Message-ID: <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> Please have a review to new webrev as below, some code refactoring on lib parts and enhanced DNSServer to handle retry request which will make the tests more stable, thanks http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ Regards, Chris > On 22 Mar 2018, at 11:16 AM, Chris Yin wrote: > > Please review the changes to add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/, thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8198882 > webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ > > Regards, > Chris From claes.redestad at oracle.com Thu Jul 12 11:15:57 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 12 Jul 2018 13:15:57 +0200 Subject: [11] RFR 8207027 Lookup.accessClass fails for an array type in the same package when assertions are enabled In-Reply-To: <689C8A42-F06D-4820-87C3-3A5C6A7CB1EB@oracle.com> References: <689C8A42-F06D-4820-87C3-3A5C6A7CB1EB@oracle.com> Message-ID: +1 /Claes On 2018-07-12 02:01, Paul Sandoz wrote: > Hi, > > Please review this fix for 11: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8207027-lookup-access-class-array-type/webrev/ > > The fix removes an old and now incorrect assert statement. This statement causes MethodHandles.Lookup.accessClass to fail when system assertions are enabled and an array class in the same package as the lookup class is passed as an argument. > > Usages of MethodHandles.Lookup.accessClass such as by ConstantBootstraps.arrayVarHandle will also fail, which results in a JCK test failure (reported in bug). > > Thanks, > Paul. From mandy.chung at oracle.com Thu Jul 12 15:07:15 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 12 Jul 2018 08:07:15 -0700 Subject: [11] RFR 8207027 Lookup.accessClass fails for an array type in the same package when assertions are enabled In-Reply-To: <689C8A42-F06D-4820-87C3-3A5C6A7CB1EB@oracle.com> References: <689C8A42-F06D-4820-87C3-3A5C6A7CB1EB@oracle.com> Message-ID: <77fe8fc1-f0d2-216d-64dc-064a9d546087@oracle.com> Looks fine. Mandy On 7/11/18 5:01 PM, Paul Sandoz wrote: > Hi, > > Please review this fix for 11: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8207027-lookup-access-class-array-type/webrev/ > > The fix removes an old and now incorrect assert statement. This > statement causes MethodHandles.Lookup.accessClass to fail when system > assertions are enabled and an array class in the same package as the > lookup class is passed as an argument. > > Usages of MethodHandles.Lookup.accessClass such as by > ConstantBootstraps.arrayVarHandle will also fail, which results in a > JCK test failure (reported in bug). > > Thanks, Paul. > From ivan.gerasimov at oracle.com Thu Jul 12 15:58:34 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 12 Jul 2018 08:58:34 -0700 Subject: RFR 8207060 : Memory leak when malloc fails within WITH_UNICODE_STRING block In-Reply-To: <311e5021-e5e1-4a77-b991-6eef8516d4e9@oracle.com> References: <857818a6-a289-6e02-e576-dd9adb3c3ec0@oracle.com> <311e5021-e5e1-4a77-b991-6eef8516d4e9@oracle.com> Message-ID: Thanks Vyom for reviewing! I think I only need an approval from an official jdk reviewer to commit. With kind regards, Ivan On 7/11/18 11:40 PM, vyom tewari wrote: > Hi Ivan, > > Changes looks good to me, nice cleanup. > > Thanks, > > Vyom > > > On Wednesday 11 July 2018 09:45 PM, Ivan Gerasimov wrote: >> Hello! >> >> File src/java.base/windows/native/libjava/io_util_md.c >> >> In the function pathToNTPath(), memory is allocated with malloc() >> within a block of macros WITH_UNICODE_STRING / END_UNICODE_STRING. >> >> In an unlikely event of malloc() failure, the function returns, >> failing to release the string array acquired with WITH_UNICODE_STRING. >> >> Would you please help review the fix? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207060 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8207060/00/webrev/ >> >> Thanks in advance! >> > > -- With kind regards, Ivan Gerasimov From naoto.sato at oracle.com Thu Jul 12 17:28:18 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 12 Jul 2018 10:28:18 -0700 Subject: [12] RFR: 8193444: SimpleDateFormat throws ArrayIndexOutOfBoundsException when format contains long sequences of unicode characters In-Reply-To: <800c0442-b44e-4353-22cd-25bf2a8b5916@oracle.com> References: <800c0442-b44e-4353-22cd-25bf2a8b5916@oracle.com> Message-ID: <4b7c0243-1d64-4540-c649-0a27b42954d5@oracle.com> Looks good to me. Naoto On 7/12/18 1:26 AM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8193444 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8193444 > Webrev: http://cr.openjdk.java.net/~nishjain/8193444/webrev.02/ > > Cause: If a format pattern contains a sequence of over 256 non-ASCII > unicode characters, the length field overflows as it is stored as 8-bit. > Fix: The handling of long_length (> 254) for non-ASCII unicode > characters is missing in the compilation phase of the format pattern, > added that in the fix. > > Regards, > Nishit Jain From naoto.sato at oracle.com Thu Jul 12 17:51:39 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 12 Jul 2018 10:51:39 -0700 Subject: [11] RFR: 8207152: Placeholder for Japanese new era should be two characters Message-ID: <89660abe-519a-a07b-8d31-4145dc088501@oracle.com> Hello, Please review the changes to the following issue: https://bugs.openjdk.java.net/browse/JDK-8207152 The proposed fix is located at: http://cr.openjdk.java.net/~naoto/8207152/webrev.00/ The fix is to simply change the name from "???" to "??". From the API standpoint, there is no restriction on the length of a Era name (in fact, era names in English localization have more than 2 characters even for existing eras). However, it would be desirable to avoid any possible unwanted regressions in customer's apps even for the placeholder. Naoto From Roger.Riggs at Oracle.com Thu Jul 12 18:01:14 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 12 Jul 2018 14:01:14 -0400 Subject: RFR 8207060 : Memory leak when malloc fails within WITH_UNICODE_STRING block In-Reply-To: <311e5021-e5e1-4a77-b991-6eef8516d4e9@oracle.com> References: <857818a6-a289-6e02-e576-dd9adb3c3ec0@oracle.com> <311e5021-e5e1-4a77-b991-6eef8516d4e9@oracle.com> Message-ID: <7ebfcc6b-29b1-f5c8-1be3-131809850026@Oracle.com> Looks fine Ivan, Thanks, Roger On 7/12/2018 2:40 AM, vyom tewari wrote: > Hi Ivan, > > Changes looks good to me, nice cleanup. > > Thanks, > > Vyom > > > On Wednesday 11 July 2018 09:45 PM, Ivan Gerasimov wrote: >> Hello! >> >> File src/java.base/windows/native/libjava/io_util_md.c >> >> In the function pathToNTPath(), memory is allocated with malloc() >> within a block of macros WITH_UNICODE_STRING / END_UNICODE_STRING. >> >> In an unlikely event of malloc() failure, the function returns, >> failing to release the string array acquired with WITH_UNICODE_STRING. >> >> Would you please help review the fix? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207060 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8207060/00/webrev/ >> >> Thanks in advance! >> > From Roger.Riggs at Oracle.com Thu Jul 12 18:16:42 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 12 Jul 2018 14:16:42 -0400 Subject: [12] RFR: 8193444: SimpleDateFormat throws ArrayIndexOutOfBoundsException when format contains long sequences of unicode characters In-Reply-To: <4b7c0243-1d64-4540-c649-0a27b42954d5@oracle.com> References: <800c0442-b44e-4353-22cd-25bf2a8b5916@oracle.com> <4b7c0243-1d64-4540-c649-0a27b42954d5@oracle.com> Message-ID: <38e09380-594e-f70c-3094-ee0f992e6948@Oracle.com> +1 On 7/12/2018 1:28 PM, Naoto Sato wrote: > Looks good to me. > > Naoto > > On 7/12/18 1:26 AM, Nishit Jain wrote: >> Hi, >> >> Please review the fix for JDK-8193444 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8193444 >> Webrev: http://cr.openjdk.java.net/~nishjain/8193444/webrev.02/ >> >> Cause: If a format pattern contains a sequence of over 256 non-ASCII >> unicode characters, the length field overflows as it is stored as 8-bit. >> Fix: The handling of long_length (> 254) for non-ASCII unicode >> characters is missing in the compilation phase of the format pattern, >> added that in the fix. >> >> Regards, >> Nishit Jain From Roger.Riggs at Oracle.com Thu Jul 12 18:18:50 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 12 Jul 2018 14:18:50 -0400 Subject: [11] RFR: 8207152: Placeholder for Japanese new era should be two characters In-Reply-To: <89660abe-519a-a07b-8d31-4145dc088501@oracle.com> References: <89660abe-519a-a07b-8d31-4145dc088501@oracle.com> Message-ID: Looks fine Roger On 7/12/2018 1:51 PM, Naoto Sato wrote: > Hello, > > Please review the changes to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8207152 > > The proposed fix is located at: > > http://cr.openjdk.java.net/~naoto/8207152/webrev.00/ > > The fix is to simply change the name from "???" to "??". From the API > standpoint, there is no restriction on the length of a Era name (in > fact, era names in English localization have more than 2 characters > even for existing eras). However, it would be desirable to avoid any > possible unwanted regressions in customer's apps even for the > placeholder. > > Naoto From ivan.gerasimov at oracle.com Thu Jul 12 18:20:00 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 12 Jul 2018 11:20:00 -0700 Subject: RFR 8207060 : Memory leak when malloc fails within WITH_UNICODE_STRING block In-Reply-To: <7ebfcc6b-29b1-f5c8-1be3-131809850026@Oracle.com> References: <857818a6-a289-6e02-e576-dd9adb3c3ec0@oracle.com> <311e5021-e5e1-4a77-b991-6eef8516d4e9@oracle.com> <7ebfcc6b-29b1-f5c8-1be3-131809850026@Oracle.com> Message-ID: Thanks Vyom and Roger for reviewing! On 7/12/18 11:01 AM, Roger Riggs wrote: > Looks fine Ivan, > > Thanks, Roger > > > On 7/12/2018 2:40 AM, vyom tewari wrote: >> Hi Ivan, >> >> Changes looks good to me, nice cleanup. >> >> Thanks, >> >> Vyom >> >> >> On Wednesday 11 July 2018 09:45 PM, Ivan Gerasimov wrote: >>> Hello! >>> >>> File src/java.base/windows/native/libjava/io_util_md.c >>> >>> In the function pathToNTPath(), memory is allocated with malloc() >>> within a block of macros WITH_UNICODE_STRING / END_UNICODE_STRING. >>> >>> In an unlikely event of malloc() failure, the function returns, >>> failing to release the string array acquired with WITH_UNICODE_STRING. >>> >>> Would you please help review the fix? >>> >>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207060 >>> WEBREV: http://cr.openjdk.java.net/~igerasim/8207060/00/webrev/ >>> >>> Thanks in advance! >>> >> > > -- With kind regards, Ivan Gerasimov From huizhe.wang at oracle.com Thu Jul 12 19:07:51 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 12 Jul 2018 12:07:51 -0700 Subject: RFR(JDK12/JAXP/java.xml) 8194680: StartElement#getAttributes and getNamespaces refer to incorrect package In-Reply-To: References: <5B4540F7.5080107@oracle.com> <52d2d7da-0d0c-f22c-381d-98d5054a9181@oracle.com> <5B457518.2010108@oracle.com> Message-ID: <5B47A707.10007@oracle.com> Thanks Lance, and Jon! -Joe On 7/11/18, 10:49 AM, Lance Andersen wrote: > Hi Joe, > > The updates like fine >> On Jul 10, 2018, at 11:10 PM, Joe Wang > > wrote: >> >> >> >> On 7/10/18, 4:39 PM, Jonathan Gibbons wrote: >>> >>> >>> >>> On 7/10/18 4:27 PM, Joe Wang wrote: >>>> Hi, >>>> >>>> Please review a javadoc fix for a couple of incorrect references to >>>> the javax.xml.stream package. Attribute and Namespace are both in >>>> the javax.xml.stream.events, and so is StartElement where they are >>>> referenced. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8194680 >>>> webrevs: http://cr.openjdk.java.net/~joehw/jdk12/8194680/webrev/ >>>> >>>> >>>> Thanks, >>>> Joe >>> The following text does not read well: >>> 54 * Attributes are fundamentally unordered and may not be reported >>> 55 * in any order. >>> >>> It would read better if it said: >>> >>> 54 * Attributes are fundamentally unordered and may be reported >>> 55 * in any order. >>> >>> (i.e. delete "not") >> >> Done (deleted "not"). >> >> Thanks, >> Joe >> >>> >>> -- Jon > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From vyom.tewari at oracle.com Fri Jul 13 05:46:32 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Fri, 13 Jul 2018 11:16:32 +0530 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> Message-ID: Hi Chris, Thanks for doing this overall looks good to me, few minor comments. 1->? DNSTestUtils.java, please start the server first and then set the "TEST_DNS_SERVER_THREAD". This will not make much difference but we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if server fails to start. 129 env.put(TEST_DNS_SERVER_THREAD, inst); 130 inst.start(); 2->? I noticed that? copyright date (Copyright (c) 2000, 2018,) but webrev tells all the tests are new, please fix copyright date as well. Thanks, Vyom On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: > Please have a review to new webrev as below, some code refactoring on > lib parts and enhanced DNSServer to handle retry request which will > make the tests more stable, thanks > > http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ > > > Regards, > Chris > >> On 22 Mar 2018, at 11:16 AM, Chris Yin > > wrote: >> >> Please review the changes to?add 10 JNDI tests to >> com/sun/jndi/dns/AttributeTests/, thanks >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 >> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ >> >> >> Regards, >> Chris > From xu.y.yin at oracle.com Fri Jul 13 06:14:49 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Fri, 13 Jul 2018 14:14:49 +0800 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> Message-ID: Hi, Vyom Thank you for the review and comments, update webrev as below and comment inline webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.02/ > On 13 Jul 2018, at 1:46 PM, vyom tewari wrote: > > Hi Chris, > > Thanks for doing this overall looks good to me, few minor comments. > 1-> DNSTestUtils.java, please start the server first and then set the "TEST_DNS_SERVER_THREAD". This will not make much difference but we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if server fails to start. > 129 env.put(TEST_DNS_SERVER_THREAD, inst); > 130 inst.start(); Fixed, thanks > > 2-> I noticed that copyright date (Copyright (c) 2000, 2018,) but webrev tells all the tests are new, please fix copyright date as well. Thanks for checking this. Since this task is part of umbrella enhancement JDK-8191011 JNDI SQE tests co-location, for those added tests which are migrated from SQE tests, the copyright date will follow the guidance SQE Test copyright year + migration copyright year if the 2 year are not same, for dump files (like *.dns) are new added under our new framework so just use current copyright year, hope that explains :), thanks Regards, Chris > > Thanks, > Vyom > > On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: >> Please have a review to new webrev as below, some code refactoring on lib parts and enhanced DNSServer to handle retry request which will make the tests more stable, thanks >> >> http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ >> >> Regards, >> Chris >> >>> On 22 Mar 2018, at 11:16 AM, Chris Yin > wrote: >>> >>> Please review the changes to add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/, thanks >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 >>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ >>> >>> Regards, >>> Chris >> > From vyom.tewari at oracle.com Fri Jul 13 09:15:31 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Fri, 13 Jul 2018 14:45:31 +0530 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> Message-ID: <296d1f3e-bec3-bea5-77b7-0ffcae83f48a@oracle.com> Hi Chris, latest webrev looks good to me, thanks for explanation about copyright date. Thanks, Vyom On Friday 13 July 2018 11:44 AM, Chris Yin wrote: > Hi, Vyom > > Thank you for the review and comments, update webrev as below and > comment inline > > webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.02/ > > > >> On 13 Jul 2018, at 1:46 PM, vyom tewari > > wrote: >> >> Hi Chris, >> >> Thanks for doing this overall looks good to me, few minor comments. >> >> 1->? DNSTestUtils.java, please start the server first and then set >> the "TEST_DNS_SERVER_THREAD". This will not make much difference but >> we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if server >> fails to start. >> >> 129 env.put(TEST_DNS_SERVER_THREAD, inst); >> 130 inst.start(); > Fixed, thanks > >> >> 2->? I noticed that? copyright date (Copyright (c) 2000, 2018,) but >> webrev tells all the tests are new, please fix copyright date as well. > > Thanks for checking this. Since this task is part of?umbrella > enhancement JDK-8191011 > ?JNDI SQE tests > co-location, for those added tests which are migrated from SQE tests, > the copyright date will follow the guidance SQE Test copyright year + > migration copyright year if the 2 year are not same, for dump files > (like *.dns) are new added under our new framework so just use current > copyright year, hope that explains :), thanks > > Regards, > Chris > >> >> Thanks, >> Vyom >> >> On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: >>> Please have a review to new webrev as below, some code refactoring >>> on lib parts and enhanced DNSServer to handle retry request which >>> will make the tests more stable, thanks >>> >>> http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ >>> >>> >>> Regards, >>> Chris >>> >>>> On 22 Mar 2018, at 11:16 AM, Chris Yin >>> > wrote: >>>> >>>> Please review the changes to?add 10 JNDI tests to >>>> com/sun/jndi/dns/AttributeTests/, thanks >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 >>>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ >>>> >>>> >>>> Regards, >>>> Chris >>> >> > From goetz.lindenmaier at sap.com Fri Jul 13 10:54:35 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 13 Jul 2018 10:54:35 +0000 Subject: RFR(S): 8207233: Minor improvements of jdk C-coding Message-ID: <34185929affa44fa838a61ac26b1cfdb@sap.com> Hi, I ran coverity on the jdk11 jdk sources and want to propose the following fixes. I scanned the linux x86_64 build. Some issues are similar to previous parfait fixes (check for NULL). I also identified some issues I consider real problems. If you think some are tooo conservative, I'm happy to remove them. I posted this to core-libs-dev and awt-dev, if you think this should be discussed on other lists please tell me. http://cr.openjdk.java.net/~goetz/wr18/8207233-covJDK/01/ In detail: Real issues: ------------ transport.c Loop overruns the array, it iterates to 8. Only two iterations are intended. Unix.c getgroups can return -1. This is handled below, but not here. Return as for other errors. Useful code improvements. ------------------------- zip_util.c pmsg is compared to null above. Thus, don't dereference it unconditionally below. I would assume pmsg is always != NULL, so that the check above could as well be turned into a guarantee. This fix is more safe, though. fontpath.c This is a real error, but harmless as the same size is returned. pcsc.c If size is 0, mszReaders is not allocated, but accessed below. return if size is 0. Here, too, I would assume that one could turn the if(size) check into a guarantee, but this way it's more safe. ecl_muilt.c This block calls point_mul, which requires the kt.flag is initialized. unpack.cpp lo is checked for null. If it is null, the dereference below fails. Return if lo == Null similar as above. Alternatively, one could turn the if (lo != null) check into a guarantee. From Roger.Riggs at Oracle.com Fri Jul 13 15:23:56 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 13 Jul 2018 11:23:56 -0400 Subject: RFR(S): 8207233: Minor improvements of jdk C-coding In-Reply-To: <34185929affa44fa838a61ac26b1cfdb@sap.com> References: <34185929affa44fa838a61ac26b1cfdb@sap.com> Message-ID: <04333301-33e1-f334-b924-1fda2cdab985@Oracle.com> Hi Goetz, Thanks for the cleanup; looks fine with or without the suggestion below. src/jdk.crypto.ec/share/native/libsunec/impl/ecl_mult.c: 84, 89 ?? Since the assignment is done in both branches of the if, it could be moved up. Regards, Roger On 7/13/2018 6:54 AM, Lindenmaier, Goetz wrote: > Hi, > > I ran coverity on the jdk11 jdk sources and want to propose the following fixes. I scanned the linux x86_64 build. Some issues are similar to previous parfait fixes (check for NULL). I also identified some issues I consider real problems. If you think some are tooo conservative, I'm happy to remove them. > I posted this to core-libs-dev and awt-dev, if you think this should > be discussed on other lists please tell me. > > http://cr.openjdk.java.net/~goetz/wr18/8207233-covJDK/01/ > > In detail: > > Real issues: > ------------ > > transport.c > Loop overruns the array, it iterates to 8. Only > two iterations are intended. > > Unix.c > getgroups can return -1. This is handled below, > but not here. Return as for other errors. > > Useful code improvements. > ------------------------- > > zip_util.c > pmsg is compared to null above. Thus, don't > dereference it unconditionally below. > I would assume pmsg is always != NULL, so that the > check above could as well be turned into a guarantee. > This fix is more safe, though. > > fontpath.c > This is a real error, but harmless as the same size is > returned. > > pcsc.c > If size is 0, mszReaders is not allocated, but accessed > below. return if size is 0. > Here, too, I would assume that one could turn the if(size) > check into a guarantee, but this way it's more safe. > > ecl_muilt.c > This block calls point_mul, which requires the kt.flag > is initialized. > > unpack.cpp > lo is checked for null. If it is null, the dereference > below fails. > Return if lo == Null similar as above. > Alternatively, one could turn the if (lo != null) check into > a guarantee. From goetz.lindenmaier at sap.com Fri Jul 13 15:40:36 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 13 Jul 2018 15:40:36 +0000 Subject: RFR(S): 8207233: Minor improvements of jdk C-coding In-Reply-To: <04333301-33e1-f334-b924-1fda2cdab985@Oracle.com> References: <34185929affa44fa838a61ac26b1cfdb@sap.com> <04333301-33e1-f334-b924-1fda2cdab985@Oracle.com> Message-ID: Hi Roger, Thanks for looking at this! > Since the assignment is done in both branches of the if, it could be > moved up. You're right, that's better. Done. Best regards, Goetz. > -----Original Message----- > From: core-libs-dev On Behalf > Of Roger Riggs > Sent: Friday, July 13, 2018 5:24 PM > To: core-libs-dev at openjdk.java.net > Subject: Re: RFR(S): 8207233: Minor improvements of jdk C-coding > > Hi Goetz, > > Thanks for the cleanup; looks fine with or without the suggestion below. > > src/jdk.crypto.ec/share/native/libsunec/impl/ecl_mult.c: 84, 89 > ?? Since the assignment is done in both branches of the if, it could be > moved up. > > Regards, Roger > > > On 7/13/2018 6:54 AM, Lindenmaier, Goetz wrote: > > Hi, > > > > I ran coverity on the jdk11 jdk sources and want to propose the following > fixes. I scanned the linux x86_64 build. Some issues are similar to previous > parfait fixes (check for NULL). I also identified some issues I consider real > problems. If you think some are tooo conservative, I'm happy to remove > them. > > I posted this to core-libs-dev and awt-dev, if you think this should > > be discussed on other lists please tell me. > > > > http://cr.openjdk.java.net/~goetz/wr18/8207233-covJDK/01/ > > > > In detail: > > > > Real issues: > > ------------ > > > > transport.c > > Loop overruns the array, it iterates to 8. Only > > two iterations are intended. > > > > Unix.c > > getgroups can return -1. This is handled below, > > but not here. Return as for other errors. > > > > Useful code improvements. > > ------------------------- > > > > zip_util.c > > pmsg is compared to null above. Thus, don't > > dereference it unconditionally below. > > I would assume pmsg is always != NULL, so that the > > check above could as well be turned into a guarantee. > > This fix is more safe, though. > > > > fontpath.c > > This is a real error, but harmless as the same size is > > returned. > > > > pcsc.c > > If size is 0, mszReaders is not allocated, but accessed > > below. return if size is 0. > > Here, too, I would assume that one could turn the if(size) > > check into a guarantee, but this way it's more safe. > > > > ecl_muilt.c > > This block calls point_mul, which requires the kt.flag > > is initialized. > > > > unpack.cpp > > lo is checked for null. If it is null, the dereference > > below fails. > > Return if lo == Null similar as above. > > Alternatively, one could turn the if (lo != null) check into > > a guarantee. From philip.race at oracle.com Fri Jul 13 15:46:56 2018 From: philip.race at oracle.com (Philip Race) Date: Fri, 13 Jul 2018 08:46:56 -0700 Subject: RFR(S): 8207233: Minor improvements of jdk C-coding In-Reply-To: References: <34185929affa44fa838a61ac26b1cfdb@sap.com> <04333301-33e1-f334-b924-1fda2cdab985@Oracle.com> Message-ID: <5B48C970.3070001@oracle.com> font change .. and all the rest look fine to me. -phil. On 7/13/18, 8:40 AM, Lindenmaier, Goetz wrote: > Hi Roger, > > Thanks for looking at this! > >> Since the assignment is done in both branches of the if, it could be >> moved up. > You're right, that's better. Done. > > Best regards, > Goetz. > >> -----Original Message----- >> From: core-libs-dev On Behalf >> Of Roger Riggs >> Sent: Friday, July 13, 2018 5:24 PM >> To: core-libs-dev at openjdk.java.net >> Subject: Re: RFR(S): 8207233: Minor improvements of jdk C-coding >> >> Hi Goetz, >> >> Thanks for the cleanup; looks fine with or without the suggestion below. >> >> src/jdk.crypto.ec/share/native/libsunec/impl/ecl_mult.c: 84, 89 >> Since the assignment is done in both branches of the if, it could be >> moved up. >> >> Regards, Roger >> >> >> On 7/13/2018 6:54 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> I ran coverity on the jdk11 jdk sources and want to propose the following >> fixes. I scanned the linux x86_64 build. Some issues are similar to previous >> parfait fixes (check for NULL). I also identified some issues I consider real >> problems. If you think some are tooo conservative, I'm happy to remove >> them. >>> I posted this to core-libs-dev and awt-dev, if you think this should >>> be discussed on other lists please tell me. >>> >>> http://cr.openjdk.java.net/~goetz/wr18/8207233-covJDK/01/ >>> >>> In detail: >>> >>> Real issues: >>> ------------ >>> >>> transport.c >>> Loop overruns the array, it iterates to 8. Only >>> two iterations are intended. >>> >>> Unix.c >>> getgroups can return -1. This is handled below, >>> but not here. Return as for other errors. >>> >>> Useful code improvements. >>> ------------------------- >>> >>> zip_util.c >>> pmsg is compared to null above. Thus, don't >>> dereference it unconditionally below. >>> I would assume pmsg is always != NULL, so that the >>> check above could as well be turned into a guarantee. >>> This fix is more safe, though. >>> >>> fontpath.c >>> This is a real error, but harmless as the same size is >>> returned. >>> >>> pcsc.c >>> If size is 0, mszReaders is not allocated, but accessed >>> below. return if size is 0. >>> Here, too, I would assume that one could turn the if(size) >>> check into a guarantee, but this way it's more safe. >>> >>> ecl_muilt.c >>> This block calls point_mul, which requires the kt.flag >>> is initialized. >>> >>> unpack.cpp >>> lo is checked for null. If it is null, the dereference >>> below fails. >>> Return if lo == Null similar as above. >>> Alternatively, one could turn the if (lo != null) check into >>> a guarantee. From rasbold at google.com Fri Jul 13 16:45:15 2018 From: rasbold at google.com (Chuck Rasbold) Date: Fri, 13 Jul 2018 09:45:15 -0700 Subject: RFR: 8206863: A closed JarVerifier.VerifierStream should throw IOException Message-ID: Looks good to me. > From: Martin Buchholz > Date: Mon, Jul 9, 2018 at 11:17 AM > Subject: RFR: 8206863: A closed JarVerifier.VerifierStream should throw > IOException > To: core-libs-dev , Xueming Shen < > xueming.shen at oracle.com>, Tobias Thierer > > > Here's another fix from Tobias and myself: > > 8206863: A closed JarVerifier.VerifierStream should throw IOException > http://cr.openjdk.java.net/~martin/webrevs/jdk/JarVerifier-stream-closed/ > https://bugs.openjdk.java.net/browse/JDK-8206863 > > > From forax at univ-mlv.fr Fri Jul 13 18:31:48 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 13 Jul 2018 20:31:48 +0200 (CEST) Subject: javac --release 11 not support by jdk 12 Message-ID: <1603767512.929008.1531506708261.JavaMail.zimbra@u-pem.fr> There is a weird bug with javac of jdk 12. /usr/jdk/jdk-12/bin/javac --release 11 error: release version 11 not supported Usage: javac use --help for a list of possible options while javac --release 10 or javac --release 12 works ? R?mi From joe.darcy at oracle.com Fri Jul 13 19:05:35 2018 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 13 Jul 2018 12:05:35 -0700 Subject: javac --release 11 not support by jdk 12 In-Reply-To: <1603767512.929008.1531506708261.JavaMail.zimbra@u-pem.fr> References: <1603767512.929008.1531506708261.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi, On 7/13/2018 11:31 AM, Remi Forax wrote: > There is a weird bug with javac of jdk 12. > > /usr/jdk/jdk-12/bin/javac --release 11 > error: release version 11 not supported > Usage: javac > use --help for a list of possible options > > while javac --release 10 or javac --release 12 works ? That is an expected but temporary situation given the way --release is implemented at present. Once JDK 11 ships, the exact API changes be be encoding so that --release will pick them up. HTH, -Joe From jonathan.gibbons at oracle.com Fri Jul 13 19:06:00 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 13 Jul 2018 12:06:00 -0700 Subject: javac --release 11 not support by jdk 12 In-Reply-To: <1603767512.929008.1531506708261.JavaMail.zimbra@u-pem.fr> References: <1603767512.929008.1531506708261.JavaMail.zimbra@u-pem.fr> Message-ID: <5B48F818.8080602@oracle.com> Remi, This should go to compiler-dev. It requires an update to make the historical data available for an earlier release. -- Jon On 07/13/2018 11:31 AM, Remi Forax wrote: > There is a weird bug with javac of jdk 12. > > /usr/jdk/jdk-12/bin/javac --release 11 > error: release version 11 not supported > Usage: javac > use --help for a list of possible options > > while javac --release 10 or javac --release 12 works ? > > R?mi > From Roger.Riggs at Oracle.com Fri Jul 13 19:07:32 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 13 Jul 2018 15:07:32 -0400 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline Message-ID: Please review a doc change to improve the description of ProcessBuilder.startPipeline. The clarifications are not intended to imply any change in behavior and only focus the description on the steps taken for each command in the pipeline. (A CSR will be needed also when the wording is finalized). Webrev: ?? http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ Issue: ?? https://bugs.openjdk.java.net/browse/JDK-8189717 Thanks, Roger From lance.andersen at oracle.com Fri Jul 13 19:26:44 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 13 Jul 2018 15:26:44 -0400 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline In-Reply-To: References: Message-ID: <91E0C567-701E-4697-8AE7-21AA72ED19C4@oracle.com> Hi Roger, I am struggling with the wording at: ??? * Subsequent modifications to any of process builders in the {@code builders} 1198 * argument will not affect the returned {@link Process}. ???? Maybe specified/provided process builders Not sure, will think about it a bit. Otherwise I think the changes are fine > On Jul 13, 2018, at 3:07 PM, Roger Riggs wrote: > > Please review a doc change to improve the description of ProcessBuilder.startPipeline. > The clarifications are not intended to imply any change in behavior and only focus the description > on the steps taken for each command in the pipeline. > (A CSR will be needed also when the wording is finalized). > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8189717 > > Thanks, Roger > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From brian.burkhalter at oracle.com Fri Jul 13 19:27:45 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 13 Jul 2018 12:27:45 -0700 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline In-Reply-To: References: Message-ID: <8892001E-1A6E-4DDD-90CD-EFEE27EBDE5A@oracle.com> Hi Roger, +1 Brian On Jul 13, 2018, at 12:07 PM, Roger Riggs wrote: > Please review a doc change to improve the description of ProcessBuilder.startPipeline. > The clarifications are not intended to imply any change in behavior and only focus the description > on the steps taken for each command in the pipeline. > (A CSR will be needed also when the wording is finalized). > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8189717 From Roger.Riggs at Oracle.com Fri Jul 13 19:48:15 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 13 Jul 2018 15:48:15 -0400 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline In-Reply-To: <91E0C567-701E-4697-8AE7-21AA72ED19C4@oracle.com> References: <91E0C567-701E-4697-8AE7-21AA72ED19C4@oracle.com> Message-ID: Hi Lance, How about: Subsequent modifications to any of {@code builder}swill not affect the returned {@link Process}. Roger On 7/13/2018 3:26 PM, Lance Andersen wrote: > Hi Roger, > > I am struggling with the wording at: > > ??? > * Subsequent modifications to any of process builders in the {@code > builders} > 1198 * argument will not affect the returned {@link Process}. > > ???? > > Maybe specified/provided ?process builders > > Not sure, will think about it a bit. > > Otherwise I think the changes are fine >> On Jul 13, 2018, at 3:07 PM, Roger Riggs > > wrote: >> >> Please review a doc change to improve the description of >> ProcessBuilder.startPipeline. >> The clarifications are not intended to imply any change in behavior >> and only focus the description >> on the steps taken for each command in the pipeline. >> (A CSR will be needed also when the wording is finalized). >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8189717 >> >> Thanks, Roger >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From lance.andersen at oracle.com Fri Jul 13 20:31:53 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 13 Jul 2018 16:31:53 -0400 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline In-Reply-To: References: <91E0C567-701E-4697-8AE7-21AA72ED19C4@oracle.com> Message-ID: <547AAD0E-0D4B-4869-BB3A-C945C279326B@oracle.com> Hi Roger, > On Jul 13, 2018, at 3:48 PM, Roger Riggs wrote: > > Hi Lance, > > How about: > Subsequent modifications to any of {@code builder}s will not affect > the returned {@link Process}. ?.. to any of the specified builders ?. At a minimum, you need ?the? added to the above > Roger > > > On 7/13/2018 3:26 PM, Lance Andersen wrote: >> Hi Roger, >> >> I am struggling with the wording at: >> >> ??? >> * Subsequent modifications to any of process builders in the {@code builders} >> 1198 * argument will not affect the returned {@link Process}. >> >> ???? >> >> Maybe specified/provided process builders >> >> Not sure, will think about it a bit. >> >> Otherwise I think the changes are fine >>> On Jul 13, 2018, at 3:07 PM, Roger Riggs > wrote: >>> >>> Please review a doc change to improve the description of ProcessBuilder.startPipeline. >>> The clarifications are not intended to imply any change in behavior and only focus the description >>> on the steps taken for each command in the pipeline. >>> (A CSR will be needed also when the wording is finalized). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8189717 >>> >>> Thanks, Roger >>> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Fri Jul 13 20:38:06 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 13 Jul 2018 16:38:06 -0400 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline In-Reply-To: <547AAD0E-0D4B-4869-BB3A-C945C279326B@oracle.com> References: <91E0C567-701E-4697-8AE7-21AA72ED19C4@oracle.com> <547AAD0E-0D4B-4869-BB3A-C945C279326B@oracle.com> Message-ID: <4ae3362e-f08c-cdf5-745a-2bb245ad8e31@Oracle.com> Hi Lance, Updated in place with your suggestion. http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ Thanks, Roger On 7/13/2018 4:31 PM, Lance Andersen wrote: > Hi Roger, > >> On Jul 13, 2018, at 3:48 PM, Roger Riggs > > wrote: >> >> Hi Lance, >> >> How about: >> Subsequent modifications to any of {@code builder}swill not affect >> the returned {@link Process}. > ?.. to any of the specified builders ?. > > At a minimum, you need ??the? added to the above > >> Roger >> >> >> On 7/13/2018 3:26 PM, Lance Andersen wrote: >>> Hi Roger, >>> >>> I am struggling with the wording at: >>> >>> ??? >>> * Subsequent modifications to any of process builders in the {@code >>> builders} >>> 1198 * argument will not affect the returned {@link Process}. >>> >>> ???? >>> >>> Maybe specified/provided ?process builders >>> >>> Not sure, will think about it a bit. >>> >>> Otherwise I think the changes are fine >>>> On Jul 13, 2018, at 3:07 PM, Roger Riggs >>> > wrote: >>>> >>>> Please review a doc change to improve the description of >>>> ProcessBuilder.startPipeline. >>>> The clarifications are not intended to imply any change in behavior >>>> and only focus the description >>>> on the steps taken for each command in the pipeline. >>>> (A CSR will be needed also when the wording is finalized). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8189717 >>>> >>>> Thanks, Roger >>>> >>> >>> >>> >>> >>> Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle?Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From lance.andersen at oracle.com Fri Jul 13 20:50:36 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 13 Jul 2018 16:50:36 -0400 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline In-Reply-To: <4ae3362e-f08c-cdf5-745a-2bb245ad8e31@Oracle.com> References: <91E0C567-701E-4697-8AE7-21AA72ED19C4@oracle.com> <547AAD0E-0D4B-4869-BB3A-C945C279326B@oracle.com> <4ae3362e-f08c-cdf5-745a-2bb245ad8e31@Oracle.com> Message-ID: <4FC1E7DC-A7DC-4B69-9CB3-F5EF8A94D00C@oracle.com> Hi Roger, +1 Have a great weekend! Best Lance > On Jul 13, 2018, at 4:38 PM, Roger Riggs wrote: > > Hi Lance, > > Updated in place with your suggestion. > http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ > > Thanks, Roger > > On 7/13/2018 4:31 PM, Lance Andersen wrote: >> Hi Roger, >> >>> On Jul 13, 2018, at 3:48 PM, Roger Riggs > wrote: >>> >>> Hi Lance, >>> >>> How about: >>> Subsequent modifications to any of {@code builder}swill not affect the returned {@link Process}. >> ?.. to any of the specified builders ?. >> >> At a minimum, you need ?the? added to the above >> >>> Roger >>> >>> >>> On 7/13/2018 3:26 PM, Lance Andersen wrote: >>>> Hi Roger, >>>> >>>> I am struggling with the wording at: >>>> >>>> ??? >>>> * Subsequent modifications to any of process builders in the {@code builders} >>>> 1198 * argument will not affect the returned {@link Process}. >>>> >>>> ???? >>>> >>>> Maybe specified/provided process builders >>>> >>>> Not sure, will think about it a bit. >>>> >>>> Otherwise I think the changes are fine >>>>> On Jul 13, 2018, at 3:07 PM, Roger Riggs > wrote: >>>>> >>>>> Please review a doc change to improve the description of ProcessBuilder.startPipeline. >>>>> The clarifications are not intended to imply any change in behavior and only focus the description >>>>> on the steps taken for each command in the pipeline. >>>>> (A CSR will be needed also when the wording is finalized). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8189717 >>>>> >>>>> Thanks, Roger >>>>> >>>> >>>> >>>> >>>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>> Oracle Java Engineering >>>> 1 Network Drive >>>> Burlington, MA 01803 >>>> Lance.Andersen at oracle.com >>>> >>>> >>>> >>> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From ivan.gerasimov at oracle.com Sat Jul 14 05:22:52 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 13 Jul 2018 22:22:52 -0700 Subject: RFR 8207314 : Unnecessary reallocation when constructing WeakHashMap from a large Map Message-ID: Hello! When a WeakHashMap is constructed from another Map m, then the initial capacity is calculated as (int) (m.size() / DEFAULT_LOAD_FACTOR) + 1. For large values of m.size() this becomes negative due to integer overflow. The result is that the WeakHashMap is initially constructed with the default initial capacity 16, and then is immediately resized. Would you please help review a trivial patch? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207314 WEBREV: http://cr.openjdk.java.net/~igerasim/8207314/00/webrev/ Thanks in advance! -- With kind regards, Ivan Gerasimov From martinrb at google.com Sun Jul 15 16:14:00 2018 From: martinrb at google.com (Martin Buchholz) Date: Sun, 15 Jul 2018 09:14:00 -0700 Subject: RFR 8207314 : Unnecessary reallocation when constructing WeakHashMap from a large Map In-Reply-To: References: Message-ID: Hi Ivan, Here are some thoughts while looking at this: --- WeakHashMap promises to have similar "capacity" handling to HashMap, but implementations (and doc?) seem more different than necessary (diverged over time?), and there don't seem to be any tests. HashMap seems to deal with this problem by doing computations using float not int. Choose the best one and use it in both source files. float ft = ((float)s / loadFactor) + 1.0F; int t = ((ft < (float)MAXIMUM_CAPACITY) ? (int)ft : MAXIMUM_CAPACITY); --- Consider adding a WhiteBox test. An existing one for ConcurrentHashMap could be modified to test internal table maintenance. PriorityBlockingQueue/WhiteBox.java is an example of a test that ensures two implementations stay in sync. --- i see if (loadFactor <= 0 || Float.isNaN(loadFactor)) but that nan check doesn't have much value. It can be removed using if (! (loadFactor > 0)) --- HashMap's resize() doesn't take an arg, while WeakHashMap's does. Why? As a result, I see in HashMap.putMapEntries else if (s > threshold) resize(); which suggests that if you create a fresh HashMap, then putAll(hugeMap) it will repeatedly resize instead of resizing to the target capacity in one step. Which seems like a HashMap bug. On Fri, Jul 13, 2018 at 10:22 PM, Ivan Gerasimov wrote: > Hello! > > When a WeakHashMap is constructed from another Map m, then the initial > capacity is calculated as > > (int) (m.size() / DEFAULT_LOAD_FACTOR) + 1. > > For large values of m.size() this becomes negative due to integer overflow. > > The result is that the WeakHashMap is initially constructed with the > default initial capacity 16, and then is immediately resized. > > Would you please help review a trivial patch? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207314 > WEBREV: http://cr.openjdk.java.net/~igerasim/8207314/00/webrev/ > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > From xu.y.yin at oracle.com Mon Jul 16 00:50:39 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 16 Jul 2018 08:50:39 +0800 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: <296d1f3e-bec3-bea5-77b7-0ffcae83f48a@oracle.com> References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> <296d1f3e-bec3-bea5-77b7-0ffcae83f48a@oracle.com> Message-ID: <9F443F23-8F39-4D2B-97FB-F54B0041EC35@oracle.com> Thank you, Vyom Regards, Chris > On 13 Jul 2018, at 5:15 PM, vyom tewari wrote: > > Hi Chris, > > latest webrev looks good to me, thanks for explanation about copyright date. > > Thanks, > > Vyom > > On Friday 13 July 2018 11:44 AM, Chris Yin wrote: >> Hi, Vyom >> >> Thank you for the review and comments, update webrev as below and comment inline >> >> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.02/ >> >> >>> On 13 Jul 2018, at 1:46 PM, vyom tewari > wrote: >>> >>> Hi Chris, >>> >>> Thanks for doing this overall looks good to me, few minor comments. >>> 1-> DNSTestUtils.java, please start the server first and then set the "TEST_DNS_SERVER_THREAD". This will not make much difference but we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if server fails to start. >>> 129 env.put(TEST_DNS_SERVER_THREAD, inst); >>> 130 inst.start(); >> Fixed, thanks >> >>> >>> 2-> I noticed that copyright date (Copyright (c) 2000, 2018,) but webrev tells all the tests are new, please fix copyright date as well. >> >> Thanks for checking this. Since this task is part of umbrella enhancement JDK-8191011 JNDI SQE tests co-location, for those added tests which are migrated from SQE tests, the copyright date will follow the guidance SQE Test copyright year + migration copyright year if the 2 year are not same, for dump files (like *.dns) are new added under our new framework so just use current copyright year, hope that explains :), thanks >> >> Regards, >> Chris >> >>> >>> Thanks, >>> Vyom >>> >>> On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: >>>> Please have a review to new webrev as below, some code refactoring on lib parts and enhanced DNSServer to handle retry request which will make the tests more stable, thanks >>>> >>>> http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ >>>> >>>> Regards, >>>> Chris >>>> >>>>> On 22 Mar 2018, at 11:16 AM, Chris Yin > wrote: >>>>> >>>>> Please review the changes to add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/, thanks >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 >>>>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ >>>>> >>>>> Regards, >>>>> Chris >>>> >>> >> > From Alan.Bateman at oracle.com Mon Jul 16 12:05:33 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 16 Jul 2018 13:05:33 +0100 Subject: RFR 8207016 : [win] Avoid redundant native memory allocation in getFinalPath() In-Reply-To: References: Message-ID: On 11/07/2018 03:33, Ivan Gerasimov wrote: > Hello! > > File src/java.base/windows/native/libjava/WinNTFileSystem_md.c > The function getFinalPath() is wrapper for > GetFinalPathNameByHandleW(), which is used to resolve the symlinks on > Windows. > > If the resulting string begins with a prefix "\\?\" or "\\?\UNC", a > copy without the prefix is created and returned. > > Instead of creating the copy, we can override the result with > memmove() to avoid allocating extra native memory. > The result of getFinalPath() is short living anyway, so a few unused > bytes at the end of the buffer should not be a problem. > > The advantage is a shorter code and minus one point of possible failure. > > Would you please help review it? > > BUG: https://bugs.openjdk.java.net/browse/JDK-8207016 > Webrev: http://cr.openjdk.java.net/~igerasim/8207016/00/webrev/ I think this looks okay. -Alan From Alan.Bateman at oracle.com Mon Jul 16 12:10:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 16 Jul 2018 13:10:38 +0100 Subject: RFR (Unraised): JDK8 ResourceBundle vulnerable to GC In-Reply-To: References: Message-ID: <33de7bf7-d313-9f6d-0a7c-d1dd7147cb76@oracle.com> On 11/07/2018 14:27, Adam Farley8 wrote: > Hi All, > > -- Summary: > > When calling "ResourceBundle.getBundle(String, Locale, ClassLoader)" on > JDK8, the ClassLoader can get GC'd before we're finished with it. > > This can result in us getting the wrong result back, like if we asked for > "Stuff" with the locale "fr, CA" and got back "Stuff_fr.class" instead of > "Stuff_fr_CA.class". > > -- Likely Cause: > > Inside the CacheKey object in the getBundleImpl method, we use a soft > reference that can result in the ClassLoader object getting GC'd before > we're done with it. > > -- Example Fix: > > One fix proven to work is to create a static, volatile Object with global > scope in ResourceBundle, and compare it ("==" style) to the loader right > before it gets returned from getBundleImpl. > > The result of the compare can be something direct like "throw new > Error("Unexpected error.")", as this compare should never return true. > > -- Only seen on... > > This defect is only seen on JDK8, on ppcle and zLinux, and hasn't yet been > seen using the Hotspot VM (defect found using OpenJ9 VM). > ResourceBundle has changed significantly since JDK 8 so I'm curious if you have studied the implementation in the mainline (jdk/jdk) to see if the issue is possible there. If it's still a bug then I assume we would want to fix it in jdk/jdk first before thinking about a back port. -Alan From peter.levart at gmail.com Mon Jul 16 13:12:32 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 16 Jul 2018 15:12:32 +0200 Subject: RFR (Unraised): JDK8 ResourceBundle vulnerable to GC In-Reply-To: <33de7bf7-d313-9f6d-0a7c-d1dd7147cb76@oracle.com> References: <33de7bf7-d313-9f6d-0a7c-d1dd7147cb76@oracle.com> Message-ID: Hi, On 07/16/2018 02:10 PM, Alan Bateman wrote: > On 11/07/2018 14:27, Adam Farley8 wrote: >> Hi All, >> >> -- Summary: >> >> When calling "ResourceBundle.getBundle(String, Locale, ClassLoader)" on >> JDK8, the ClassLoader can get GC'd before we're finished with it. >> >> This can result in us getting the wrong result back, like if we asked >> for >> "Stuff" with the locale "fr, CA" and got back "Stuff_fr.class" >> instead of >> "Stuff_fr_CA.class". >> >> -- Likely Cause: >> >> Inside the CacheKey object in the getBundleImpl method, we use a soft >> reference that can result in the ClassLoader object getting GC'd before >> we're done with it. >> >> -- Example Fix: >> >> One fix proven to work is to create a static, volatile Object with >> global >> scope in ResourceBundle, and compare it ("==" style) to the loader right >> before it gets returned from getBundleImpl. >> >> The result of the compare can be something direct like "throw new >> Error("Unexpected error.")", as this compare should never return true. >> >> -- Only seen on... >> >> This defect is only seen on JDK8, on ppcle and zLinux, and hasn't yet >> been >> seen using the Hotspot VM (defect found using OpenJ9 VM). >> > ResourceBundle has changed significantly since JDK 8 so I'm curious if > you have studied the implementation in the mainline (jdk/jdk) to see > if the issue is possible there. If it's still a bug then I assume we > would want to fix it in jdk/jdk first before thinking about a back port. > > -Alan Checking the differences shows that instead of WeakReference, CacheKey uses two WeakReference instead in JDK 9+. One is the caller module and the other is the module of the bundle (unnamed module if ClassLoader is used to search for the bundle). The caller module is guaranteed to stay reachable for the entire call (as it holds the calling code), but the Module of the bundle could go out of scope during the call, so I think there is a theoretical possibility that the lookup CacheKey becomes non-equal to a key in the cache before the cache lookup is executed. Placing Reference.reachabilityFence(module) at the right place(s) should help eliminate the possibility. The fix for JDK 8 can not use Reference.reachabilityFence(module) as it is JDK 9+ API, but as we learned, an empty static method taking an Object parameter is equivalent to Reference.reachabilityFence() at least in the HotSpot VM. That might not be true for J9 VM though... Regards, Peter From matthias.baesken at sap.com Mon Jul 16 13:53:29 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 16 Jul 2018 13:53:29 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> Message-ID: Hello, after latest comments from Alan and Jaikiran I created a new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8205525.2/ The jar file path is now printed in case jdk.includeInExceptions contains jarpath (this approach is "borrowed" from the enhanced socket exceptions ) . The line number is always printed . Best regards, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Dienstag, 10. Juli 2018 11:53 > To: 'Alan Bateman' ; core-libs- > dev at openjdk.java.net; 'jai.forums2013 at gmail.com' > > Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > Hi Alan, thanks for commenting on this . > > Jaikiran mentioned that printing just the jar file name and not file with > path might be okay : > > > I am not a reviewer and neither do I have enough knowledge about > whether > > jar/file _names_ are considered security sensitive. However, the patch > > that's proposed for this change, prints the file _path_ (and not just > > the name). That I believe is security sensitive. > > What do you think ? > > Best regards, Matthias > > > > -----Original Message----- > > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > > Sent: Sonntag, 8. Juli 2018 09:36 > > To: Baesken, Matthias ; core-libs- > > dev at openjdk.java.net > > Cc: Lindenmaier, Goetz > > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > > parsing of jar archives > > > > On 06/07/2018 13:44, Baesken, Matthias wrote: > > > Hi Alan ,so it looks like JDK-8204233 added a switch (system property) > to > > enable the enhanced socket IOException messages . > > > > > > That would be an option as well for 8205525 . > > Yes, it's documented in conf/security/java.security and something > > equivalent could be done here. The giveaway in your original patch is > > that it needed a privileged block to create the exception message. > > > > > > > > 8205525 adds the jar file name and the line number info to the > > exception message . > > > > > > In case that only the jar file name would be considered sensitive , I > would > > prefer to just output the line number (and omit the system property ). > > > > > That should be okay (I can't think of any concerns). > > > > -Alan From Alan.Bateman at oracle.com Mon Jul 16 14:00:16 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 16 Jul 2018 15:00:16 +0100 Subject: RFR (Unraised): JDK8 ResourceBundle vulnerable to GC In-Reply-To: References: <33de7bf7-d313-9f6d-0a7c-d1dd7147cb76@oracle.com> Message-ID: On 16/07/2018 14:12, Peter Levart wrote: > > Checking the differences shows that instead of > WeakReference, CacheKey uses two WeakReference > instead in JDK 9+. One is the caller module and the other is the > module of the bundle (unnamed module if ClassLoader is used to search > for the bundle). The caller module is guaranteed to stay reachable for > the entire call (as it holds the calling code), but the Module of the > bundle could go out of scope during the call, so I think there is a > theoretical possibility that the lookup CacheKey becomes non-equal to > a key in the cache before the cache lookup is executed. This would require module layers or custom class loaders in the picture for that to happen but I agree we should fix it. -Alan From adam.farley at uk.ibm.com Mon Jul 16 15:13:13 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Mon, 16 Jul 2018 16:13:13 +0100 Subject: RFR (Unraised): JDK8 ResourceBundle vulnerable to GC In-Reply-To: References: <33de7bf7-d313-9f6d-0a7c-d1dd7147cb76@oracle.com> Message-ID: > Alan Bateman wrote on 16/07/2018 15:00:16: > From: Alan Bateman > To: Peter Levart , Adam Farley8 > , core-libs-dev > Date: 16/07/2018 14:59 > Subject: Re: RFR (Unraised): JDK8 ResourceBundle vulnerable to GC > > On 16/07/2018 14:12, Peter Levart wrote: > > > > Checking the differences shows that instead of > > WeakReference, CacheKey uses two WeakReference > > instead in JDK 9+. One is the caller module and the other is the > > module of the bundle (unnamed module if ClassLoader is used to search > > for the bundle). The caller module is guaranteed to stay reachable for > > the entire call (as it holds the calling code), but the Module of the > > bundle could go out of scope during the call, so I think there is a > > theoretical possibility that the lookup CacheKey becomes non-equal to > > a key in the cache before the cache lookup is executed. > This would require module layers or custom class loaders in the picture > for that to happen but I agree we should fix it. > > -Alan > So the JDK8 fix could be introduced as proposed (==). As for the JDK9+ issue, I believe that has already been solved. See the line at the end of the Module-Module-String-Locale-Control getBundleImpl, where it says: Reference.reachabilityFence(module); As for JDK8, to confirm, are you agreeing that the correct fix for this is the originally proposed == (where we compare the Class Loader with a blank static volatile Object (defined outside the method scope) at the end of the getBundleImpl class)? So maybe: ----------------------------------------- +1322 /** * volatile reference object to guard the ClassLoader object * being garbage collected before getBundleImpl() method completes * the caching and retrieving of requested Resourcebundle object */ private static volatile Object vo = new Object(); +1400 //Should never be true. Using the loader here prevents it being GC'd. if (loader == vo) throw new Error("Unexpected error."); ----------------------------------------- Except in a proper webrev. - Adam Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From peter.levart at gmail.com Mon Jul 16 15:29:22 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 16 Jul 2018 17:29:22 +0200 Subject: RFR (Unraised): JDK8 ResourceBundle vulnerable to GC In-Reply-To: References: <33de7bf7-d313-9f6d-0a7c-d1dd7147cb76@oracle.com> Message-ID: <61378544-dec7-3412-8079-69c04bbbf824@gmail.com> On 07/16/2018 05:13 PM, Adam Farley8 wrote: > As for the JDK9+ > issue, I believe that has already been solved. See the line at the end of > the Module-Module-String-Locale-Control getBundleImpl, where it says: > > Reference.reachabilityFence(module); You're right, Adam. I missed that. Regards, Peter From adam.farley at uk.ibm.com Mon Jul 16 16:03:31 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Mon, 16 Jul 2018 17:03:31 +0100 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: <9c1af95d-2c0f-b96d-76a7-5b40228996de@redhat.com> References: <9c1af95d-2c0f-b96d-76a7-5b40228996de@redhat.com> Message-ID: Hi Folks, > Zhengyu Gu wrote on 06/06/2018 01:58:18: > From: Zhengyu Gu > To: "Thomas St?fe" , Adam Farley8 > > Cc: "hotspot-dev at openjdk.java.net developers" dev at openjdk.java.net>, core-libs-dev > Date: 06/06/2018 01:58 > Subject: Re: RFR Bug-pending: Enable Hotspot to Track Native Memory > Usage for Direct Byte Buffers > > On 06/05/2018 12:10 PM, Thomas St?fe wrote:> On Tue, Jun 5, 2018 at 3:46 > PM, Adam Farley8 wrote: > >> Hi All, > >> > >> Native memory allocation for DBBs is tracked in java.nio.Bits, but that > >> only includes what the user thinks they are allocating. > >> > > > > Which is exactly what I would expect as a user... A debugger poring over the values in the file may prefer total accuracy in at least one of the variables, so they can find out where all their memory went in the event of a leak. Even if this variable is not accessible via a getter method, and is only readable via a system core. > > > > I agree with Thomas, there is no point for a user to aware of tracking > overhead, and the overhead only incurs when native memory tracking is > on. As a matter of fact, it can really confuse user that values can be > varied, depending on whether native memory tracking is on. > > Thanks, > > -Zhengyu I agree that the casual user shouldn't have to worry. This accuracy would be for analysis after the fact, via system cores. The standard variables can stay as they are. I suggest the addition of a single AtomicLong that shows an accurate value for the sole purpose of aiding debugging. > > > >> When the VM adds extra memory to the allocation amount this extra bit is > >> not represented in the Bits total. A cursory glance > >> shows, minimum, that we round the requested memory quantity up to the heap > >> word size in the Unsafe.allocateMemory code > > > > which I do not understand either - why do we do this? After all, > > normal allocations from inside hotspot do not get aligned up in size, > > and the java doc to Unsafe allocateMemory does not state anything > > about the size being aligned. > > > > In addition to questioning the align up of the user requested size, I > > would be in favor of adding a new NMT tag for these, maybe "mtUnsafe"? > > That would be an easy fix. > > > >> , and > >> something to do with nmt_header_size in os:malloc() (os.cpp) too. > > > > That is mighty unspecific and also wrong. The align-up mentioned above > > goes into the size reported by Bits; the nmt header size does not. > > I believe we agree here too. My point is that, for the sake of accuracy, we *should* have this information in Bits. This is part of the debugger-aid change that I am suggesting. > >> > >> On its own, and in small quantities, align_up(sz, HeapWordSize) isn't that > >> big of an issue. But when you allocate a lot of DBBs, > >> and coupled with the nmt_header_size business, it makes the Bits values > >> wrong. The more DBB allocations, the more inaccurate those > >> numbers will be. > > > > To be annoyingly precise, it will never be more wrong than 1:7 on > > 64bit machines :) - if all memory requested via Unsafe.allocateMemory > > would be of size 1 byte. Sounds like the sort of thing I'd do. Once a stress tester, always a stress tester. :) > > > >> > >> To get the "+X", it seems to me that the best option would be to introduce > >> an native method in Bits that fetches "X" directly > >> from Hotspot, using the same code that Hotspot uses (so we'd have to > >> abstract-out the Hotspot logic that adds X to the memory > >> quantity). This way, anyone modifying the Hotspot logic won't risk > >> rendering the Bits logic wrong again. > > > > I don't follow that. I was trying to describe one method to enable the VM to tell Bits how much memory will actually be reserved for a given amount of DBB. E.g. Bits says it has a DBB 7 bytes in size, and it tells the VM. The VM replies with "OK, if you came to me and asked for 7 bytes, I'd reserve 8.", and then Bits can update that debugging variable I mentioned. If we abstract out the logic, then Bits and the VM would be using the exact same code when telling Bits how much memory will *hypothetically* be added, as we do when the VM is determining how much overhead it needs when actually reserving the memory. Is that clearer? > > > >> > >> That's only one way to fix the accuracy problem here though. Suggestions > >> welcome. > > > > You are throwing two effects together: > > > > - As mentioned above, I consider the align-up of the user requested > > size to be at least questionable. It shows up as user size in NMT > > which should not be. I also fail to see a compelling reason for it, > > but maybe someone else can enlighten me. Well, if we got rid of it then that's one way to make the Bits variables accurate. :) > > > > - But anything else - NMT headers, overwriter guards, etc added by the > > VM I consider in the same class as any other overhead incurred e.g. by > > the CRT or the OS when calling malloc (e.g. malloc allocator bucket > > size). Basically, rss will go up by more than size requested by > > malloc. Something maybe worth noting, but IMHO not as part of the > > numbers returned by java.nio.Bits. We agree again. No need to confuse things by altering the return values. Simply store the accurate information internally as a debugging aid. > > > > Just my 2 cents. And they are appreciated. Apologies for the delay in my response. - Adam > > > > Best Regards, Thomas > > > >> > >> Best Regards > >> > >> Adam Farley > >> Unless stated otherwise above: > >> IBM United Kingdom Limited - Registered in England and Wales with number > >> 741598. > >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From erik.joelsson at oracle.com Mon Jul 16 16:41:35 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 16 Jul 2018 09:41:35 -0700 Subject: RFR: JDK-8207243: Fix translation filtering to also support zh_HK and zh_TW Message-ID: <04583614-db30-91ef-d478-bca9b195a726@oracle.com> The zh_HK translation is mostly generated by copying the zh_TW translation. Because of this, there are different types of rules defined for generating those and the recently added filtering mechanism won't work for it. This patch fixes that as well as adds those translations to the Oracle filter list. It also further modifies the logging/LocalizedLevelName test to only use translations provided by Oracle. Webrev: http://cr.openjdk.java.net/~erikj/8207243/webrev.01/index.html Bug: https://bugs.openjdk.java.net/browse/JDK-8207243 /Erik From tim.bell at oracle.com Mon Jul 16 17:35:38 2018 From: tim.bell at oracle.com (Tim Bell) Date: Mon, 16 Jul 2018 10:35:38 -0700 Subject: RFR: JDK-8207243: Fix translation filtering to also support zh_HK and zh_TW In-Reply-To: <04583614-db30-91ef-d478-bca9b195a726@oracle.com> References: <04583614-db30-91ef-d478-bca9b195a726@oracle.com> Message-ID: <5B4CD76A.40206@oracle.com> Erik: > The zh_HK translation is mostly generated by copying the zh_TW > translation. Because of this, there are different types of rules defined > for generating those and the recently added filtering mechanism won't > work for it. This patch fixes that as well as adds those translations to > the Oracle filter list. It also further modifies the > logging/LocalizedLevelName test to only use translations provided by > Oracle. > > Webrev: http://cr.openjdk.java.net/~erikj/8207243/webrev.01/index.html > > Bug: https://bugs.openjdk.java.net/browse/JDK-8207243 Looks good. /Tim From ivan.gerasimov at oracle.com Mon Jul 16 17:44:46 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 16 Jul 2018 10:44:46 -0700 Subject: RFR 8207016 : [win] Avoid redundant native memory allocation in getFinalPath() In-Reply-To: References: Message-ID: <66426df6-10aa-bbca-f97a-7b832dd0e052@oracle.com> Thank you Alan for review! On 7/16/18 5:05 AM, Alan Bateman wrote: > On 11/07/2018 03:33, Ivan Gerasimov wrote: >> Hello! >> >> File src/java.base/windows/native/libjava/WinNTFileSystem_md.c >> The function getFinalPath() is wrapper for >> GetFinalPathNameByHandleW(), which is used to resolve the symlinks on >> Windows. >> >> If the resulting string begins with a prefix "\\?\" or "\\?\UNC", a >> copy without the prefix is created and returned. >> >> Instead of creating the copy, we can override the result with >> memmove() to avoid allocating extra native memory. >> The result of getFinalPath() is short living anyway, so a few unused >> bytes at the end of the buffer should not be a problem. >> >> The advantage is a shorter code and minus one point of possible failure. >> >> Would you please help review it? >> >> BUG: https://bugs.openjdk.java.net/browse/JDK-8207016 >> Webrev: http://cr.openjdk.java.net/~igerasim/8207016/00/webrev/ > I think this looks okay. > > -Alan > -- With kind regards, Ivan Gerasimov From brian.burkhalter at oracle.com Tue Jul 17 03:57:21 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 16 Jul 2018 20:57:21 -0700 Subject: 8202794: Native Unix code should use readdir rather than readdir_r Message-ID: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8202794 Continues thread [1] with subject line updated to match the issue summary. Part of this general problem was addressed in [2]. The patch is in effect that proposed in thread [1] less the changes in [2]: http://cr.openjdk.java.net/~bpb/8202794/webrev.00/ Thanks to Bernard Blaser for the patch. There were no build errors on Linux, macOS, nor Solaris, nor in the usual set of regression tests on these platforms. Thanks, Brian [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/052991.html [2] http://mail.openjdk.java.net/pipermail/nio-dev/2018-July/005325.html From Alan.Bateman at oracle.com Tue Jul 17 07:46:17 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Jul 2018 08:46:17 +0100 Subject: 8202794: Native Unix code should use readdir rather than readdir_r In-Reply-To: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> References: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> Message-ID: On 17/07/2018 04:57, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8202794 > > Continues thread [1] with subject line updated to match the issue summary. Part of this general problem was addressed in [2]. > > The patch is in effect that proposed in thread [1] less the changes in [2]: > > http://cr.openjdk.java.net/~bpb/8202794/webrev.00/ > > Thanks to Bernard Blaser for the patch. > > There were no build errors on Linux, macOS, nor Solaris, nor in the usual set of regression tests on these platforms. > For serviceability-dev but I assume OperatingSystemImpl.c should move to readdir64 to be consistent with the other usages. Separately, we have few inconsistencies (including in TimeZone_md.c and UnixFileSystem_md.c) where opendir/closedir is used instead of opendir64/closedir64. Looks like the AIX port may have run into issues as it redefines them. Something for a separate issue of course. -Alan From Alan.Bateman at oracle.com Tue Jul 17 10:46:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Jul 2018 11:46:06 +0100 Subject: 8207393: ServiceLoader class description improvements Message-ID: [1] JDK-8207393 is suggestions from Alex Buckley to improve ServiceLoader's class description and specifically to add a strong recommendation that an application should not require the modules that provide implementations of the service. The diffs for jdk/jdk are inlined below. I have already Reviewed the changes. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8207393 diff --git a/src/java.base/share/classes/java/util/ServiceLoader.java b/src/java.base/share/classes/java/util/ServiceLoader.java --- a/src/java.base/share/classes/java/util/ServiceLoader.java +++ b/src/java.base/share/classes/java/util/ServiceLoader.java @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2005, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2005, 2018, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -65,19 +65,21 @@ ? * interface or class. A {@code ServiceLoader} is an object that locates and ? * loads service providers deployed in the run time environment at a time of an ? * application's choosing. Application code refers only to the service, not to - * service providers, and is assumed to be capable of differentiating between - * multiple service providers as well as handling the possibility that no service - * providers are located. + * service providers, and is assumed to be capable of choosing between multiple + * service providers (based on the functionality they expose through the service), + * and handling the possibility that no service providers are located. ? * ? *

Obtaining a service loader

? * ? *

An application obtains a service loader for a given service by invoking - * one of the static {@code load} methods of ServiceLoader. If the application - * is a module, then its module declaration must have a uses directive - * that specifies the service; this helps to locate providers and ensure they - * will execute reliably. In addition, if the service is not in the application - * module, then the module declaration must have a requires directive - * that specifies the module which exports the service. + * one of the static {@code load} methods of {@code ServiceLoader}. If the + * application is a module, then its module declaration must have a uses + * directive that specifies the service; this helps to locate providers and ensure + * they will execute reliably. In addition, if the application module does not + * contain the service, then its module declaration must have a requires + * directive that specifies the module which exports the service. It is strongly + * recommended that the application module does not require modules which + * contain providers of the service. ? * ? *

A service loader can be used to locate and instantiate providers of the ? * service by means of the {@link #iterator() iterator} method. {@code ServiceLoader} From matthias.baesken at sap.com Tue Jul 17 11:15:24 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 17 Jul 2018 11:15:24 +0000 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] Message-ID: <49a61af46449420ba82075433ab835c5@sap.com> Please review this small fix for allowing windows UNC paths in the jar -C parameter. Currently passing a UNC path to a directory with -C

fails : c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir README.txt Illegal option: s Try `jar --help' for more information. With the patch it works . webrev http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ bug https://bugs.openjdk.java.net/browse/JDK-8207395 Thanks, Matthias From Alan.Bateman at oracle.com Tue Jul 17 11:39:02 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Jul 2018 12:39:02 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> Message-ID: <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> On 16/07/2018 14:53, Baesken, Matthias wrote: > Hello, after latest comments from Alan and Jaikiran I created a new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205525.2/ > > The jar file path is now printed in case jdk.includeInExceptions contains jarpath (this approach is "borrowed" from the enhanced socket exceptions ) . > The line number is always printed . > The general approach seems okay and consistent with the agreement on how to reveal host names in socket exceptions. The name of the security/system property will need discussion as "jdk.includeInExceptions" is very general. If we have something general then we'll need a good name and replace the existing jdk.net.includeInExceptions. It might be better to go with something specific for the area such as "jdk.util.jar.includeInExceptions=jarpath" (to be consistent with other jdk.* properties in this code). A CSR will be needed for this too as the property will be documented in the java.security file. As regards the patch then it mostly looks okay but I think the changes in Attributes will need cleanup to get it consistent (esp. the line lengths) with the existing code. -Alan From enasser at in.ibm.com Tue Jul 17 13:48:01 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Tue, 17 Jul 2018 19:18:01 +0530 Subject: Adding new IBM extended charsets In-Reply-To: <4cba7633-33b7-f29a-fb34-05a06b224057@oracle.com> References: <5B3E4AC8.3070307@oracle.com> <4cba7633-33b7-f29a-fb34-05a06b224057@oracle.com> Message-ID: Hi Alan, Thank you for your inputs. I would like to clarify that all the IBM charsets (IBMXXXX) in jdk.charsets are not IBM platform specific charsets. For example, only 43 charsets out of 72 IBMXXXX in jdk.charsets are EBCDIC or IBM platform specific charsets. Similarly, many charsets in the list of 75 charsets which we would like to contribute are not EBCDIC charsets. I feel we should have a standard guideline for the extended charsets. If we are keeping the extended charsets in the JDK, then we may want to consider all ICU/IANA approved charsets in JDK. Otherwise, we may want to keep only the standard charsets in JDK and remove all the extended charsets so that all extended charsets can be taken from third party libraries like ICU4J. If we decided to keep the extended charsets, then may be we can classify the extended charsets as ASCII and EBCDIC and the corresponding modules as jdk.ascii.charset and jdk.ebcdic.charset. Then, depends upon the platform, we can consider including either of the charset module or both. Please advise. Thank you, Nasser Ebrahim From: Alan Bateman To: Nasser Ebrahim , Xueming Shen , core-libs-dev at openjdk.java.net Date: 07/09/2018 01:25 AM Subject: Re: Adding new IBM extended charsets On 06/07/2018 14:56, Nasser Ebrahim wrote: > : > I understood you preferred option is 3 [Remove all extended charsets from > JDK (keep only default charsets) and use the extended charsets from third > party like ICU4J]. Just to confirm, so you meant we need to keep only the > standard charsets in the JDK and remove all the extended charsets from JDK > and use them from ICU4J OR you meant apply that only for the new extended > charsets. I think it is better to keep the consistency - either take all > extended charsets from ICU4J or maintain all extended charsets with JDK. > Keeping some extended charsets within JDK and use ICU4J for other extended > charsets may confuse the Java user. I think the suggestion in Sherman's mail is to drop the 70 or so IBM charsets from jdk.charsets. This will reduce the size of jdk.charsets and eliminate the need to maintain these charsets (at least on non-AIX builds). If developers need these charsets, say when connecting to database on an IBM system, then they can deploy the ICU4J provider on the class path or module path. I don't think the suggestion impacts the 11 IBM charsets in java.base on non-AIX builds or the non-IBM charsets in jdk.charsets. They may be opportunities to drop some of these but that can be looked at separately. Also I don't think the suggestion impacts the additional 12 IBM charsets that are included in the AIX build of java.base at this time. From the review threads, it seems there are supported locales on AIX that map to these charsets so this is why they are in java.base. -Alan. From bob.vandette at oracle.com Tue Jul 17 14:00:08 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 17 Jul 2018 10:00:08 -0400 Subject: RFR: 8206456 - [TESTBUG] docker jtreg tests fail on systems without cpuset.effective_cpus / cpuset.effective_mem Message-ID: Please review this fix which eliminates some docker/cgroup test failures when running on older Linux kernels with missing cgroup metric files. BUGS: https://bugs.openjdk.java.net/browse/JDK-8206456 WEBREV: http://cr.openjdk.java.net/~bobv/8206456/webrev/ This fix has been verified by the reporter of the issue. Bob. From Roger.Riggs at Oracle.com Tue Jul 17 14:09:33 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 17 Jul 2018 10:09:33 -0400 Subject: RFR 8189717: Too much documentation of ProcessBuilder.start copied to ProcessBuilder.startPipeline In-Reply-To: <4FC1E7DC-A7DC-4B69-9CB3-F5EF8A94D00C@oracle.com> References: <91E0C567-701E-4697-8AE7-21AA72ED19C4@oracle.com> <547AAD0E-0D4B-4869-BB3A-C945C279326B@oracle.com> <4ae3362e-f08c-cdf5-745a-2bb245ad8e31@Oracle.com> <4FC1E7DC-A7DC-4B69-9CB3-F5EF8A94D00C@oracle.com> Message-ID: The corresponding CSR is: ? https://bugs.openjdk.java.net/browse/JDK-8207403 With a reviewer, it can be finalized sooner. Thanks, Roger On 7/13/2018 4:50 PM, Lance Andersen wrote: > Hi Roger, > > +1 > > Have a great weekend! > > Best > Lance >> On Jul 13, 2018, at 4:38 PM, Roger Riggs > > wrote: >> >> Hi Lance, >> >> Updated in place with your suggestion. >> http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ >> >> >> Thanks, Roger >> >> On 7/13/2018 4:31 PM, Lance Andersen wrote: >>> Hi Roger, >>> >>>> On Jul 13, 2018, at 3:48 PM, Roger Riggs >>> > wrote: >>>> >>>> Hi Lance, >>>> >>>> How about: >>>> Subsequent modifications to any of {@code builder}swill not affect >>>> the returned {@link Process}. >>> ?.. to any of the specified builders ?. >>> >>> At a minimum, you need ??the? added to the above >>> >>>> Roger >>>> >>>> >>>> On 7/13/2018 3:26 PM, Lance Andersen wrote: >>>>> Hi Roger, >>>>> >>>>> I am struggling with the wording at: >>>>> >>>>> ??? >>>>> * Subsequent modifications to any of process builders in the >>>>> {@code builders} >>>>> 1198 * argument will not affect the returned {@link Process}. >>>>> >>>>> ???? >>>>> >>>>> Maybe specified/provided ?process builders >>>>> >>>>> Not sure, will think about it a bit. >>>>> >>>>> Otherwise I think the changes are fine >>>>>> On Jul 13, 2018, at 3:07 PM, Roger Riggs >>>>> > wrote: >>>>>> >>>>>> Please review a doc change to improve the description of >>>>>> ProcessBuilder.startPipeline. >>>>>> The clarifications are not intended to imply any change in >>>>>> behavior and only focus the description >>>>>> on the steps taken for each command in the pipeline. >>>>>> (A CSR will be needed also when the wording is finalized). >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-process-8189717/ >>>>>> >>>>>> Issue: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8189717 >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Lance >>>>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>> Oracle?Java Engineering >>>>> 1 Network Drive >>>>> Burlington, MA 01803 >>>>> Lance.Andersen at oracle.com >>>>> >>>>> >>>>> >>>> >>> >>> >>> >>> Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle?Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From matthias.baesken at sap.com Tue Jul 17 14:13:08 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 17 Jul 2018 14:13:08 +0000 Subject: 8206456 - [TESTBUG] docker jtreg tests fail on systems without cpuset.effective_cpus / cpuset.effective_mem In-Reply-To: References: Message-ID: Hi Bob, looks good (I am not a Reviewer however) ! The reported issues occured on a SUSE Linux 12 SP1 system , where /sys/fs/cgroup/cpuset/cpuset.effective_cpus and /sys/fs/cgroup/cpuset/cpuset.effective_mems are not present . I applied Bobs patch , now the jdk/internal/platform/docker - jtreg tests do not fail any more on the mentioned system . Thanks, Matthias > -----Original Message----- > From: Bob Vandette [mailto:bob.vandette at oracle.com] > Sent: Dienstag, 17. Juli 2018 16:00 > To: serviceability-dev at openjdk.java.net serviceability- > dev at openjdk.java.net ; core-libs- > dev > Cc: Baesken, Matthias ; Schmidt, Lutz > > Subject: RFR: 8206456 - [TESTBUG] docker jtreg tests fail on systems without > cpuset.effective_cpus / cpuset.effective_mem > > Please review this fix which eliminates some docker/cgroup test failures > when running on older > Linux kernels with missing cgroup metric files. > > BUGS: > https://bugs.openjdk.java.net/browse/JDK-8206456 > > WEBREV: > http://cr.openjdk.java.net/~bobv/8206456/webrev/ > > This fix has been verified by the reporter of the issue. > > Bob. > > > From Roger.Riggs at Oracle.com Tue Jul 17 14:16:59 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 17 Jul 2018 10:16:59 -0400 Subject: RFR 8205610: [TESTLIB] Improve listing of open file descriptors Message-ID: Please review a suggested improvement to the test library that lists open file descriptors, adding support for Solaris pfiles. webrev: ? ? http://cr.openjdk.java.net/~rriggs/webrev-lsof-8205610/ Issue: ??? https://bugs.openjdk.java.net/browse/JDK-8205610 Thanks, Roger From lance.andersen at oracle.com Tue Jul 17 15:58:25 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 17 Jul 2018 11:58:25 -0400 Subject: RFR 8205610: [TESTLIB] Improve listing of open file descriptors In-Reply-To: References: Message-ID: <3CA94244-BD61-4A08-88F3-5991962529C6@oracle.com> The changes look good Roger > On Jul 17, 2018, at 10:16 AM, Roger Riggs wrote: > > Please review a suggested improvement to the test library that lists open file descriptors, > adding support for Solaris pfiles. > > webrev: > http://cr.openjdk.java.net/~rriggs/webrev-lsof-8205610/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8205610 > > Thanks, Roger > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From bsrbnd at gmail.com Tue Jul 17 16:05:03 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Tue, 17 Jul 2018 18:05:03 +0200 Subject: 8202794: Native Unix code should use readdir rather than readdir_r In-Reply-To: References: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> Message-ID: On 17 July 2018 at 09:46, Alan Bateman wrote: > On 17/07/2018 04:57, Brian Burkhalter wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8202794 >> >> Continues thread [1] with subject line updated to match the issue summary. >> Part of this general problem was addressed in [2]. >> >> The patch is in effect that proposed in thread [1] less the changes in >> [2]: >> >> http://cr.openjdk.java.net/~bpb/8202794/webrev.00/ >> >> Thanks to Bernard Blaser for the patch. >> >> There were no build errors on Linux, macOS, nor Solaris, nor in the usual >> set of regression tests on these platforms. >> > For serviceability-dev but I assume OperatingSystemImpl.c should move to > readdir64 to be consistent with the other usages. > > Separately, we have few inconsistencies (including in TimeZone_md.c and > UnixFileSystem_md.c) where opendir/closedir is used instead of > opendir64/closedir64. Looks like the AIX port may have run into issues as it > redefines them. Something for a separate issue of course. > > -Alan Looks good to me too. I've also noted the small inconsistencies between readdir64 and opendir/closedir to be addressed in a separate issue. Thanks, Bernard From martinrb at google.com Tue Jul 17 16:46:33 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 17 Jul 2018 09:46:33 -0700 Subject: Adding new IBM extended charsets In-Reply-To: References: <5B3E4AC8.3070307@oracle.com> <4cba7633-33b7-f29a-fb34-05a06b224057@oracle.com> Message-ID: History: I recall 15 years ago wondering why no one from IBM was maintaining the IBM charsets in the jdk. Today anything but UTF-* is increasingly legacy, so the case for including them is weaker than back then. Probably today the IBM charsets are best maintained outside the jdk sources as a third party package, especially considering that openjdk has been kicking some other components like corba out of the nest. But there is some synergy - some of the openjdk charset tests check invariants that apply to all charset implementations. I may have even fixed some IBM charset code many years ago. From alexfoster at hotmail.ca Mon Jul 16 17:36:07 2018 From: alexfoster at hotmail.ca (Alex Foster) Date: Mon, 16 Jul 2018 17:36:07 +0000 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: , Message-ID: Hi again, I updated ArrayDeque with your last idea (adding all list methods but not implementing List) and added a subclass ArrayDeque.List which overrides equals and hashcode and implements List. There is also a subclass ArrayDeque.WithNulls that accepts null elements. ArrayDeque has removeAt(int index) instead of remove(index) to avoid overloading remove(Object). I also added shared empty arrays similar to Arraylist, and a reallocate method which can do the same things as trimToSize and ensure capacity in Arraylist. It also allows you to trim to a specific capacity other than the size or skip trimming if the capacity is within a specified distance of the target capacity. Also the bulk add methods call collection.toArray, then check the array for illegal elements, then add the array, which means that a collection could keep the array it returns from toArray and modify it from another thread after it has been checked but before it has been added which could lead to illegal elements being added to the ArrayDeque. We could maybe avoid this by cloning the array or checking the elements after adding them but I'm not sure if it's worth it... What do you think? I also changed the WhiteBox test a bit: --- a/test/jdk/java/util/ArrayDeque/WhiteBox.java +++ b/test/jdk/java/util/ArrayDeque/WhiteBox.java @@ -88,7 +88,10 @@ @Test public void defaultConstructor() { - checkCapacity(new ArrayDeque(), 16); + ArrayDeque d = new ArrayDeque(); + d.add(new Object()); + d.clear(); + checkCapacity(d, 16); } @Test @@ -131,7 +134,7 @@ if (rnd.nextBoolean()) d.add(99); ArrayDeque clone = serialClone(d); assertInvariants(clone); - assertNotSame(elements(d), elements(clone)); + assertTrue(d.isEmpty() || elements(d) != elements(clone)); assertEquals(d, clone); } } Alex From brian.burkhalter at oracle.com Tue Jul 17 17:13:46 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 17 Jul 2018 10:13:46 -0700 Subject: 8202794: Native Unix code should use readdir rather than readdir_r In-Reply-To: References: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> Message-ID: <92926BC2-A2A0-4713-B444-355A6C1D6737@oracle.com> On Jul 17, 2018, at 9:05 AM, B. Blaser wrote: > On 17 July 2018 at 09:46, Alan Bateman wrote: >> >> For serviceability-dev but I assume OperatingSystemImpl.c should move to >> readdir64 to be consistent with the other usages. I made this small change: http://cr.openjdk.java.net/~bpb/8202794/webrev.01/ >> Separately, we have few inconsistencies (including in TimeZone_md.c and >> UnixFileSystem_md.c) where opendir/closedir is used instead of >> opendir64/closedir64. Looks like the AIX port may have run into issues as it >> redefines them. Something for a separate issue of course. No changes made on these points. > Looks good to me too. > I've also noted the small inconsistencies between readdir64 and > opendir/closedir to be addressed in a separate issue. Is there any issue on file for these things? I could file one but perhaps someone else already knows more detail. Thanks, Brian From brian.burkhalter at oracle.com Tue Jul 17 17:45:33 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 17 Jul 2018 10:45:33 -0700 Subject: RFR 8205610: [TESTLIB] Improve listing of open file descriptors In-Reply-To: References: Message-ID: Looks good, Roger Brian On Jul 17, 2018, at 7:16 AM, Roger Riggs wrote: > Please review a suggested improvement to the test library that lists open file descriptors, > adding support for Solaris pfiles. > > webrev: > http://cr.openjdk.java.net/~rriggs/webrev-lsof-8205610/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8205610 From lance.andersen at oracle.com Tue Jul 17 18:34:44 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 17 Jul 2018 14:34:44 -0400 Subject: 8207393: ServiceLoader class description improvements In-Reply-To: References: Message-ID: Hi Alan, The changes read well Best Lance > On Jul 17, 2018, at 6:46 AM, Alan Bateman wrote: > > [1] JDK-8207393 is suggestions from Alex Buckley to improve ServiceLoader's class description and specifically to add a strong recommendation that an application should not require the modules that provide implementations of the service. > > The diffs for jdk/jdk are inlined below. I have already Reviewed the changes. > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8207393 > > > diff --git a/src/java.base/share/classes/java/util/ServiceLoader.java b/src/java.base/share/classes/java/util/ServiceLoader.java > --- a/src/java.base/share/classes/java/util/ServiceLoader.java > +++ b/src/java.base/share/classes/java/util/ServiceLoader.java > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2005, 2017, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2005, 2018, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -65,19 +65,21 @@ > * interface or class. A {@code ServiceLoader} is an object that locates and > * loads service providers deployed in the run time environment at a time of an > * application's choosing. Application code refers only to the service, not to > - * service providers, and is assumed to be capable of differentiating between > - * multiple service providers as well as handling the possibility that no service > - * providers are located. > + * service providers, and is assumed to be capable of choosing between multiple > + * service providers (based on the functionality they expose through the service), > + * and handling the possibility that no service providers are located. > * > *

Obtaining a service loader

> * > *

An application obtains a service loader for a given service by invoking > - * one of the static {@code load} methods of ServiceLoader. If the application > - * is a module, then its module declaration must have a uses directive > - * that specifies the service; this helps to locate providers and ensure they > - * will execute reliably. In addition, if the service is not in the application > - * module, then the module declaration must have a requires directive > - * that specifies the module which exports the service. > + * one of the static {@code load} methods of {@code ServiceLoader}. If the > + * application is a module, then its module declaration must have a uses > + * directive that specifies the service; this helps to locate providers and ensure > + * they will execute reliably. In addition, if the application module does not > + * contain the service, then its module declaration must have a requires > + * directive that specifies the module which exports the service. It is strongly > + * recommended that the application module does not require modules which > + * contain providers of the service. > * > *

A service loader can be used to locate and instantiate providers of the > * service by means of the {@link #iterator() iterator} method. {@code ServiceLoader} > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Alan.Bateman at oracle.com Tue Jul 17 18:45:49 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Jul 2018 19:45:49 +0100 Subject: 8202794: Native Unix code should use readdir rather than readdir_r In-Reply-To: <92926BC2-A2A0-4713-B444-355A6C1D6737@oracle.com> References: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> <92926BC2-A2A0-4713-B444-355A6C1D6737@oracle.com> Message-ID: <748cdd1e-f86c-9dea-bcf9-2cc236a2f666@oracle.com> On 17/07/2018 18:13, Brian Burkhalter wrote: > > I made this small change: > > http://cr.openjdk.java.net/~bpb/8202794/webrev.01/ > Looks okay. > : > > Is there any issue on file for these things? I could file one but > perhaps someone else already knows more detail. > It would be good to create an issue on this. -Alan From brian.burkhalter at oracle.com Tue Jul 17 19:01:34 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 17 Jul 2018 12:01:34 -0700 Subject: 8202794: Native Unix code should use readdir rather than readdir_r In-Reply-To: <748cdd1e-f86c-9dea-bcf9-2cc236a2f666@oracle.com> References: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> <92926BC2-A2A0-4713-B444-355A6C1D6737@oracle.com> <748cdd1e-f86c-9dea-bcf9-2cc236a2f666@oracle.com> Message-ID: <96F0CC88-900F-47E7-B309-6698F772DDCE@oracle.com> On Jul 17, 2018, at 11:45 AM, Alan Bateman wrote: > On 17/07/2018 18:13, Brian Burkhalter wrote: >> >> I made this small change: >> >> http://cr.openjdk.java.net/~bpb/8202794/webrev.01/ > Looks okay. Thanks. >> : >> >> Is there any issue on file for these things? I could file one but perhaps someone else already knows more detail. >> > It would be good to create an issue on this. Done: https://bugs.openjdk.java.net/browse/JDK-8207744 Brian From bsrbnd at gmail.com Tue Jul 17 19:07:18 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Tue, 17 Jul 2018 21:07:18 +0200 Subject: 8202794: Native Unix code should use readdir rather than readdir_r In-Reply-To: <92926BC2-A2A0-4713-B444-355A6C1D6737@oracle.com> References: <292C5A78-273A-47C4-85C1-124AEE19B680@oracle.com> <92926BC2-A2A0-4713-B444-355A6C1D6737@oracle.com> Message-ID: On 17 July 2018 at 19:13, Brian Burkhalter wrote: > > On Jul 17, 2018, at 9:05 AM, B. Blaser wrote: > > On 17 July 2018 at 09:46, Alan Bateman wrote: > > For serviceability-dev but I assume OperatingSystemImpl.c should move to > readdir64 to be consistent with the other usages. > > > I made this small change: > > http://cr.openjdk.java.net/~bpb/8202794/webrev.01/ Looks good to me too. Thanks, Bernard From brian.burkhalter at oracle.com Tue Jul 17 23:18:07 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 17 Jul 2018 16:18:07 -0700 Subject: RFR [12] 8207748: Fix for 8202794 breaks tier1 builds Message-ID: <2B4ADB03-D6F7-45EA-ABD0-887B2DBD7107@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8207748.01 Diff included below. Build verified on Linux, macOS, and Solaris. Thanks, Brian --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c @@ -75,6 +75,7 @@ #endif /* _ALLBSD_SOURCE */ #if defined(_ALLBSD_SOURCE) + #define dirent64 dirent #define readdir64 readdir #endif @@ -422,7 +423,7 @@ return (100); #else /* solaris/linux */ DIR *dirp; - struct dirent* dentp; + struct dirent64* dentp; jlong fds = 0; From joe.darcy at oracle.com Tue Jul 17 23:19:26 2018 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 17 Jul 2018 16:19:26 -0700 Subject: JDK 12 RFR of JDK-8207751: Remove misleading serialVersionUID from JulienFields.Field Message-ID: <7dde869b-584a-47ef-7074-262004cf08bc@oracle.com> Hello, The nested enum java.time.temporal.JulianFields.Field declare a serialVersionUID field. Per the serialization spec, a serialVersionUID field in an enum is ignored. "The process by which enum constants are serialized cannot be customized: any class-specific writeObject, readObject, readObjectNoData, writeReplace, and readResolve methods defined by enum types are ignored during serialization and deserialization. Similarly, any serialPersistentFields or serialVersionUID field declarations are also ignored--all enum types have a fixed serialVersionUID of 0L. Documenting serializable fields and data for enum types is unnecessary, since there is no variation in the type of data sent." https://docs.oracle.com/javase/10/docs/specs/serialization/serial-arch.html#serialization-of-enum-constants Therefore, this misleading field should be deleted; please review the patch below which does this. Thanks, -Joe --- a/src/java.base/share/classes/java/time/temporal/JulianFields.java Tue Jul 17 17:14:03 2018 -0400 +++ b/src/java.base/share/classes/java/time/temporal/JulianFields.java Tue Jul 17 16:07:22 2018 -0700 @@ -222,8 +222,6 @@ ???????? MODIFIED_JULIAN_DAY("ModifiedJulianDay", DAYS, FOREVER, 40587L), ???????? RATA_DIE("RataDie", DAYS, FOREVER, 719163L); -??????? private static final long serialVersionUID = -7501623920830201812L; - ???????? private final transient String name; ???????? private final transient TemporalUnit baseUnit; ???????? private final transient TemporalUnit rangeUnit; From joe.darcy at oracle.com Tue Jul 17 23:20:39 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 17 Jul 2018 16:20:39 -0700 Subject: RFR [12] 8207748: Fix for 8202794 breaks tier1 builds In-Reply-To: <2B4ADB03-D6F7-45EA-ABD0-887B2DBD7107@oracle.com> References: <2B4ADB03-D6F7-45EA-ABD0-887B2DBD7107@oracle.com> Message-ID: <5B4E79C7.5010900@oracle.com> +1 -Joe On 7/17/2018 4:18 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8207748.01 > > Diff included below. Build verified on Linux, macOS, and Solaris. > > Thanks, > > Brian > > --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c > +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c > @@ -75,6 +75,7 @@ > #endif /* _ALLBSD_SOURCE */ > > #if defined(_ALLBSD_SOURCE) > + #define dirent64 dirent > #define readdir64 readdir > #endif > > @@ -422,7 +423,7 @@ > return (100); > #else /* solaris/linux */ > DIR *dirp; > - struct dirent* dentp; > + struct dirent64* dentp; > jlong fds = 0; > From kim.barrett at oracle.com Tue Jul 17 23:23:06 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 17 Jul 2018 19:23:06 -0400 Subject: RFR [12] 8207748: Fix for 8202794 breaks tier1 builds In-Reply-To: <2B4ADB03-D6F7-45EA-ABD0-887B2DBD7107@oracle.com> References: <2B4ADB03-D6F7-45EA-ABD0-887B2DBD7107@oracle.com> Message-ID: <45004710-5CD0-4424-911F-E1C590E685EE@oracle.com> > On Jul 17, 2018, at 7:18 PM, Brian Burkhalter wrote: > > https://bugs.openjdk.java.net/browse/JDK-8207748.01 > > Diff included below. Build verified on Linux, macOS, and Solaris. > > Thanks, > > Brian > > --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c > +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c > @@ -75,6 +75,7 @@ > #endif /* _ALLBSD_SOURCE */ > > #if defined(_ALLBSD_SOURCE) > + #define dirent64 dirent > #define readdir64 readdir > #endif > > @@ -422,7 +423,7 @@ > return (100); > #else /* solaris/linux */ > DIR *dirp; > - struct dirent* dentp; > + struct dirent64* dentp; > jlong fds = 0; Looks good. I think AIX may have also been broken by 8202794, and not fixed by this. If so, that can be addressed separately. From lance.andersen at oracle.com Tue Jul 17 23:25:28 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 17 Jul 2018 19:25:28 -0400 Subject: JDK 12 RFR of JDK-8207751: Remove misleading serialVersionUID from JulienFields.Field In-Reply-To: <7dde869b-584a-47ef-7074-262004cf08bc@oracle.com> References: <7dde869b-584a-47ef-7074-262004cf08bc@oracle.com> Message-ID: <99C01C2D-F809-40FF-99E8-F5B53A42F616@oracle.com> Hi Joe, +1 I assume we do not need to add a specific test for this clean-up? > On Jul 17, 2018, at 7:19 PM, joe darcy wrote: > > Hello, > > The nested enum java.time.temporal.JulianFields.Field declare a serialVersionUID field. Per the serialization spec, a serialVersionUID field in an enum is ignored. > > "The process by which enum constants are serialized cannot be customized: any class-specific writeObject, readObject, readObjectNoData, writeReplace, and readResolve methods defined by enum types are ignored during serialization and deserialization. Similarly, any serialPersistentFields or serialVersionUID field declarations are also ignored--all enum types have a fixed serialVersionUID of 0L. Documenting serializable fields and data for enum types is unnecessary, since there is no variation in the type of data sent." > > https://docs.oracle.com/javase/10/docs/specs/serialization/serial-arch.html#serialization-of-enum-constants > > Therefore, this misleading field should be deleted; please review the patch below which does this. > > Thanks, > > -Joe > > --- a/src/java.base/share/classes/java/time/temporal/JulianFields.java Tue Jul 17 17:14:03 2018 -0400 > +++ b/src/java.base/share/classes/java/time/temporal/JulianFields.java Tue Jul 17 16:07:22 2018 -0700 > @@ -222,8 +222,6 @@ > MODIFIED_JULIAN_DAY("ModifiedJulianDay", DAYS, FOREVER, 40587L), > RATA_DIE("RataDie", DAYS, FOREVER, 719163L); > > - private static final long serialVersionUID = -7501623920830201812L; > - > private final transient String name; > private final transient TemporalUnit baseUnit; > private final transient TemporalUnit rangeUnit; > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From joe.darcy at oracle.com Tue Jul 17 23:32:43 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 17 Jul 2018 16:32:43 -0700 Subject: JDK 12 RFR of JDK-8207751: Remove misleading serialVersionUID from JulienFields.Field In-Reply-To: <99C01C2D-F809-40FF-99E8-F5B53A42F616@oracle.com> References: <7dde869b-584a-47ef-7074-262004cf08bc@oracle.com> <99C01C2D-F809-40FF-99E8-F5B53A42F616@oracle.com> Message-ID: <5B4E7C9B.5020607@oracle.com> Hi Lance, On 7/17/2018 4:25 PM, Lance Andersen wrote: > Hi Joe, > > +1 > > I assume we do not need to add a specific test for this clean-up? I put a "noreg-hard" label on the bug; I don't think it merits a dedicated test. Thanks, -Joe From lance.andersen at oracle.com Tue Jul 17 23:53:28 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 17 Jul 2018 19:53:28 -0400 Subject: JDK 12 RFR of JDK-8207751: Remove misleading serialVersionUID from JulienFields.Field In-Reply-To: <5B4E7C9B.5020607@oracle.com> References: <7dde869b-584a-47ef-7074-262004cf08bc@oracle.com> <99C01C2D-F809-40FF-99E8-F5B53A42F616@oracle.com> <5B4E7C9B.5020607@oracle.com> Message-ID: Hi Joe > On Jul 17, 2018, at 7:32 PM, Joseph D. Darcy wrote: > > Hi Lance, > > On 7/17/2018 4:25 PM, Lance Andersen wrote: >> Hi Joe, >> >> +1 >> >> I assume we do not need to add a specific test for this clean-up? > > I put a "noreg-hard" label on the bug; I don't think it merits a dedicated test. I figured as much but figured it couldn?t hurt ask ;-) Best Lance > > Thanks, > > -Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From ivan.gerasimov at oracle.com Wed Jul 18 00:01:43 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 17 Jul 2018 17:01:43 -0700 Subject: RFR 8207750 : Native handle leak in java.io.WinNTFileSystem.list() Message-ID: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> Hello! In the file WinNTFileSystem_md.c there is a potential leak of native handle obtained from FindFirstFileW() in the unfortunate event of memory allocation failure. Would you please help review a simple fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207750 WEBREV: http://cr.openjdk.java.net/~igerasim/8207750/00/webrev/ Thanks in advance! -- With kind regards, Ivan Gerasimov From mandy.chung at oracle.com Wed Jul 18 00:07:03 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 17 Jul 2018 17:07:03 -0700 Subject: RFR: 8206456 - [TESTBUG] docker jtreg tests fail on systems without cpuset.effective_cpus / cpuset.effective_mem In-Reply-To: References: Message-ID: <598c9af3-2041-0be5-f177-b3a031b2ef61@oracle.com> On 7/17/18 7:00 AM, Bob Vandette wrote: > Please review this fix which eliminates some docker/cgroup test failures when running on older > Linux kernels with missing cgroup metric files. > > BUGS: > https://bugs.openjdk.java.net/browse/JDK-8206456 > > WEBREV: > http://cr.openjdk.java.net/~bobv/8206456/webrev/ Nit: It would be clearer to check for the specific metrics: int[] cpusets = metrics.getEffectiveCpuSetCpus(); if (cpusets.length != 0) { .... } Same applies to getEffectiveCpuSetMems. No need for a new webrev. Mandy P.S. I am not sure the conversion from the primitive to boxed type is necessary. But this is not related to this issue. You may want to take a look at that. From brian.burkhalter at oracle.com Wed Jul 18 00:08:20 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 17 Jul 2018 17:08:20 -0700 Subject: RFR 8207750 : Native handle leak in java.io.WinNTFileSystem.list() In-Reply-To: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> References: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> Message-ID: <65676035-F1DD-4922-836D-68FF064ECE56@oracle.com> Hi Ivan, This looks all right to me. Thanks, Brian On Jul 17, 2018, at 5:01 PM, Ivan Gerasimov wrote: > In the file WinNTFileSystem_md.c there is a potential leak of native handle obtained from FindFirstFileW() in the unfortunate event of memory allocation failure. > > Would you please help review a simple fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207750 > WEBREV:http://cr.openjdk.java.net/~igerasim/8207750/00/webrev/ From joe.darcy at oracle.com Wed Jul 18 00:09:13 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 17 Jul 2018 17:09:13 -0700 Subject: JDK 12 RFR of JDK-8207751: Remove misleading serialVersionUID from JulienFields.Field In-Reply-To: References: <7dde869b-584a-47ef-7074-262004cf08bc@oracle.com> <99C01C2D-F809-40FF-99E8-F5B53A42F616@oracle.com> <5B4E7C9B.5020607@oracle.com> Message-ID: <5B4E8529.2000409@oracle.com> PS This change is part of preparatory work to allow stricter checking of serial-related properties by the javac -Xlint:serial checks. So in the fullness of time, the work intended to be done under JDK-8202056: "Expand serial warning to check for bad overloads of serial-related methods" will check for these kinds of properties as part of the build. HTH, -Joe On 7/17/2018 4:53 PM, Lance Andersen wrote: > Hi Joe >> On Jul 17, 2018, at 7:32 PM, Joseph D. Darcy > > wrote: >> >> Hi Lance, >> >> On 7/17/2018 4:25 PM, Lance Andersen wrote: >>> Hi Joe, >>> >>> +1 >>> >>> I assume we do not need to add a specific test for this clean-up? >> >> I put a "noreg-hard" label on the bug; I don't think it merits a >> dedicated test. > > I figured as much but figured it couldn?t hurt ask ;-) > From ivan.gerasimov at oracle.com Wed Jul 18 00:19:04 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 17 Jul 2018 17:19:04 -0700 Subject: RFR 8207750 : Native handle leak in java.io.WinNTFileSystem.list() In-Reply-To: <65676035-F1DD-4922-836D-68FF064ECE56@oracle.com> References: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> <65676035-F1DD-4922-836D-68FF064ECE56@oracle.com> Message-ID: Thank you Brian for review! On 7/17/18 5:08 PM, Brian Burkhalter wrote: > Hi Ivan, > > This looks all right to me. > > Thanks, > > Brian > > On Jul 17, 2018, at 5:01 PM, Ivan Gerasimov > wrote: > >> In the file WinNTFileSystem_md.c there is a potential leak of native >> handle obtained from FindFirstFileW() in the unfortunate event of >> memory allocation failure. >> >> Would you please help review a simple fix? >> >> BUGURL:https://bugs.openjdk.java.net/browse/JDK-8207750 >> WEBREV:http://cr.openjdk.java.net/~igerasim/8207750/00/webrev/ >> > -- With kind regards, Ivan Gerasimov From vyom.tewari at oracle.com Wed Jul 18 05:22:24 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Wed, 18 Jul 2018 10:52:24 +0530 Subject: RFR 8207750 : Native handle leak in java.io.WinNTFileSystem.list() In-Reply-To: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> References: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> Message-ID: <0671112c-55e4-b56c-051b-837515d38de6@oracle.com> Hi Ivan, looks good to me, although i am not the official reviewer. Thanks, Vyom On Wednesday 18 July 2018 05:31 AM, Ivan Gerasimov wrote: > Hello! > > In the file WinNTFileSystem_md.c there is a potential leak of native > handle obtained from FindFirstFileW() in the unfortunate event of > memory allocation failure. > > Would you please help review a simple fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207750 > WEBREV: http://cr.openjdk.java.net/~igerasim/8207750/00/webrev/ > > Thanks in advance! > From thomas.stuefe at gmail.com Wed Jul 18 05:29:13 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 18 Jul 2018 07:29:13 +0200 Subject: RFR 8207750 : Native handle leak in java.io.WinNTFileSystem.list() In-Reply-To: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> References: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com> Message-ID: Hi Ivan, looks good. Best Regards, Thomas On Wed, Jul 18, 2018 at 2:01 AM, Ivan Gerasimov wrote: > Hello! > > In the file WinNTFileSystem_md.c there is a potential leak of native handle > obtained from FindFirstFileW() in the unfortunate event of memory allocation > failure. > > Would you please help review a simple fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207750 > WEBREV: http://cr.openjdk.java.net/~igerasim/8207750/00/webrev/ > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > From goetz.lindenmaier at sap.com Wed Jul 18 07:27:53 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 18 Jul 2018 07:27:53 +0000 Subject: RFR(S): 8207766: [testbug] Adapt tests for Aix. Message-ID: Hi, I would like to fix these tests to run on aix: http://cr.openjdk.java.net/~goetz/wr18/8207766-aixTestFixes/01/ MulticastSocket tests Opened bug and asked IBM to fix. Put on ProblemList. See https://bugs.openjdk.java.net/browse/JDK-8207404 EvalArraysAsList.sh Quotes need to be escaped on AIX. WrappedToolkitTest.sh Check for AIX missing. PKCS11Test.java getNSSLibDir can return null. If so, static initialization of badNSSVersion fails: java.io.FileNotFoundException: nulllibsoftokn3.so (No such file or directory) TestNssDbSqlite.java Don't run this on AIX. There is no NSS on basic installations. launcher/SourceMode.java AIX does not support arguments to the shebang. Best regards, Goetz. From matthias.baesken at sap.com Wed Jul 18 08:21:33 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 18 Jul 2018 08:21:33 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> Message-ID: <6cfa4b0266de4fdf85d6c959c450043c@sap.com> > The name of the security/system property will need discussion as > "jdk.includeInExceptions" is very general. If we have something general > then we'll need a good name and replace the existing > jdk.net.includeInExceptions. It might be better to go with something > specific for the area such as "jdk.util.jar.includeInExceptions=jarpath" > (to be consistent with other jdk.* properties in this code). A CSR will > be needed for this too as the property will be documented in the > java.security file. Hi Alan, I'll prepare a CSR . I selected a more general name "jdk.includeInExceptions" , because there is the idea to enhance more exceptions with additional output . In such a case " jdk.util.jar.includeInExceptions" would not really help . However so far it is still a bit unclear how many exceptions we would like to enhance , so this has to be checked first . Best regards, Matthias > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Dienstag, 17. Juli 2018 13:39 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > On 16/07/2018 14:53, Baesken, Matthias wrote: > > Hello, after latest comments from Alan and Jaikiran I created a new > webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8205525.2/ > > > > The jar file path is now printed in case jdk.includeInExceptions contains > jarpath (this approach is "borrowed" from the enhanced socket > exceptions ) . > > The line number is always printed . > > > The general approach seems okay and consistent with the agreement on > how > to reveal host names in socket exceptions. > > The name of the security/system property will need discussion as > "jdk.includeInExceptions" is very general. If we have something general > then we'll need a good name and replace the existing > jdk.net.includeInExceptions. It might be better to go with something > specific for the area such as "jdk.util.jar.includeInExceptions=jarpath" > (to be consistent with other jdk.* properties in this code). A CSR will > be needed for this too as the property will be documented in the > java.security file. > > As regards the patch then it mostly looks okay but I think the changes > in Attributes will need cleanup to get it consistent (esp. the line > lengths) with the existing code. > > -Alan From goetz.lindenmaier at sap.com Wed Jul 18 09:31:51 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 18 Jul 2018 09:31:51 +0000 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: <49a61af46449420ba82075433ab835c5@sap.com> References: <49a61af46449420ba82075433ab835c5@sap.com> Message-ID: <68104e97f8e042b0badef5fe579c86b3@sap.com> Hi Matthias, thanks for doing this fix. I think this can be noted down a bit better, avoiding duplicating the loop. Also, please remove the redundant dir.replace(File.separatorChar, '/'). dir = dir.replace(File.separatorChar, '/'); + String unc = (dir.startsWith("//") && (File.separatorChar == '\\')) ? "/" : ""; while (dir.indexOf("//") > -1) { dir = dir.replace("//", "/"); } - pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); + // Restore the second leading '/' needed for the Windows UNC path. + dir = unc + dir; + pathsMap.get(version).add(dir); nameBuf[k++] = dir + args[++i]; Best regards, Goetz. > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Baesken, Matthias > Sent: Dienstag, 17. Juli 2018 13:15 > To: core-libs-dev at openjdk.java.net > Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path arguments > for the jar -C parameter [windows] > > Please review this small fix for allowing windows UNC paths in the jar -C > parameter. > Currently passing a UNC path to a directory with -C

fails : > > c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir > README.txt > Illegal option: s > Try `jar --help' for more information. > > With the patch it works . > > webrev > > http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ > > bug > > https://bugs.openjdk.java.net/browse/JDK-8207395 > > > > > Thanks, Matthias From matthias.baesken at sap.com Wed Jul 18 11:18:36 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 18 Jul 2018 11:18:36 +0000 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: <68104e97f8e042b0badef5fe579c86b3@sap.com> References: <49a61af46449420ba82075433ab835c5@sap.com> <68104e97f8e042b0badef5fe579c86b3@sap.com> Message-ID: <445d240e44b04b26af9421a782208235@sap.com> Hi G?tz, thanks for the input ! Should we maybe use dir.replaceAll and not the while loop ? > while (dir.indexOf("//") > -1) { > dir = dir.replace("//", "/"); > } Best regards, Matthias > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Mittwoch, 18. Juli 2018 11:32 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Subject: RE: RFR : 8207395: jar has issues with UNC-path arguments for the > jar -C parameter [windows] > > Hi Matthias, > > thanks for doing this fix. > > I think this can be noted down a bit better, avoiding duplicating the loop. > Also, please remove the redundant dir.replace(File.separatorChar, '/'). > > dir = dir.replace(File.separatorChar, '/'); > + String unc = (dir.startsWith("//") && (File.separatorChar == '\\')) > ? "/" : ""; > while (dir.indexOf("//") > -1) { > dir = dir.replace("//", "/"); > } > - pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); > + // Restore the second leading '/' needed for the Windows UNC > path. > + dir = unc + dir; > + pathsMap.get(version).add(dir); > nameBuf[k++] = dir + args[++i]; > > Best regards, > Goetz. > > > -----Original Message----- > > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > > Behalf Of Baesken, Matthias > > Sent: Dienstag, 17. Juli 2018 13:15 > > To: core-libs-dev at openjdk.java.net > > Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path arguments > > for the jar -C parameter [windows] > > > > Please review this small fix for allowing windows UNC paths in the jar -C > > parameter. > > Currently passing a UNC path to a directory with -C fails : > > > > c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir > > README.txt > > Illegal option: s > > Try `jar --help' for more information. > > > > With the patch it works . > > > > webrev > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ > > > > bug > > > > https://bugs.openjdk.java.net/browse/JDK-8207395 > > > > > > > > > > Thanks, Matthias From swpalmer at gmail.com Wed Jul 18 13:04:41 2018 From: swpalmer at gmail.com (Scott Palmer) Date: Wed, 18 Jul 2018 09:04:41 -0400 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: <445d240e44b04b26af9421a782208235@sap.com> References: <49a61af46449420ba82075433ab835c5@sap.com> <68104e97f8e042b0badef5fe579c86b3@sap.com> <445d240e44b04b26af9421a782208235@sap.com> Message-ID: That gives a different result. Original: "///" -> ?/" replaceAll: "///" -> ?//" > On Jul 18, 2018, at 7:18 AM, Baesken, Matthias wrote: > > Hi G?tz, thanks for the input ! > Should we maybe use > > dir.replaceAll > > and not the while loop ? > >> while (dir.indexOf("//") > -1) { >> dir = dir.replace("//", "/"); >> } > > Best regards, Matthias > > >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Mittwoch, 18. Juli 2018 11:32 >> To: Baesken, Matthias ; core-libs- >> dev at openjdk.java.net >> Subject: RE: RFR : 8207395: jar has issues with UNC-path arguments for the >> jar -C parameter [windows] >> >> Hi Matthias, >> >> thanks for doing this fix. >> >> I think this can be noted down a bit better, avoiding duplicating the loop. >> Also, please remove the redundant dir.replace(File.separatorChar, '/'). >> >> dir = dir.replace(File.separatorChar, '/'); >> + String unc = (dir.startsWith("//") && (File.separatorChar == '\\')) >> ? "/" : ""; >> while (dir.indexOf("//") > -1) { >> dir = dir.replace("//", "/"); >> } >> - pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); >> + // Restore the second leading '/' needed for the Windows UNC >> path. >> + dir = unc + dir; >> + pathsMap.get(version).add(dir); >> nameBuf[k++] = dir + args[++i]; >> >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >>> Behalf Of Baesken, Matthias >>> Sent: Dienstag, 17. Juli 2018 13:15 >>> To: core-libs-dev at openjdk.java.net >>> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path arguments >>> for the jar -C parameter [windows] >>> >>> Please review this small fix for allowing windows UNC paths in the jar -C >>> parameter. >>> Currently passing a UNC path to a directory with -C fails : >>> >>> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir >>> README.txt >>> Illegal option: s >>> Try `jar --help' for more information. >>> >>> With the patch it works . >>> >>> webrev >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ >>> >>> bug >>> >>> https://bugs.openjdk.java.net/browse/JDK-8207395 >>> >>> >>> >>> >>> Thanks, Matthias > From Roger.Riggs at Oracle.com Wed Jul 18 13:44:03 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 18 Jul 2018 09:44:03 -0400 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: <68104e97f8e042b0badef5fe579c86b3@sap.com> References: <49a61af46449420ba82075433ab835c5@sap.com> <68104e97f8e042b0badef5fe579c86b3@sap.com> Message-ID: <7583c970-f982-f71e-ad98-706ea181698e@Oracle.com> Hi, Is there any chance that just using java.nio.file.Path.of will do the needed cleanup?? (or Paths.get) It seems a shame to spread this kind of adhoc fixup around? $0.02, Roger On 7/18/2018 5:31 AM, Lindenmaier, Goetz wrote: > Hi Matthias, > > thanks for doing this fix. > > I think this can be noted down a bit better, avoiding duplicating the loop. > Also, please remove the redundant dir.replace(File.separatorChar, '/'). > > dir = dir.replace(File.separatorChar, '/'); > + String unc = (dir.startsWith("//") && (File.separatorChar == '\\')) ? "/" : ""; > while (dir.indexOf("//") > -1) { > dir = dir.replace("//", "/"); > } > - pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); > + // Restore the second leading '/' needed for the Windows UNC path. > + dir = unc + dir; > + pathsMap.get(version).add(dir); > nameBuf[k++] = dir + args[++i]; > > Best regards, > Goetz. > >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of Baesken, Matthias >> Sent: Dienstag, 17. Juli 2018 13:15 >> To: core-libs-dev at openjdk.java.net >> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path arguments >> for the jar -C parameter [windows] >> >> Please review this small fix for allowing windows UNC paths in the jar -C >> parameter. >> Currently passing a UNC path to a directory with -C fails : >> >> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir >> README.txt >> Illegal option: s >> Try `jar --help' for more information. >> >> With the patch it works . >> >> webrev >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ >> >> bug >> >> https://bugs.openjdk.java.net/browse/JDK-8207395 >> >> >> >> >> Thanks, Matthias From isaac.r.levy at gmail.com Wed Jul 18 14:20:49 2018 From: isaac.r.levy at gmail.com (Isaac Levy) Date: Wed, 18 Jul 2018 10:20:49 -0400 Subject: [PATCH] regex matcher opt: remove redundant StringBuilder In-Reply-To: References: <5ADF60FA.2020109@oracle.com> <5ADFDC21.2040909@oracle.com> Message-ID: I still think it would be valuable to land a patch for replaceAll to avoid temporary StringBuilders, is there anyone who wants to help me land it? Isaac On Sun, Apr 29, 2018 at 10:24 PM, Isaac Levy wrote: > Your patch looks good to me, and will get the majority of performance > benefits without any API issues. > > Isaac > > > On Tue, Apr 24, 2018 at 9:38 PM, Xueming Shen wrote: >> for String based replaceAll/First() it might be worth doing something like >> >> http://cr.openjdk.java.net/~sherman/regex_removesb/webrev/ >> >> >> On 4/24/18, 10:47 AM, Isaac Levy wrote: >>> >>> Hi Sherman, >>> >>> Thanks for clarifying. Looks like exceptions are caused by invalid >>> format string. I wouldn't expect most programs to be catching this and >>> preserving their buffer, but dunno. >>> >>> How much does it affect perf? Well it depends on use case, a jmh of >>> replaceAll with a length 200 string of digits and regex "(\w)" shows >>> about 30% speedup. >>> >>> [info] Benchmark Mode Cnt Score Error Units >>> [info] origM avgt 10 11.669 ? 0.211 us/op >>> [info] newM avgt 10 8.926 ? 0.095 us/op >>> >>> Isaac >>> >>> >>> On Tue, Apr 24, 2018 at 12:53 PM, Xueming Shen >>> wrote: >>>> >>>> Hi Isaac, >>>> >>>> I actually meant to say "we are not supposed to output the partial text >>>> into >>>> the output buffer in case of an exception". It has nothing to do with the >>>> changeset you cited. This has been the behavior since day one/JDK1.4, >>>> though it is not specified explicitly in the API doc. The newly added >>>> StringBuilder variant simply follows this behavior. If it's really >>>> desired >>>> it >>>> is kinda doable to save that StringBuilder without the "incompatible" >>>> behavior >>>> change but just wonder if it is really worth the effort. >>>> >>>> Thanks, >>>> Sherman >>>> >>>> >>>> On 4/24/18, 9:11 AM, Isaac Levy wrote: >>>>> >>>>> (moving this to a separate discussion) >>>>> >>>>> >>>>> --- a/src/java.base/share/classes/java/util/regex/Matcher.java >>>>> +++ b/src/java.base/share/classes/java/util/regex/Matcher.java >>>>> @@ -993,13 +993,11 @@ >>>>> public Matcher appendReplacement(StringBuilder sb, String >>>>> replacement) { >>>>> // If no match, return error >>>>> if (first< 0) >>>>> throw new IllegalStateException("No match available"); >>>>> - StringBuilder result = new StringBuilder(); >>>>> - appendExpandedReplacement(replacement, result); >>>>> // Append the intervening text >>>>> sb.append(text, lastAppendPosition, first); >>>>> // Append the match substitution >>>>> + appendExpandedReplacement(replacement, sb); >>>>> - sb.append(result); >>>>> >>>>> >>>>> >>>>> On Mon, Apr 23, 2018 at 5:05 PM Xueming Shen >>>>> wrote: >>>>>> >>>>>> >>>>>> I would assume in case of an exception thrown from >>>>>> appendExpandedReplacement() we don't >>>>>> want "text" to be pushed into the "sb". >>>>>> >>>>>> -sherman >>>>> >>>>> >>>>> Perhaps. Though the behavior under exception is undefined and this >>>>> function is probably primarily used though the replaceAll API, which >>>>> wouldn?t return the intermediate sb under the case of exception. >>>>> >>>>> My reading of the blame was the temp StringBuilder was an artifact of >>>>> the api previously using StringBuffer externally. See >>>>> http://hg.openjdk.java.net/jdk9/dev/jdk/rev/763c564451b3 >>>> >>>> >> From isaac.r.levy at gmail.com Wed Jul 18 14:29:21 2018 From: isaac.r.levy at gmail.com (Isaac Levy) Date: Wed, 18 Jul 2018 10:29:21 -0400 Subject: [RFC] StringBuilder.toCharArray Message-ID: Is there any interest in a toCharArray method for StringBuilder? I'm unable to to make a bug for it. Just a bit of sugar: char[] toCharArray() { int length = length(); char[] dst = new char[length]; getChars(0, length, dst, 0); return dst; } Regards, Isaac From isaac.r.levy at gmail.com Wed Jul 18 14:49:50 2018 From: isaac.r.levy at gmail.com (Isaac Levy) Date: Wed, 18 Jul 2018 10:49:50 -0400 Subject: [PATCH] AbstractStringBuilder.append() In-Reply-To: References: <66BCA06A-D1BC-4F67-9331-BD3874E47493@oracle.com> Message-ID: I think adding a dedicated method would help clarify and encourage performant code. For example, I sped up the snippet below after seeing that StringBuilder.append() has a fast path when the arg is another StringBuilder, which isn't clear from the javadoc. public class ScalaSB implements java.lang.CharSequence { final StringBuilder sb; ... public append(ScalaSB other) { - sb.append(other); + sb.append(other.sb); } } From matthias.baesken at sap.com Wed Jul 18 14:51:55 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 18 Jul 2018 14:51:55 +0000 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: References: <49a61af46449420ba82075433ab835c5@sap.com> <68104e97f8e042b0badef5fe579c86b3@sap.com> <445d240e44b04b26af9421a782208235@sap.com> Message-ID: <9c54927eeb0f4ca285fd17fa0cfcf67c@sap.com> Good point, I think I better keep the "good old" loop . Second webrev containing the suggestions of Goetz : http://cr.openjdk.java.net/~mbaesken/webrevs/8207395.1/ Best regards, Matthias > -----Original Message----- > From: Scott Palmer [mailto:swpalmer at gmail.com] > Sent: Mittwoch, 18. Juli 2018 15:05 > To: Baesken, Matthias > Cc: Lindenmaier, Goetz ; core-libs- > dev at openjdk.java.net > Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for the > jar -C parameter [windows] > > That gives a different result. > > Original: > "///" -> ?/" > > replaceAll: > "///" -> ?//" > > > > On Jul 18, 2018, at 7:18 AM, Baesken, Matthias > wrote: > > > > Hi G?tz, thanks for the input ! > > Should we maybe use > > > > dir.replaceAll > > > > and not the while loop ? > > > >> while (dir.indexOf("//") > -1) { > >> dir = dir.replace("//", "/"); > >> } > > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: Lindenmaier, Goetz > >> Sent: Mittwoch, 18. Juli 2018 11:32 > >> To: Baesken, Matthias ; core-libs- > >> dev at openjdk.java.net > >> Subject: RE: RFR : 8207395: jar has issues with UNC-path arguments for > the > >> jar -C parameter [windows] > >> > >> Hi Matthias, > >> > >> thanks for doing this fix. > >> > >> I think this can be noted down a bit better, avoiding duplicating the loop. > >> Also, please remove the redundant dir.replace(File.separatorChar, '/'). > >> > >> dir = dir.replace(File.separatorChar, '/'); > >> + String unc = (dir.startsWith("//") && (File.separatorChar == > '\\')) > >> ? "/" : ""; > >> while (dir.indexOf("//") > -1) { > >> dir = dir.replace("//", "/"); > >> } > >> - pathsMap.get(version).add(dir.replace(File.separatorChar, > '/')); > >> + // Restore the second leading '/' needed for the Windows > UNC > >> path. > >> + dir = unc + dir; > >> + pathsMap.get(version).add(dir); > >> nameBuf[k++] = dir + args[++i]; > >> > >> Best regards, > >> Goetz. > >> > >>> -----Original Message----- > >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] > On > >>> Behalf Of Baesken, Matthias > >>> Sent: Dienstag, 17. Juli 2018 13:15 > >>> To: core-libs-dev at openjdk.java.net > >>> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path > arguments > >>> for the jar -C parameter [windows] > >>> > >>> Please review this small fix for allowing windows UNC paths in the jar - > C > >>> parameter. > >>> Currently passing a UNC path to a directory with -C fails : > >>> > >>> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir > >>> README.txt > >>> Illegal option: s > >>> Try `jar --help' for more information. > >>> > >>> With the patch it works . > >>> > >>> webrev > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ > >>> > >>> bug > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8207395 > >>> > >>> > >>> > >>> > >>> Thanks, Matthias > > From Alan.Bateman at oracle.com Wed Jul 18 17:43:33 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 18 Jul 2018 18:43:33 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <6cfa4b0266de4fdf85d6c959c450043c@sap.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> Message-ID: <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> On 18/07/2018 09:21, Baesken, Matthias wrote: > Hi Alan, I'll prepare a CSR . > I selected a more general name "jdk.includeInExceptions" , because there is the idea to enhance more exceptions with additional output . > In such a case " jdk.util.jar.includeInExceptions" would not really help . > However so far it is still a bit unclear how many exceptions we would like to enhance , so this has to be checked first . > jdk.includeInExceptions expands the scope. That might be okay but we will need to re-visit jdk.net.includeInExceptions and also move the support to somewhere in jdk.internal so that it can be used by other code in java.base. -Alan From xueming.shen at oracle.com Wed Jul 18 18:37:34 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 18 Jul 2018 11:37:34 -0700 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: <7583c970-f982-f71e-ad98-706ea181698e@Oracle.com> References: <49a61af46449420ba82075433ab835c5@sap.com> <68104e97f8e042b0badef5fe579c86b3@sap.com> <7583c970-f982-f71e-ad98-706ea181698e@Oracle.com> Message-ID: <5B4F88EE.8080608@oracle.com> Ideally it would be preferred to move jar to a complete nio implementation and we then can leave the WindowsPath to take care of this stuff. That said that would be a relative big change. I do have a version on my disk but need clean up, test, performance measurement and review to get it. So for now I think it's fine to add this small fix to support the unc. now sure if the following is better? 637 boolean hasUNC = File.separatorChar == '\\'&& dir.startsWith("//"); 638 while (dir.indexOf("//")> -1) { 639 dir = dir.replace("//", "/"); 640 } 641 if (isUNC) // Restore Windows UNC path. 642 dir = "/" + dir; -Sherman On 7/18/18, 6:44 AM, Roger Riggs wrote: > Hi, > > Is there any chance that just using java.nio.file.Path.of will do the > needed cleanup? (or Paths.get) > It seems a shame to spread this kind of adhoc fixup around? > > $0.02, Roger > > On 7/18/2018 5:31 AM, Lindenmaier, Goetz wrote: >> Hi Matthias, >> >> thanks for doing this fix. >> >> I think this can be noted down a bit better, avoiding duplicating the >> loop. >> Also, please remove the redundant dir.replace(File.separatorChar, '/'). >> >> dir = dir.replace(File.separatorChar, '/'); >> + String unc = (dir.startsWith("//") && >> (File.separatorChar == '\\')) ? "/" : ""; >> while (dir.indexOf("//") > -1) { >> dir = dir.replace("//", "/"); >> } >> - >> pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); >> + // Restore the second leading '/' needed for >> the Windows UNC path. >> + dir = unc + dir; >> + pathsMap.get(version).add(dir); >> nameBuf[k++] = dir + args[++i]; >> >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >>> Behalf Of Baesken, Matthias >>> Sent: Dienstag, 17. Juli 2018 13:15 >>> To: core-libs-dev at openjdk.java.net >>> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path >>> arguments >>> for the jar -C parameter [windows] >>> >>> Please review this small fix for allowing windows UNC paths in >>> the jar -C >>> parameter. >>> Currently passing a UNC path to a directory with -C fails : >>> >>> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir >>> README.txt >>> Illegal option: s >>> Try `jar --help' for more information. >>> >>> With the patch it works . >>> >>> webrev >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ >>> >>> bug >>> >>> https://bugs.openjdk.java.net/browse/JDK-8207395 >>> >>> >>> >>> >>> Thanks, Matthias > From martinrb at google.com Wed Jul 18 23:29:39 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 18 Jul 2018 16:29:39 -0700 Subject: [RFC] StringBuilder.toCharArray In-Reply-To: References: Message-ID: You can already copy a StringBuilder's contents into a String or another StringBuilder. What would Josh say? "Doesn't pull its weight." ?! On Wed, Jul 18, 2018 at 7:29 AM, Isaac Levy wrote: > Is there any interest in a toCharArray method for StringBuilder? I'm > unable to to make a bug for it. > > Just a bit of sugar: > > char[] toCharArray() { > int length = length(); > char[] dst = new char[length]; > getChars(0, length, dst, 0); > return dst; > } > > Regards, > Isaac > From ivan.gerasimov at oracle.com Thu Jul 19 03:19:00 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 18 Jul 2018 20:19:00 -0700 Subject: RFR 8207753 : Handle to process snapshot is leaked if Process32First() fails Message-ID: Hello! In native functions java.lang.ProcessHandleImpl.getProcessPids0() and java.lang.ProcessHandleImpl.parent0() a handle to a snapshot of all processes in the system is obtained with CreateToolhelp32Snapshot(). This handle isn't releases if the subsequent call to Process32First() fails. Would you please help review a trivial fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207753 WEBREV: http://cr.openjdk.java.net/~igerasim/8207753/00/webrev/ Thanks in advance! -- With kind regards, Ivan Gerasimov From dhiller at twitter.com Thu Jul 19 04:23:06 2018 From: dhiller at twitter.com (Dean Hiller) Date: Wed, 18 Jul 2018 22:23:06 -0600 Subject: any movement on fixing the breakage of logback MDC with CompletableFutures? Message-ID: I wanted to revisit the fact that MDC of logback broke in java's CompletableFuture while it works just fine with scala twitter Future because of their Local.scala class that acts like a ThreadLocal BUT for Futures transferring context. There is basically no way to fix it except by threading in some sort of LocalFuture.java into the CompletableFutures which would rock. thanks for any info there, (I would rather not switch this whole project to scala and having the MDC in logback broken is extremely annoying) https://stackoverflow.com/questions/37933713/does-completablefuture-have-a-corresponding-local-context thanks Dean From ecki at zusammenkunft.net Thu Jul 19 06:43:22 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 19 Jul 2018 06:43:22 +0000 Subject: RFR 8207750 : Native handle leak in java.io.WinNTFileSystem.list() In-Reply-To: <65676035-F1DD-4922-836D-68FF064ECE56@oracle.com> References: <534098c0-aaab-bc93-ec29-23d5d626a396@oracle.com>, <65676035-F1DD-4922-836D-68FF064ECE56@oracle.com> Message-ID: Hello, I am not sure about the JNi conventions, but does the jObjectArray rv need to be released as local reference in some of the early returns as well? Gruss Bernd Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: -1031249216m Auftrag von Gesendet: Mittwoch, Juli 18, 2018 4:38 AM An: Ivan Gerasimov Cc: core-libs-dev Betreff: Re: RFR 8207750 : Native handle leak in java.io.WinNTFileSystem.list() Hi Ivan, This looks all right to me. Thanks, Brian On Jul 17, 2018, at 5:01 PM, Ivan Gerasimov wrote: > In the file WinNTFileSystem_md.c there is a potential leak of native handle obtained from FindFirstFileW() in the unfortunate event of memory allocation failure. > > Would you please help review a simple fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207750 > WEBREV:http://cr.openjdk.java.net/~igerasim/8207750/00/webrev/ From christoph.langer at sap.com Thu Jul 19 06:59:49 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 19 Jul 2018 06:59:49 +0000 Subject: RFR(S): 8207766: [testbug] Adapt tests for Aix. In-Reply-To: References: Message-ID: Hi Goetz, Are you sure the quotation change in EvalArraysAsList.sh works well on other platforms? Other than that the changes look good. Reviewed. Best regards Christoph > -----Original Message----- > From: core-libs-dev On Behalf > Of Lindenmaier, Goetz > Sent: Mittwoch, 18. Juli 2018 09:28 > To: core-libs-dev at openjdk.java.net > Subject: [CAUTION] RFR(S): 8207766: [testbug] Adapt tests for Aix. > > Hi, > > I would like to fix these tests to run on aix: > http://cr.openjdk.java.net/~goetz/wr18/8207766-aixTestFixes/01/ > > > MulticastSocket tests > Opened bug and asked IBM to fix. Put on ProblemList. See > https://bugs.openjdk.java.net/browse/JDK-8207404 > > EvalArraysAsList.sh > Quotes need to be escaped on AIX. > > WrappedToolkitTest.sh > Check for AIX missing. > > PKCS11Test.java > getNSSLibDir can return null. If so, static initialization of badNSSVersion fails: > java.io.FileNotFoundException: nulllibsoftokn3.so (No such file or directory) > > TestNssDbSqlite.java > Don't run this on AIX. There is no NSS on basic installations. > > launcher/SourceMode.java > AIX does not support arguments to the shebang. > > Best regards, > Goetz. From xueming.shen at oracle.com Thu Jul 19 07:27:37 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 19 Jul 2018 00:27:37 -0700 Subject: Adding new IBM extended charsets In-Reply-To: References: <5B3E4AC8.3070307@oracle.com> <4cba7633-33b7-f29a-fb34-05a06b224057@oracle.com> Message-ID: <5B503D69.6010004@oracle.com> Hi Nasser, From openjdk's perspective It would be preferred to direct the develop to use the charset implementation provided by IBM, or the reliable third party that has the appropriate knowledge, experience and resource to support/maintain those charsets such as the icu4j charset project. I have been pulling the data from that huge icu-charset-data file and implement/maintain them based on my best knowledge, but I'm sure engineers from IBM or the icu project probably can do a much better job to implement/maintain/update those charsets going forward. As first step we can separate those IBM charsets from the jdk.charset into a separate package somewhere and configure them to be built into java.base and jdk.charsets, for aix platform only. Then we can further discuss the best way to handle/distribute those charsets that are not needed for the java.base module (for vm startup). As I said, it would be ideal if we can remove them from the openjdk repo/binaries complete and direct the developer/user to use the icu4j charset provider for those encodings, when needed. But given the possible compatibility concern, we might want to phase this work out gradually in next major release. Thanks, Sherman On 7/17/18, 6:48 AM, Nasser Ebrahim wrote: > Hi Alan, > > Thank you for your inputs. I would like to clarify that all the IBM > charsets (IBMXXXX) in jdk.charsets are not IBM platform specific > charsets. For example, only 43 charsets out of 72 IBMXXXX in > jdk.charsets are EBCDIC or IBM platform specific charsets. Similarly, > many charsets in the list of 75 charsets which we would like to > contribute are not EBCDIC charsets. > > I feel we should have a standard guideline for the extended charsets. > If we are keeping the extended charsets in the JDK, then we may want > to consider all ICU/IANA approved charsets in JDK. Otherwise, we may > want to keep only the standard charsets in JDK and remove all the > extended charsets so that all extended charsets can be taken from > third party libraries like ICU4J. > > If we decided to keep the extended charsets, then may be we can > classify the extended charsets as ASCII and EBCDIC and the > corresponding modules as jdk.ascii.charset and jdk.ebcdic.charset. > Then, depends upon the platform, we can consider including either of > the charset module or both. > > Please advise. > > Thank you, > Nasser Ebrahim > > > > > From: Alan Bateman > To: Nasser Ebrahim , Xueming Shen > , core-libs-dev at openjdk.java.net > Date: 07/09/2018 01:25 AM > Subject: Re: Adding new IBM extended charsets > ------------------------------------------------------------------------ > > > > On 06/07/2018 14:56, Nasser Ebrahim wrote: > > : > > I understood you preferred option is 3 [Remove all extended charsets > from > > JDK (keep only default charsets) and use the extended charsets from > third > > party like ICU4J]. Just to confirm, so you meant we need to keep > only the > > standard charsets in the JDK and remove all the extended charsets > from JDK > > and use them from ICU4J OR you meant apply that only for the new > extended > > charsets. I think it is better to keep the consistency - either take all > > extended charsets from ICU4J or maintain all extended charsets with JDK. > > Keeping some extended charsets within JDK and use ICU4J for other > extended > > charsets may confuse the Java user. > I think the suggestion in Sherman's mail is to drop the 70 or so IBM > charsets from jdk.charsets. This will reduce the size of jdk.charsets > and eliminate the need to maintain these charsets (at least on non-AIX > builds). If developers need these charsets, say when connecting to > database on an IBM system, then they can deploy the ICU4J provider on > the class path or module path. > > I don't think the suggestion impacts the 11 IBM charsets in java.base on > non-AIX builds or the non-IBM charsets in jdk.charsets. They may be > opportunities to drop some of these but that can be looked at separately. > > Also I don't think the suggestion impacts the additional 12 IBM charsets > that are included in the AIX build of java.base at this time. From the > review threads, it seems there are supported locales on AIX that map to > these charsets so this is why they are in java.base. > > -Alan. > > > > From matthias.baesken at sap.com Thu Jul 19 08:07:40 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 19 Jul 2018 08:07:40 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> Message-ID: <60acb2950d9147e5adc8e246de7d7b08@sap.com> Hello, in the meantime I prepared a CSR : https://bugs.openjdk.java.net/browse/JDK-8207768 > jdk.includeInExceptions expands the scope. That might be okay but we > will need to re-visit jdk.net.includeInExceptions and also move the > support to somewhere in jdk.internal so that it can be used by other > code in java.base. Is there some support code for " jdk.net.includeInExceptions " or do you just mean the name of the property ? Best regards, Matthias > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Mittwoch, 18. Juli 2018 19:44 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net; Lindenmaier, Goetz > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > On 18/07/2018 09:21, Baesken, Matthias wrote: > > Hi Alan, I'll prepare a CSR . > > I selected a more general name "jdk.includeInExceptions" , because > there is the idea to enhance more exceptions with additional output . > > In such a case " jdk.util.jar.includeInExceptions" would not really help . > > However so far it is still a bit unclear how many exceptions we would like to > enhance , so this has to be checked first . > > > jdk.includeInExceptions expands the scope. That might be okay but we > will need to re-visit jdk.net.includeInExceptions and also move the > support to somewhere in jdk.internal so that it can be used by other > code in java.base. > > -Alan From matthias.baesken at sap.com Thu Jul 19 09:15:00 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 19 Jul 2018 09:15:00 +0000 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] Message-ID: <48bdeddc592645dbb5ef046ab7226c67@sap.com> Hi Sherman , I like your proposal , I prepared this webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8207395.2/ (and looking forward to your future move of jar to a complete nio implementation !) Best regards, Matthias > > Message: 2 > Date: Wed, 18 Jul 2018 11:37:34 -0700 > From: Xueming Shen > To: core-libs-dev at openjdk.java.net > Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for > the jar -C parameter [windows] > Message-ID: <5B4F88EE.8080608 at oracle.com> > Content-Type: text/plain; charset=UTF-8; format=flowed > > > Ideally it would be preferred to move jar to a complete nio > implementation and we then > can leave the WindowsPath to take care of this stuff. That said that > would be a relative > big change. I do have a version on my disk but need clean up, test, > performance measurement > and review to get it. > > So for now I think it's fine to add this small fix to support the unc. > > now sure if the following is better? > > 637 boolean hasUNC = File.separatorChar == '\\'&& > dir.startsWith("//"); > 638 while (dir.indexOf("//")> -1) { > 639 dir = dir.replace("//", "/"); > 640 } > 641 if (isUNC) // Restore Windows UNC path. > 642 dir = "/" + dir; > > -Sherman > > > On 7/18/18, 6:44 AM, Roger Riggs wrote: > > Hi, > > > > Is there any chance that just using java.nio.file.Path.of will do the > > needed cleanup? (or Paths.get) > > It seems a shame to spread this kind of adhoc fixup around? > > > > $0.02, Roger > > > > On 7/18/2018 5:31 AM, Lindenmaier, Goetz wrote: > >> Hi Matthias, > >> > >> thanks for doing this fix. > >> > >> I think this can be noted down a bit better, avoiding duplicating the > >> loop. > >> Also, please remove the redundant dir.replace(File.separatorChar, '/'). > >> > >> dir = dir.replace(File.separatorChar, '/'); > >> + String unc = (dir.startsWith("//") && > >> (File.separatorChar == '\\')) ? "/" : ""; > >> while (dir.indexOf("//") > -1) { > >> dir = dir.replace("//", "/"); > >> } > >> - > >> pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); > >> + // Restore the second leading '/' needed for > >> the Windows UNC path. > >> + dir = unc + dir; > >> + pathsMap.get(version).add(dir); > >> nameBuf[k++] = dir + args[++i]; > >> > >> Best regards, > >> Goetz. > >> > >>> -----Original Message----- > >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] > On > >>> Behalf Of Baesken, Matthias > >>> Sent: Dienstag, 17. Juli 2018 13:15 > >>> To: core-libs-dev at openjdk.java.net > >>> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path > >>> arguments > >>> for the jar -C parameter [windows] > >>> > >>> Please review this small fix for allowing windows UNC paths in > >>> the jar -C > >>> parameter. > >>> Currently passing a UNC path to a directory with -C fails : > >>> > >>> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir > >>> README.txt > >>> Illegal option: s > >>> Try `jar --help' for more information. > >>> > >>> With the patch it works . > >>> > >>> webrev > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ > >>> > >>> bug > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8207395 > >>> > >>> > >>> > >>> > >>> Thanks, Matthias > > From chris.hegarty at oracle.com Thu Jul 19 10:07:34 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 19 Jul 2018 11:07:34 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <60acb2950d9147e5adc8e246de7d7b08@sap.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> Message-ID: <1CF5550B-90E6-4864-B544-990EEFAC03A4@oracle.com> > On 19 Jul 2018, at 09:07, Baesken, Matthias wrote: > > Hello, in the meantime I prepared a CSR : > > https://bugs.openjdk.java.net/browse/JDK-8207768 This CSR seems a little premature. Matthias said: "However so far it is still a bit unclear how many exceptions we would like to enhance , so this has to be checked first ?. Has this been checked? I have seen no update on this or any other email thread. JDK-8204233 [1] was a pragmatic and targeted solution to a long standing complaint. It was not intended, as is obvious by the newly added property name, to set precedent for doing similar all over the platform. If, after a broader discussion, this approach is to be applied to several different areas of the platform, then maybe JDK-8204233 should be revisited ( noting that it is JDK 11, which is currently in RDP 1 ). I think we should try to avoid a whole plethora of these properties. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8204233 > >> jdk.includeInExceptions expands the scope. That might be okay but we >> will need to re-visit jdk.net.includeInExceptions and also move the >> support to somewhere in jdk.internal so that it can be used by other >> code in java.base. > > Is there some support code for " jdk.net.includeInExceptions " or do you just mean the name of the property ? > > > Best regards, Matthias > > > >> -----Original Message----- >> From: Alan Bateman [mailto:Alan.Bateman at oracle.com] >> Sent: Mittwoch, 18. Juli 2018 19:44 >> To: Baesken, Matthias ; core-libs- >> dev at openjdk.java.net; Lindenmaier, Goetz >> Subject: Re: [RFR] 8205525 : Improve exception messages during manifest >> parsing of jar archives >> >> On 18/07/2018 09:21, Baesken, Matthias wrote: >>> Hi Alan, I'll prepare a CSR . >>> I selected a more general name "jdk.includeInExceptions" , because >> there is the idea to enhance more exceptions with additional output . >>> In such a case " jdk.util.jar.includeInExceptions" would not really help . >>> However so far it is still a bit unclear how many exceptions we would like to >> enhance , so this has to be checked first . >>> >> jdk.includeInExceptions expands the scope. That might be okay but we >> will need to re-visit jdk.net.includeInExceptions and also move the >> support to somewhere in jdk.internal so that it can be used by other >> code in java.base. >> >> -Alan From Alan.Bateman at oracle.com Thu Jul 19 10:14:15 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 19 Jul 2018 11:14:15 +0100 Subject: Adding new IBM extended charsets In-Reply-To: <5B503D69.6010004@oracle.com> References: <5B3E4AC8.3070307@oracle.com> <4cba7633-33b7-f29a-fb34-05a06b224057@oracle.com> <5B503D69.6010004@oracle.com> Message-ID: <4ed347e6-1e64-5ef6-be48-8d12b0adac6f@oracle.com> On 19/07/2018 08:27, Xueming Shen wrote: > Hi Nasser, > > From openjdk's perspective It would be preferred to direct the develop > to use the charset > implementation provided by IBM, or the reliable third party that has > the appropriate knowledge, > experience and resource to support/maintain those charsets such as the > icu4j charset > project. I have been pulling the data from that huge icu-charset-data > file and implement/maintain > them based on my best knowledge, but I'm sure engineers from IBM or > the icu project probably > can do a much better job to implement/maintain/update those charsets > going forward. > > As first step we can separate those IBM charsets from the jdk.charset > into a separate package > somewhere and configure them to be built into java.base and > jdk.charsets, for aix platform only. > Then we can further discuss the best way to handle/distribute those > charsets that are not needed > for the java.base module (for vm startup). As I said, it would be > ideal if we can remove them from the > openjdk repo/binaries complete and direct the developer/user to use > the icu4j charset provider > for those encodings, when needed. But given the possible compatibility > concern, we might want to > phase this work out gradually in next major release. I agree and in terms of phasing then I don't think it would be too disruptive if the EBCDIC charsets were dropped from jdk.charsets in JDK 12, at least on the main stream platforms. As we've established in this thread, the ICU4J project does seem to publish its charset provider to Maven so there are alternatives for applications that really need these charsets Nasser - do you do any testing with the ICU4J charsets? I quickly tried 62.1 and it seems to work fine on the class path. I didn't check for any compatibility differences or compare the performance but maybe you have. It's a bit awkward to test this provider as an automatic module due to the unusual naming of these JAR files. They may not have looked at modules yet but the ability to link thee icu4h.charsets module into a run-time image seems something that people may want to do in the future. -Alan From Alan.Bateman at oracle.com Thu Jul 19 10:46:47 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 19 Jul 2018 11:46:47 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <60acb2950d9147e5adc8e246de7d7b08@sap.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> Message-ID: <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> On 19/07/2018 09:07, Baesken, Matthias wrote: > Hello, in the meantime I prepared a CSR : > > https://bugs.openjdk.java.net/browse/JDK-8207768 > > >> jdk.includeInExceptions expands the scope. That might be okay but we >> will need to re-visit jdk.net.includeInExceptions and also move the >> support to somewhere in jdk.internal so that it can be used by other >> code in java.base. > Is there some support code for " jdk.net.includeInExceptions " or do you just mean the name of the property ? > Chris is right that it's a bit premature to submit a CSR while the question on whether to rename the existing security property is on the table. I have no objection to doing that. It probably means we need the code to read this property moved to a jdk.internal package and change the networking, NIO, and JAR code to use that. It introduces a small compatibility issue with JDK 11 jdk.net property so that will need to be handled too. -Alan From chris.hegarty at oracle.com Thu Jul 19 12:54:28 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 19 Jul 2018 13:54:28 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> Message-ID: <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> > On 19 Jul 2018, at 11:46, Alan Bateman wrote: > > On 19/07/2018 09:07, Baesken, Matthias wrote: >> Hello, in the meantime I prepared a CSR : >> >> https://bugs.openjdk.java.net/browse/JDK-8207768 >> >> >>> jdk.includeInExceptions expands the scope. That might be okay but we >>> will need to re-visit jdk.net.includeInExceptions and also move the >>> support to somewhere in jdk.internal so that it can be used by other >>> code in java.base. >> Is there some support code for " jdk.net.includeInExceptions " or do you just mean the name of the property ? >> > Chris is right that it's a bit premature to submit a CSR while the question on whether to rename the existing security property is on the table. I have no objection to doing that. I filed the following issue to generalize the `includeInExceptions` security property: https://bugs.openjdk.java.net/browse/JDK-8207846 I would like to resolve 8207846 first, then 8205525 can propose to add the new argument value, `jarPath`. -Chris. From roger.riggs at oracle.com Thu Jul 19 13:15:48 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Thu, 19 Jul 2018 09:15:48 -0400 Subject: RFR 8207753 : Handle to process snapshot is leaked if Process32First() fails In-Reply-To: References: Message-ID: Looks good. Thanks for catching and fixing this. Roger On 7/18/18 11:19 PM, Ivan Gerasimov wrote: > Hello! > > In native functions java.lang.ProcessHandleImpl.getProcessPids0() and > java.lang.ProcessHandleImpl.parent0() a handle to a snapshot of all > processes in the system is obtained with CreateToolhelp32Snapshot(). > > This handle isn't releases if the subsequent call to Process32First() > fails. > > Would you please help review a trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207753 > WEBREV: http://cr.openjdk.java.net/~igerasim/8207753/00/webrev/ > > Thanks in advance! > From sean.mullan at oracle.com Thu Jul 19 14:42:49 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 19 Jul 2018 10:42:49 -0400 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> Message-ID: On 7/19/18 8:54 AM, Chris Hegarty wrote: > >> On 19 Jul 2018, at 11:46, Alan Bateman wrote: >> >> On 19/07/2018 09:07, Baesken, Matthias wrote: >>> Hello, in the meantime I prepared a CSR : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8207768 >>> >>> >>>> jdk.includeInExceptions expands the scope. That might be okay but we >>>> will need to re-visit jdk.net.includeInExceptions and also move the >>>> support to somewhere in jdk.internal so that it can be used by other >>>> code in java.base. >>> Is there some support code for " jdk.net.includeInExceptions " or do you just mean the name of the property ? >>> >> Chris is right that it's a bit premature to submit a CSR while the question on whether to rename the existing security property is on the table. I have no objection to doing that. > > I filed the following issue to generalize the `includeInExceptions` security > property: > https://bugs.openjdk.java.net/browse/JDK-8207846 > > I would like to resolve 8207846 first, then 8205525 can propose to add the > new argument value, `jarPath`. Note that making this a security property for all general cases may have performance implications in certain scenarios since the java.security file will need to be loaded and fully parsed before it can be used. If you are already parsing the java.security file for something else (which is probably the case for jdk.net.includeInExceptions), then it should be fine, but that may not always be the case if this is made into a more general property. Just something to think about ... --Sean From bob.vandette at oracle.com Thu Jul 19 15:34:33 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 19 Jul 2018 11:34:33 -0400 Subject: RFR: 8206456 - [TESTBUG] docker jtreg tests fail on systems without cpuset.effective_cpus / cpuset.effective_mem In-Reply-To: <598c9af3-2041-0be5-f177-b3a031b2ef61@oracle.com> References: <598c9af3-2041-0be5-f177-b3a031b2ef61@oracle.com> Message-ID: <9766EFD8-9220-4AC1-A5D2-71A8F9568FC4@oracle.com> > On Jul 17, 2018, at 8:07 PM, mandy chung wrote: > > > > On 7/17/18 7:00 AM, Bob Vandette wrote: >> Please review this fix which eliminates some docker/cgroup test failures when running on older >> Linux kernels with missing cgroup metric files. >> BUGS: >> https://bugs.openjdk.java.net/browse/JDK-8206456 >> WEBREV: >> http://cr.openjdk.java.net/~bobv/8206456/webrev/ > > Nit: It would be clearer to check for the specific metrics: > > int[] cpusets = metrics.getEffectiveCpuSetCpus(); > if (cpusets.length != 0) { > .... > } > > Same applies to getEffectiveCpuSetMems. No need for a new webrev. Thanks, I?ll do that cleanup. > > Mandy > P.S. I am not sure the conversion from the primitive to boxed type > is necessary. But this is not related to this issue. You may > want to take a look at that. I?ll defer this issue to Harsha who wrote these tests since changing that is out of scope for this fix. Thanks, Bob. From naoto.sato at oracle.com Thu Jul 19 16:57:42 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 19 Jul 2018 09:57:42 -0700 Subject: [12] RFR: 8206886: Java does not set the default format locale correctly on mac10.13 Message-ID: <461d0a63-051d-cd59-a95a-b13768ed27d2@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8206886 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8206886/webrev.00/ macOS has removed the locale selection for formats since 10.12 (detail in the bug report). The fix intends to use the user's preferred language for Java's default format locale on 10.12 or later. Naoto From Roger.Riggs at Oracle.com Thu Jul 19 17:51:42 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 19 Jul 2018 13:51:42 -0400 Subject: [12] RFR: 8206886: Java does not set the default format locale correctly on mac10.13 In-Reply-To: <461d0a63-051d-cd59-a95a-b13768ed27d2@oracle.com> References: <461d0a63-051d-cd59-a95a-b13768ed27d2@oracle.com> Message-ID: <46f8caf4-b10a-a541-87bc-605f67f51fe7@Oracle.com> Hi Naoto, Looks ok, (I see the label noreg-hard covers the need for manual tests). Regards, Roger On 7/19/2018 12:57 PM, Naoto Sato wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8206886 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8206886/webrev.00/ > > macOS has removed the locale selection for formats since 10.12 (detail > in the bug report). The fix intends to use the user's preferred > language for Java's default format locale on 10.12 or later. > > Naoto From takiguc at linux.vnet.ibm.com Thu Jul 19 18:06:06 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 20 Jul 2018 03:06:06 +0900 Subject: Console.readPassword() was failed on Linux s390x platform Message-ID: <76e6aabf100ac4e5a01fcdf95c52c1d8@linux.vnet.ibm.com> Hello. I ran following testcase on Linux s390x platform. ------ import java.io.Console; public class ConsoleTestC { public static void main(String[] args) { Console console = System.console(); char password[] = console.readPassword("password> "); System.out.println(password); } } ------ Test instruction 1. Compile and run ConsoleTestC on terminal $ java ConsoleTestC password> 2. Type "abc", then "abc" is displayed 3. Type "ls", but nothing is displayed 4. Type "stty echo" to change back the setting I wrote following program for PD. ------ import java.io.*; import java.lang.reflect.*; public class ConsoleTestB { public static void main(String[] args) throws Exception { Console console = System.console(); Method echo = Console.class.getDeclaredMethod("echo", boolean.class); echo.setAccessible(true); System.out.println("echo false"); System.out.println(echo.invoke(console, false)); console.reader().read(); System.out.println("echo true"); System.out.println(echo.invoke(console, true)); } } ------ Test instruction 1. Compile and run ConsoleTestB on terminal $ java ConsoleTestB ...... echo false false 2. Last output should be "true", so Java could not read echo flag properly 3. Press return key echo true false $ Console.readPassword() could not read initial echo flag, so move back to "-echo" instead of "echo" Fix candidate is as follows: ====== diff -r 14708e1acdc3 src/java.base/unix/native/libjava/Console_md.c --- a/src/java.base/unix/native/libjava/Console_md.c Tue Jul 03 09:27:41 2018 +0800 +++ b/src/java.base/unix/native/libjava/Console_md.c Fri Jul 20 02:52:38 2018 +0900 @@ -56,7 +56,7 @@ JNU_ThrowIOExceptionWithLastError(env, "tcgetattr failed"); return !on; } - old = (tio.c_lflag & ECHO); + old = (tio.c_lflag & ECHO) == ECHO ? JNI_TRUE : JNI_FALSE; if (on) { tio.c_lflag |= ECHO; } else { ====== I'd like to obtain a sponsor for this patch. Could you put it into JDK11 ? Thanks, Ichiroh Takiguchi IBM Japan, Ltd. From xueming.shen at oracle.com Thu Jul 19 18:36:19 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 19 Jul 2018 11:36:19 -0700 Subject: Console.readPassword() was failed on Linux s390x platform In-Reply-To: <76e6aabf100ac4e5a01fcdf95c52c1d8@linux.vnet.ibm.com> References: <76e6aabf100ac4e5a01fcdf95c52c1d8@linux.vnet.ibm.com> Message-ID: <5B50DA23.9000108@oracle.com> Hi Ichiroh, Does the test case always fail? how about the previous version? The Console class was updated slightly in 11 though. does "old = (tio.c_lflag & ECHO) != 0" fix the issue as well? Please file a issue with appropriate P value. Not sure if it can make 11. Thanks, Sherman old = (tio.c_lflag & ECHO) == ECHO ? JNI_TRUE : JNI_FALSE; On 7/19/18, 11:06 AM, Ichiroh Takiguchi wrote: > Hello. > > I ran following testcase on Linux s390x platform. > ------ > import java.io.Console; > > public class ConsoleTestC { > public static void main(String[] args) { > Console console = System.console(); > char password[] = console.readPassword("password> "); > System.out.println(password); > } > } > ------ > > Test instruction > 1. Compile and run ConsoleTestC on terminal > $ java ConsoleTestC > password> > 2. Type "abc", then "abc" is displayed > 3. Type "ls", but nothing is displayed > 4. Type "stty echo" to change back the setting > > I wrote following program for PD. > ------ > import java.io.*; > import java.lang.reflect.*; > > public class ConsoleTestB { > public static void main(String[] args) throws Exception { > Console console = System.console(); > Method echo = Console.class.getDeclaredMethod("echo", boolean.class); > echo.setAccessible(true); > System.out.println("echo false"); > System.out.println(echo.invoke(console, false)); > console.reader().read(); > System.out.println("echo true"); > System.out.println(echo.invoke(console, true)); > } > } > ------ > > Test instruction > 1. Compile and run ConsoleTestB on terminal > $ java ConsoleTestB > ...... > echo false > false > 2. Last output should be "true", so Java could not read echo flag > properly > 3. Press return key > echo true > false > $ > > Console.readPassword() could not read initial echo flag, > so move back to "-echo" instead of "echo" > > Fix candidate is as follows: > ====== > diff -r 14708e1acdc3 src/java.base/unix/native/libjava/Console_md.c > --- a/src/java.base/unix/native/libjava/Console_md.c Tue Jul 03 > 09:27:41 2018 +0800 > +++ b/src/java.base/unix/native/libjava/Console_md.c Fri Jul 20 > 02:52:38 2018 +0900 > @@ -56,7 +56,7 @@ > JNU_ThrowIOExceptionWithLastError(env, "tcgetattr failed"); > return !on; > } > - old = (tio.c_lflag & ECHO); > + old = (tio.c_lflag & ECHO) == ECHO ? JNI_TRUE : JNI_FALSE; > if (on) { > tio.c_lflag |= ECHO; > } else { > ====== > > I'd like to obtain a sponsor for this patch. > Could you put it into JDK11 ? > > Thanks, > Ichiroh Takiguchi > IBM Japan, Ltd. > From Roger.Riggs at Oracle.com Thu Jul 19 19:39:14 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 19 Jul 2018 15:39:14 -0400 Subject: API for parsing path strings Message-ID: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> Hi, Parsing of paths such as the values of properties java.class.path and java.library.path do not have? direct support in the Java API. There are several implementations within OpenJDK and a few variations exist with regard to treatment of empty path elements and on Windows support for quoting. Would it be useful to have an API such as the following: /** * Returns a list of paths parsed from a string separated by {@link File#pathSeparator}. * All characters except the {@link File#pathSeparator} are allowed. * Some operating systems support quoting segments of the string * potentially including {@code pathSeparator} characters. * Empty paths, identified by leading, trailing, and adjacent separator characters, * can be omitted or replaced with a non-null provided path. * * @implNote * On Windows, zero or more characters between double-quotes ({@code "}) are included * literally in the path, regardless of whether they are valid in a path. * * @param path a string containing paths separated by the {@link File#pathSeparator} * @param emptyPath a path to replace an empty path or {@code null} to ignore empty paths * @return an immutable list of strings for each path; {@code non-null} */ public static List parsePathStrings(String path, String emptyPath) { ...} There are of course, possible variations and convenience versions but the API should be kept to the essentials. With java.util.streams it becomes easy to process each component or to map the strings to Files or Path. Suggestions welcome, Roger From brian.burkhalter at oracle.com Thu Jul 19 20:20:21 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 19 Jul 2018 13:20:21 -0700 Subject: API for parsing path strings In-Reply-To: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> References: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> Message-ID: Hi Roger, I was just wondering whether you have some specific use cases in mind. Thanks, Brian On Jul 19, 2018, at 12:39 PM, Roger Riggs wrote: > Parsing of paths such as the values of properties java.class.path and java.library.path > do not have direct support in the Java API. > There are several implementations within OpenJDK and a few variations exist with regard to > treatment of empty path elements and on Windows support for quoting. > > Would it be useful to have an API such as the following: > > [?] > > Suggestions welcome, Roger From Roger.Riggs at Oracle.com Thu Jul 19 20:30:30 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 19 Jul 2018 16:30:30 -0400 Subject: API for parsing path strings In-Reply-To: References: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> Message-ID: <23db9ee1-ecac-9981-ee9d-f3609f9c0776@Oracle.com> Hi Brian, No particular novel use cases. There are quite a few places in OpenJDK tools, etc that naively use string.split(pathSeparator) that will misbehave on Windows in the edge cases and may not be paying attention to the empty path component case. It would provide a measure of regularity and predictability to path parsing. There was a query on stack Overflow about such an API. Thanks, Roger On 7/19/2018 4:20 PM, Brian Burkhalter wrote: > Hi Roger, > > I was just wondering whether you have some specific use cases in mind. > > Thanks, > > Brian > > On Jul 19, 2018, at 12:39 PM, Roger Riggs > wrote: > >> Parsing of paths such as the values of properties java.class.path and >> java.library.path >> do not have? direct support in the Java API. >> There are several implementations within OpenJDK and a few variations >> exist with regard to >> treatment of empty path elements and on Windows support for quoting. >> >> Would it be useful to have an API such as the following: >> >> [?] >> >> Suggestions welcome, Roger > From brian.burkhalter at oracle.com Thu Jul 19 20:37:53 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 19 Jul 2018 13:37:53 -0700 Subject: API for parsing path strings In-Reply-To: <23db9ee1-ecac-9981-ee9d-f3609f9c0776@Oracle.com> References: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> <23db9ee1-ecac-9981-ee9d-f3609f9c0776@Oracle.com> Message-ID: <5739A17C-EEC4-4692-9F12-5690B25F89AC@oracle.com> Hi Roger, Well if there is some developer interest in it and it would consolidate and make code more robust in the JDK then it seems worth pursuing. Thanks, Brian On Jul 19, 2018, at 1:30 PM, Roger Riggs wrote: > No particular novel use cases. > There are quite a few places in OpenJDK tools, etc that naively use string.split(pathSeparator) that > will misbehave on Windows in the edge cases and may not be paying attention to the empty path component case. > It would provide a measure of regularity and predictability to path parsing. > There was a query on stack Overflow about such an API. From ivan.gerasimov at oracle.com Thu Jul 19 20:43:30 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 19 Jul 2018 13:43:30 -0700 Subject: RFR 8207753 : Handle to process snapshot is leaked if Process32First() fails In-Reply-To: References: Message-ID: Thank you Roger for review! On 7/19/18 6:15 AM, Roger Riggs wrote: > Looks good. > > Thanks for catching and fixing this. > > Roger > > On 7/18/18 11:19 PM, Ivan Gerasimov wrote: >> Hello! >> >> In native functions java.lang.ProcessHandleImpl.getProcessPids0() and >> java.lang.ProcessHandleImpl.parent0() a handle to a snapshot of all >> processes in the system is obtained with CreateToolhelp32Snapshot(). >> >> This handle isn't releases if the subsequent call to Process32First() >> fails. >> >> Would you please help review a trivial fix? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207753 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8207753/00/webrev/ >> >> Thanks in advance! >> > > -- With kind regards, Ivan Gerasimov From stuart.marks at oracle.com Thu Jul 19 21:22:05 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 19 Jul 2018 14:22:05 -0700 Subject: JDK 11 RFR(s) JDK-8206865 RMI activation tests fail with InvalidClassException Message-ID: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> Hi all, Please review this fix for some failing RMI activation tests. Some reflective code needed adjustment to take nestmates into account. This also removes the failing tests from the problem list. Webrev: http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8206865 Thanks, s'marks From joe.darcy at oracle.com Thu Jul 19 21:24:24 2018 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 19 Jul 2018 14:24:24 -0700 Subject: JDK 11 RFR(s) JDK-8206865 RMI activation tests fail with InvalidClassException In-Reply-To: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> References: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> Message-ID: <4bfff216-5e88-01b4-47f6-ab93a6f22a2a@oracle.com> Looks fine Stuart; cheers, -Joe On 7/19/2018 2:22 PM, Stuart Marks wrote: > Hi all, > > Please review this fix for some failing RMI activation tests. Some > reflective code needed adjustment to take nestmates into account. This > also removes the failing tests from the problem list. > > Webrev: > > ????http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.0/ > > Bug: > ????https://bugs.openjdk.java.net/browse/JDK-8206865 > > Thanks, > > s'marks > From takiguc at linux.vnet.ibm.com Fri Jul 20 06:52:24 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 20 Jul 2018 15:52:24 +0900 Subject: Console.readPassword() was failed on Linux s390x platform In-Reply-To: <5B50DA23.9000108@oracle.com> References: <76e6aabf100ac4e5a01fcdf95c52c1d8@linux.vnet.ibm.com> <5B50DA23.9000108@oracle.com> Message-ID: <20d5b8151ce27b23aa0101f57fbce3fb@linux.vnet.ibm.com> Hello. I ran ConsTestB on AdoptOpenJDK prebuilt. =========================== $ ~/jdk8u172-b11/jre/bin/java -showversion ConsoleTestB openjdk version "1.8.0-adoptopenjdk" OpenJDK Runtime Environment (build 1.8.0-adoptopenjdk-jenkins_2018_05_18_21_00-b00) OpenJDK 64-Bit Zero VM (build 25.71-b00, interpreted mode) echo false true echo true false $ ~/jdk-10+46/bin/java -showversion ConsoleTestB openjdk version "10-adoptopenjdk" 2018-03-20 OpenJDK Runtime Environment (build 10-adoptopenjdk+0-adhoc.jenkins.openjdk) Eclipse OpenJ9 VM (build master-2954f9d4, JRE 10 Linux s390x-64 Compressed References 20180329_15 (JIT enabled, AOT enabled) OpenJ9 - 2954f9d4 OMR - 963e23a2 JCL - 5692cdfa3a based on ) ... echo false true echo true false =========================== Both build worked fine. I applied following change. =========================== diff -r 14708e1acdc3 src/java.base/unix/native/libjava/Console_md.c --- a/src/java.base/unix/native/libjava/Console_md.c Tue Jul 03 09:27:41 2018 +0800 +++ b/src/java.base/unix/native/libjava/Console_md.c Fri Jul 20 15:37:59 2018 +0900 @@ -56,7 +56,7 @@ JNU_ThrowIOExceptionWithLastError(env, "tcgetattr failed"); return !on; } - old = (tio.c_lflag & ECHO); + old = (tio.c_lflag & ECHO) != 0; if (on) { tio.c_lflag |= ECHO; } else { =========================== It worked fine. =========================== $ java ConsoleTestB ... echo false true echo true false =========================== On 2018-07-20 03:36, Xueming Shen wrote: > Hi Ichiroh, > > Does the test case always fail? how about the previous version? The > Console class was updated slightly in 11 though. > > does "old = (tio.c_lflag & ECHO) != 0" fix the issue as well? > > Please file a issue with appropriate P value. Not sure if it can make > 11. > > Thanks, > Sherman > > > > old = (tio.c_lflag & ECHO) == ECHO ? JNI_TRUE : JNI_FALSE; > > On 7/19/18, 11:06 AM, Ichiroh Takiguchi wrote: >> Hello. >> >> I ran following testcase on Linux s390x platform. >> ------ >> import java.io.Console; >> >> public class ConsoleTestC { >> public static void main(String[] args) { >> Console console = System.console(); >> char password[] = console.readPassword("password> "); >> System.out.println(password); >> } >> } >> ------ >> >> Test instruction >> 1. Compile and run ConsoleTestC on terminal >> $ java ConsoleTestC >> password> >> 2. Type "abc", then "abc" is displayed >> 3. Type "ls", but nothing is displayed >> 4. Type "stty echo" to change back the setting >> >> I wrote following program for PD. >> ------ >> import java.io.*; >> import java.lang.reflect.*; >> >> public class ConsoleTestB { >> public static void main(String[] args) throws Exception { >> Console console = System.console(); >> Method echo = Console.class.getDeclaredMethod("echo", >> boolean.class); >> echo.setAccessible(true); >> System.out.println("echo false"); >> System.out.println(echo.invoke(console, false)); >> console.reader().read(); >> System.out.println("echo true"); >> System.out.println(echo.invoke(console, true)); >> } >> } >> ------ >> >> Test instruction >> 1. Compile and run ConsoleTestB on terminal >> $ java ConsoleTestB >> ...... >> echo false >> false >> 2. Last output should be "true", so Java could not read echo flag >> properly >> 3. Press return key >> echo true >> false >> $ >> >> Console.readPassword() could not read initial echo flag, >> so move back to "-echo" instead of "echo" >> >> Fix candidate is as follows: >> ====== >> diff -r 14708e1acdc3 src/java.base/unix/native/libjava/Console_md.c >> --- a/src/java.base/unix/native/libjava/Console_md.c Tue Jul 03 >> 09:27:41 2018 +0800 >> +++ b/src/java.base/unix/native/libjava/Console_md.c Fri Jul 20 >> 02:52:38 2018 +0900 >> @@ -56,7 +56,7 @@ >> JNU_ThrowIOExceptionWithLastError(env, "tcgetattr failed"); >> return !on; >> } >> - old = (tio.c_lflag & ECHO); >> + old = (tio.c_lflag & ECHO) == ECHO ? JNI_TRUE : JNI_FALSE; >> if (on) { >> tio.c_lflag |= ECHO; >> } else { >> ====== >> >> I'd like to obtain a sponsor for this patch. >> Could you put it into JDK11 ? >> >> Thanks, >> Ichiroh Takiguchi >> IBM Japan, Ltd. >> From goetz.lindenmaier at sap.com Fri Jul 20 06:51:41 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 20 Jul 2018 06:51:41 +0000 Subject: RFR(S): 8207766: [testbug] Adapt tests for Aix. In-Reply-To: References: Message-ID: <7d349bd669ee44e1b42abe9be075ea4c@sap.com> Hi Christoph, thanks for looking at my change. I had run that test on all the platforms and it passed. But I double checked, on linux it makes no difference whether I spell \" or \\", in the output file I fine \". But on aix I see \\" --> \" and \" --> " Best regards, Goetz. > -----Original Message----- > From: Langer, Christoph > Sent: Donnerstag, 19. Juli 2018 09:00 > To: Lindenmaier, Goetz ; core-libs- > dev at openjdk.java.net > Subject: RE: RFR(S): 8207766: [testbug] Adapt tests for Aix. > > Hi Goetz, > > Are you sure the quotation change in EvalArraysAsList.sh works well on other > platforms? > > Other than that the changes look good. Reviewed. > > Best regards > Christoph > > > -----Original Message----- > > From: core-libs-dev On Behalf > > Of Lindenmaier, Goetz > > Sent: Mittwoch, 18. Juli 2018 09:28 > > To: core-libs-dev at openjdk.java.net > > Subject: [CAUTION] RFR(S): 8207766: [testbug] Adapt tests for Aix. > > > > Hi, > > > > I would like to fix these tests to run on aix: > > http://cr.openjdk.java.net/~goetz/wr18/8207766-aixTestFixes/01/ > > > > > > MulticastSocket tests > > Opened bug and asked IBM to fix. Put on ProblemList. See > > https://bugs.openjdk.java.net/browse/JDK-8207404 > > > > EvalArraysAsList.sh > > Quotes need to be escaped on AIX. > > > > WrappedToolkitTest.sh > > Check for AIX missing. > > > > PKCS11Test.java > > getNSSLibDir can return null. If so, static initialization of badNSSVersion > fails: > > java.io.FileNotFoundException: nulllibsoftokn3.so (No such file or > directory) > > > > TestNssDbSqlite.java > > Don't run this on AIX. There is no NSS on basic installations. > > > > launcher/SourceMode.java > > AIX does not support arguments to the shebang. > > > > Best regards, > > Goetz. From Alan.Bateman at oracle.com Fri Jul 20 06:55:10 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 20 Jul 2018 07:55:10 +0100 Subject: JDK 11 RFR(s) JDK-8206865 RMI activation tests fail with InvalidClassException In-Reply-To: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> References: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> Message-ID: <7cdab27c-3bce-5d7a-362e-aa3eaeb5bcbc@oracle.com> On 19/07/2018 22:22, Stuart Marks wrote: > Hi all, > > Please review this fix for some failing RMI activation tests. Some > reflective code needed adjustment to take nestmates into account. This > also removes the failing tests from the problem list. > > Webrev: > > ????http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.0/ > > Bug: > ????https://bugs.openjdk.java.net/browse/JDK-8206865 Reflection.areNestMates is a native method and the number of cases where it will return true in this context is small so I think better to move this into the block that handles the case of the classes in the same run-time package. -Alan From matthias.baesken at sap.com Fri Jul 20 07:03:23 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 20 Jul 2018 07:03:23 +0000 Subject: RFR(S): 8207766: [testbug] Adapt tests for Aix. Message-ID: <8768ec5ea3744869aac0a0c955533ca3@sap.com> Looks good to me ( however I am not a Reviewer). Best regards, Matthias > -----Original Message----- > From: core-libs-dev > On Behalf > Of Lindenmaier, Goetz > Sent: Mittwoch, 18. Juli 2018 09:28 > To: core-libs-dev at openjdk.java.net > Subject: [CAUTION] RFR(S): 8207766: [testbug] Adapt tests for Aix. > > Hi, > > I would like to fix these tests to run on aix: > http://cr.openjdk.java.net/~goetz/wr18/8207766-aixTestFixes/01/ > > > MulticastSocket tests > Opened bug and asked IBM to fix. Put on ProblemList. See > https://bugs.openjdk.java.net/browse/JDK-8207404 > > EvalArraysAsList.sh > Quotes need to be escaped on AIX. > > WrappedToolkitTest.sh > Check for AIX missing. > > PKCS11Test.java > getNSSLibDir can return null. If so, static initialization of badNSSVersion fails: > java.io.FileNotFoundException: nulllibsoftokn3.so (No such file or directory) > > TestNssDbSqlite.java > Don't run this on AIX. There is no NSS on basic installations. > > launcher/SourceMode.java > AIX does not support arguments to the shebang. > > Best regards, > Goetz. From goetz.lindenmaier at sap.com Fri Jul 20 07:03:54 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 20 Jul 2018 07:03:54 +0000 Subject: RFR(S): 8207766: [testbug] Adapt tests for Aix. In-Reply-To: <8768ec5ea3744869aac0a0c955533ca3@sap.com> References: <8768ec5ea3744869aac0a0c955533ca3@sap.com> Message-ID: Hi Matthias, thanks for lookin at this! Best regards, Goetz. > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 20. Juli 2018 09:03 > To: core-libs-dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: RE : RFR(S): 8207766: [testbug] Adapt tests for Aix. > > Looks good to me ( however I am not a Reviewer). > > > > Best regards, Matthias > > > > > > > > > -----Original Message----- > > > From: core-libs-dev > On Behalf > > > Of Lindenmaier, Goetz > > > Sent: Mittwoch, 18. Juli 2018 09:28 > > > To: core-libs-dev at openjdk.java.net dev at openjdk.java.net> > > > Subject: [CAUTION] RFR(S): 8207766: [testbug] Adapt tests for Aix. > > > > > > Hi, > > > > > > I would like to fix these tests to run on aix: > > > http://cr.openjdk.java.net/~goetz/wr18/8207766-aixTestFixes/01/ > > > > > > > > > > MulticastSocket tests > > > Opened bug and asked IBM to fix. Put on ProblemList. See > > > https://bugs.openjdk.java.net/browse/JDK-8207404 > > > > > > > EvalArraysAsList.sh > > > Quotes need to be escaped on AIX. > > > > > > WrappedToolkitTest.sh > > > Check for AIX missing. > > > > > > PKCS11Test.java > > > getNSSLibDir can return null. If so, static initialization of badNSSVersion > fails: > > > java.io.FileNotFoundException: nulllibsoftokn3.so (No such file or > directory) > > > > > > TestNssDbSqlite.java > > > Don't run this on AIX. There is no NSS on basic installations. > > > > > > launcher/SourceMode.java > > > AIX does not support arguments to the shebang. > > > > > > Best regards, > > > Goetz. > > From goetz.lindenmaier at sap.com Fri Jul 20 07:38:13 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 20 Jul 2018 07:38:13 +0000 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: <48bdeddc592645dbb5ef046ab7226c67@sap.com> References: <48bdeddc592645dbb5ef046ab7226c67@sap.com> Message-ID: Looks good. Best regards, Goetz. > -----Original Message----- > From: Baesken, Matthias > Sent: Donnerstag, 19. Juli 2018 11:15 > To: core-libs-dev at openjdk.java.net; xueming.shen at oracle.com > Cc: Lindenmaier, Goetz > Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for the > jar -C parameter [windows] > > Hi Sherman , I like your proposal , I prepared this webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8207395.2/ > > > (and looking forward to your future move of jar to a complete nio > implementation !) > > Best regards, Matthias > > > > > > Message: 2 > > Date: Wed, 18 Jul 2018 11:37:34 -0700 > > From: Xueming Shen > > To: core-libs-dev at openjdk.java.net > > Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for > > the jar -C parameter [windows] > > Message-ID: <5B4F88EE.8080608 at oracle.com> > > Content-Type: text/plain; charset=UTF-8; format=flowed > > > > > > Ideally it would be preferred to move jar to a complete nio > > implementation and we then > > can leave the WindowsPath to take care of this stuff. That said that > > would be a relative > > big change. I do have a version on my disk but need clean up, test, > > performance measurement > > and review to get it. > > > > So for now I think it's fine to add this small fix to support the unc. > > > > now sure if the following is better? > > > > 637 boolean hasUNC = File.separatorChar == '\\'&& > > dir.startsWith("//"); > > 638 while (dir.indexOf("//")> -1) { > > 639 dir = dir.replace("//", "/"); > > 640 } > > 641 if (isUNC) // Restore Windows UNC path. > > 642 dir = "/" + dir; > > > > -Sherman > > > > > > On 7/18/18, 6:44 AM, Roger Riggs wrote: > > > Hi, > > > > > > Is there any chance that just using java.nio.file.Path.of will do the > > > needed cleanup? (or Paths.get) > > > It seems a shame to spread this kind of adhoc fixup around? > > > > > > $0.02, Roger > > > > > > On 7/18/2018 5:31 AM, Lindenmaier, Goetz wrote: > > >> Hi Matthias, > > >> > > >> thanks for doing this fix. > > >> > > >> I think this can be noted down a bit better, avoiding duplicating the > > >> loop. > > >> Also, please remove the redundant dir.replace(File.separatorChar, '/'). > > >> > > >> dir = dir.replace(File.separatorChar, '/'); > > >> + String unc = (dir.startsWith("//") && > > >> (File.separatorChar == '\\')) ? "/" : ""; > > >> while (dir.indexOf("//") > -1) { > > >> dir = dir.replace("//", "/"); > > >> } > > >> - > > >> pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); > > >> + // Restore the second leading '/' needed for > > >> the Windows UNC path. > > >> + dir = unc + dir; > > >> + pathsMap.get(version).add(dir); > > >> nameBuf[k++] = dir + args[++i]; > > >> > > >> Best regards, > > >> Goetz. > > >> > > >>> -----Original Message----- > > >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] > > On > > >>> Behalf Of Baesken, Matthias > > >>> Sent: Dienstag, 17. Juli 2018 13:15 > > >>> To: core-libs-dev at openjdk.java.net > > >>> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path > > >>> arguments > > >>> for the jar -C parameter [windows] > > >>> > > >>> Please review this small fix for allowing windows UNC paths in > > >>> the jar -C > > >>> parameter. > > >>> Currently passing a UNC path to a directory with -C fails : > > >>> > > >>> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C \\MYMACHINE\subdir > > >>> README.txt > > >>> Illegal option: s > > >>> Try `jar --help' for more information. > > >>> > > >>> With the patch it works . > > >>> > > >>> webrev > > >>> > > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ > > >>> > > >>> bug > > >>> > > >>> https://bugs.openjdk.java.net/browse/JDK-8207395 > > >>> > > >>> > > >>> > > >>> > > >>> Thanks, Matthias > > > From matthias.baesken at sap.com Fri Jul 20 08:08:21 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 20 Jul 2018 08:08:21 +0000 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: References: <48bdeddc592645dbb5ef046ab7226c67@sap.com> Message-ID: Thanks ! Sherman - are you fine with the latest version , if so I would push it . Best regards, Matthias > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Freitag, 20. Juli 2018 09:38 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net; xueming.shen at oracle.com > Subject: RE: RFR : 8207395: jar has issues with UNC-path arguments for the > jar -C parameter [windows] > > Looks good. > > Best regards, > Goetz. > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Donnerstag, 19. Juli 2018 11:15 > > To: core-libs-dev at openjdk.java.net; xueming.shen at oracle.com > > Cc: Lindenmaier, Goetz > > Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for the > > jar -C parameter [windows] > > > > Hi Sherman , I like your proposal , I prepared this webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8207395.2/ > > > > > > (and looking forward to your future move of jar to a complete nio > > implementation !) > > > > Best regards, Matthias > > > > > > > > > > Message: 2 > > > Date: Wed, 18 Jul 2018 11:37:34 -0700 > > > From: Xueming Shen > > > To: core-libs-dev at openjdk.java.net > > > Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for > > > the jar -C parameter [windows] > > > Message-ID: <5B4F88EE.8080608 at oracle.com> > > > Content-Type: text/plain; charset=UTF-8; format=flowed > > > > > > > > > Ideally it would be preferred to move jar to a complete nio > > > implementation and we then > > > can leave the WindowsPath to take care of this stuff. That said that > > > would be a relative > > > big change. I do have a version on my disk but need clean up, test, > > > performance measurement > > > and review to get it. > > > > > > So for now I think it's fine to add this small fix to support the unc. > > > > > > now sure if the following is better? > > > > > > 637 boolean hasUNC = File.separatorChar == '\\'&& > > > dir.startsWith("//"); > > > 638 while (dir.indexOf("//")> -1) { > > > 639 dir = dir.replace("//", "/"); > > > 640 } > > > 641 if (isUNC) // Restore Windows UNC path. > > > 642 dir = "/" + dir; > > > > > > -Sherman > > > > > > > > > On 7/18/18, 6:44 AM, Roger Riggs wrote: > > > > Hi, > > > > > > > > Is there any chance that just using java.nio.file.Path.of will do the > > > > needed cleanup? (or Paths.get) > > > > It seems a shame to spread this kind of adhoc fixup around? > > > > > > > > $0.02, Roger > > > > > > > > On 7/18/2018 5:31 AM, Lindenmaier, Goetz wrote: > > > >> Hi Matthias, > > > >> > > > >> thanks for doing this fix. > > > >> > > > >> I think this can be noted down a bit better, avoiding duplicating the > > > >> loop. > > > >> Also, please remove the redundant dir.replace(File.separatorChar, '/'). > > > >> > > > >> dir = dir.replace(File.separatorChar, '/'); > > > >> + String unc = (dir.startsWith("//") && > > > >> (File.separatorChar == '\\')) ? "/" : ""; > > > >> while (dir.indexOf("//") > -1) { > > > >> dir = dir.replace("//", "/"); > > > >> } > > > >> - > > > >> pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); > > > >> + // Restore the second leading '/' needed for > > > >> the Windows UNC path. > > > >> + dir = unc + dir; > > > >> + pathsMap.get(version).add(dir); > > > >> nameBuf[k++] = dir + args[++i]; > > > >> > > > >> Best regards, > > > >> Goetz. > > > >> > > > >>> -----Original Message----- > > > >>> From: core-libs-dev [mailto:core-libs-dev- > bounces at openjdk.java.net] > > > On > > > >>> Behalf Of Baesken, Matthias > > > >>> Sent: Dienstag, 17. Juli 2018 13:15 > > > >>> To: core-libs-dev at openjdk.java.net > > > >>> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path > > > >>> arguments > > > >>> for the jar -C parameter [windows] > > > >>> > > > >>> Please review this small fix for allowing windows UNC paths in > > > >>> the jar -C > > > >>> parameter. > > > >>> Currently passing a UNC path to a directory with -C fails : > > > >>> > > > >>> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C > \\MYMACHINE\subdir > > > >>> README.txt > > > >>> Illegal option: s > > > >>> Try `jar --help' for more information. > > > >>> > > > >>> With the patch it works . > > > >>> > > > >>> webrev > > > >>> > > > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ > > > >>> > > > >>> bug > > > >>> > > > >>> https://bugs.openjdk.java.net/browse/JDK-8207395 > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> Thanks, Matthias > > > > From jai.forums2013 at gmail.com Fri Jul 20 10:04:16 2018 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Fri, 20 Jul 2018 15:34:16 +0530 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <1CF5550B-90E6-4864-B544-990EEFAC03A4@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> <1CF5550B-90E6-4864-B544-990EEFAC03A4@oracle.com> Message-ID: <6c38b5c3-931e-3e0c-c449-77cdbd8db474@gmail.com> I agree with Chris. Unlike the connection failure error messages where the IP/port being part of the error message did add value, I don't think including the "path" to the jar (something like /opt/private/foo/bar/helloworld.jar) in case of MANIFEST parsing errors is really necessary. I think, just the filename (helloworld.jar) and the line number should be good enough in such failures. I have been trying to think in what cases the entire path would be necessary, to understand what's causing the parsing failure and address the issue. So far, I haven't been able to come up with such a use case. I do understand, as Alan already noted in this thread, the ZipFile APIs accept a file "path". So, directly using those paths in the error messages won't work out. Perhaps this changeset can be changed to do the necessary work to just get hold of the file name (while constructing the error message)? That way it wouldn't need any of these security configurations? -Jaikiran On 19/07/18 3:37 PM, Chris Hegarty wrote: >> On 19 Jul 2018, at 09:07, Baesken, Matthias >> wrote: Hello, in the meantime I prepared a >> CSR : https://bugs.openjdk.java.net/browse/JDK-8207768 > This CSR seems a little premature. Matthias said: "However so far it > is still a bit unclear how many exceptions we would like to enhance , > so this has to be checked first ?. Has this been checked? I have seen > no update on this or any other email thread. JDK-8204233 [1] was a > pragmatic and targeted solution to a long standing complaint. It was > not intended, as is obvious by the newly added property name, to set > precedent for doing similar all over the platform. If, after a broader > discussion, this approach is to be applied to several different areas > of the platform, then maybe JDK-8204233 should be revisited ( noting > that it is JDK 11, which is currently in RDP 1 ). I think we should > try to avoid a whole plethora of these properties. -Chris. [1] > https://bugs.openjdk.java.net/browse/JDK-8204233 >>> jdk.includeInExceptions expands the scope. That might be okay but we >>> will need to re-visit jdk.net.includeInExceptions and also move the >>> support to somewhere in jdk.internal so that it can be used by other >>> code in java.base. >> Is there some support code for " jdk.net.includeInExceptions " or do >> you just mean the name of the property ? Best regards, Matthias >>> -----Original Message----- From: Alan Bateman >>> [mailto:Alan.Bateman at oracle.com] Sent: Mittwoch, 18. Juli 2018 19:44 >>> To: Baesken, Matthias ; core-libs- >>> dev at openjdk.java.net; Lindenmaier, Goetz >>> Subject: Re: [RFR] 8205525 : Improve exception messages during >>> manifest parsing of jar archives On 18/07/2018 09:21, Baesken, >>> Matthias wrote: >>>> Hi Alan, I'll prepare a CSR . I selected a more general name >>>> "jdk.includeInExceptions" , because >>> there is the idea to enhance more exceptions with additional output . >>>> In such a case " jdk.util.jar.includeInExceptions" would not really >>>> help . However so far it is still a bit unclear how many exceptions >>>> we would like to >>> enhance , so this has to be checked first . >>> jdk.includeInExceptions expands the scope. That might be okay but we >>> will need to re-visit jdk.net.includeInExceptions and also move the >>> support to somewhere in jdk.internal so that it can be used by other >>> code in java.base. -Alan > From chris.hegarty at oracle.com Fri Jul 20 10:24:11 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 20 Jul 2018 11:24:11 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> Message-ID: > On 19 Jul 2018, at 15:42, Sean Mullan wrote: > >> ... > > Note that making this a security property for all general cases may have performance implications in certain scenarios since the java.security file will need to be loaded and fully parsed before it can be used. If you are already parsing the java.security file for something else (which is probably the case for jdk.net.includeInExceptions), then it should be fine, but that may not always be the case if this is made into a more general property. Just something to think about ? Good point Sean. My concern is more along the lines of a whole plethora of specific properties being added, rather than the implications of such. That should be scrutinised separately. What I am suggesting is to generalise the existing property so that it can, if applicable, be used in other areas. Not that it must be used in all other areas. The current proposal, to add a new property, already has this concern. What I am suggesting does not cause the file to be parsed any more than it would already. -Chris. From chris.hegarty at oracle.com Fri Jul 20 11:38:05 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 20 Jul 2018 12:38:05 +0100 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property Message-ID: JDK-8204233 added a new security property, `jdk.net.includeInExceptions`, to include additional, potentially security sensitive, information in exception detail messages in the networking area. The property accepts a comma separated list of values that specifies the particular type of extra detail information to add. Since its addition, in JDK 11, further uses have arisen to include additional, potentially security sensitive, information in exception detail messages in other areas, namely the java.util.jar APIs. See JDK-8205525, and http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054284.html Given that this mechanism will likely be used more generally across different parts of the platform, it seem prudent to rename the property to be less area-specific, thus allowing for additional argument values to be specified, like for example `jarPath`. The following are the suggested changes to the java.security file: $ hg extdiff -p diff -o -C1 src/java.base/share/conf/security/java.security *** 1062,1074 **** # ! # Enhanced exception message text # ! # By default, socket exception messages do not include potentially sensitive ! # information such as hostnames or port numbers. This property may be set to one ! # or more values, separated by commas, and with no white-space. Each value ! # represents a category of enhanced information. Currently, the only category defined ! # is "hostInfo" which enables more detailed information in the IOExceptions ! # thrown by java.net.Socket and also the socket types in the java.nio.channels package. ! # The setting in this file can be overridden by a system property of the same name ! # and with the same syntax and possible values. ! #jdk.net.includeInExceptions=hostInfo --- 1062,1084 ---- + + # + # Enhanced exception message information + # + # By default, several exception messages do not include potentially sensitive + # information such as file names, host names, or port numbers. This property may + # be used to enable categories of enhanced information in exception messages. + # The property accepts one or more comma separated values, each of which + # represents a category of enhanced exception message information to enable. + # Values are case-insensitive. Leading and trailing whitespaces, surrounding + # each value, are ignored. Unknown values are ignored. + # + # The categories, to enable enhanced exception message information, are: + # + # hostInfo - IOExceptions thrown by java.net.Socket and also the socket types + # in the java.nio.channels package will contain enhanced exception + # message information # ! # The property setting in this file can be overridden by a system property of ! # the same name, with the same syntax and possible values. # ! #jdk.includeInExceptions=hostInfo Full webrev: http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ -Chris. P.S. It appears that jtreg does not support quoted system property values with spaces on the @run line. I?ll file an issue against jtreg for this. From chris.hegarty at oracle.com Fri Jul 20 11:40:41 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 20 Jul 2018 12:40:41 +0100 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> Message-ID: > On 19 Jul 2018, at 13:54, Chris Hegarty wrote: > .... > > I filed the following issue to generalize the `includeInExceptions` security > property: > https://bugs.openjdk.java.net/browse/JDK-8207846 I sent out an RFR for 8207846, since I think it is worth proceeding with regardless of the outcome of 8205525. http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054497.html -Chris. From matthias.baesken at sap.com Fri Jul 20 11:47:28 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 20 Jul 2018 11:47:28 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> Message-ID: <3b73a16ae8bd4072acbb43958d22ccfc@sap.com> Hi Chris, thanks for starting the new RFR for to generalization of the `includeInExceptions` security property: Best regards, Matthias > -----Original Message----- > From: Chris Hegarty [mailto:chris.hegarty at oracle.com] > Sent: Freitag, 20. Juli 2018 13:41 > To: Alan Bateman ; Baesken, Matthias > > Cc: core-libs-dev at openjdk.java.net > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > > > On 19 Jul 2018, at 13:54, Chris Hegarty wrote: > > .... > > > > I filed the following issue to generalize the `includeInExceptions` security > > property: > > https://bugs.openjdk.java.net/browse/JDK-8207846 > > I sent out an RFR for 8207846, since I think it is worth proceeding with > regardless of the outcome of 8205525. > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054497.html > > -Chris. From goetz.lindenmaier at sap.com Fri Jul 20 12:30:15 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 20 Jul 2018 12:30:15 +0000 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: References: Message-ID: <66ccf1fc97b64438b5c2d3c03fcd1948@sap.com> Hi, I scanned all the changes we did to exception messages in our internal VM, see below. We print paths and sockets in a row of places, but also other information. It's wide spread, while most is in java.base. We plan to contribute these messages in the near future. Thus it'll be useful if the property can be reused whenever some of this information is rated security relevant. Best regards, Goetz. java/io/FileOutputStream.java java/io/RandomAccessFile.java path java.base/unix/native/libjava/UnixFileSystem_md.c java.base/windows/native/libjava/WinNTFileSystem_md.c path java/util/jar/Manifest.java java/util/jar/Attributes.java path, position in file java/util/Timer.java print the errornous values (delay, period, time) jdk/internal/org/objectweb/asm/MethodWriter.java length of too long bytecode code libverify/check_code.c position of error, errroneous types java.base/share/native/libzip/zip_util.c path to zipped file java.base/share/native/libzip/zlib/inflate.c zip version information java.base/unix/native/libnet/Inet6AddressImpl.c java.base/share/native/libnet/net_util.c java.base/unix/native/libnet/PlainDatagramSocketImpl.c java.base/unix/native/libnet/PlainSocketImpl.c java.base/unix/native/libnet/SocketInputStream.c java.base/unix/native/libnet/SocketOutputStream.c java.base/windows/native/libnet/DualStackPlainDatagramSocketImpl.c java.base/windows/native/libnet/DualStackPlainSocketImpl.c Information about sockets java.base/unix/native/libnio/ch/DatagramChannelImpl.c java.base/unix/native/libnio/ch/DatagramDispatcher.c socket java.base/unix/native/libnio/ch/FileChannelImpl.c size, protection codes of memory, file, java.desktop/unix/native/libawt_xawt/awt/awt_GraphicsEnv.c more detailed information about DISPLAY jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java inet address jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystem.java path > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Chris Hegarty > Sent: Freitag, 20. Juli 2018 13:38 > To: core-libs-dev ; Security Dev OpenJDK > > Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions > security property > > JDK-8204233 added a new security property, `jdk.net.includeInExceptions`, > to include additional, potentially security sensitive, information in > exception detail messages in the networking area. The property accepts a > comma separated list of values that specifies the particular type of > extra detail information to add. > > Since its addition, in JDK 11, further uses have arisen to include > additional, potentially security sensitive, information in exception > detail messages in other areas, namely the java.util.jar APIs. See > JDK-8205525, and http://mail.openjdk.java.net/pipermail/core-libs- > dev/2018-July/054284.html > > Given that this mechanism will likely be used more generally across > different parts of the platform, it seem prudent to rename the property > to be less area-specific, thus allowing for additional argument values > to be specified, like for example `jarPath`. > > The following are the suggested changes to the java.security file: > > $ hg extdiff -p diff -o -C1 src/java.base/share/conf/security/java.security > *** 1062,1074 **** > > # > ! # Enhanced exception message text > # > ! # By default, socket exception messages do not include potentially sensitive > ! # information such as hostnames or port numbers. This property may be set > to one > ! # or more values, separated by commas, and with no white-space. Each > value > ! # represents a category of enhanced information. Currently, the only > category defined > ! # is "hostInfo" which enables more detailed information in the IOExceptions > ! # thrown by java.net.Socket and also the socket types in the > java.nio.channels package. > ! # The setting in this file can be overridden by a system property of the same > name > ! # and with the same syntax and possible values. > ! #jdk.net.includeInExceptions=hostInfo > --- 1062,1084 ---- > > + > + # > + # Enhanced exception message information > + # > + # By default, several exception messages do not include potentially > sensitive > + # information such as file names, host names, or port numbers. This > property may > + # be used to enable categories of enhanced information in exception > messages. > + # The property accepts one or more comma separated values, each of > which > + # represents a category of enhanced exception message information to > enable. > + # Values are case-insensitive. Leading and trailing whitespaces, surrounding > + # each value, are ignored. Unknown values are ignored. > + # > + # The categories, to enable enhanced exception message information, are: > + # > + # hostInfo - IOExceptions thrown by java.net.Socket and also the socket > types > + # in the java.nio.channels package will contain enhanced exception > + # message information > # > ! # The property setting in this file can be overridden by a system property of > ! # the same name, with the same syntax and possible values. > # > ! #jdk.includeInExceptions=hostInfo > > > Full webrev: > http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ > > -Chris. > > P.S. It appears that jtreg does not support quoted system property values > with spaces on the @run line. I?ll file an issue against jtreg for this. From Alan.Bateman at oracle.com Fri Jul 20 12:55:23 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 20 Jul 2018 13:55:23 +0100 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: References: Message-ID: <1c8a463d-63eb-d2e9-118e-e37823942de8@oracle.com> On 20/07/2018 12:38, Chris Hegarty wrote: > JDK-8204233 added a new security property, `jdk.net.includeInExceptions`, > to include additional, potentially security sensitive, information in > exception detail messages in the networking area. The property accepts a > comma separated list of values that specifies the particular type of > extra detail information to add. > > : > > > Full webrev: > http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ > This looks good. For the follow-on jarPath then I think we should look at moving the parsing to somewhere in java.base so that it can be used by other code in the module. -Alan From adinn at redhat.com Fri Jul 20 13:50:22 2018 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 20 Jul 2018 14:50:22 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory Message-ID: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> I have finally managed to draft a JEP to formally present the proposal I circulated a month or two back regarding support for MappedByteBuffer access to non-volatile memory. JEP JIRA: https://bugs.openjdk.java.net/browse/JDK-8207851 The JEP references a re-implementation of the proposed API: http://cr.openjdk.java.net/~adinn/pmem/webrev.01/ The rework addresses some of the feedback provided in the initial discussion: Previous Thread: (starts in May archive) http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053317.html (continues into June archive at) http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053673.html In particular, the re-implementation has split the original Hotspot intrinsic used to guarantee writeback an address range into 3 intrinsics relocated into class Unsafe which write back a single cache line perform any necessary memory sync pre-writeback perform any necessary memory sync post-writeback Also, the implementation has been extended from Linux on x86_64 to also coater for Linux on AArch64. The latter version compiles and generates correct code but has not yet been tested because I have not yet obtained access to a box with a suitably up to date kernel. Comments and suggestions for improvement would be very welcome. n.v. Jonathan Halliday is about to get his hands on an x86 box with a real NVM dimm so we may soon be able to try this out with NVM proper rather than testing with volatile memory pretending to be NVM. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Fri Jul 20 14:05:57 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 20 Jul 2018 15:05:57 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> Message-ID: <2baace5b-fcdb-913e-0e18-b54706e434ea@oracle.com> On 20/07/2018 14:50, Andrew Dinn wrote: > I have finally managed to draft a JEP to formally present the proposal I > circulated a month or two back regarding support for MappedByteBuffer > access to non-volatile memory. > > JEP JIRA: > https://bugs.openjdk.java.net/browse/JDK-8207851 > > The JEP references a re-implementation of the proposed API: > > http://cr.openjdk.java.net/~adinn/pmem/webrev.01/ > The JEP proposes adding a map_persistent method to FileChannel. An alternative may be to just add new MapModes that you can specify to the existing map method. It would reduce the API surface and I think fit better with the existing API. -Alan From chris.hegarty at oracle.com Fri Jul 20 14:18:47 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 20 Jul 2018 15:18:47 +0100 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: References: Message-ID: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> Roger, > On 20 Jul 2018, at 14:52, Roger Riggs wrote: > > Hi Chris, > > It is very unusual for the processing of system properties to do *any* parsing except for delimiters > including removing spaces, etc. It complicates the handling and sets a bad precedent > that makes it more complex for users and developers to know how to set property values. > The whitespace trimming should be removed. The addition of the whitespace trimming was to clarify the intent of the existing wording regarding whitespace. It was not intended to set a precedent, good or bad. Regardless of any subjective opinion, whitespace are allowable within system property values, so it?s a matter of whether or not we want to deal with them explicitly or just leave it as an implementation detail. I?m ok to drop the trimming if you feel strongly about it. But just to note that this now diverges even more from the original. -Chris. > $.02, Roger > >> http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ >> >> -Chris. >> >> P.S. It appears that jtreg does not support quoted system property values >> with spaces on the @run line. I?ll file an issue against jtreg for this. >> > From adinn at redhat.com Fri Jul 20 14:31:22 2018 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 20 Jul 2018 15:31:22 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <2baace5b-fcdb-913e-0e18-b54706e434ea@oracle.com> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <2baace5b-fcdb-913e-0e18-b54706e434ea@oracle.com> Message-ID: <81493953-10f9-e79c-1000-22adc67d3dfa@redhat.com> On 20/07/18 15:05, Alan Bateman wrote: > On 20/07/2018 14:50, Andrew Dinn wrote: >> I have finally managed to draft a JEP to formally present the proposal I >> circulated a month or two back regarding support for MappedByteBuffer >> access to non-volatile memory. >> >> JEP JIRA: >> ?? https://bugs.openjdk.java.net/browse/JDK-8207851 >> >> The JEP references a re-implementation of the proposed API: >> >> ?? http://cr.openjdk.java.net/~adinn/pmem/webrev.01/ >> > The JEP proposes adding a map_persistent method to FileChannel. An > alternative may be to just add new MapModes that you can specify to the > existing map method. It would reduce the API surface and I think fit > better with the existing API. Doh! that's a good idea (I wish I had thought of that :-) I'll admit that it looks a tad counter-intuitive to specify the storage characteristics of the data as part the access mode e.g. class MapMode { public static final MapMode READ_ONLY = ... public static final MapMode READ_WRITE = ... public static final MapMode PRIVATE = ... public static final MapMode READ_ONLY_PERSISTENT = ... public static final MapMode READ_WRITE_PERSISTENT = ... . . . } However, it would make the API a damn sight neater. Also, there is still the question as to whether existing FileChannel implementations would correctly reject these new modes if passed them i.e. if (mode == MapMode.READ_ONLY) { ... } else if (mode == MapMode.READ_WRITE) { ... } else { ... } Thoughts? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From roger.riggs at oracle.com Fri Jul 20 14:36:54 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 20 Jul 2018 10:36:54 -0400 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> Message-ID: <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> Hi Chris, It is important to be clear about how whitespace is treated and within the java.security file there are other uses that explicitly define how whitespace is used. I am more concerned about how command line properties are understood and used how we have to document them. Allowing whitespace quickly gets bogged down in how shells handle quotes, telling people they have to quote them and when/whether you have to quote the quotes. Having a consistent treatment of command line and security properties keeps the story simple and easier to support. The jdk.serialFilter property had the same issue and is explicit in the java.security file that spaces are just another character and are not treated specially. Its a slippery slope, if we start compensating/ignoring whitespace in some properties then we will have to keep explaining how some are treated differently. I would keep the original non-whitespace description. Case-insensistive compares are another slippery slope but make a bit more sense for usability. $.02, Roger On 7/20/18 10:18 AM, Chris Hegarty wrote: > Roger, > >> On 20 Jul 2018, at 14:52, Roger Riggs wrote: >> >> Hi Chris, >> >> It is very unusual for the processing of system properties to do *any* parsing except for delimiters >> including removing spaces, etc. It complicates the handling and sets a bad precedent >> that makes it more complex for users and developers to know how to set property values. >> The whitespace trimming should be removed. > The addition of the whitespace trimming was to clarify the intent of > the existing wording regarding whitespace. It was not intended to > set a precedent, good or bad. > > Regardless of any subjective opinion, whitespace are allowable > within system property values, so it?s a matter of whether or not > we want to deal with them explicitly or just leave it as an > implementation detail. > > I?m ok to drop the trimming if you feel strongly about it. But just > to note that this now diverges even more from the original. > > -Chris. > >> $.02, Roger >> >>> http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ >>> >>> -Chris. >>> >>> P.S. It appears that jtreg does not support quoted system property values >>> with spaces on the @run line. I?ll file an issue against jtreg for this. >>> From chris.hegarty at oracle.com Fri Jul 20 15:08:21 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 20 Jul 2018 16:08:21 +0100 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> Message-ID: <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> Roger, > On 20 Jul 2018, at 15:36, Roger Riggs wrote: > > Hi Chris, > > It is important to be clear about how whitespace is treated and within the java.security file > there are other uses that explicitly define how whitespace is used. Right, and the usages are already inconsistent. Nothing we can do about that now. > I am more concerned about how command line properties are understood and used how we have to document them. > Allowing whitespace quickly gets bogged down in how shells handle quotes, telling people they have to > quote them and when/whether you have to quote the quotes. You cannot disallow whitespace, simple ignore them or consider them part of the value. > Having a consistent treatment of command line and security properties keeps the > story simple and easier to support. This file is already inconsistent, trimming happens in some cases. Whitespaces are either trimmed, ignored, or considered as like any other character. > The jdk.serialFilter property had the same issue and is explicit in the java.security file > that spaces are just another character and are not treated specially. This is a reasonable position. > Its a slippery slope, if we start compensating/ignoring whitespace in some properties > then we will have to keep explaining how some are treated differently. > I would keep the original non-whitespace description. Original: "This property may be set to one or more values, separated by commas, and with no white-space? This is ambiguous, and needs to be clarified. Surely, it is better to use the same wording as the serial filter: "Whitespace is significant and is considered part of the value." > Case-insensistive compares are another slippery slope but make a bit more sense for usability. The complete updated text: # # Enhanced exception message information # # By default, several exception messages do not include potentially sensitive # information such as file names, host names, or port numbers. This property may # be used to enable categories of enhanced information in exception messages. # The property accepts one or more comma separated values, each of which # represents a category of enhanced exception message information to enable. # Values are case-insensitive. Whitespace is significant and is considered part # of the value. Unknown values are ignored. # # The categories, to enable enhanced exception message information, are: # # hostInfo - IOExceptions thrown by java.net.Socket and also the socket types # in the java.nio.channels package will contain enhanced exception # message information # # The property setting in this file can be overridden by a system property of # the same name, with the same syntax and possible values. # #jdk.includeInExceptions=hostInfo -Chris. From roger.riggs at oracle.com Fri Jul 20 15:15:20 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 20 Jul 2018 11:15:20 -0400 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> Message-ID: <675acacb-1c17-3880-b5f2-1f0565ddb0ee@oracle.com> Hi Chris, The updated text is fine. Thanks for your consideration. The overlapping of configure options between java.security, net.properties, etc. and command line options is something to keep an eye on. Roger On 7/20/18 11:08 AM, Chris Hegarty wrote: > Roger, > >> On 20 Jul 2018, at 15:36, Roger Riggs wrote: >> >> Hi Chris, >> >> It is important to be clear about how whitespace is treated and within the java.security file >> there are other uses that explicitly define how whitespace is used. > Right, and the usages are already inconsistent. Nothing we can > do about that now. > >> I am more concerned about how command line properties are understood and used how we have to document them. >> Allowing whitespace quickly gets bogged down in how shells handle quotes, telling people they have to >> quote them and when/whether you have to quote the quotes. > You cannot disallow whitespace, simple ignore them or consider > them part of the value. > >> Having a consistent treatment of command line and security properties keeps the >> story simple and easier to support. > This file is already inconsistent, trimming happens in some cases. > Whitespaces are either trimmed, ignored, or considered as like > any other character. > >> The jdk.serialFilter property had the same issue and is explicit in the java.security file >> that spaces are just another character and are not treated specially. > This is a reasonable position. > >> Its a slippery slope, if we start compensating/ignoring whitespace in some properties >> then we will have to keep explaining how some are treated differently. >> I would keep the original non-whitespace description. > Original: "This property may be set to one or more values, > separated by commas, and with no white-space? > > This is ambiguous, and needs to be clarified. Surely, it is > better to use the same wording as the serial filter: > > "Whitespace is significant and is considered part of the value." > >> Case-insensistive compares are another slippery slope but make a bit more sense for usability. > The complete updated text: > > # > # Enhanced exception message information > # > # By default, several exception messages do not include potentially sensitive > # information such as file names, host names, or port numbers. This property may > # be used to enable categories of enhanced information in exception messages. > # The property accepts one or more comma separated values, each of which > # represents a category of enhanced exception message information to enable. > # Values are case-insensitive. Whitespace is significant and is considered part > # of the value. Unknown values are ignored. > # > # The categories, to enable enhanced exception message information, are: > # > # hostInfo - IOExceptions thrown by java.net.Socket and also the socket types > # in the java.nio.channels package will contain enhanced exception > # message information > # > # The property setting in this file can be overridden by a system property of > # the same name, with the same syntax and possible values. > # > #jdk.includeInExceptions=hostInfo > > -Chris. > From chris.hegarty at oracle.com Fri Jul 20 15:33:06 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 20 Jul 2018 16:33:06 +0100 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <675acacb-1c17-3880-b5f2-1f0565ddb0ee@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> <675acacb-1c17-3880-b5f2-1f0565ddb0ee@oracle.com> Message-ID: <348DBF4E-7767-4CF1-B8D9-8AAFE937332C@oracle.com> > On 20 Jul 2018, at 16:15, Roger Riggs wrote: > > Hi Chris, > > The updated text is fine. > Thanks for your consideration. Updated webrev as discussed. http://cr.openjdk.java.net/~chegar/8207846/webrev.01/ -Chris. From roger.riggs at oracle.com Fri Jul 20 15:58:02 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 20 Jul 2018 11:58:02 -0400 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <348DBF4E-7767-4CF1-B8D9-8AAFE937332C@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> <675acacb-1c17-3880-b5f2-1f0565ddb0ee@oracle.com> <348DBF4E-7767-4CF1-B8D9-8AAFE937332C@oracle.com> Message-ID: <3ad1b085-66e0-ddfa-d095-d88cd0201ceb@oracle.com> +1 Thanks, Roger On 7/20/18 11:33 AM, Chris Hegarty wrote: >> On 20 Jul 2018, at 16:15, Roger Riggs wrote: >> >> Hi Chris, >> >> The updated text is fine. >> Thanks for your consideration. > Updated webrev as discussed. > http://cr.openjdk.java.net/~chegar/8207846/webrev.01/ > > -Chris. From xueming.shen at oracle.com Fri Jul 20 16:58:40 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 20 Jul 2018 09:58:40 -0700 Subject: RFR : 8207395: jar has issues with UNC-path arguments for the jar -C parameter [windows] In-Reply-To: References: <48bdeddc592645dbb5ef046ab7226c67@sap.com> Message-ID: <5B5214C0.9070209@oracle.com> +1 On 7/20/18, 1:08 AM, Baesken, Matthias wrote: > Thanks ! > Sherman - are you fine with the latest version , if so I would push it . > > Best regards, Matthias > > >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Freitag, 20. Juli 2018 09:38 >> To: Baesken, Matthias; core-libs- >> dev at openjdk.java.net; xueming.shen at oracle.com >> Subject: RE: RFR : 8207395: jar has issues with UNC-path arguments for the >> jar -C parameter [windows] >> >> Looks good. >> >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: Baesken, Matthias >>> Sent: Donnerstag, 19. Juli 2018 11:15 >>> To: core-libs-dev at openjdk.java.net; xueming.shen at oracle.com >>> Cc: Lindenmaier, Goetz >>> Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for the >>> jar -C parameter [windows] >>> >>> Hi Sherman , I like your proposal , I prepared this webrev : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395.2/ >>> >>> >>> (and looking forward to your future move of jar to a complete nio >>> implementation !) >>> >>> Best regards, Matthias >>> >>> >>>> Message: 2 >>>> Date: Wed, 18 Jul 2018 11:37:34 -0700 >>>> From: Xueming Shen >>>> To: core-libs-dev at openjdk.java.net >>>> Subject: Re: RFR : 8207395: jar has issues with UNC-path arguments for >>>> the jar -C parameter [windows] >>>> Message-ID:<5B4F88EE.8080608 at oracle.com> >>>> Content-Type: text/plain; charset=UTF-8; format=flowed >>>> >>>> >>>> Ideally it would be preferred to move jar to a complete nio >>>> implementation and we then >>>> can leave the WindowsPath to take care of this stuff. That said that >>>> would be a relative >>>> big change. I do have a version on my disk but need clean up, test, >>>> performance measurement >>>> and review to get it. >>>> >>>> So for now I think it's fine to add this small fix to support the unc. >>>> >>>> now sure if the following is better? >>>> >>>> 637 boolean hasUNC = File.separatorChar == '\\'&& >>>> dir.startsWith("//"); >>>> 638 while (dir.indexOf("//")> -1) { >>>> 639 dir = dir.replace("//", "/"); >>>> 640 } >>>> 641 if (isUNC) // Restore Windows UNC path. >>>> 642 dir = "/" + dir; >>>> >>>> -Sherman >>>> >>>> >>>> On 7/18/18, 6:44 AM, Roger Riggs wrote: >>>>> Hi, >>>>> >>>>> Is there any chance that just using java.nio.file.Path.of will do the >>>>> needed cleanup? (or Paths.get) >>>>> It seems a shame to spread this kind of adhoc fixup around? >>>>> >>>>> $0.02, Roger >>>>> >>>>> On 7/18/2018 5:31 AM, Lindenmaier, Goetz wrote: >>>>>> Hi Matthias, >>>>>> >>>>>> thanks for doing this fix. >>>>>> >>>>>> I think this can be noted down a bit better, avoiding duplicating the >>>>>> loop. >>>>>> Also, please remove the redundant dir.replace(File.separatorChar, '/'). >>>>>> >>>>>> dir = dir.replace(File.separatorChar, '/'); >>>>>> + String unc = (dir.startsWith("//")&& >>>>>> (File.separatorChar == '\\')) ? "/" : ""; >>>>>> while (dir.indexOf("//")> -1) { >>>>>> dir = dir.replace("//", "/"); >>>>>> } >>>>>> - >>>>>> pathsMap.get(version).add(dir.replace(File.separatorChar, '/')); >>>>>> + // Restore the second leading '/' needed for >>>>>> the Windows UNC path. >>>>>> + dir = unc + dir; >>>>>> + pathsMap.get(version).add(dir); >>>>>> nameBuf[k++] = dir + args[++i]; >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: core-libs-dev [mailto:core-libs-dev- >> bounces at openjdk.java.net] >>>> On >>>>>>> Behalf Of Baesken, Matthias >>>>>>> Sent: Dienstag, 17. Juli 2018 13:15 >>>>>>> To: core-libs-dev at openjdk.java.net >>>>>>> Subject: [CAUTION] RFR : 8207395: jar has issues with UNC-path >>>>>>> arguments >>>>>>> for the jar -C parameter [windows] >>>>>>> >>>>>>> Please review this small fix for allowing windows UNC paths in >>>>>>> the jar -C >>>>>>> parameter. >>>>>>> Currently passing a UNC path to a directory with -C fails : >>>>>>> >>>>>>> c:\testdir>c:\tools\jdk10\bin\jar.exe test.jar -C >> \\MYMACHINE\subdir >>>>>>> README.txt >>>>>>> Illegal option: s >>>>>>> Try `jar --help' for more information. >>>>>>> >>>>>>> With the patch it works . >>>>>>> >>>>>>> webrev >>>>>>> >>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8207395/ >>>>>>> >>>>>>> bug >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8207395 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, Matthias From sean.mullan at oracle.com Fri Jul 20 17:07:51 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 20 Jul 2018 13:07:51 -0400 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> Message-ID: <3c5c5465-492b-b740-77fc-f5a5c042d283@oracle.com> On 7/20/18 11:08 AM, Chris Hegarty wrote: > This is ambiguous, and needs to be clarified. Surely, it is > better to use the same wording as the serial filter: > > "Whitespace is significant and is considered part of the value." Kind of on the fence on that one. If this were a general property/value format, I would agree, but these values are fixed and not potentially complicated expressions. For example, does this mean: jdk.includeInExceptions=hostInfo,jarInfo and jdk.includeInExceptions=hostInfo, jarInfo are different? And I assume the latter " jarInfo" would be ignored? If you are strongly in favor of this, I would highly recommend logging a warning when there is an unknown value, otherwise typos and such would be hard to detect (although this doesn't necessarily need to be in the specification). --Sean From stuart.marks at oracle.com Fri Jul 20 17:27:34 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 20 Jul 2018 10:27:34 -0700 Subject: JDK 11 RFR(s) JDK-8206865 RMI activation tests fail with InvalidClassException In-Reply-To: <7cdab27c-3bce-5d7a-362e-aa3eaeb5bcbc@oracle.com> References: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> <7cdab27c-3bce-5d7a-362e-aa3eaeb5bcbc@oracle.com> Message-ID: <29dd7c64-a8ce-916b-d5b5-f1b8f2b00024@oracle.com> On 7/19/18 11:55 PM, Alan Bateman wrote: > On 19/07/2018 22:22, Stuart Marks wrote: >> Please review this fix for some failing RMI activation tests. Some reflective >> code needed adjustment to take nestmates into account. This also removes the >> failing tests from the problem list. >> >> Webrev: >> >> ????http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.0/ >> >> Bug: >> ????https://bugs.openjdk.java.net/browse/JDK-8206865 > Reflection.areNestMates is a native method and the number of cases where it will > return true in this context is small so I think better to move this into the > block that handles the case of the classes in the same run-time package. OK. Updated webrev: http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.1/ Thanks, s'marks From mandy.chung at oracle.com Fri Jul 20 17:36:30 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 20 Jul 2018 10:36:30 -0700 Subject: JDK 11 RFR(s) JDK-8206865 RMI activation tests fail with InvalidClassException In-Reply-To: <29dd7c64-a8ce-916b-d5b5-f1b8f2b00024@oracle.com> References: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> <7cdab27c-3bce-5d7a-362e-aa3eaeb5bcbc@oracle.com> <29dd7c64-a8ce-916b-d5b5-f1b8f2b00024@oracle.com> Message-ID: On 7/20/18 10:27 AM, Stuart Marks wrote: > On 7/19/18 11:55 PM, Alan Bateman wrote: >> On 19/07/2018 22:22, Stuart Marks wrote: >>> Please review this fix for some failing RMI activation tests. Some >>> reflective code needed adjustment to take nestmates into account. >>> This also removes the failing tests from the problem list. >>> >>> Webrev: >>> >>> ????http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.0/ >>> >>> Bug: >>> ????https://bugs.openjdk.java.net/browse/JDK-8206865 > >> Reflection.areNestMates is a native method and the number of cases >> where it will return true in this context is small so I think better >> to move this into the block that handles the case of the classes in >> the same run-time package. > > OK. Updated webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.1/ This looks good. Mandy From fw at deneb.enyo.de Fri Jul 20 17:55:52 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Fri, 20 Jul 2018 19:55:52 +0200 Subject: Long chains created by direct Buffer::slice Message-ID: <87o9f1n653.fsf@mid.deneb.enyo.de> src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template contains this, unfortunately without further comments: | public $Type$Buffer slice() { | int pos = this.position(); | int lim = this.limit(); | assert (pos <= lim); | int rem = (pos <= lim ? lim - pos : 0); | int off = (pos << $LG_BYTES_PER_VALUE$); | assert (off >= 0); | return new Direct$Type$Buffer$RW$$BO$(this, -1, 0, rem, rem, off); | } ByteBuffer::duplicate and ByteBuffer::asReadOnlyBuffer have similar code. The constructor invoked: | // For duplicates and slices | // | Direct$Type$Buffer$RW$$BO$(DirectBuffer db, // package-private | int mark, int pos, int lim, int cap, | int off) | { | #if[rw] | super(mark, pos, lim, cap); | address = db.address() + off; | #if[byte] | cleaner = null; | #end[byte] | att = db; | #else[rw] | super(db, mark, pos, lim, cap, off); | this.isReadOnly = true; | #end[rw] | } The key part is the assignment to the att member. If I understand this correctly, it is needed to keep the backing object alive during the lifetime of this buffer. However, it causes the creation of a long chain of buffer objects. With -Xmx100m or so, the following test will OOM fairly quickly for this reason: | volatile ByteBuffer buffer; | ? | buffer = ByteBuffer.allocateDirect(16384); | while (true) { | buffer = buffer.duplicate(); | } I wonder if it would be possible to change the setting of the att member to this instead: | if (db.att == null) { | att = db; | } else { | att = db.att; | } This would only keep the object alive which actually owns the backing storage, as if Buffer::slice had been invoked on it directly. From Alan.Bateman at oracle.com Fri Jul 20 18:14:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 20 Jul 2018 19:14:06 +0100 Subject: JDK 11 RFR(s) JDK-8206865 RMI activation tests fail with InvalidClassException In-Reply-To: <29dd7c64-a8ce-916b-d5b5-f1b8f2b00024@oracle.com> References: <7628b8a6-f523-aa29-f2ac-78401cf1f425@oracle.com> <7cdab27c-3bce-5d7a-362e-aa3eaeb5bcbc@oracle.com> <29dd7c64-a8ce-916b-d5b5-f1b8f2b00024@oracle.com> Message-ID: On 20/07/2018 18:27, Stuart Marks wrote: > > OK. Updated webrev: > > http://cr.openjdk.java.net/~smarks/reviews/8206865/webrev.1/ Looks good. From mark.reinhold at oracle.com Fri Jul 20 18:15:59 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 20 Jul 2018 11:15:59 -0700 Subject: Long chains created by direct Buffer::slice In-Reply-To: <87o9f1n653.fsf@mid.deneb.enyo.de> References: <87o9f1n653.fsf@mid.deneb.enyo.de> Message-ID: <20180720111559.849741094@eggemoggin.niobe.net> 2018/7/20 10:55:52 -0700, Florian Weimer : > ... > > The constructor invoked: > > | // For duplicates and slices > | // > | Direct$Type$Buffer$RW$$BO$(DirectBuffer db, // package-private > | int mark, int pos, int lim, int cap, > | int off) > | { > | #if[rw] > | super(mark, pos, lim, cap); > | address = db.address() + off; > | #if[byte] > | cleaner = null; > | #end[byte] > | att = db; > | #else[rw] > | super(db, mark, pos, lim, cap, off); > | this.isReadOnly = true; > | #end[rw] > | } > > The key part is the assignment to the att member. If I understand > this correctly, it is needed to keep the backing object alive during > the lifetime of this buffer. Correct. > However, it causes the creation of a > long chain of buffer objects. With -Xmx100m or so, the following test > will OOM fairly quickly for this reason: > > | volatile ByteBuffer buffer; > | ? > | buffer = ByteBuffer.allocateDirect(16384); > | while (true) { > | buffer = buffer.duplicate(); > | } Well spotted! This bug has been lurking there for sixteen years. > > I wonder if it would be possible to change the setting of the att > member to this instead: > > | if (db.att == null) { > | att = db; > | } else { > | att = db.att; > | } > > This would only keep the object alive which actually owns the backing > storage, as if Buffer::slice had been invoked on it directly. Your suggested fix looks fine. - Mark From jiangli.zhou at oracle.com Fri Jul 20 18:31:57 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 20 Jul 2018 11:31:57 -0700 Subject: RFR: 8207263: Store the Configuration for system modules into CDS archive Message-ID: Please review the following webrev that archives the system module boot layer Configuration (including all java objects reachable from the Configuration) in CDS archive. This is built on top of the earlier change for JDK-8202035 (https://bugs.openjdk.java.net/browse/JDK-8202035), which provides a framework for object sub-graph archiving. The boot layer Configuration is created in ModuleBootstrap.boot() (similar to the archived system ModuleDescriptor objects, etc) and is unchanged after construction. With archived boot layer Configuration, it allows runtime to bypass the work for creating the configuration. Currently, this is only supported when the initial module is unnamed module. Measurements indicate archiving the boot layer Configuration improves the startup time by 1% ~ 1.5% (on linux-x64) when running HelloWorld from -cp at runtime. Many thanks to Alan and Claes for discussions and contributions to this change! Webrev: http://cr.openjdk.java.net/~jiangli/8207263/webrev.00/ RFE: https://bugs.openjdk.java.net/browse/JDK-8207263 Tested with tier1 - tier5 tests via mach5. Thanks, Jiangli From joe.darcy at oracle.com Fri Jul 20 21:18:55 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Fri, 20 Jul 2018 14:18:55 -0700 Subject: JDK 12 RFR of JDK-8208060: Additional corrections of serial-related declarations Message-ID: <5B5251BF.6020005@oracle.com> Hello, Another round of cleaning up the declarations of serial-related fields and methods: JDK-8208060: Additional corrections of serial-related declarations http://cr.openjdk.java.net/~darcy/8208060.0/ Summary of the issues being addressed: src/java.base/share/classes/java/net/InetAddress.java Extra thrown types for readObjectNoData. src/java.management/share/classes/java/lang/management/MemoryType.java A serialVesionUID field in an enum type is a no-op. src/java.rmi/share/classes/java/rmi/activation/ActivationID.java src/java.rmi/share/classes/java/rmi/server/RemoteObject.java src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialArray.java src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialBlob.java rc/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialRef.java src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialStruct.java src/java.sql/share/classes/java/sql/BatchUpdateException.java A writeObject method doesn't throw ClassNotFoundException. Thanks, From roger.riggs at oracle.com Fri Jul 20 21:26:22 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 20 Jul 2018 17:26:22 -0400 Subject: JDK 12 RFR of JDK-8208060: Additional corrections of serial-related declarations In-Reply-To: <5B5251BF.6020005@oracle.com> References: <5B5251BF.6020005@oracle.com> Message-ID: <866ef310-bfa5-5f90-6492-0823a55e73c6@oracle.com> Looks fine Joe, Thanks for the cleanup. Roger On 7/20/18 5:18 PM, Joseph D. Darcy wrote: > Hello, > > Another round of cleaning up the declarations of serial-related fields > and methods: > > ??? JDK-8208060: Additional corrections of serial-related declarations > ??? http://cr.openjdk.java.net/~darcy/8208060.0/ > > Summary of the issues being addressed: > > ??? src/java.base/share/classes/java/net/InetAddress.java > > Extra thrown types for readObjectNoData. > > src/java.management/share/classes/java/lang/management/MemoryType.java > > A serialVesionUID field in an enum type is a no-op. > > src/java.rmi/share/classes/java/rmi/activation/ActivationID.java > ??? src/java.rmi/share/classes/java/rmi/server/RemoteObject.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialArray.java > > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialBlob.java > rc/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialRef.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialStruct.java > > ??? src/java.sql/share/classes/java/sql/BatchUpdateException.java > > A writeObject method doesn't throw ClassNotFoundException. > > Thanks, From lance.andersen at oracle.com Fri Jul 20 21:27:39 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 20 Jul 2018 17:27:39 -0400 Subject: JDK 12 RFR of JDK-8208060: Additional corrections of serial-related declarations In-Reply-To: <5B5251BF.6020005@oracle.com> References: <5B5251BF.6020005@oracle.com> Message-ID: <48A75C45-7D2C-4550-8BA9-CB583ED56123@oracle.com> Hi Joe, Looks fine > On Jul 20, 2018, at 5:18 PM, Joseph D. Darcy wrote: > > Hello, > > Another round of cleaning up the declarations of serial-related fields and methods: > > JDK-8208060: Additional corrections of serial-related declarations > http://cr.openjdk.java.net/~darcy/8208060.0/ > > Summary of the issues being addressed: > > src/java.base/share/classes/java/net/InetAddress.java > > Extra thrown types for readObjectNoData. > > src/java.management/share/classes/java/lang/management/MemoryType.java > > A serialVesionUID field in an enum type is a no-op. > > src/java.rmi/share/classes/java/rmi/activation/ActivationID.java > src/java.rmi/share/classes/java/rmi/server/RemoteObject.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialArray.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialBlob.java > rc/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialRef.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialStruct.java > src/java.sql/share/classes/java/sql/BatchUpdateException.java > > A writeObject method doesn't throw ClassNotFoundException. > > Thanks, Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From mandy.chung at oracle.com Fri Jul 20 22:49:58 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 20 Jul 2018 15:49:58 -0700 Subject: JDK 12 RFR of JDK-8208060: Additional corrections of serial-related declarations In-Reply-To: <5B5251BF.6020005@oracle.com> References: <5B5251BF.6020005@oracle.com> Message-ID: <66ae33c4-ad4f-ec0a-db62-4513e64c0a96@oracle.com> Looks fine. Mandy On 7/20/18 2:18 PM, Joseph D. Darcy wrote: > Hello, > > Another round of cleaning up the declarations of serial-related fields > and methods: > > ??? JDK-8208060: Additional corrections of serial-related declarations > ??? http://cr.openjdk.java.net/~darcy/8208060.0/ > > Summary of the issues being addressed: > > ??? src/java.base/share/classes/java/net/InetAddress.java > > Extra thrown types for readObjectNoData. > > src/java.management/share/classes/java/lang/management/MemoryType.java > > A serialVesionUID field in an enum type is a no-op. > > ??? src/java.rmi/share/classes/java/rmi/activation/ActivationID.java > ??? src/java.rmi/share/classes/java/rmi/server/RemoteObject.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialArray.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialBlob.java > rc/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialRef.java > src/java.sql.rowset/share/classes/javax/sql/rowset/serial/SerialStruct.java > ??? src/java.sql/share/classes/java/sql/BatchUpdateException.java > > A writeObject method doesn't throw ClassNotFoundException. > > Thanks, From fw at deneb.enyo.de Fri Jul 20 22:15:50 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 21 Jul 2018 00:15:50 +0200 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> (Andrew Dinn's message of "Fri, 20 Jul 2018 14:50:22 +0100") References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> Message-ID: <87o9f1lfjd.fsf@mid.deneb.enyo.de> * Andrew Dinn: > Comments and suggestions for improvement would be very welcome. There are a bunch of typos in the JEP (?susbet?, ?Rntime?). Formatting of the headlines looks wrong to me, too. On the implementation side, I find this a bit concerning: + // TODO - remove the following defines + // for testing we need to define the extra MAP_XXX flags used for + // persistent memory. they will eventually get defined by the + // system and included via sys/mman.h Do you really want to change implementation behavior based on which glibc headers (or kernel headers) were used to build the JDK? It's likely more appropriate to define the constants yourself if they are not available. There is some architecture variance for the MAP_* constants, but in case of MAP_SHARED_VALIDATE, even HPPA has the same definition. How does the MappedByteBuffer::force(long, long) method interact with the memory model? From ivan.gerasimov at oracle.com Sat Jul 21 01:42:04 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 20 Jul 2018 18:42:04 -0700 Subject: RFR 8207314 : Unnecessary reallocation when constructing WeakHashMap from a large Map In-Reply-To: References: Message-ID: Hi Martin, Thank you for looking into this! Please see the comments inline. On 7/15/18 9:14 AM, Martin Buchholz wrote: > Hi Ivan, > > Here are some thoughts while looking at this: > > --- > > WeakHashMap promises to have similar "capacity" handling to HashMap, > but implementations (and doc?) seem more different than necessary > (diverged over time?), and there don't seem to be any tests. > > HashMap seems to deal with this problem by doing computations using > float not int. Choose the best one and use it in both source files. > > float ft = ((float)s / loadFactor) + 1.0F; > int t = ((ft < (float)MAXIMUM_CAPACITY) ? > (int)ft : MAXIMUM_CAPACITY); > I compared the bytecode generated for 1) (int) ((float)s / loadFactor) + 1.0F 2) (int) (s / loadFactor + 1); and the results are actually exactly the same: 0: iload_0 1: i2f 2: ldc #2 // float 0.75f 4: fdiv 5: fconst_1 6: fadd 7: f2i So, I just modified the webrev to make it look more like what we currently have in HashMap. > > --- > > Consider adding a WhiteBox test. An existing one for > ConcurrentHashMap could be modified to test internal table maintenance. > PriorityBlockingQueue/WhiteBox.java is an example of a test that > ensures two implementations stay in sync. > I cannot see how to verify this fix easily, and I didn't really think that such a trivial fix is worth a dedicated test. If you disagree, I can try to write a test that will pass a custom map as an argument to WeakHashMap constructor. From the custom map's size() we can retrieve a reference to the WeakHashMap being constructed and make sure that its table is either null or doesn't have the default size. Sounds fragile though, so I would prefer to just set noreg-hard label :) > --- > > i see > if (loadFactor <= 0 || Float.isNaN(loadFactor)) > but that nan check doesn't have much value. It can be removed using > if (! (loadFactor > 0)) > This pattern is used in several places (HashMap, HashSet, Hashtable, WeakHashMap). If simplifying it in the way you suggest gives a performance gain, I wonder if JVM can be taught to optimize checks for NaN via combining them with comparisons nearby? > --- > > HashMap's resize() doesn't take an arg, while WeakHashMap's does. Why? > As a result, I see in HashMap.putMapEntries > > else if (s > threshold) > resize(); > > which suggests that if you create a fresh HashMap, then > putAll(hugeMap) it will repeatedly resize instead of resizing to the > target capacity in one step. Which seems like a HashMap bug. > I agree that adding a hint to HashMap.resize() may help to allocate the new table more optimally. I think is should be a separate enhancement though. Here's the updated webrev with only stylistic change, comparing to the previous one: http://cr.openjdk.java.net/~igerasim/8207314/01/webrev/ With kind regards, Ivan > > On Fri, Jul 13, 2018 at 10:22 PM, Ivan Gerasimov > > wrote: > > Hello! > > When a WeakHashMap is constructed from another Map m, then the > initial capacity is calculated as > > (int) (m.size() / DEFAULT_LOAD_FACTOR) + 1. > > For large values of m.size() this becomes negative due to integer > overflow. > > The result is that the WeakHashMap is initially constructed with > the default initial capacity 16, and then is immediately resized. > > Would you please help review a trivial patch? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207314 > > WEBREV: http://cr.openjdk.java.net/~igerasim/8207314/00/webrev/ > > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > -- With kind regards, Ivan Gerasimov From chris.hegarty at oracle.com Mon Jul 23 10:09:16 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 23 Jul 2018 11:09:16 +0100 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <3c5c5465-492b-b740-77fc-f5a5c042d283@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> <3c5c5465-492b-b740-77fc-f5a5c042d283@oracle.com> Message-ID: Sean, > On 20 Jul 2018, at 18:07, Sean Mullan wrote: > > On 7/20/18 11:08 AM, Chris Hegarty wrote: >> This is ambiguous, and needs to be clarified. Surely, it is >> better to use the same wording as the serial filter: >> "Whitespace is significant and is considered part of the value." > > Kind of on the fence on that one. If this were a general property/value format, I would agree, but these values are fixed and not potentially complicated expressions. Sure, they are simple strings consisting of only printable characters. > For example, does this mean: > > jdk.includeInExceptions=hostInfo,jarInfo > > and > > jdk.includeInExceptions=hostInfo, jarInfo > > are different? And I assume the latter " jarInfo" would be ignored? > > If you are strongly in favor of this, I would highly recommend logging a warning when there is an unknown value, otherwise typos and such would be hard to detect (although this doesn't necessarily need to be in the specification). My original whitespace handling avoided the potential issue where a rogue lead or trailing whitespace accidentally found its way into a value, therefore avoiding the scenario above ( and potentially emitting a warning ). Whitespace is significant and should be specified as such, since ... jdk.includeInExceptions=?host Info? IS NOT equal to jdk.includeInExceptions=?hostInfo? and system property values can contain spaces jdk.includeInExceptions=?foo bar,hostInfo,jar Info,? After given this some more thought, I now think that I gave in to the comment to change whitespace handing too easy. While maybe not consistent, with the already inconsistent, whitespace handling in java.security, I think ( for this particular case ) the original - trim leading and trailing - is the right thing to do. It avoids your above scenario where someone accidentally adds a leading space, which could be difficult to debug/find without a warning - which we should avoid if possible. I?d like to re-propose the original webrev for consideration ( whitespace handling is the only change ): http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ -Chris. From Joerg.Buchberger at pruftechnik.com Mon Jul 23 11:47:56 2018 From: Joerg.Buchberger at pruftechnik.com (Buchberger, Joerg) Date: Mon, 23 Jul 2018 11:47:56 +0000 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: <7c2de54d-793d-77a9-556e-34f5871bdcc8@oracle.com> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <75106ab90b974292890c71b6d320b56c@DEISMMXS02.pt.local> <7c2de54d-793d-77a9-556e-34f5871bdcc8@oracle.com> Message-ID: <6f33f11d7423409c905a94dca38ecd1a@DEISMMXS02.pt.local> Thanks for the insight. @Alexey: Then, how much work do you see in reactivating service bundler? Cheers J?rg -----Original Message----- From: Kevin Rushforth [mailto:kevin.rushforth at oracle.com] Sent: Donnerstag, 12. Juli 2018 01:09 To: Buchberger, Joerg ; core-libs-dev at openjdk.java.net Cc: Alexey Semenyuk ; Andy Herrick Subject: Re: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] We will likely be able to deliver the .exe installer (with its dependency on Inno Setup). As for the service bundler, this will be a "nice to have" (a stretch goal) for this version, but isn't on the list of committed features. Alexsei might be able to comment further on how much work it would be to provide it, including documenting and testing it. This might give you, and other interested developers, a sense of how likely this is to make it for this version. -- Kevin On 7/10/2018 4:35 AM, Buchberger, Joerg wrote: > Hi > > thanks for the update/info. I had a quick look at the changes. So, here some thoughts... > > As described in JDK-8200758, and therefore expected, WinExeBundler has been removed in favor of putting focus on WinMsiBundler. > (Although, I regret that decision - since my personal experience has > been that InnoSetup based WinExeBundler has worked much better than > wix based WinMsiBundler for our use cases - I can live with that.) > > What is much more disturbing: WinServiceBundler has also been actively removed, although that was working fine together with both wix/msi and exe/iss. Why has service wrapping been removed as well, while the command line option for it is kept in place? > > Is there any chance of service bundler coming back into scope of JDK-8200758 or coming back in at all? > > Cheers > J?rg > > > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Kevin Rushforth > Sent: Freitag, 6. Juli 2018 22:14 > To: core-libs-dev at openjdk.java.net > Cc: Alexey Semenyuk ; Andy Herrick > > Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP > proposal: JDK-8200758: Packaging Tool] > > An initial prototype of the jpackager tool has been pushed to a new 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is interested in taking a look, you can clone it as follows: > > ??? hg clone https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=jWWENz_KIkmyh-9-kQQvoZ0BwBymwQ-BKx8hG3F5Iy0&e= > ??? cd ./sandbox > ??? hg update JDK-8200758-branch > > I plan to reply to the feedback already provided, and update the JEP [2] next week some time, but in the mean time if you have additional questions or comments, feel free to reply. > > -- Kevin > > [1] > https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.ne > t_jdk_sandbox_shortlog_JDK-2D8200758-2Dbranch&d=DwIFaQ&c=uD3W7j5M6i1jD > eSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJK > hVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=F-CqPAWlz-Cfb0k > ae2FBEj4Ncd3ZBVu7BeOVY1AM-cA&e= [2] > https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java > .net_browse_JDK-2D8200758&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxR > W_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHq > hAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=DFIAHtCR1o--KMLuBzurIzx5MDu67NgtUrE > dQ22wI9I&e= > > > On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >> We're aiming to get this into JDK 12 early enough so that an EA build >> would be available around the time JDK 11 ships. That will allow you >> to take a jlinked image with JDK 11 and package it up using (the EA) >> jpackager. >> >> We will create a development branch in the JDK sandbox [1] some time >> in the next week or so so you can follow the development. >> >> Also, thank you to those who have provided feedback. I'll reply to >> feedback soon and then incorporate it into an updated JEP. >> >> -- Kevin From srinivas.dama at oracle.com Mon Jul 23 14:16:09 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 23 Jul 2018 07:16:09 -0700 (PDT) Subject: [11] RFR: 8206445: JImageListTest.java failed in Windows Message-ID: <5cfe3a6e-be92-4547-beb3-1e08a6e10d83@default> Hi Alan, As discussed in private conversations, I am pushing below fix for the issue. webrev: http://cr.openjdk.java.net/~sdama/8206445/webrev.04/ Bug: https://bugs.openjdk.java.net/browse/JDK-8206445 Ran all mach5 tests successfully. Regards, Srinivas ----- Original Message ----- From: Alan.Bateman at oracle.com To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net Sent: Monday, 9 July, 2018 4:23:48 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [11] RFR: 8206445: JImageListTest.java failed in Windows On 09/07/2018 11:38, Srinivas Dama wrote: > Hi, > > Please review > webrev: http://cr.openjdk.java.net/~sdama/8206445/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8206445 > testListEmptyFile looks okay but you can reduce the code if if you use Path.of("."). testListNotAnImage can use Path.of(".") too. Also it can use Files.writeString which will avoid the test trying to list the contents of the image while the file is open. Will you make sure to test this on all platforms before pushing? These tests have been on and off the exclude list several times in the last week and it would be good to check that this one is stable before it runs again. -Alan From adinn at redhat.com Mon Jul 23 11:01:17 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 23 Jul 2018 12:01:17 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <87o9f1lfjd.fsf@mid.deneb.enyo.de> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <87o9f1lfjd.fsf@mid.deneb.enyo.de> Message-ID: Hi Florian, Thank you for the feedback. Comments inline below (extensive comments in the case of the memory model question - it is complex). regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander On 20/07/18 23:15, Florian Weimer wrote: > There are a bunch of typos in the JEP (?susbet?, ?Rntime?). > Formatting of the headlines looks wrong to me, too. Thanks. Yes, the spelling and, more importantly, format need a lot of work. The JIRA format does not exactly look compelling and the format in the JEP listing is even worse. I am currently working to remedy this. > On the implementation side, I find this a bit concerning: > > + // TODO - remove the following defines > + // for testing we need to define the extra MAP_XXX flags used for > + // persistent memory. they will eventually get defined by the > + // system and included via sys/mman.h > > Do you really want to change implementation behavior based on which > glibc headers (or kernel headers) were used to build the JDK? It's > likely more appropriate to define the constants yourself if they are > not available. There is some architecture variance for the MAP_* > constants, but in case of MAP_SHARED_VALIDATE, even HPPA has the same > definition. No, I guess I don't really want to change implementation behaviour based on which glibc headers (or kernel headers) were used to build the JDK. So, I think I need to modify this so the constants are always defined by FileChannelImpl.c I have followed the Intel libpmem code in defining the values for these constants. Can you provide details of the 'architecture variance' you refer to above? I guess it doesn't matter too much while the code is being compiled only for Linux on x86_64/AArch64. > How does the MappedByteBuffer::force(long, long) method interact with > the memory model? That's a very good question, obviously with different answers for AArch64 and x86_64. I'll note before citing the relevant documentation that in the current design I have explicitly allowed for independent pre and post sync operations to implement whatever memory sync is required wrt to non-writeback memory operations preceding or following the writebacks. This is to make it easy to revise the current barrier generation choices if needed (and this flexibility comes at no cost once the relevant sync methods are intrinsified by the JIT). I have currently implemented both pre and post sync by an mfence on x86_64 whatever the operation used to flush memory, which is playing safe and this choice merits reviewing. This is a stronger memory sync than that employed by the libpmem code for x86_64 which also relies on mfence (rather than sfence) but i) omits the mfence for the post-sync and ii) when using CLFLUSH to implement the writeback also omits the pre-sync mfence. It is not clear to me from the docs that a post-sync can safely be omitted for CLFLUSHOPT or CLWB but I do read it as confirming that pre- and post-sync can be safely omitted when using CLFLUSH. Also, the docs suggest only an sfence is needed rather than an mfence -- which sounds right. However, I have followed libpmem and employed mfence for now (more below). On AArch64 I have currently implemented both pre and post sync by a StoreStore barrier -- a dmb(ISHST). This is weaker than the libpmem pre-sync for AArch64 which emits a dmb(ISH) but also stronger as regards the post-sync in that libpmem does not emit a post-sync fence. On my reading of the ARM Architecture Reference Manual (ARM ARM) I believe dmb(ISHST) pre and post is what is needed and all that is needed (again see below for more). It's worth stating up front that I added the post-sync for both architectures to ensure that the thread doing the writeback cannot update any other NVM before the writeback has completed. This may perhaps be overkill. However, I envisaged the following possibility for error to arise. Imagine that a writeback to 'data' state e.g. a log record might be followed by a write to a NVM memory location that marks the log record as 'live' (it might, say, be linked into the tail of a list of valid log records). The danger here is that the memory system might, independent of any writeback scheduled by the application, flush the cache line for the 'live' write before the prior 'data' writeback was complete. A crash at this point might leave the log record live but, potentially, incomplete/inconsistent. Ok, now here is what the relevant documentation says with my gloss on what it means for the current draft implementation. x86_64: The latest Intel documentation available at https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf describes all 3 available instructions for implementing the cache line writeback: CLFLUSH, CLFLUSHOPT and CLWB. Vol. 2A 3-139 CLFLUSH "The memory attribute of the page containing the affected line has no effect on the behavior of this instruction. It should be noted that processors are free to speculatively fetch and cache data from system memory regions assigned a memory-type allowing for speculative reads (such as, the WB, WC, and WT memory types). PREFETCHh instructions can be used to provide the processor with hints for this speculative behavior. Because this speculative fetching can occur at any time and is not tied to instruction execution, the CLFLUSH instruction is not ordered with respect to PREFETCHh instructions or any of the speculative fetching mechanisms (that is, data can be speculatively loaded into a cache line just before, during, or after the execution of a CLFLUSH instruction that references the cache line). "Executions of the CLFLUSH instruction are ordered with respect to each other and with respect to writes, locked read-modify-write instructions, fence instructions, and executions of CLFLUSHOPT to the same cache line. They are not ordered with respect to executions of CLFLUSHOPT to different cache lines." n.b. the prefetch clauses above (and also repeated in the next two citations) are not important since all a prefetch might do is bring the same flushed data back into cache without affecting validity of completion of the writeback. Ordering of loads can also be ignored for similar reasons. So, I believe the above text indicates that there is no need for a pre-sync or post-sync memory barrier when using CLFLUSH. All writes executed before the CLFLUSH will be completed before the flush is performed, all CLFLUSH operations complete in order of execution before each subsequent CLFLUSH and the last CLFLUSH completes before any subsequently executed writes can complete and become visible. Vol. 2A 3-141 CLFLUSHOPT [note the references to CLFLUSH below /appear/ exactly as occurs in the manual -- I believe the two occurences in the first paragraph are meant to read CLFLUSHOPT] "The memory attribute of the page containing the affected line has no effect on the behavior of this instruction. It should be noted that processors are free to speculatively fetch and cache data from system memory regions assigned a memory-type allowing for speculative reads (such as, the WB, WC, and WT memory types). PREFETCHh instructions can be used to provide the processor with hints for this speculative behavior. Because this speculative fetching can occur at any time and is not tied to instruction execution, the CLFLUSH [sic] instruction is not ordered with respect to PREFETCHh instructions or any of the speculative fetching mechanisms (that is, data can be speculatively loaded into a cache line just before, during, or after the execution of a CLFLUSH [sic] instruction that references the cache line). "Executions of the CLFLUSHOPT instruction are ordered with respect to fence instructions and to locked read- modify-write instructions; they are also ordered with respect to the following accesses to the cache line being invalidated: writes, executions of CLFLUSH, and executions of CLFLUSHOPT. They are not ordered with respect to writes, executions of CLFLUSH, or executions of CLFLUSHOPT that access other cache lines; to enforce ordering with such an operation, software can insert an SFENCE instruction between CFLUSHOPT and that operation." I believe this indicates that an SFENCE (or a stronger MFENCE) is needed to ensure all writes executed before the first CLFLUSHOPT are visible. Successive CLFLUSHOPT instructions can then complete out of order and may not have completed before any subsequently executed write becomes visible. I think this risks the situation I described above where a controlling update may be flushed before a flush of the data it controls. Hence I think we require a post-sync MFENCE (or SFENCE). Vol. 2A 3-146 CLWB "The memory attribute of the page containing the affected line has no effect on the behavior of this instruction. It should be noted that processors are free to speculatively fetch and cache data from system memory regions that are assigned a memory-type allowing for speculative reads (such as, the WB, WC, and WT memory types). PREFETCHh instructions can be used to provide the processor with hints for this speculative behavior. Because this speculative fetching can occur at any time and is not tied to instruction execution, the CLWB instruction is not ordered with respect to PREFETCHh instructions or any of the speculative fetching mechanisms (that is, data can be speculatively loaded into a cache line just before, during, or after the execution of a CLWB instruction that references the cache line). "CLWB instruction is ordered only by store-fencing operations. For example, software can use an SFENCE, MFENCE, XCHG, or LOCK-prefixed instructions to ensure that previous stores are included in the write-back. CLWB instruction need not be ordered by another CLWB or CLFLUSHOPT instruction. CLWB is implicitly ordered with older stores executed by the logical processor to the same address." I think this is the same story as for CLFLUSHOPT i.e. an SFENCE is needed before any CLWB instructions are executed to ensure that pending writes are visible but then successive CLWB instructions can complete out of order and may not have completed before any subsequently executed write become visible. So, I also think this may need an MFENCE (or SFENCE) after the CLWB instructions. AArch64: The latest ARM ARM I have is ARM ARM C available at https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile I am following libpmem in using instruction dc(CVAC) to writeback the data cache lines (clear virtual address to point of coherency). I think it would be preferable if dc(CVAP) were used (clear virtual address to point of persistence). However 1) support for dc(CVAP) does not seem to be a mandatory part of the architecture 2) user-space access to CVAP instruction is not guaranteed and not currently configured by default on Linux 3) CVAC is expected normally to guarantee writeback to physical memory Ordering of cache management instructions is specified in section D3.4.8 under heading "Ordering and completion of data and instruction cache instructions" on page D3-2069 as follows: "All data cache instructions, other than DC ZVA, that specify an address: - Execute in program order relative to loads or stores that access an address in Normal memory with either Inner Write Through or Inner Write Back attributes within the same cache line of minimum size, as indicated by CTR_EL0.DMinLine. - Can execute in any order relative to loads or stores that access any address with the Device memory attribute, or with Normal memory with Inner Non-cacheable attribute unless a DMB or DSB is executed between the instructions. - Execute in program order relative to other data cache maintenance instructions, other than DC ZVA, that specify an address within the same cache line of minimum size, as indicated by CTR_EL0.DMinLine. - Can execute in any order relative to loads or stores that access an address in a different cache line of minimum size, as indicated by CTR_EL0.DMinLine, unless a DMB or DSB is executed between the instructions. - Can execute in any order relative to other data cache maintenance instructions, other than DC ZVA , that specify an address in a different cache line of minimum size, as indicated by CTR_EL0.DMinLine, unless a DMB or DSB is executed between the instructions. - Can execute in any order relative to data cache maintenance instructions that do not specify an address unless a DMB or DSB is executed between the instructions. - Can execute in any order relative to instruction cache maintenance instructions unless a DSB is executed between the instructions." . . . "All data cache maintenance instructions that do not specify an address: - Can execute in any order relative to data cache maintenance instructions that do not specify an address unless a DMB or DSB is executed between the instructions. - Can execute in any order relative to data cache maintenance instructions that specify an address, other than Data Cache Zero, unless a DMB or DSB is executed between the instructions. - Can execute in any order relative to loads or stores unless a DMB or DSB is executed between the instructions. - Can execute in any order relative to instruction cache maintenance instructions unless a DSB is executed between the instructions. All instruction cache instructions can execute in any order relative to other instruction cache instructions, data cache instructions, loads, and stores unless a DSB is executed between the instructions. A cache maintenance instruction can complete at any time after it is executed, but is only guaranteed to be complete, and its effects visible to other observers, following a DSB instruction executed by the PE that executed the cache maintenance instruction. See also the requirements for cache maintenance instructions in Completion and endpoint ordering on page B2-97. In all cases, where the text in this section refers to a DMB or a DSB , this means a DMB or DSB whose required access type is both loads and stores." The link to B2-97 provides some further clarification of what it means for the above operations to be 'complete' (I have omitted the clauses regarding completion of translation table walks and TLB invalidates). "For all memory, the completion rules are defined as: - A read R 1 to a Location is complete for a shareability domain when all of the following are true: - Any write to the same Location by an Observer within the shareability domain will be Coherence-after R 1 . - Any translation table walks associated with R 1 are complete for that shareability domain. - A write W 1 to a Location is complete for a shareability domain when all of the following are true: - Any write to the same Location by an Observer within the shareability domain will be Coherence-after W 1 . - Any read to the same Location by an Observer within the shareability domain will either Reads-from W 1 or Reads-from a write that is Coherence-after W 1 . - Any translation table walks associated with the write are complete for that shareability domain. . . . - A cache maintenance instruction is complete for a shareability domain when the memory effects of the instruction are complete for that shareability domain, and any translation table walks that arise from the instruction are complete for that shareability domain. . . . The completion of any cache or TLB maintenance instruction includes its completion on all PEs that are affected by both the instruction and the DSB operation that is required to guarantee visibility of the maintenance instruction." So, my reading of this is i) a DMB is needed to ensure that writes executed before the dc(CVAC) are completed before the writeback starts ii) a DMB(ISHST) is all that is needed to achieve this pre-ordering iii) a DMB is needed to ensure that pending writebacks are completed before any subsequently executed writes can become visible. iv) a DMB(ISHST) is all that is needed to achieve this post-ordering From vyom.tewari at oracle.com Mon Jul 23 15:02:18 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Mon, 23 Jul 2018 20:32:18 +0530 Subject: [11] RFR: 8206445: JImageListTest.java failed in Windows In-Reply-To: <5cfe3a6e-be92-4547-beb3-1e08a6e10d83@default> References: <5cfe3a6e-be92-4547-beb3-1e08a6e10d83@default> Message-ID: Hi Srinivas, If runTests() throws exception then? "System.gc()" will not? be invoked. Thanks, Vyom On Monday 23 July 2018 07:46 PM, Srinivas Dama wrote: > Hi Alan, > > As discussed in private conversations, I am pushing below fix for the issue. > webrev: http://cr.openjdk.java.net/~sdama/8206445/webrev.04/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8206445 > > Ran all mach5 tests successfully. > > Regards, > Srinivas > > ----- Original Message ----- > From: Alan.Bateman at oracle.com > To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net > Sent: Monday, 9 July, 2018 4:23:48 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: Re: [11] RFR: 8206445: JImageListTest.java failed in Windows > > On 09/07/2018 11:38, Srinivas Dama wrote: >> Hi, >> >> Please review >> webrev: http://cr.openjdk.java.net/~sdama/8206445/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8206445 >> > testListEmptyFile looks okay but you can reduce the code if if you use > Path.of("."). testListNotAnImage can use Path.of(".") too. Also it can > use Files.writeString which will avoid the test trying to list the > contents of the image while the file is open. > > Will you make sure to test this on all platforms before pushing? These > tests have been on and off the exclude list several times in the last > week and it would be good to check that this one is stable before it > runs again. > > -Alan From Alan.Bateman at oracle.com Mon Jul 23 15:10:44 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 23 Jul 2018 16:10:44 +0100 Subject: [11] RFR: 8206445: JImageListTest.java failed in Windows In-Reply-To: References: <5cfe3a6e-be92-4547-beb3-1e08a6e10d83@default> Message-ID: <840244b3-479f-3ee7-9a12-42b4aaddfce5@oracle.com> On 23/07/2018 16:02, vyom tewari wrote: > Hi Srinivas, > > If runTests() throws exception then? "System.gc()" will not? be invoked. which probably doesn't matter because the test will fail anyway (this issue is about jtreg doing cleanup when the test passes). -Alan From sean.mullan at oracle.com Mon Jul 23 15:58:41 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 23 Jul 2018 11:58:41 -0400 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> <3c5c5465-492b-b740-77fc-f5a5c042d283@oracle.com> Message-ID: <60a2405d-baae-9881-5872-cec0bca615d9@oracle.com> On 7/23/18 6:09 AM, Chris Hegarty wrote: > After given this some more thought, I now think that I gave in to the > comment to change whitespace handing too easy. While maybe not > consistent, with the already inconsistent, whitespace handling in > java.security, I think ( for this particular case ) the original - trim > leading and trailing - is the right thing to do. It avoids your above > scenario where someone accidentally adds a leading space, which could be > difficult to debug/find without a warning - which we should avoid if > possible. Thanks for making that change. > I?d like to re-propose the original webrev for consideration ( whitespace > handling is the only change ): > > http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ A few nits and wording suggestions in the java.security file: "By default, several exception messages do not include potentially sensitive information such as file names, host names, or port numbers." I think the following sounds a bit better: "By default, exception messages should not include potentially sensitive information such as file names, host names, or port numbers." Also, the 2nd and 3rd sentences basically say the same thing. I would remove the 2nd sentence. "The categories, to enable enhanced exception message information, are:" I would remove ", to enable enhanced exception message information," since it seems redundant (and I believe is grammatically incorrect). hostInfo - IOExceptions thrown by java.net.Socket and also the ... Remove "also" (not really necessary). --Sean From chris.hegarty at oracle.com Mon Jul 23 16:10:33 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 23 Jul 2018 17:10:33 +0100 Subject: RFR [11] 8207846: Generalize the jdk.net.includeInExceptions security property In-Reply-To: <60a2405d-baae-9881-5872-cec0bca615d9@oracle.com> References: <803E606C-CC41-47C4-9BD9-769208AAB5A9@oracle.com> <1caeb924-26d2-14f6-9b0d-6f1cfc81a9ea@oracle.com> <0A75D4BE-B31B-4AD4-A526-25F32162062A@oracle.com> <3c5c5465-492b-b740-77fc-f5a5c042d283@oracle.com> <60a2405d-baae-9881-5872-cec0bca615d9@oracle.com> Message-ID: Thanks for the review Sean, > On 23 Jul 2018, at 16:58, Sean Mullan wrote: > ... >> http://cr.openjdk.java.net/~chegar/8207846/webrev.00/ > > A few nits and wording suggestions in the java.security file: > > "By default, several exception messages do not include potentially sensitive information such as file names, host names, or port numbers." > > I think the following sounds a bit better: > > "By default, exception messages should not include potentially sensitive > information such as file names, host names, or port numbers." > > Also, the 2nd and 3rd sentences basically say the same thing. I would remove the 2nd sentence. > > "The categories, to enable enhanced exception message information, are:" > > I would remove ", to enable enhanced exception message information," since it seems redundant (and I believe is grammatically incorrect). > > hostInfo - IOExceptions thrown by java.net.Socket and also the ... > > Remove "also" (not really necessary). Agreed. Here?s where this ended up. # # Enhanced exception message information # # By default, exception messages should not include potentially sensitive # information such as file names, host names, or port numbers. This property # accepts one or more comma separated values, each of which represents a # category of enhanced exception message information to enable. Values are # case-insensitive. Leading and trailing whitespaces, surrounding each value, # are ignored. Unknown values are ignored. # # The categories are: # # hostInfo - IOExceptions thrown by java.net.Socket and the socket types in the # java.nio.channels package will contain enhanced exception # message information # # The property setting in this file can be overridden by a system property of # the same name, with the same syntax and possible values. # #jdk.includeInExceptions=hostInfo -Chris From roger.riggs at oracle.com Mon Jul 23 17:05:17 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 23 Jul 2018 13:05:17 -0400 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> Message-ID: Hi Chris, Sorry to be late to the review cycle. The use of initializer blocks instead of constructors reduces the readability of the code. A simple fix is to add the constructor declarations ahead of the initializer to make the code more readable. GetAttrsBase: 69: The signature of setMandatoryAttrs could be setMandatoryAttrs(String...) would allow the mandatory strings to be passed without explicit array creation and would still allow String[] to be passed where more convenient. GetAttrsBase: 53: The exception message would be more readable as: "Attributes to be verified is null" GetAttrsNotFound: 50: throw the fail exception here; hiding it in the verifyAttributes method is harder to understand GetNonstandard: 87, 93: Printing the expected and actual arrays on failure (regardless of debug) will help diagnose failures more quickly GetNumericIRRs/GetNumericRRs: Tests like this are so abstract that is it difficult to see what is being tested. For example, where does getIndex() get its value? Either more comments are needed or more expressive method names. DNSTestUtils: 94:? Don't encourage the usage allowing separation of -D from the argument. Its not allowed by the normal command line parsing and should not be supported. (Yes, I know it is/was not changed by this review). :153:? cleanupClosableRes does not appear to be used;? if it is not needed, remove it A lot of the test framework is embodied in these DNSTestUtils without explanation or comment. It will help future maintenance, if there is a class level description (prose) describing the sequence of events for tests. Regards, Roger On 7/13/18 2:14 AM, Chris Yin wrote: > Hi, Vyom > > Thank you for the review and comments, update webrev as below and comment inline > > webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.02/ > > >> On 13 Jul 2018, at 1:46 PM, vyom tewari wrote: >> >> Hi Chris, >> >> Thanks for doing this overall looks good to me, few minor comments. >> 1-> DNSTestUtils.java, please start the server first and then set the "TEST_DNS_SERVER_THREAD". This will not make much difference but we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if server fails to start. >> 129 env.put(TEST_DNS_SERVER_THREAD, inst); >> 130 inst.start(); > Fixed, thanks > >> 2-> I noticed that copyright date (Copyright (c) 2000, 2018,) but webrev tells all the tests are new, please fix copyright date as well. > Thanks for checking this. Since this task is part of umbrella enhancement JDK-8191011 JNDI SQE tests co-location, for those added tests which are migrated from SQE tests, the copyright date will follow the guidance SQE Test copyright year + migration copyright year if the 2 year are not same, for dump files (like *.dns) are new added under our new framework so just use current copyright year, hope that explains :), thanks > > Regards, > Chris > >> Thanks, >> Vyom >> >> On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: >>> Please have a review to new webrev as below, some code refactoring on lib parts and enhanced DNSServer to handle retry request which will make the tests more stable, thanks >>> >>> http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ >>> >>> Regards, >>> Chris >>> >>>> On 22 Mar 2018, at 11:16 AM, Chris Yin > wrote: >>>> >>>> Please review the changes to add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/, thanks >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 >>>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ >>>> >>>> Regards, >>>> Chris From jonathan.gibbons at oracle.com Mon Jul 23 17:53:32 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 23 Jul 2018 10:53:32 -0700 Subject: API for parsing path strings In-Reply-To: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> References: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> Message-ID: Roger, Having written internal library code for both javac and jtreg to do what you suggest, I would support such an API. I note that as well as parsing system properties, it would also be a useful API for command-line tools that accept paths as options. However, I would suggest that it is more appropriate to return a List than a List.? It is better to bias the API towards an appropriate type (i.e. Path) than to leave it to the user to create a stream to do the conversion.?? Those folk that still want a File can still create the stream to do the conversion if they so desire. By converting to Paths as part of the API, it becomes a clearer point to specify and document the possibility of InvalidPathException. -- Jon On 7/19/18 12:39 PM, Roger Riggs wrote: > Hi, > > Parsing of paths such as the values of properties java.class.path and > java.library.path > do not have? direct support in the Java API. > There are several implementations within OpenJDK and a few variations > exist with regard to > treatment of empty path elements and on Windows support for quoting. > > Would it be useful to have an API such as the following: > > ??? /** > ???? * Returns a list of paths parsed from a string separated by > {@link File#pathSeparator}. > ???? * All characters except the {@link File#pathSeparator} are allowed. > ???? * Some operating systems support quoting segments of the string > ???? * potentially including {@code pathSeparator} characters. > ???? * Empty paths, identified by leading, trailing, and adjacent > separator characters, > ???? * can be omitted or replaced with a non-null provided path. > ???? * > ???? * @implNote > ???? * On Windows, zero or more characters between double-quotes > ({@code "}) are included > ???? * literally in the path, regardless of whether they are valid in > a path. > ???? * > ???? * @param path a string containing paths separated by the {@link > File#pathSeparator} > ???? * @param emptyPath a path to replace an empty path or {@code > null} to ignore empty paths > ???? * @return an immutable list of strings for each path; {@code > non-null} > ???? */ > ??? public static List parsePathStrings(String path, String > emptyPath) { ...} > > There are of course, possible variations and convenience versions but > the API should be kept to the essentials. > With java.util.streams it becomes easy to process each component or to > map the strings to Files or Path. > > Suggestions welcome, Roger > From roger.riggs at oracle.com Mon Jul 23 18:04:18 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 23 Jul 2018 14:04:18 -0400 Subject: API for parsing path strings In-Reply-To: References: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> Message-ID: <917fa123-1dee-cf51-deaf-df75298e2474@oracle.com> Hi Jon, I agree that supporting Paths leads people in the right direction. Two considerations led me to propose the simpler String version first. 1.? Early parsing of paths can occur before the file systems that support Path have been initialized. 2.? There is an additional exception that can occur when creating a Path (InvalidPathException) and the caller may want to handle it individually. Adding another method that returns List is a straightforward addition. Keeping the string version may make adoption easier. Thanks, Roger On 7/23/18 1:53 PM, Jonathan Gibbons wrote: > Roger, > > Having written internal library code for both javac and jtreg to do > what you suggest, I would support such an API. > > I note that as well as parsing system properties, it would also be a > useful API for command-line tools that accept paths as options. > > However, I would suggest that it is more appropriate to return a > List than a List.? It is better to bias the API towards > an appropriate type (i.e. Path) than to leave it to the user to create > a stream to do the conversion.?? Those folk that still want a File can > still create the stream to do the conversion if they so desire. > > By converting to Paths as part of the API, it becomes a clearer point > to specify and document the possibility of InvalidPathException. > > -- Jon > > > On 7/19/18 12:39 PM, Roger Riggs wrote: >> Hi, >> >> Parsing of paths such as the values of properties java.class.path and >> java.library.path >> do not have? direct support in the Java API. >> There are several implementations within OpenJDK and a few variations >> exist with regard to >> treatment of empty path elements and on Windows support for quoting. >> >> Would it be useful to have an API such as the following: >> >> ??? /** >> ???? * Returns a list of paths parsed from a string separated by >> {@link File#pathSeparator}. >> ???? * All characters except the {@link File#pathSeparator} are allowed. >> ???? * Some operating systems support quoting segments of the string >> ???? * potentially including {@code pathSeparator} characters. >> ???? * Empty paths, identified by leading, trailing, and adjacent >> separator characters, >> ???? * can be omitted or replaced with a non-null provided path. >> ???? * >> ???? * @implNote >> ???? * On Windows, zero or more characters between double-quotes >> ({@code "}) are included >> ???? * literally in the path, regardless of whether they are valid in >> a path. >> ???? * >> ???? * @param path a string containing paths separated by the {@link >> File#pathSeparator} >> ???? * @param emptyPath a path to replace an empty path or {@code >> null} to ignore empty paths >> ???? * @return an immutable list of strings for each path; {@code >> non-null} >> ???? */ >> ??? public static List parsePathStrings(String path, String >> emptyPath) { ...} >> >> There are of course, possible variations and convenience versions but >> the API should be kept to the essentials. >> With java.util.streams it becomes easy to process each component or >> to map the strings to Files or Path. >> >> Suggestions welcome, Roger >> > From jonathan.gibbons at oracle.com Mon Jul 23 18:17:22 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 23 Jul 2018 11:17:22 -0700 Subject: API for parsing path strings In-Reply-To: <917fa123-1dee-cf51-deaf-df75298e2474@oracle.com> References: <46734250-094f-b4b3-d0c0-931d32e96833@Oracle.com> <917fa123-1dee-cf51-deaf-df75298e2474@oracle.com> Message-ID: Hi Roger, The point about early parsing is a weak one, because it would mostly just seem to apply to JDK itself. That being said, a variant of this argument is that it allows folk to choose how to map strings to Paths, e.g. using a custom (non-default) file system, and that argument is more compelling. The point about handling the individual possible InvalidPathException is a good one. Having an extra method that returns List and which specifies that the character sequences in between the path separator characters must be valid paths would also be good. -- Jon On 7/23/18 11:04 AM, Roger Riggs wrote: > Hi Jon, > > I agree that supporting Paths leads people in the right direction. > Two considerations led me to propose the simpler String version first. > 1.? Early parsing of paths can occur before the file systems that > support Path have been initialized. > 2.? There is an additional exception that can occur when creating a > Path (InvalidPathException) > and the caller may want to handle it individually. > > Adding another method that returns List is a straightforward > addition. > Keeping the string version may make adoption easier. > > Thanks, Roger > > On 7/23/18 1:53 PM, Jonathan Gibbons wrote: >> Roger, >> >> Having written internal library code for both javac and jtreg to do >> what you suggest, I would support such an API. >> >> I note that as well as parsing system properties, it would also be a >> useful API for command-line tools that accept paths as options. >> >> However, I would suggest that it is more appropriate to return a >> List than a List.? It is better to bias the API towards >> an appropriate type (i.e. Path) than to leave it to the user to >> create a stream to do the conversion. Those folk that still want a >> File can still create the stream to do the conversion if they so desire. >> >> By converting to Paths as part of the API, it becomes a clearer point >> to specify and document the possibility of InvalidPathException. >> >> -- Jon >> >> >> On 7/19/18 12:39 PM, Roger Riggs wrote: >>> Hi, >>> >>> Parsing of paths such as the values of properties java.class.path >>> and java.library.path >>> do not have? direct support in the Java API. >>> There are several implementations within OpenJDK and a few >>> variations exist with regard to >>> treatment of empty path elements and on Windows support for quoting. >>> >>> Would it be useful to have an API such as the following: >>> >>> ??? /** >>> ???? * Returns a list of paths parsed from a string separated by >>> {@link File#pathSeparator}. >>> ???? * All characters except the {@link File#pathSeparator} are >>> allowed. >>> ???? * Some operating systems support quoting segments of the string >>> ???? * potentially including {@code pathSeparator} characters. >>> ???? * Empty paths, identified by leading, trailing, and adjacent >>> separator characters, >>> ???? * can be omitted or replaced with a non-null provided path. >>> ???? * >>> ???? * @implNote >>> ???? * On Windows, zero or more characters between double-quotes >>> ({@code "}) are included >>> ???? * literally in the path, regardless of whether they are valid >>> in a path. >>> ???? * >>> ???? * @param path a string containing paths separated by the {@link >>> File#pathSeparator} >>> ???? * @param emptyPath a path to replace an empty path or {@code >>> null} to ignore empty paths >>> ???? * @return an immutable list of strings for each path; {@code >>> non-null} >>> ???? */ >>> ??? public static List parsePathStrings(String path, String >>> emptyPath) { ...} >>> >>> There are of course, possible variations and convenience versions >>> but the API should be kept to the essentials. >>> With java.util.streams it becomes easy to process each component or >>> to map the strings to Files or Path. >>> >>> Suggestions welcome, Roger >>> >> > From brian.burkhalter at oracle.com Mon Jul 23 20:34:23 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 23 Jul 2018 13:34:23 -0700 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity Message-ID: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8206403 http://cr.openjdk.java.net/~bpb/8206403/webrev.00/ Disallow creating an internal array of size larger than Integer.MAX_VALUE - 8. The change if approved as-is will require a CSR for the @implNote. Thanks, Brian From roger.riggs at oracle.com Mon Jul 23 21:10:26 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 23 Jul 2018 17:10:26 -0400 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> Message-ID: <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> Hi Brian, You might want to add an @requires of 8Gb or whatever so the test only runs on a system it can succeed on. I don't see the @randomness in the test.? (Other than perhaps available heap). ByteArrayOutputStream:121: A message indicating the nature of the error would be useful. In the test, I don't think you need to fill the array, writing 0 is just as good as 0xff. Roger On 7/23/18 4:34 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8206403 > http://cr.openjdk.java.net/~bpb/8206403/webrev.00/ > > Disallow creating an internal array of size larger than Integer.MAX_VALUE - 8. > > The change if approved as-is will require a CSR for the @implNote. > > Thanks, > > Brian From brian.burkhalter at oracle.com Mon Jul 23 21:49:21 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 23 Jul 2018 14:49:21 -0700 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> Message-ID: Hi Roger, Updated version: http://cr.openjdk.java.net/~bpb/8206403/webrev.01/ On Jul 23, 2018, at 2:10 PM, Roger Riggs wrote: > You might want to add an @requires of 8Gb or whatever so the test only runs on a system it can succeed on. Re-tested and changed to @requires 2g and dropped the @ignore. > I don't see the @randomness in the test. (Other than perhaps available heap). That was vestigial from copying the header from elsewhere. > ByteArrayOutputStream:121: A message indicating the nature of the error would be useful. There was no message in the original file but I concur that one is better so I added one. > In the test, I don't think you need to fill the array, writing 0 is just as good as 0xff. Changed. Thanks for the review. Brian From martinrb at google.com Tue Jul 24 00:26:21 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 23 Jul 2018 17:26:21 -0700 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> Message-ID: I'm the author of most of the MAX_ARRAY_SIZE code in the jdk. We should be as consistent as we can given the history. It all looks pretty similar. --- + if (minCapacity < 0) // overflow + throw new OutOfMemoryError("Memory capacity overflow: " + + minCapacity); + I deliberately tried to push this cold code into hugeCapacity. (yeah, not worth much ...) --- I deliberately tried to avoid a hard limit within the grow code itself, instead letting the VM fail with its OOME. Perhaps the VM will surprise us by allowing the creation of the array of size Integer.MAX_VALUE, as I think it should. + * @implNote + * The maximum size permitted by this implementation is + * {@code Integer.MAX_VALUE - 8}. --- So unsurprisingly I prefer the status quo. We don't know for sure that """ ByteArrayOutputStream hugeCapacity method can return invalid capacity""" On Mon, Jul 23, 2018 at 2:49 PM, Brian Burkhalter < brian.burkhalter at oracle.com> wrote: > Hi Roger, > > Updated version: http://cr.openjdk.java.net/~bpb/8206403/webrev.01/ > > On Jul 23, 2018, at 2:10 PM, Roger Riggs wrote: > > > You might want to add an @requires of 8Gb or whatever so the test only > runs on a system it can succeed on. > > Re-tested and changed to @requires 2g and dropped the @ignore. > > > I don't see the @randomness in the test. (Other than perhaps available > heap). > > That was vestigial from copying the header from elsewhere. > > > ByteArrayOutputStream:121: A message indicating the nature of the error > would be useful. > > There was no message in the original file but I concur that one is better > so I added one. > > > In the test, I don't think you need to fill the array, writing 0 is just > as good as 0xff. > > Changed. > > Thanks for the review. > > Brian From brian.burkhalter at oracle.com Tue Jul 24 00:40:48 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 23 Jul 2018 17:40:48 -0700 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> Message-ID: <14345E10-C6A1-46F9-8B21-66F182BA1554@oracle.com> On Jul 23, 2018, at 5:26 PM, Martin Buchholz wrote: > I'm the author of most of the MAX_ARRAY_SIZE code in the jdk. > We should be as consistent as we can given the history. Thanks for the history lesson. > It all looks pretty similar. There are in fact a number of them in java.base: private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/io/ByteArrayOutputStream.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/lang/AbstractStringBuilder.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/AbstractCollection.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/ArrayDeque.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/ArrayList.java static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/concurrent/ConcurrentHashMap.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/concurrent/PriorityBlockingQueue.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/Hashtable.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/PriorityQueue.java static final long MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/stream/Nodes.java private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; src/java.base/share/classes/java/util/Vector.java > So unsurprisingly I prefer the status quo. I am wondering whether MAX_ARRAY_SIZE is even still necessary in current VMs? I don?t know the low level details enough to comment on that. Thanks, Brian From martinrb at google.com Tue Jul 24 01:04:41 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 23 Jul 2018 18:04:41 -0700 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: <14345E10-C6A1-46F9-8B21-66F182BA1554@oracle.com> References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> <14345E10-C6A1-46F9-8B21-66F182BA1554@oracle.com> Message-ID: Latest hotspot code still has the limit. It's dependent on element type and VM specifics like header size, so we can't just try to see what the limit is empirically and adjust 8 to 2. // Return the maximum length of an array of BasicType. The length can passed // to typeArrayOop::object_size(scale, length, header_size) without causing an // overflow. We also need to make sure that this will not overflow a size_t on // 32 bit platforms when we convert it to a byte size. static int32_t max_array_length(BasicType type) { assert(type >= 0 && type < T_CONFLICT, "wrong type"); assert(type2aelembytes(type) != 0, "wrong type"); const size_t max_element_words_per_size_t = align_down((SIZE_MAX/HeapWordSize - header_size(type)), MinObjAlignment); const size_t max_elements_per_size_t = HeapWordSize * max_element_words_per_size_t / type2aelembytes(type); if ((size_t)max_jint < max_elements_per_size_t) { // It should be ok to return max_jint here, but parts of the code // (CollectedHeap, Klass::oop_oop_iterate(), and more) uses an int for // passing around the size (in words) of an object. So, we need to avoid // overflowing an int when we add the header. See CRs 4718400 and 7110613. return align_down(max_jint - header_size(type), MinObjAlignment); } return (int32_t)max_elements_per_size_t; } On Mon, Jul 23, 2018 at 5:40 PM, Brian Burkhalter < brian.burkhalter at oracle.com> wrote: > > On Jul 23, 2018, at 5:26 PM, Martin Buchholz wrote: > > I'm the author of most of the MAX_ARRAY_SIZE code in the jdk. > We should be as consistent as we can given the history. > > > Thanks for the history lesson. > > It all looks pretty similar. > > > There are in fact a number of them in java.base: > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/io/ByteArrayOutputStream.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/lang/AbstractStringBuilder.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/AbstractCollection.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/ArrayDeque.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/ArrayList.java > > static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/concurrent/ConcurrentHashMap.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/concurrent/ > PriorityBlockingQueue.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/Hashtable.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/PriorityQueue.java > > static final long MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/stream/Nodes.java > > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > src/java.base/share/classes/java/util/Vector.java > > So unsurprisingly I prefer the status quo. > > > I am wondering whether MAX_ARRAY_SIZE is even still necessary in current > VMs? I don?t know the low level details enough to comment on that. > > Thanks, > > Brian > From martinrb at google.com Tue Jul 24 03:02:00 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 23 Jul 2018 20:02:00 -0700 Subject: RFR 8207314 : Unnecessary reallocation when constructing WeakHashMap from a large Map In-Reply-To: References: Message-ID: Thanks Ivan, what you have now looks good for this bug. Capacity bugs are common because they can only be tested via whitebox tests, which are onerous to write. For future work, the HashMap resize seems important enough to fix, and that one may deserve a whitebox test. Explicit NaN checking is distracting in the source code and totally unimportant in practice, so I'd like to just not see it. On Fri, Jul 20, 2018 at 6:42 PM, Ivan Gerasimov wrote: > Hi Martin, > > Thank you for looking into this! Please see the comments inline. > > On 7/15/18 9:14 AM, Martin Buchholz wrote: > > Hi Ivan, > > Here are some thoughts while looking at this: > > --- > > WeakHashMap promises to have similar "capacity" handling to HashMap, but > implementations (and doc?) seem more different than necessary (diverged > over time?), and there don't seem to be any tests. > > HashMap seems to deal with this problem by doing computations using float > not int. Choose the best one and use it in both source files. > > float ft = ((float)s / loadFactor) + 1.0F; > int t = ((ft < (float)MAXIMUM_CAPACITY) ? > (int)ft : MAXIMUM_CAPACITY); > > I compared the bytecode generated for > 1) (int) ((float)s / loadFactor) + 1.0F > 2) (int) (s / loadFactor + 1); > > and the results are actually exactly the same: > 0: iload_0 > 1: i2f > 2: ldc #2 // float 0.75f > 4: fdiv > 5: fconst_1 > 6: fadd > 7: f2i > > So, I just modified the webrev to make it look more like what we currently > have in HashMap. > > > --- > > Consider adding a WhiteBox test. An existing one for ConcurrentHashMap > could be modified to test internal table maintenance. > PriorityBlockingQueue/WhiteBox.java is an example of a test that ensures > two implementations stay in sync. > > I cannot see how to verify this fix easily, and I didn't really think that > such a trivial fix is worth a dedicated test. > > If you disagree, I can try to write a test that will pass a custom map as > an argument to WeakHashMap constructor. > From the custom map's size() we can retrieve a reference to the > WeakHashMap being constructed and make sure that its table is either null > or doesn't have the default size. > Sounds fragile though, so I would prefer to just set noreg-hard label :) > > --- > > i see > if (loadFactor <= 0 || Float.isNaN(loadFactor)) > but that nan check doesn't have much value. It can be removed using > if (! (loadFactor > 0)) > > This pattern is used in several places (HashMap, HashSet, Hashtable, > WeakHashMap). > If simplifying it in the way you suggest gives a performance gain, I > wonder if JVM can be taught to optimize checks for NaN via combining them > with comparisons nearby? > > --- > > HashMap's resize() doesn't take an arg, while WeakHashMap's does. Why? > As a result, I see in HashMap.putMapEntries > > else if (s > threshold) > resize(); > > which suggests that if you create a fresh HashMap, then putAll(hugeMap) it > will repeatedly resize instead of resizing to the target capacity in one > step. Which seems like a HashMap bug. > > I agree that adding a hint to HashMap.resize() may help to allocate the > new table more optimally. > I think is should be a separate enhancement though. > > Here's the updated webrev with only stylistic change, comparing to the > previous one: > http://cr.openjdk.java.net/~igerasim/8207314/01/webrev/ > > With kind regards, > Ivan > > > On Fri, Jul 13, 2018 at 10:22 PM, Ivan Gerasimov < > ivan.gerasimov at oracle.com> wrote: > >> Hello! >> >> When a WeakHashMap is constructed from another Map m, then the initial >> capacity is calculated as >> >> (int) (m.size() / DEFAULT_LOAD_FACTOR) + 1. >> >> For large values of m.size() this becomes negative due to integer >> overflow. >> >> The result is that the WeakHashMap is initially constructed with the >> default initial capacity 16, and then is immediately resized. >> >> Would you please help review a trivial patch? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8207314 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8207314/00/webrev/ >> >> Thanks in advance! >> >> -- >> With kind regards, >> Ivan Gerasimov >> >> > > -- > With kind regards, > Ivan Gerasimov > > From martinrb at google.com Tue Jul 24 03:46:52 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 23 Jul 2018 20:46:52 -0700 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: Message-ID: (As usual, I don't have enough time to devote to this at the moment) I sort of like the idea of adding all of those List methods to ArrayDeque, but it's __weird__ for java code to do that, and remove(int) is a problem when you have ArrayDeque, so it will probably end up rejected. --- Similarly, the idea of an ArrayDeque subclass that accepts nulls will be unpopular - null elements have fallen out of favor. While Deque implementations are not strictly required to prohibit the insertion of null elements, they are strongly encouraged to do so. Users of any Deque implementations that do allow null elements are strongly encouraged *not* to take advantage of the ability to insert nulls. This is so because null is used as a special return value by various methods to indicate that the deque is empty. --- It makes some sense for ArrayDeque's growth policy to be very similar to ArrayList's - that should be done as an independent change. Are there any lessons to be learned from ArrayList's experience? Is the world filled with empty ArrayDeques that could share a common backing array? --- I do like the idea of having ArrayDeque's List-implementing subclass be a nestmate of ArrayDeque, to share implementation details and to help discovery and naming. I'm not thrilled with reusing "List" in ArrayDeque.List but I don't have a great alternative to suggest. On Mon, Jul 16, 2018 at 10:36 AM, Alex Foster wrote: > Hi again, > > I updated ArrayDeque with your last idea (adding all list methods but not > implementing List) and added a subclass ArrayDeque.List which overrides > equals and hashcode and implements List. There is also a subclass > ArrayDeque.WithNulls that accepts null elements. ArrayDeque has > removeAt(int index) instead of remove(index) to avoid overloading > remove(Object). > > I also added shared empty arrays similar to Arraylist, and a reallocate > method which can do the same things as trimToSize and ensure capacity in > Arraylist. It also allows you to trim to a specific capacity other than the > size or skip trimming if the capacity is within a specified distance of the > target capacity. > > Also the bulk add methods call collection.toArray, then check the array > for illegal elements, then add the array, which means that a collection > could keep the array it returns from toArray and modify it from another > thread after it has been checked but before it has been added which could > lead to illegal elements being added to the ArrayDeque. We could maybe > avoid this by cloning the array or checking the elements after adding them > but I'm not sure if it's worth it... > > What do you think? > > I also changed the WhiteBox test a bit: > > --- a/test/jdk/java/util/ArrayDeque/WhiteBox.java > +++ b/test/jdk/java/util/ArrayDeque/WhiteBox.java > @@ -88,7 +88,10 @@ > > @Test > public void defaultConstructor() { > - checkCapacity(new ArrayDeque(), 16); > + ArrayDeque d = new ArrayDeque(); > + d.add(new Object()); > + d.clear(); > + checkCapacity(d, 16); > } > > @Test > @@ -131,7 +134,7 @@ > if (rnd.nextBoolean()) d.add(99); > ArrayDeque clone = serialClone(d); > assertInvariants(clone); > - assertNotSame(elements(d), elements(clone)); > + assertTrue(d.isEmpty() || elements(d) != elements(clone)); > assertEquals(d, clone); > } > } > > Alex > From xu.y.yin at oracle.com Tue Jul 24 06:12:31 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 24 Jul 2018 14:12:31 +0800 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> Message-ID: Hi, Roger Thanks a lot for your review and comments, inline and update new webrev as below, please have a review http://cr.openjdk.java.net/~xyin/8198882/webrev.03/ > On 24 Jul 2018, at 1:05 AM, Roger Riggs wrote: > > Hi Chris, > > Sorry to be late to the review cycle. > > The use of initializer blocks instead of constructors reduces the readability of the code. > A simple fix is to add the constructor declarations ahead of the initializer to make the code more readable. Fixed, use constructor now instead of initialization block, thanks > > GetAttrsBase: 69: The signature of setMandatoryAttrs could be setMandatoryAttrs(String...) would allow > the mandatory strings to be passed without explicit array creation and would still allow > String[] to be passed where more convenient. Changed with your suggested style > > GetAttrsBase: 53: The exception message would be more readable as: "Attributes to be verified is null? Sure, changed the exception msg as you suggested > > GetAttrsNotFound: 50: throw the fail exception here; hiding it in the verifyAttributes > method is harder to understand Fixed, removed verifyAttributes override method and throw exception directly in runTest() > > GetNonstandard: 87, 93: Printing the expected and actual arrays on failure (regardless of debug) will help diagnose failures more quickly Sure, put expected and actual arrays in exception msg and simplify condition check > > GetNumericIRRs/GetNumericRRs: Tests like this are so abstract that is it difficult to see what > is being tested. For example, where does getIndex() get its value? > Either more comments are needed or more expressive method names. Yep, let me try to expand the key test method into test directly, there will be some redundancy, but should be more clear to see what we want to test > > DNSTestUtils: 94: Don't encourage the usage allowing separation of -D from the argument. > Its not allowed by the normal command line parsing and should not be supported. > (Yes, I know it is/was not changed by this review). Sure, removed this kind of usage support > > :153: cleanupClosableRes does not appear to be used; if it is not needed, remove it Yes, removed > > A lot of the test framework is embodied in these DNSTestUtils without explanation or comment. > It will help future maintenance, if there is a class level description (prose) describing > the sequence of events for tests. I add some javadoc to those DNSTestUtils, base sequence in TestBase class description, hope that will be helpful Thanks & Regards, Chris > > Regards, Roger > > > On 7/13/18 2:14 AM, Chris Yin wrote: >> Hi, Vyom >> >> Thank you for the review and comments, update webrev as below and comment inline >> >> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.02/ >> >> >>> On 13 Jul 2018, at 1:46 PM, vyom tewari wrote: >>> >>> Hi Chris, >>> >>> Thanks for doing this overall looks good to me, few minor comments. >>> 1-> DNSTestUtils.java, please start the server first and then set the "TEST_DNS_SERVER_THREAD". This will not make much difference but we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if server fails to start. >>> 129 env.put(TEST_DNS_SERVER_THREAD, inst); >>> 130 inst.start(); >> Fixed, thanks >> >>> 2-> I noticed that copyright date (Copyright (c) 2000, 2018,) but webrev tells all the tests are new, please fix copyright date as well. >> Thanks for checking this. Since this task is part of umbrella enhancement JDK-8191011 JNDI SQE tests co-location, for those added tests which are migrated from SQE tests, the copyright date will follow the guidance SQE Test copyright year + migration copyright year if the 2 year are not same, for dump files (like *.dns) are new added under our new framework so just use current copyright year, hope that explains :), thanks >> >> Regards, >> Chris >> >>> Thanks, >>> Vyom >>> >>> On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: >>>> Please have a review to new webrev as below, some code refactoring on lib parts and enhanced DNSServer to handle retry request which will make the tests more stable, thanks >>>> >>>> http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ >>>> >>>> Regards, >>>> Chris >>>> >>>>> On 22 Mar 2018, at 11:16 AM, Chris Yin > wrote: >>>>> >>>>> Please review the changes to add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/, thanks >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 >>>>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ >>>>> >>>>> Regards, >>>>> Chris > From enasser at in.ibm.com Tue Jul 24 08:56:50 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Tue, 24 Jul 2018 08:56:50 +0000 Subject: Adding new IBM extended charsets In-Reply-To: <4ed347e6-1e64-5ef6-be48-8d12b0adac6f@oracle.com> References: <5B3E4AC8.3070307@oracle.com> <4cba7633-33b7-f29a-fb34-05a06b224057@oracle.com> <5B503D69.6010004@oracle.com> <4ed347e6-1e64-5ef6-be48-8d12b0adac6f@oracle.com> Message-ID: Thank you Martin, Sherman and Alan for your valuable inputs. I have done some initial analysis on the ICU4J. There are some compatibility issues on the ICU4J charsets with JDK charsets but am more concerned about its performance as JDK optimization do no exist in that implementation. I think we need to work with the ICU4J community to resolve those issues before we remove those charsets from JDK. The primary reason we are interested to contribute the charsets to openjdk is that Java users of all locales to get a seamless experience when they move between openjdk and other implementations. I agree it is good from footprint and maintenance perspective if we are able to reduce the number of charsets. I believe the maintenance effort on the charsets are usually less as we hardly make any changes to the charsets once developed. Also, the charsets are usually independent to each other and hence usually will not affect the Java users unless they are used. As more team members from my team would like to actively participate in the openjdk community, I hope maintenance of any issues reported on IBM charsets may not be an issue going forward. As we discussed before, the footprint issue can be avoided if we enable the IBM charsets on a need basis with a build flag. As you advised, we can enable the IBM charsets only for AIX platform by default and user can enable them on other platforms on a need basis. If all of you agree, we can start working on moving all IBM charsets from jdk.charsets to a different module jdk.ibm.charsets and enable them only for AIX platform by default. We can consider removing them from JDK in future if community found them as an overhead or not adding value. Please advise. Thank you, Nasser Ebrahim From: Alan Bateman To: Xueming Shen , Nasser Ebrahim Cc: core-libs-dev at openjdk.java.net Date: 07/19/2018 03:44 PM Subject: Re: Adding new IBM extended charsets On 19/07/2018 08:27, Xueming Shen wrote: > Hi Nasser, > > From openjdk's perspective It would be preferred to direct the develop > to use the charset > implementation provided by IBM, or the reliable third party that has > the appropriate knowledge, > experience and resource to support/maintain those charsets such as the > icu4j charset > project. I have been pulling the data from that huge icu-charset-data > file and implement/maintain > them based on my best knowledge, but I'm sure engineers from IBM or > the icu project probably > can do a much better job to implement/maintain/update those charsets > going forward. > > As first step we can separate those IBM charsets from the jdk.charset > into a separate package > somewhere and configure them to be built into java.base and > jdk.charsets, for aix platform only. > Then we can further discuss the best way to handle/distribute those > charsets that are not needed > for the java.base module (for vm startup). As I said, it would be > ideal if we can remove them from the > openjdk repo/binaries complete and direct the developer/user to use > the icu4j charset provider > for those encodings, when needed. But given the possible compatibility > concern, we might want to > phase this work out gradually in next major release. I agree and in terms of phasing then I don't think it would be too disruptive if the EBCDIC charsets were dropped from jdk.charsets in JDK 12, at least on the main stream platforms. As we've established in this thread, the ICU4J project does seem to publish its charset provider to Maven so there are alternatives for applications that really need these charsets Nasser - do you do any testing with the ICU4J charsets? I quickly tried 62.1 and it seems to work fine on the class path. I didn't check for any compatibility differences or compare the performance but maybe you have. It's a bit awkward to test this provider as an automatic module due to the unusual naming of these JAR files. They may not have looked at modules yet but the ability to link thee icu4h.charsets module into a run-time image seems something that people may want to do in the future. -Alan From adinn at redhat.com Tue Jul 24 12:17:44 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 24 Jul 2018 13:17:44 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <87o9f1lfjd.fsf@mid.deneb.enyo.de> Message-ID: <6a98d387-c23f-7d3c-0d78-cf1064d5d11e@redhat.com> On 23/07/18 12:01, Andrew Dinn wrote: > Hi Florian, > > Thank you for the feedback. Comments inline below (extensive comments in > the case of the memory model question - it is complex). Having written up what I assumed was needed, of course I then went back to have another look at the Intel code and as a result I think I have i) corrected my mistaken understanding of what it does ii) come to the conclusion that libpmem is doing the right thing No surprise ;-] Firstly, I was mistaken in thinking that the libpmem code was executing a memory barrier before the writeback operations (I was confused by talk of pre-drain, hw-drain and drain at various points in the code). The definition of the critical function pmem_persist is as follows void pmem_persist(const void *addr, size_t len) { pmem_flush(addr, len); pmem_drain(); } where pmem_flush is implemented via clflush/clflushopt/clwb or dc(CVAC) pmem_drain is implemented as an sfence or dmb(ISH) So, libpmem is emitting a memory barrier /after/ writeback and this barrier serves to to ensure that writeback is complete before any subsequent writes can proceed to completion and become visible. That deals with the data + control update issue I was concerned about. On the other hand, libpmem does /not/ execute any memory barriers prior to executing the writeback instructions. I had assumed that a pre-sync was also necessary to ensure that prior writes completed before the writeback was initiated. However, looking at the details provided in the documentation this seems to be a spurious requirement of my own invention. On x86_64 writeback operations proceeding via CLFLUSHOPT or CLWB on some specific cache line are ordered wrt to writes on that same cache line. The relevant text is: "Executions of the CLFLUSHOPT instruction are [...] ordered with respect to the following accesses to the cache line being invalidated: writes, executions of CLFLUSH, and executions of CLFLUSHOPT." "CLWB is implicitly ordered with older stores executed by the logical processor to the same address" So, there is no need to execute a memory barrier to avoid the possibility of writing back a cache line before it has been updated by an in-progress write to that same cache line. Of course, neither is there a need to be concerned about in-progress writes to cache lines that are not going to be flushed. They won't have any effect on the correctness of the writeback. So, a pre-barrier is unnecessary. The same consideration seem to apply for AArch64. "All data cache instructions, other than DC ZVA, that specify an address: - Execute in program order relative to loads or stores that access an address in Normal memory with either Inner Write Through or Inner Write Back attributes within the same cache line of minimum size, as indicated by CTR_EL0.DMinLine" So, once again a DMB is required after but not before writeback is initiated. I'm also now unsure that a DMB(ISHST)is actually adequate. The relevant text from the ARM ARM is at the end of the text I previously cited. "In all cases, where the text in this section refers to a DMB or a DSB , this means a DMB or DSB whose required access type is both loads and stores." I think that means that the pmem_drain barrier has to be a DMB(ISH) not a DMB(ISHST). I'm not really sure why but it does seem to be the implication. So, to sum up Unsafe.writebackPreSync0 should be implemented as a no-op on x86_64 a no-op on AArch64 Unsafe.writebackPosSync0 should be implemented as a no-op on x86_64 when using clflush an sfence on x86_64 when using clflushopt or clwb a dmb(ISH) on AArch64 I think it may still be worth retaining Unsafe.writebackPreSync0 (even though it currently translates to a no-op) in case new hardware requires some extra preparatory operation before writeback commences. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From roger.riggs at oracle.com Tue Jul 24 14:12:04 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 24 Jul 2018 10:12:04 -0400 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> Message-ID: <91eb84d7-8760-9c57-07e8-4819602f72a5@oracle.com> Hi Chris, Thanks for the improvements and javadoc. Conventionally, the first sentence javadoc comments end in "." (period). It makes it more natural to read. The closing ")" in method calls looks odd to be on a line by itself. For example, GetAttrBase.java: line 35 Please fix these but no further review is needed. Thanks, Roger On 7/24/18 2:12 AM, Chris Yin wrote: > Hi, Roger > > Thanks a lot for your review and comments, inline and update new > webrev as below, please have a review > > http://cr.openjdk.java.net/~xyin/8198882/webrev.03/ > > > >> On 24 Jul 2018, at 1:05 AM, Roger Riggs > > wrote: >> >> Hi Chris, >> >> Sorry to be late to the review cycle. >> >> The use of initializer blocks instead of constructors reduces the >> readability of the code. >> A simple fix is to add the constructor declarations ahead of the >> initializer to make the code more readable. > > Fixed, use constructor now instead of initialization block, thanks > >> >> GetAttrsBase: 69: The signature of setMandatoryAttrs could be >> setMandatoryAttrs(String...) would allow >> the mandatory strings to be passed without explicit array creation >> and would still allow >> String[] to be passed where more convenient. > > Changed with your suggested style > >> >> GetAttrsBase: 53: The exception message would be more readable as: >> "Attributes to be verified is null? > > Sure, changed the exception msg as you suggested > >> >> GetAttrsNotFound: 50: throw the fail exception here; hiding it in the >> verifyAttributes >> method is harder to understand > > Fixed, removed verifyAttributes override method and throw exception > directly in runTest() > >> >> GetNonstandard: 87, 93: Printing the expected and actual arrays on >> failure (regardless of debug) will help diagnose failures more quickly > > Sure, put expected and actual arrays in exception msg and simplify > condition check > >> >> GetNumericIRRs/GetNumericRRs: Tests like this are so abstract that is >> it difficult to see what >> is being tested. For example, where does getIndex() get its value? >> Either more comments are needed or more expressive method names. > > Yep, let me try to expand the key test method into test directly, > there will be some redundancy, but should be more clear to see what we > want to test > >> >> DNSTestUtils: 94:? Don't encourage the usage allowing separation of >> -D from the argument. >> Its not allowed by the normal command line parsing and should not be >> supported. >> (Yes, I know it is/was not changed by this review). > > Sure, removed this kind of usage support > >> >> :153:? cleanupClosableRes does not appear to be used;? if it is not >> needed, remove it > > Yes, removed > >> >> A lot of the test framework is embodied in these DNSTestUtils without >> explanation or comment. >> It will help future maintenance, if there is a class level >> description (prose) describing >> the sequence of events for tests. > > I add some javadoc to those DNSTestUtils, base sequence in TestBase > class description, hope that will be helpful > > Thanks & Regards, > Chris > >> >> Regards, Roger >> >> >> On 7/13/18 2:14 AM, Chris Yin wrote: >>> Hi, Vyom >>> >>> Thank you for the review and comments, update webrev as below and >>> comment inline >>> >>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.02/ >>> >>> >> > >>> >>> >>>> On 13 Jul 2018, at 1:46 PM, vyom tewari >>> > wrote: >>>> >>>> Hi Chris, >>>> >>>> Thanks for doing this overall looks good to me, few minor comments. >>>> 1-> ?DNSTestUtils.java, please start the server first and then set >>>> the "TEST_DNS_SERVER_THREAD". This will not make much difference >>>> but we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if >>>> server fails to start. >>>> 129 ????????????env.put(TEST_DNS_SERVER_THREAD, inst); >>>> ?130 ????????????inst.start(); >>> Fixed, thanks >>> >>>> 2-> ?I noticed that ?copyright date (Copyright (c) 2000, 2018,) but >>>> webrev tells all the tests are new, please fix copyright date as well. >>> Thanks for checking this. Since this task is part of umbrella >>> enhancement JDK-8191011 >>> JNDI SQE tests >>> co-location, for those added tests which are migrated from SQE >>> tests, the copyright date will follow the guidance SQE Test >>> copyright year + migration copyright year if the 2 year are not >>> same, for dump files (like *.dns) are new added under our new >>> framework so just use current copyright year, hope that explains :), >>> thanks >>> >>> Regards, >>> Chris >>> >>>> Thanks, >>>> Vyom >>>> >>>> On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: >>>>> Please have a review to new webrev as below, some code refactoring >>>>> on lib parts and enhanced DNSServer to handle retry request which >>>>> will make the tests more stable, thanks >>>>> >>>>> http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ >>>>> >>>>> >>>>> >>>>> Regards, >>>>> Chris >>>>> >>>>>> On 22 Mar 2018, at 11:16 AM, Chris Yin >>>>> > wrote: >>>>>> >>>>>> Please review the changes to add 10 JNDI tests to >>>>>> com/sun/jndi/dns/AttributeTests/, thanks >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ >>>>>> >>>>>> >>>>>> >>>>>> Regards, >>>>>> Chris >> > From roger.riggs at oracle.com Tue Jul 24 14:26:05 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 24 Jul 2018 10:26:05 -0400 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> Message-ID: Hi Brian, The update looks fine. Roger On 7/23/18 5:49 PM, Brian Burkhalter wrote: > Hi Roger, > > Updated version: http://cr.openjdk.java.net/~bpb/8206403/webrev.01/ > > > On Jul 23, 2018, at 2:10 PM, Roger Riggs > wrote: > >> You might want to add an @requires of 8Gb or whatever so the test >> only runs on a system it can succeed on. > > Re-tested and changed to @requires 2g and dropped the @ignore. > >> I don't see the @randomness in the test.? (Other than perhaps >> available heap). > > That was vestigial from copying the header from elsewhere. > >> ByteArrayOutputStream:121: A message indicating the nature of the >> error would be useful. > > There was no message in the original file but I concur that one is > better so I added one. > >> In the test, I don't think you need to fill the array, writing 0 is >> just as good as 0xff. > > Changed. > > Thanks for the review. > > Brian From roger.riggs at oracle.com Tue Jul 24 15:37:25 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 24 Jul 2018 11:37:25 -0400 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> Message-ID: <3a03c617-48b6-b3c1-3fcc-8b2fd05f9334@oracle.com> Hi Brian, A followup on the issues raised by Martin. The original issue[1] was that the resize by doubling approach failed to take advantage of nearly 1G of potential buffer space. The new issue is raised against getting the last additional 2-6 bytes of buffer space before the hitting the VM's implementation limit. I don't think its worth the effort to try to ensure those last few bytes are available before throwing OOM. Reconsidering, I would close the issue as WillNotFix for the reason that the application is encountering an implementation limit. Regards, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8055949 On 7/24/18 10:26 AM, Roger Riggs wrote: > Hi Brian, > > The update looks fine. > > Roger > > > On 7/23/18 5:49 PM, Brian Burkhalter wrote: >> Hi Roger, >> >> Updated version: http://cr.openjdk.java.net/~bpb/8206403/webrev.01/ >> >> >> On Jul 23, 2018, at 2:10 PM, Roger Riggs > > wrote: >> >>> You might want to add an @requires of 8Gb or whatever so the test >>> only runs on a system it can succeed on. >> >> Re-tested and changed to @requires 2g and dropped the @ignore. >> >>> I don't see the @randomness in the test.? (Other than perhaps >>> available heap). >> >> That was vestigial from copying the header from elsewhere. >> >>> ByteArrayOutputStream:121: A message indicating the nature of the >>> error would be useful. >> >> There was no message in the original file but I concur that one is >> better so I added one. >> >>> In the test, I don't think you need to fill the array, writing 0 is >>> just as good as 0xff. >> >> Changed. >> >> Thanks for the review. >> >> Brian > From brian.burkhalter at oracle.com Tue Jul 24 16:46:48 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 24 Jul 2018 09:46:48 -0700 Subject: [12] 8206403: ByteArrayOutputStream hugeCapacity method can return invalid capacity In-Reply-To: <3a03c617-48b6-b3c1-3fcc-8b2fd05f9334@oracle.com> References: <976E1B8F-AA4A-478B-A160-4020C0957220@oracle.com> <0f5943b4-5e61-aa69-c566-a6d187985c31@oracle.com> <3a03c617-48b6-b3c1-3fcc-8b2fd05f9334@oracle.com> Message-ID: <24EECC36-157E-424C-BCB2-41A7B9BC5151@oracle.com> Hi Roger, I tend to agree with you. Also, I think that if MAX_ARRAY_SIZE were to be dealt with it should be so globally for consistency and not just in this one class. Thanks for the follow up and the historical link (context). Brian On Jul 24, 2018, at 8:37 AM, Roger Riggs wrote: > A followup on the issues raised by Martin. > > The original issue[1] was that the resize by doubling approach failed to take advantage > of nearly 1G of potential buffer space. > The new issue is raised against getting the last additional 2-6 bytes of buffer space before > the hitting the VM's implementation limit. > > I don't think its worth the effort to try to ensure those last few bytes are available before throwing OOM. > Reconsidering, I would close the issue as WillNotFix for the reason that the application is > encountering an implementation limit. > > Regards, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8055949 From aph at redhat.com Tue Jul 24 17:00:35 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 24 Jul 2018 18:00:35 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <6a98d387-c23f-7d3c-0d78-cf1064d5d11e@redhat.com> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <87o9f1lfjd.fsf@mid.deneb.enyo.de> <6a98d387-c23f-7d3c-0d78-cf1064d5d11e@redhat.com> Message-ID: <3c9e1a8d-2964-3556-dd20-60db5ee9264f@redhat.com> On 07/24/2018 01:17 PM, Andrew Dinn wrote: > I'm also now unsure that a DMB(ISHST)is actually adequate. The relevant > text from the ARM ARM is at the end of the text I previously cited. > > "In all cases, where the text in this section refers to a DMB or a DSB , > this means a DMB or DSB whose required access type is both loads and > stores." > > I think that means that the pmem_drain barrier has to be a DMB(ISH) not > a DMB(ISHST). I'm not really sure why but it does seem to be the > implication. Probably, but that's OK. DMB(ISHST) is only a StoreStore, and I suspect you want LoadStore as well anyway. StoreStore on its own is only in practice correct if what you're storing before the barrier instruction is constant data, and that's really not true in this case. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From alexandre.iline at oracle.com Tue Jul 24 22:48:33 2018 From: alexandre.iline at oracle.com (Alexandre (Shura) Iline) Date: Tue, 24 Jul 2018 15:48:33 -0700 Subject: RFR: 8208157: requires.VMProps throws NPE for missing properties in "release" file Message-ID: <390EE6A8-331D-4194-8A77-CC2DB44DAF2D@oracle.com> Hi, Could you please tale a quick look on this simple fix? diff --git a/test/jtreg-ext/requires/VMProps.java b/test/jtreg-ext/requires/VMProps.java --- a/test/jtreg-ext/requires/VMProps.java +++ b/test/jtreg-ext/requires/VMProps.java @@ -432,7 +432,8 @@ System.getProperty("java.home") + "/release"))) { Properties properties = new Properties(); properties.load(in); - return properties.getProperty("IMPLEMENTOR").replace("\"", ""); + String implementorProperty = properties.getProperty("IMPLEMENTOR"); + return (implementorProperty == null) ? "null" : implementorProperty.replace("\"", ""); } catch (IOException e) { e.printStackTrace(); } Thank you. Shura From igor.ignatyev at oracle.com Tue Jul 24 22:51:35 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 24 Jul 2018 15:51:35 -0700 Subject: RFR: 8208157: requires.VMProps throws NPE for missing properties in "release" file In-Reply-To: <390EE6A8-331D-4194-8A77-CC2DB44DAF2D@oracle.com> References: <390EE6A8-331D-4194-8A77-CC2DB44DAF2D@oracle.com> Message-ID: <34394F2F-02B2-4D2E-B3C4-FD64CBF59B02@oracle.com> looks good to me. -- Igor > On Jul 24, 2018, at 3:48 PM, Alexandre (Shura) Iline wrote: > > Hi, > > Could you please tale a quick look on this simple fix? > > diff --git a/test/jtreg-ext/requires/VMProps.java b/test/jtreg-ext/requires/VMProps.java > --- a/test/jtreg-ext/requires/VMProps.java > +++ b/test/jtreg-ext/requires/VMProps.java > @@ -432,7 +432,8 @@ > System.getProperty("java.home") + "/release"))) { > Properties properties = new Properties(); > properties.load(in); > - return properties.getProperty("IMPLEMENTOR").replace("\"", ""); > + String implementorProperty = properties.getProperty("IMPLEMENTOR"); > + return (implementorProperty == null) ? "null" : implementorProperty.replace("\"", ""); > } catch (IOException e) { > e.printStackTrace(); > } > > > Thank you. > > Shura From lance.andersen at oracle.com Tue Jul 24 22:52:23 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 24 Jul 2018 18:52:23 -0400 Subject: RFR: 8208157: requires.VMProps throws NPE for missing properties in "release" file In-Reply-To: <390EE6A8-331D-4194-8A77-CC2DB44DAF2D@oracle.com> References: <390EE6A8-331D-4194-8A77-CC2DB44DAF2D@oracle.com> Message-ID: <70CA3F01-1116-4C0E-88FB-2E8E94791DF6@oracle.com> Looks fine shura > On Jul 24, 2018, at 6:48 PM, Alexandre (Shura) Iline wrote: > > Hi, > > Could you please tale a quick look on this simple fix? > > diff --git a/test/jtreg-ext/requires/VMProps.java b/test/jtreg-ext/requires/VMProps.java > --- a/test/jtreg-ext/requires/VMProps.java > +++ b/test/jtreg-ext/requires/VMProps.java > @@ -432,7 +432,8 @@ > System.getProperty("java.home") + "/release"))) { > Properties properties = new Properties(); > properties.load(in); > - return properties.getProperty("IMPLEMENTOR").replace("\"", ""); > + String implementorProperty = properties.getProperty("IMPLEMENTOR"); > + return (implementorProperty == null) ? "null" : implementorProperty.replace("\"", ""); > } catch (IOException e) { > e.printStackTrace(); > } > > > Thank you. > > Shura Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From erik.joelsson at oracle.com Tue Jul 24 23:50:18 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 24 Jul 2018 16:50:18 -0700 Subject: RFR: 8208157: requires.VMProps throws NPE for missing properties in "release" file In-Reply-To: <390EE6A8-331D-4194-8A77-CC2DB44DAF2D@oracle.com> References: <390EE6A8-331D-4194-8A77-CC2DB44DAF2D@oracle.com> Message-ID: <61ebd89b-2db7-bd46-4694-98a4aa8e52ea@oracle.com> Looks good. /Erik On 2018-07-24 15:48, Alexandre (Shura) Iline wrote: > Hi, > > Could you please tale a quick look on this simple fix? > > diff --git a/test/jtreg-ext/requires/VMProps.java b/test/jtreg-ext/requires/VMProps.java > --- a/test/jtreg-ext/requires/VMProps.java > +++ b/test/jtreg-ext/requires/VMProps.java > @@ -432,7 +432,8 @@ > System.getProperty("java.home") + "/release"))) { > Properties properties = new Properties(); > properties.load(in); > - return properties.getProperty("IMPLEMENTOR").replace("\"", ""); > + String implementorProperty = properties.getProperty("IMPLEMENTOR"); > + return (implementorProperty == null) ? "null" : implementorProperty.replace("\"", ""); > } catch (IOException e) { > e.printStackTrace(); > } > > > Thank you. > > Shura From xu.y.yin at oracle.com Wed Jul 25 02:14:47 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Wed, 25 Jul 2018 10:14:47 +0800 Subject: RFR 8198882: Add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/ In-Reply-To: <91eb84d7-8760-9c57-07e8-4819602f72a5@oracle.com> References: <370A8106-5EE5-4420-9C5F-F415475D2809@oracle.com> <1EC05283-3FA9-4FED-81E4-A456D122D734@oracle.com> <91eb84d7-8760-9c57-07e8-4819602f72a5@oracle.com> Message-ID: <08F00691-479C-48A4-BAB9-666850BCE7A2@oracle.com> Thank you Roger, I will fix them before push. Regards, Chris > On 24 Jul 2018, at 10:12 PM, Roger Riggs wrote: > > Hi Chris, > > Thanks for the improvements and javadoc. > > Conventionally, the first sentence javadoc comments end in "." (period). It makes it more natural to read. > > The closing ")" in method calls looks odd to be on a line by itself. > For example, GetAttrBase.java: line 35 > > Please fix these but no further review is needed. > > Thanks, Roger > > > > > On 7/24/18 2:12 AM, Chris Yin wrote: >> Hi, Roger >> >> Thanks a lot for your review and comments, inline and update new webrev as below, please have a review >> >> http://cr.openjdk.java.net/~xyin/8198882/webrev.03/ >> >> >>> On 24 Jul 2018, at 1:05 AM, Roger Riggs > wrote: >>> >>> Hi Chris, >>> >>> Sorry to be late to the review cycle. >>> >>> The use of initializer blocks instead of constructors reduces the readability of the code. >>> A simple fix is to add the constructor declarations ahead of the initializer to make the code more readable. >> >> Fixed, use constructor now instead of initialization block, thanks >> >>> >>> GetAttrsBase: 69: The signature of setMandatoryAttrs could be setMandatoryAttrs(String...) would allow >>> the mandatory strings to be passed without explicit array creation and would still allow >>> String[] to be passed where more convenient. >> >> Changed with your suggested style >> >>> >>> GetAttrsBase: 53: The exception message would be more readable as: "Attributes to be verified is null? >> >> Sure, changed the exception msg as you suggested >> >>> >>> GetAttrsNotFound: 50: throw the fail exception here; hiding it in the verifyAttributes >>> method is harder to understand >> >> Fixed, removed verifyAttributes override method and throw exception directly in runTest() >> >>> >>> GetNonstandard: 87, 93: Printing the expected and actual arrays on failure (regardless of debug) will help diagnose failures more quickly >> >> Sure, put expected and actual arrays in exception msg and simplify condition check >> >>> >>> GetNumericIRRs/GetNumericRRs: Tests like this are so abstract that is it difficult to see what >>> is being tested. For example, where does getIndex() get its value? >>> Either more comments are needed or more expressive method names. >> >> Yep, let me try to expand the key test method into test directly, there will be some redundancy, but should be more clear to see what we want to test >> >>> >>> DNSTestUtils: 94: Don't encourage the usage allowing separation of -D from the argument. >>> Its not allowed by the normal command line parsing and should not be supported. >>> (Yes, I know it is/was not changed by this review). >> >> Sure, removed this kind of usage support >> >>> >>> :153: cleanupClosableRes does not appear to be used; if it is not needed, remove it >> >> Yes, removed >> >>> >>> A lot of the test framework is embodied in these DNSTestUtils without explanation or comment. >>> It will help future maintenance, if there is a class level description (prose) describing >>> the sequence of events for tests. >> >> I add some javadoc to those DNSTestUtils, base sequence in TestBase class description, hope that will be helpful >> >> Thanks & Regards, >> Chris >> >>> >>> Regards, Roger >>> >>> >>> On 7/13/18 2:14 AM, Chris Yin wrote: >>>> Hi, Vyom >>>> >>>> Thank you for the review and comments, update webrev as below and comment inline >>>> >>>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.02/ > >>>> >>>> >>>>> On 13 Jul 2018, at 1:46 PM, vyom tewari > wrote: >>>>> >>>>> Hi Chris, >>>>> >>>>> Thanks for doing this overall looks good to me, few minor comments. >>>>> 1-> DNSTestUtils.java, please start the server first and then set the "TEST_DNS_SERVER_THREAD". This will not make much difference but we will avoid setting "TEST_DNS_SERVER_THREAD" env variable if server fails to start. >>>>> 129 env.put(TEST_DNS_SERVER_THREAD, inst); >>>>> 130 inst.start(); >>>> Fixed, thanks >>>> >>>>> 2-> I noticed that copyright date (Copyright (c) 2000, 2018,) but webrev tells all the tests are new, please fix copyright date as well. >>>> Thanks for checking this. Since this task is part of umbrella enhancement JDK-8191011 > JNDI SQE tests co-location, for those added tests which are migrated from SQE tests, the copyright date will follow the guidance SQE Test copyright year + migration copyright year if the 2 year are not same, for dump files (like *.dns) are new added under our new framework so just use current copyright year, hope that explains :), thanks >>>> >>>> Regards, >>>> Chris >>>> >>>>> Thanks, >>>>> Vyom >>>>> >>>>> On Thursday 12 July 2018 02:08 PM, Chris Yin wrote: >>>>>> Please have a review to new webrev as below, some code refactoring on lib parts and enhanced DNSServer to handle retry request which will make the tests more stable, thanks >>>>>> >>>>>> http://cr.openjdk.java.net/~xyin/8198882/webrev.01/ > >>>>>> >>>>>> Regards, >>>>>> Chris >>>>>> >>>>>>> On 22 Mar 2018, at 11:16 AM, Chris Yin >> wrote: >>>>>>> >>>>>>> Please review the changes to add 10 JNDI tests to com/sun/jndi/dns/AttributeTests/, thanks >>>>>>> >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8198882 > >>>>>>> webrev: http://cr.openjdk.java.net/~xyin/8198882/webrev.00/ > >>>>>>> >>>>>>> Regards, >>>>>>> Chris >>> >> > From xu.y.yin at oracle.com Wed Jul 25 09:11:29 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Wed, 25 Jul 2018 17:11:29 +0800 Subject: [12] RFR 8200151: Add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ Message-ID: <9FD6703A-49E8-41D8-B527-8D0D09726EB0@oracle.com> Please review the changes to add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ in OpenJDK, due to known issue 7164518, PortUnreachable.java will be problem list for 'macosx-all', thanks bug: https://bugs.openjdk.java.net/browse/JDK-8200151 webrev: http://cr.openjdk.java.net/~xyin/8200151/webrev.00/ Regards, Chris From enasser at in.ibm.com Wed Jul 25 10:52:47 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Wed, 25 Jul 2018 16:22:47 +0530 Subject: Add x-IBM-1129 charset In-Reply-To: <1ebd2f6516ee598a3a0be2f1eee99b0e@linux.vnet.ibm.com> References: <1ebd2f6516ee598a3a0be2f1eee99b0e@linux.vnet.ibm.com> Message-ID: Hello, I had kept aside this discussion for a while as there is a parallel discussion on whether to move the IBM charsets on openjdk to a different location or outside JDK[ http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054248.html ]. From that discussion, it is confirmed that the charsets in java.base will not be impacted irrespective the IBM charsets are moved to a new location or removed from openjdk. Hence, I believe we can proceed with this new charset as it requires to be in the java.base for AIX platform. Ichiroh has already reviewed and the tested the changes. Kindly request you to open a bug and review the changeset. Please let me know if you have any questions. > The webrev of the fix is available at > http://cr.openjdk.java.net/~aleonard/IBM1129/webrev.00/ Thank you, Nasser Ebrahim PS: Thank you Ichiroh for the testing and review. From: Ichiroh Takiguchi To: Nasser Ebrahim/India/IBM at IBMIN Cc: core-libs-dev at openjdk.java.net Date: 06/15/2018 12:59 PM Subject: Re: Add x-IBM-1129 charset Hello. I tested IBM-1129 charset on AIX platform. It worked fine for default encoding, $ LANG=Vi_VN ~/jdk/bin/java PrintDefaultCharset Vi_VN x-IBM1129 IBM-1129 IBM-1129 $ And also code table is fine. We would appreciate to open a bug/RFE for this change request. Ichiroh Takiguchi IBM Japan, Ltd. On 2018-05-30 10:41, Nasser Ebrahim wrote: > Hello, > > I just realized that I missed to provide the details of the test I have > conducted for the new charset IBM-1129. Please note that I have done > the > following jtreg tests to make sure the new charset is working properly. > > sun.nio.cs.TestCharsetMapping > java.nio.charset.Charset.RegisteredCharsets > sun.nio.cs.CheckHistoricalNamesTest > java.nio.charset.RemovingSunIO.SunioAlias > > Please let me know if I have to run any additional tests. > > Am aware of the suggestion from Alan and Thomas in the mail thread > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html > to move the IBM platform specific charsets to a separate module instead > of > adding to jdk.charsets. We will open another thread to discuss that > topic > as we would like to contribute many more extended charsets. However, I > believe we can still consider integrating this charset without waiting > for > the conclusion of that discussion as this charset is part of default > charset for AIX and needs to be part of the java.base. If we decided to > move the IBM charsets out of jdk.charsets, we can move this charsets as > well for non-AIX platforms. > > Kindly request you to open a bug/RFE for this change request, review > the > fix and provide your feedback. > > Thank you, > Nasser Ebrahim > > > From: Nasser Ebrahim/India/IBM > To: core-libs-dev at openjdk.java.net > Date: 05/19/2018 12:51 PM > Subject: Add x-IBM-1129 charset > > > > Hello, > > With the following three bugs, all the default locale charsets except > two > (Vi_VN.IBM-1129 & ja_JP.IBM-eucJP) are fixed for AIX platform. > > - JDK-8201540: [AIX] Extend the set of supported charsets in java.base > - JDK-8202329: Codepage mappings for IBM-943 and Big5 (aix) > - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053050.html > : [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to > stdcs > [bug not yet opened]. > > For those fixed charsets, the charsets were existing in the extended > charsets (jdk.charsets) and they were not working with default locale > charset as it did not exist in the standard charset (java.base). The > charsets correspond to the two pending locale (Vi_VN.IBM-1129 & > ja_JP.IBM-eucJP) does not exist in the jdk. They need to be added to > the > extended charsets before adding to stdcs on AIX platform. > > Here, am including the patch to fix the charset IBM-1129 for the > locale > Vi_VN.IBM-1129. We are working on the other missing charset (for > ja_JP.IBM-eucJP) which will be contributed in some time. > > The webrev of the fix is available at > http://cr.openjdk.java.net/~aleonard/IBM1129/webrev.00/ > > Kindly request you to open a bug and review the fix. Please let me know > if > you have any questions. > > Thank you, > Nasser Ebrahim From adinn at redhat.com Wed Jul 25 11:44:38 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 25 Jul 2018 12:44:38 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> Message-ID: <8e1fe763-c631-1060-e714-9f9b9481cfe2@redhat.com> Round 2 ------- I have updated the JEP and uploaded a revised webrev in the light of existing feedback JEP JIRA: https://bugs.openjdk.java.net/browse/JDK-8207851 Formatted JEP: http://openjdk.java.net/jeps/8207851 New webrev: http://cr.openjdk.java.net/~adinn/pmem/webrev.02/ I would welcome any further comments (I guess I'd be happy to push this as is but I find it hard to believe it does not require more work. Someone must have something to add :-) List of Changes --------------- I have followed Alan's request: - extending FileChannel.MapMode to add two new modes READ_ONLY_PERISTENT and READ_WRITE_PERSISTENT - retaining the single method FileChannel.map to handle the new modes (i.e. dumping the idea of a separate map_peristent API) - FileChannelImpl.map now handles the relevant XXX_PERSISTENT cases (rather than delegating to an internal method) I have followed Florian's advice: corrected spelling and format of the JEP, incuding made a clearer separation of description of API changes from implementation details modified the conditional compilation conditions and associated comments in FileChannelImpl.c to define MAP_SYNC and MAP_PRIVATE when the OS does not do so I have also brought the implementation of the pre-sync and post-sync operations in line with the libpmem code: On x86 pre-sync is a no-op post-sync is an no-op if clflush is used for writeback post-sync is an sfence if clflushopt or clwb is used for writeback On AArch64 pre-sync is a no-op post-sync is a dmb(ISH) Finally, I have updated the JEP text to reflect all the above changes. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adam.petcher at oracle.com Wed Jul 25 15:07:53 2018 From: adam.petcher at oracle.com (Adam Petcher) Date: Wed, 25 Jul 2018 11:07:53 -0400 Subject: Please review EdDSA API In-Reply-To: <449fc199-3bb8-8ada-0a23-520d30e7a4e5@oracle.com> References: <449fc199-3bb8-8ada-0a23-520d30e7a4e5@oracle.com> Message-ID: <3801e4b5-8df4-6201-6d3f-6689fdb7f00d@oracle.com> +core-libs-dev for additional API expertise. On 7/25/2018 10:29 AM, Adam Petcher wrote: > The draft CSR[1] for the EdDSA API[2] is ready for review. Please take > a look and send me any feedback you may have. Here are a few > high-level notes to explain the API: > > 1) Where possible, this API is similar to the API for X25519/X448. To > get the complete background/motivation for the API design, you can > review the discussion[3] on this topic. > 2) Similar to X25519/X448, private keys are byte arrays, and public > keys coordinates. Though we can't get by with a single BigInteger > coordinate for EdDSA, so I am using the new EdPoint class to hold the > coordinates. > 3) EdDSA has multiple signature modes defined in the RFC[4], including > some that "prehash" the input before signing. The draft API uses the > EdDSAParameterSpec class to specify parameters of these modes. The > standard does not allow an arbitrary choice of prehash function, so > the API for EdDSA does not support algorithm names like > "SHA256withEdDSA". > > [1] https://wiki.openjdk.java.net/display/csr/Main > [2] https://bugs.openjdk.java.net/browse/JDK-8190219 > [3] > http://mail.openjdk.java.net/pipermail/security-dev/2017-September/016325.html > [4] https://tools.ietf.org/html/rfc8032 > From alexey.semenyuk at oracle.com Wed Jul 25 16:00:04 2018 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 25 Jul 2018 12:00:04 -0400 Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] In-Reply-To: <6f33f11d7423409c905a94dca38ecd1a@DEISMMXS02.pt.local> References: <8aee154ce3b846d79a926e6695844b1a@DEISMMXS02.pt.local> <75106ab90b974292890c71b6d320b56c@DEISMMXS02.pt.local> <7c2de54d-793d-77a9-556e-34f5871bdcc8@oracle.com> <6f33f11d7423409c905a94dca38ecd1a@DEISMMXS02.pt.local> Message-ID: <1a688ca2-2fdb-0555-6f24-696495be3c11@oracle.com> J?rg, I don't think it will be very much work to bring service bundler code from JFX packager into OpenJDK jpackager. Though I can't give you estimates on amount of work needed to be done for this at the moment. - Alexey On 7/23/2018 7:47 AM, Buchberger, Joerg wrote: > Thanks for the insight. > > @Alexey: Then, how much work do you see in reactivating service bundler? > > Cheers > J?rg > > > > -----Original Message----- > From: Kevin Rushforth [mailto:kevin.rushforth at oracle.com] > Sent: Donnerstag, 12. Juli 2018 01:09 > To: Buchberger, Joerg ; core-libs-dev at openjdk.java.net > Cc: Alexey Semenyuk ; Andy Herrick > Subject: Re: Prototype of jpackager in jdk/sandbox [was: Draft JEP proposal: JDK-8200758: Packaging Tool] > > We will likely be able to deliver the .exe installer (with its dependency on Inno Setup). > > As for the service bundler, this will be a "nice to have" (a stretch > goal) for this version, but isn't on the list of committed features. > Alexsei might be able to comment further on how much work it would be to provide it, including documenting and testing it. This might give you, and other interested developers, a sense of how likely this is to make it for this version. > > -- Kevin > > > On 7/10/2018 4:35 AM, Buchberger, Joerg wrote: >> Hi >> >> thanks for the update/info. I had a quick look at the changes. So, here some thoughts... >> >> As described in JDK-8200758, and therefore expected, WinExeBundler has been removed in favor of putting focus on WinMsiBundler. >> (Although, I regret that decision - since my personal experience has >> been that InnoSetup based WinExeBundler has worked much better than >> wix based WinMsiBundler for our use cases - I can live with that.) >> >> What is much more disturbing: WinServiceBundler has also been actively removed, although that was working fine together with both wix/msi and exe/iss. Why has service wrapping been removed as well, while the command line option for it is kept in place? >> >> Is there any chance of service bundler coming back into scope of JDK-8200758 or coming back in at all? >> >> Cheers >> J?rg >> >> >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of Kevin Rushforth >> Sent: Freitag, 6. Juli 2018 22:14 >> To: core-libs-dev at openjdk.java.net >> Cc: Alexey Semenyuk ; Andy Herrick >> >> Subject: Prototype of jpackager in jdk/sandbox [was: Draft JEP >> proposal: JDK-8200758: Packaging Tool] >> >> An initial prototype of the jpackager tool has been pushed to a new 'JDK-8200758-branch' branch in the JDK sandbox [1]. If anyone is interested in taking a look, you can clone it as follows: >> >> ??? hg clone https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.net_jdk_sandbox&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=jWWENz_KIkmyh-9-kQQvoZ0BwBymwQ-BKx8hG3F5Iy0&e= >> ??? cd ./sandbox >> ??? hg update JDK-8200758-branch >> >> I plan to reply to the feedback already provided, and update the JEP [2] next week some time, but in the mean time if you have additional questions or comments, feel free to reply. >> >> -- Kevin >> >> [1] >> https://urldefense.proofpoint.com/v2/url?u=http-3A__hg.openjdk.java.ne >> t_jdk_sandbox_shortlog_JDK-2D8200758-2Dbranch&d=DwIFaQ&c=uD3W7j5M6i1jD >> eSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJK >> hVEy_Q&m=7aoiG26qKHqhAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=F-CqPAWlz-Cfb0k >> ae2FBEj4Ncd3ZBVu7BeOVY1AM-cA&e= [2] >> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java >> .net_browse_JDK-2D8200758&d=DwIFaQ&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxR >> W_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=7aoiG26qKHq >> hAG4Ry-hOl_c8cZ2UdmcCtrya0JOnsgg&s=DFIAHtCR1o--KMLuBzurIzx5MDu67NgtUrE >> dQ22wI9I&e= >> >> >> On 6/27/2018 3:30 PM, Kevin Rushforth wrote: >>> We're aiming to get this into JDK 12 early enough so that an EA build >>> would be available around the time JDK 11 ships. That will allow you >>> to take a jlinked image with JDK 11 and package it up using (the EA) >>> jpackager. >>> >>> We will create a development branch in the JDK sandbox [1] some time >>> in the next week or so so you can follow the development. >>> >>> Also, thank you to those who have provided feedback. I'll reply to >>> feedback soon and then incorporate it into an updated JEP. >>> >>> -- Kevin From rachna.goel at oracle.com Wed Jul 25 17:03:12 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Wed, 25 Jul 2018 22:33:12 +0530 Subject: [11]RFR:JDK-8206965: java/util/TimeZone/Bug8149452.java failed on de_DE and ja_JP locale. Message-ID: <174d08ae-e6d6-af55-e814-096c7f6a58be@oracle.com> Hi, Kindly review fix for JDK-8206965. Bug: https://bugs.openjdk.java.net/browse/JDK-8206965 patch: http://cr.openjdk.java.net/~rgoel/JDK-8206965/webrev.06/ This is a regression caused by JDK-8181157 due to which for some locales, timezone display names were not getting generated. Generating fallback names for missing timezones at runtime would solve this issue. -- Thanks, Rachna From naoto.sato at oracle.com Wed Jul 25 17:46:30 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 25 Jul 2018 10:46:30 -0700 Subject: [11]RFR:JDK-8206965: java/util/TimeZone/Bug8149452.java failed on de_DE and ja_JP locale. In-Reply-To: <174d08ae-e6d6-af55-e814-096c7f6a58be@oracle.com> References: <174d08ae-e6d6-af55-e814-096c7f6a58be@oracle.com> Message-ID: <87966df3-e147-f177-096a-d293ee3e8b03@oracle.com> Looks good. Naoto On 7/25/18 10:03 AM, Rachna Goel wrote: > Hi, > > Kindly review fix for JDK-8206965. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8206965 > patch: http://cr.openjdk.java.net/~rgoel/JDK-8206965/webrev.06/ > > This is a regression caused by JDK-8181157 due to which for some > locales,? timezone display names were not getting generated. > Generating fallback names for missing timezones at runtime would solve > this issue. > From kevin.rushforth at oracle.com Wed Jul 25 23:12:30 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 25 Jul 2018 16:12:30 -0700 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> Message-ID: <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> Thank you to all who provided feedback. I have updated the draft JEP [1], and I think I have incorporated most of the feedback I received. Specifically, I have reorganized and reworded a few things for clarity, added '.exe' and '.app in a .dmg' native package format to the list of features, and added the service bundler (daemon) feature to the "if we have time" list (at the top of that list). Please let me know if I missed an important point. I hope to submit this JEP in the next week or two. -- Kevin [1] https://bugs.openjdk.java.net/browse/JDK-8200758 On 5/30/2018 5:10 PM, Kevin Rushforth wrote: > I would like to propose the following Draft JEP [1] for discussion. > > JDK-8200758: Packaging Tool > > This is intended as a JDK-scope replacement for the existing > javapackager tool that ships with Oracle JDK 10 (and earlier Oracle > JDK releases), and was delivered as part of the JavaFX build. The > javapackager tool has been removed from Oracle JDK 11 along with the > removal of JavaFX. > > Comments on this JEP are welcome. It is currently not targeted for a > specific release, but we are aiming for JDK 12. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-8200758 > From ecki at zusammenkunft.net Thu Jul 26 02:32:09 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 26 Jul 2018 02:32:09 +0000 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com>, <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> Message-ID: Hello, Will there be: - support for displaying and asking for acceptance of license text - asking for target directory (installing outside of programfiles) - starting optional post-Installation class - menu entries - user vs. System wide Installation (on windows) - the new launchers also be available as jlink plugin - theming Installer Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: -974640832m Auftrag von Gesendet: Donnerstag, Juli 26, 2018 2:12 AM An: core-libs-dev at openjdk.java.net Betreff: Re: Draft JEP proposal: JDK-8200758: Packaging Tool Thank you to all who provided feedback. I have updated the draft JEP [1], and I think I have incorporated most of the feedback I received. Specifically, I have reorganized and reworded a few things for clarity, added '.exe' and '.app in a .dmg' native package format to the list of features, and added the service bundler (daemon) feature to the "if we have time" list (at the top of that list). Please let me know if I missed an important point. I hope to submit this JEP in the next week or two. -- Kevin [1] https://bugs.openjdk.java.net/browse/JDK-8200758 On 5/30/2018 5:10 PM, Kevin Rushforth wrote: > I would like to propose the following Draft JEP [1] for discussion. > > JDK-8200758: Packaging Tool > > This is intended as a JDK-scope replacement for the existing > javapackager tool that ships with Oracle JDK 10 (and earlier Oracle > JDK releases), and was delivered as part of the JavaFX build. The > javapackager tool has been removed from Oracle JDK 11 along with the > removal of JavaFX. > > Comments on this JEP are welcome. It is currently not targeted for a > specific release, but we are aiming for JDK 12. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-8200758 > From Alan.Bateman at oracle.com Thu Jul 26 07:15:42 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 26 Jul 2018 08:15:42 +0100 Subject: Add x-IBM-1129 charset In-Reply-To: References: <1ebd2f6516ee598a3a0be2f1eee99b0e@linux.vnet.ibm.com> Message-ID: On 25/07/2018 11:52, Nasser Ebrahim wrote: > Hello, > > I had kept aside this discussion for a while as there is a parallel > discussion on whether to move the IBM charsets on openjdk to a different > location or outside JDK[ > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054248.html > ]. From that discussion, it is confirmed that the charsets in java.base > will not be impacted irrespective the IBM charsets are moved to a new > location or removed from openjdk. Hence, I believe we can proceed with > this new charset as it requires to be in the java.base for AIX platform. and to be clear, I think what you are doing here is adding IBM-1129 to java.base on AIX builds and adding it to jdk.charsets on other platforms. There isn't any build infrastructure at this time to do otherwise and maybe that is something that will come out of the other discussion thread. -Alan From xu.y.yin at oracle.com Thu Jul 26 07:37:52 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Thu, 26 Jul 2018 15:37:52 +0800 Subject: [12] RFR 8208279: Add 8 JNDI tests to com/sun/jndi/dns/EnvTests/ Message-ID: <0EFABB87-51B9-4454-85DF-33C49C7BF805@oracle.com> Please review the changes to add another 8 JNDI tests to com/sun/jndi/dns/EnvTests/ in OpenJDK, thanks bug: https://bugs.openjdk.java.net/browse/JDK-8208279 webrev: http://cr.openjdk.java.net/~xyin/8208279/webrev.00/ Regards, Chris From nishit.jain at oracle.com Thu Jul 26 08:34:15 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Thu, 26 Jul 2018 14:04:15 +0530 Subject: [12] RFR 8021322: [Fmt-Ch] Implementation of ChoiceFormat math methods should delegate to java.lang.Math methods Message-ID: Hi, Please review the fix for JDK-8021322. CSR for the spec changes has been approved. Bug: https://bugs.openjdk.java.net/browse/JDK-8021322 Webrev: http://cr.openjdk.java.net/~nishjain/8021322/webrev.01/ CSR: https://bugs.openjdk.java.net/browse/JDK-8208071 Fix: Changed the ChoiceFormat's nextDouble(double), previousDouble(double), nextDouble(double, boolean) APIs implementation to delegate the execution to the equivalent Math APIs Regards, Nishit Jain From norman.maurer at googlemail.com Thu Jul 26 08:35:24 2018 From: norman.maurer at googlemail.com (Norman Maurer) Date: Thu, 26 Jul 2018 10:35:24 +0200 Subject: Consider making TerminatingThreadLocal public In-Reply-To: <3BAB12F3-CCBC-4286-99C1-B59C4B4635FB@googlemail.com> References: <3BAB12F3-CCBC-4286-99C1-B59C4B4635FB@googlemail.com> Message-ID: So I take this as a sign that there is no interest in this ? Bye Norman > On 10. Jul 2018, at 12:06, Norman Maurer wrote: > > Hi all, > > It came to my attention that you introduced TerminatingThreadLocal in openjdk lately as an ?internal class?: > > http://hg.openjdk.java.net/jdk/jdk/rev/106dc156ce6b > > I would like to take the chance to ask if you ever considered to make this ?non-internal? as I think it would super helpful for some use cases. To give you some more context let me give you an example of why we would love it in netty. > > So in netty we sometimes pool ?direct memory? in ThreadLocals [1] and have to do some tricks to ensure this direct memory is really released once a Thread exit. Which I think is also exactly one use-case you have here. Sure we can do this with a WeakReference + ReferenceQueue , Cleaner or finalizer but I feel exposing TerminatingThreadLocal would be a lot more elegant. Another advantage would be that we can be sure that the cleanup action would be called in the Thread that holds the value of the ThreadLocal, which is not really possible with using WeakReference + ReferenceQueue , Cleaner or finalizer. > > What we currently doing in netty is basically have our own ThreadLocal [1] implementation and out own Thread sub-class [2] that will ensure we call some method on our TheadLocal that gives us the chance to do the cleanup by wrapping the Runnable[3] that is passed in. This works great but we can not always guarantee that our sub-class of Thread is used to access our ThreadLocal implementation, which means the cleanup action will not run in this case. > > Thanks, > Norman > > [1] https://github.com/netty/netty/blob/netty-4.1.26.Final/common/src/main/java/io/netty/util/concurrent/FastThreadLocal.java > [2] https://github.com/netty/netty/blob/netty-4.1.26.Final/common/src/main/java/io/netty/util/concurrent/FastThreadLocalThread.java > [3] https://github.com/netty/netty/blob/netty-4.1.26.Final/common/src/main/java/io/netty/util/concurrent/FastThreadLocalRunnable.java > > > > > > From enasser at in.ibm.com Thu Jul 26 09:26:06 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Thu, 26 Jul 2018 14:56:06 +0530 Subject: Add x-IBM-1129 charset In-Reply-To: References: <1ebd2f6516ee598a3a0be2f1eee99b0e@linux.vnet.ibm.com> Message-ID: Thank you Alan for your response. > and to be clear, I think what you are doing here is adding IBM-1129 to > java.base on AIX builds and adding it to jdk.charsets on other > platforms. There isn't any build infrastructure at this time to do > otherwise and maybe that is something that will come out of the other > discussion thread. Yes. Currently, it will be added to jdk.charsets for non-AIX platforms. Only two charsets which has to be in java.base on AIX platform is currently missing [Vi_VN.IBM-1129 & ja_JP.IBM-eucJP]. We are trying to add those two charsets so that openjdk can start on all locales for AIX-platforms. Once we decided the approach for IBM charsets on jdk.charsets in the other discussion thread, we can move all the IBM charsets (including these two new ones) to the new location. Hope that is acceptable. Please advise. Thank you, Nasser Ebrahim From: Alan Bateman To: Nasser Ebrahim , Ichiroh Takiguchi , core-libs-dev at openjdk.java.net Date: 07/26/2018 12:45 PM Subject: Re: Add x-IBM-1129 charset On 25/07/2018 11:52, Nasser Ebrahim wrote: > Hello, > > I had kept aside this discussion for a while as there is a parallel > discussion on whether to move the IBM charsets on openjdk to a different > location or outside JDK[ > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054248.html > ]. From that discussion, it is confirmed that the charsets in java.base > will not be impacted irrespective the IBM charsets are moved to a new > location or removed from openjdk. Hence, I believe we can proceed with > this new charset as it requires to be in the java.base for AIX platform. and to be clear, I think what you are doing here is adding IBM-1129 to java.base on AIX builds and adding it to jdk.charsets on other platforms. There isn't any build infrastructure at this time to do otherwise and maybe that is something that will come out of the other discussion thread. -Alan From vyom.tewari at oracle.com Thu Jul 26 09:27:56 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Thu, 26 Jul 2018 14:57:56 +0530 Subject: [12] RFR 8200151: Add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ In-Reply-To: <9FD6703A-49E8-41D8-B527-8D0D09726EB0@oracle.com> References: <9FD6703A-49E8-41D8-B527-8D0D09726EB0@oracle.com> Message-ID: Hi Chris, Thanks for writing the tests overall code looks good to me, below are few minor comments. 1`-> Fix tag order, my Netbeans always complains :) . 2->? you make AuthRecursiveBase class as public but i can not see it is being used outside,? i will suggest you to give the default access if possible. 3-> AuthTrue.java, change the message as follows "Got exception as expected : " -> "Got expected exception: "; 4-> PortUnreachable.java , any specific reason for 1000ms or you just choose random value ? Please don't hard-code this specific value? create a? constant and use it . If possible put some comment why we choose 1 second, this will help in debugging if this test fails intermittently in future. 5-> Timeout.java, do you really need to set the env using "DNSTestUtils.initEnv" ? I see, this test is not using the DNSServer and other environments variables which were set by "DNSTestUtils.initEnv()" or i am missing something. Thanks, Vyom On Wednesday 25 July 2018 02:41 PM, Chris Yin wrote: > Please review the changes to add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ in OpenJDK, due to known issue 7164518, PortUnreachable.java will be problem list for 'macosx-all', thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8200151 > webrev: http://cr.openjdk.java.net/~xyin/8200151/webrev.00/ > > Regards, > Chris From adinn at redhat.com Thu Jul 26 09:45:54 2018 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 26 Jul 2018 10:45:54 +0100 Subject: Consider making TerminatingThreadLocal public In-Reply-To: References: <3BAB12F3-CCBC-4286-99C1-B59C4B4635FB@googlemail.com> Message-ID: Hi Norman, On 26/07/18 09:35, Norman Maurer wrote: > So I take this as a sign that there is no interest in this ? I'm not sure that is the right conclusion to draw. It may well be that those involved in making the original change are too busy to respond right now. However, there are a two other important considerations worth bearing in mind. Firstly, if you look back through the core-libs-dev archive you will find discussion that preceded introduction of this change -- primarily between Alan Bateman and Tony Printezis as I recall but I think others were involved. I believe that discussion included debate as to how this change ought to expose the new capability either as a restricted API (internal to JDK runtime) or as an external API (open to JDK client apps). Its clear what the conclusion was. Have you checked the archive? If not I would advise doing so and then coming up with counter-arguments in any argument you present. That might encourage those who were involved in the discussion to consider your suggestion. Secondly, I am afraid you are a bit late to the party with this post. I realise that it is a genuine suggestion for improvement and not simply a drive-by request for others to make your life easier. However, it is hard for developers to return to a fix that is done and dusted and meets the agreed remit because of late-expressed concerns. By that stage they will be busy chasing other important new tasks. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Thu Jul 26 10:08:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 26 Jul 2018 11:08:38 +0100 Subject: Consider making TerminatingThreadLocal public In-Reply-To: References: <3BAB12F3-CCBC-4286-99C1-B59C4B4635FB@googlemail.com> Message-ID: On 26/07/2018 10:45, Andrew Dinn wrote: > : > Firstly, if you look back through the core-libs-dev archive you will > find discussion that preceded introduction of this change -- primarily > between Alan Bateman and Tony Printezis as I recall but I think others > were involved. I believe that discussion included debate as to how this > change ought to expose the new capability either as a restricted API > (internal to JDK runtime) or as an external API (open to JDK client > apps). Its clear what the conclusion was. > Thanks Andrew. Yes, there was lengthy discussion here, mostly Tony, Peter Levart and I. We decided not to expose anything at this time, an approach that does not preclude re-visiting it in the future. In general it would need a lot of consideration, esp. when you think about cases where a thread local maintains an object with a reference to a native resource and the object is reachable by other means. We also need to think about what it would mean further down the road if/when the platform is retrofitted to support fibers as there will have implications for what we know today as thread locals. -Alan From swpalmer at gmail.com Thu Jul 26 14:38:56 2018 From: swpalmer at gmail.com (Scott Palmer) Date: Thu, 26 Jul 2018 10:38:56 -0400 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> Message-ID: <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> "The input to jpackager includes: a Java runtime image, and a Java application in one of several formats..." Will it be possible to NOT include the JRE, but specify instead a pre-existing location for the JRE? As an example use-case consider an office productivity suite where there are separate installers for a word processor and a spreadsheet application. These applications are independent and can be installed in any combination (word processor only, both, spreadsheet only). However they are part of the same versioned application suite and have been developed and tested with a particular JRE. Only a single JRE needs to be installed. The applications can share it. This is not the same as using a system-wide JRE (is that even supported for Java 11 and beyond?). But a shared private JRE controlled by the application developer. This is similar but not the same as the proposed "Multiple launchers" feature (if time allows), as the shared JRE could be used by different software packages. In many cases the JRE is a very large part of the software installation, both in terms of the size of the distributed installer package and the storage requirements of the installed application. JRE sharing can help with this. I'm thinking that eventually we could get to the point where developers could treat the JRE as a versioned dependency, also covering the case of customized JRE images. I don't propose that this jpackager tool be involved in creating or distributing such JRE images, only that it supports running applications using a pre-installed JRE where the location can be determined at installer build time or perhaps install time. This was possible with the javapackager tool. Scott > On Jul 25, 2018, at 7:12 PM, Kevin Rushforth wrote: > > Thank you to all who provided feedback. I have updated the draft JEP [1], and I think I have incorporated most of the feedback I received. Specifically, I have reorganized and reworded a few things for clarity, added '.exe' and '.app in a .dmg' native package format to the list of features, and added the service bundler (daemon) feature to the "if we have time" list (at the top of that list). > > Please let me know if I missed an important point. I hope to submit this JEP in the next week or two. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-8200758 > > > On 5/30/2018 5:10 PM, Kevin Rushforth wrote: >> I would like to propose the following Draft JEP [1] for discussion. >> >> JDK-8200758: Packaging Tool >> >> This is intended as a JDK-scope replacement for the existing javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK releases), and was delivered as part of the JavaFX build. The javapackager tool has been removed from Oracle JDK 11 along with the removal of JavaFX. >> >> Comments on this JEP are welcome. It is currently not targeted for a specific release, but we are aiming for JDK 12. >> >> -- Kevin >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >> > From naoto.sato at oracle.com Thu Jul 26 16:26:39 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 26 Jul 2018 09:26:39 -0700 Subject: [12] RFR 8021322: [Fmt-Ch] Implementation of ChoiceFormat math methods should delegate to java.lang.Math methods In-Reply-To: References: Message-ID: <77c2ebf2-4743-3ac3-be98-e282a132623b@oracle.com> Looks good. Naoto On 7/26/18 1:34 AM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8021322. CSR for the spec changes has been > approved. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8021322 > Webrev: http://cr.openjdk.java.net/~nishjain/8021322/webrev.01/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8208071 > > Fix: Changed the ChoiceFormat's nextDouble(double), > previousDouble(double), nextDouble(double, boolean) APIs implementation > to delegate the execution to the equivalent Math APIs > > Regards, > Nishit Jain From xueming.shen at oracle.com Thu Jul 26 17:04:33 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 26 Jul 2018 10:04:33 -0700 Subject: Add x-IBM-1129 charset In-Reply-To: References: Message-ID: <5B59FF21.8010808@oracle.com> The change looks fine. Btw, do you guy have better implementation for these two charsets? These two were left untouched when I did the reimplementation back to 6/7. Did not have time to figure out its mapping table (especially the compatibility concern when dealing with the mapping hard-coded inside the source and those maps from cdc repository). This was the main reason I did not move the SimpleEUCEncoder into the sun.nio.cs.DoubleByte.java. -Sherman On 5/19/18, 12:56 AM, Nasser Ebrahim wrote: > Hello, > > With the following three bugs, all the default locale charsets except two > (Vi_VN.IBM-1129& ja_JP.IBM-eucJP) are fixed for AIX platform. > > - JDK-8201540: [AIX] Extend the set of supported charsets in java.base > - JDK-8202329: Codepage mappings for IBM-943 and Big5 (aix) > - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053050.html > : [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs > [bug not yet opened]. > > For those fixed charsets, the charsets were existing in the extended > charsets (jdk.charsets) and they were not working with default locale > charset as it did not exist in the standard charset (java.base). The > charsets correspond to the two pending locale (Vi_VN.IBM-1129& > ja_JP.IBM-eucJP) does not exist in the jdk. They need to be added to the > extended charsets before adding to stdcs on AIX platform. > > Here, am including the patch to fix the charset IBM-1129 for the locale > Vi_VN.IBM-1129. We are working on the other missing charset (for > ja_JP.IBM-eucJP) which will be contributed in some time. > > The webrev of the fix is available at > http://cr.openjdk.java.net/~aleonard/IBM1129/webrev.00/ > > Kindly request you to open a bug and review the fix. Please let me know if > you have any questions. > > Thank you, > Nasser Ebrahim > From brian.burkhalter at oracle.com Thu Jul 26 20:13:02 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 26 Jul 2018 13:13:02 -0700 Subject: [12] (AIX) 8207744: Clean up inconsistent use of opendir/closedir versus opendir64/closedir64 Message-ID: https://bugs.openjdk.java.net/browse/JDK-8207744 http://cr.openjdk.java.net/~bpb/8207744/webrev.00/ This change simply adds #if defined(_AIX) #define DIR DIR64 #define opendir opendir64 #define closedir closedir64 #endif to each of src/java.base/share/native/libjli/wildcard.c src/java.base/unix/native/libjava/ProcessHandleImpl_unix.c src/java.base/unix/native/libjava/TimeZone_md.c If someone who builds OpenJDK on AIX would try this out that would be great as I can only verify that it does not cause problems on the platforms that we support (currently in progress). Thanks, Brian From joe.darcy at oracle.com Thu Jul 26 20:35:48 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 26 Jul 2018 13:35:48 -0700 Subject: [12] RFR 8021322: [Fmt-Ch] Implementation of ChoiceFormat math methods should delegate to java.lang.Math methods In-Reply-To: <77c2ebf2-4743-3ac3-be98-e282a132623b@oracle.com> References: <77c2ebf2-4743-3ac3-be98-e282a132623b@oracle.com> Message-ID: <5B5A30A4.4080902@oracle.com> Looks fine; cheers, -Joe On 7/26/2018 9:26 AM, Naoto Sato wrote: > Looks good. > > Naoto > > On 7/26/18 1:34 AM, Nishit Jain wrote: >> Hi, >> >> Please review the fix for JDK-8021322. CSR for the spec changes has >> been approved. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8021322 >> Webrev: http://cr.openjdk.java.net/~nishjain/8021322/webrev.01/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8208071 >> >> Fix: Changed the ChoiceFormat's nextDouble(double), >> previousDouble(double), nextDouble(double, boolean) APIs >> implementation to delegate the execution to the equivalent Math APIs >> >> Regards, >> Nishit Jain From paul.sandoz at oracle.com Fri Jul 27 00:02:18 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 26 Jul 2018 17:02:18 -0700 Subject: Long chains created by direct Buffer::slice In-Reply-To: <20180720111559.849741094@eggemoggin.niobe.net> References: <87o9f1n653.fsf@mid.deneb.enyo.de> <20180720111559.849741094@eggemoggin.niobe.net> Message-ID: <029E054A-7238-4271-AAC5-7904E86DB3D6@oracle.com> +1 I created this issue: https://bugs.openjdk.java.net/browse/JDK-8208362 The suggested fix requires a tweak though since an instance of a DirectBuffer interface is passed. This is required because views over direct ByteBuffers can be created. Florian, i will push tomorrow and assign you as at the contributor. Paul. diff -r 448cd909c9e2 src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Jul 26 11:53:59 2018 -0700 +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Jul 26 16:46:20 2018 -0700 @@ -194,7 +194,8 @@ #if[byte] cleaner = null; #end[byte] - att = db; + Object attachment = db.attachment(); + att = (attachment == null ? db : attachment); #else[rw] super(db, mark, pos, lim, cap, off); this.isReadOnly = true; > On Jul 20, 2018, at 11:15 AM, mark.reinhold at oracle.com wrote: > > 2018/7/20 10:55:52 -0700, Florian Weimer : >> ... >> >> The constructor invoked: >> >> | // For duplicates and slices >> | // >> | Direct$Type$Buffer$RW$$BO$(DirectBuffer db, // package-private >> | int mark, int pos, int lim, int cap, >> | int off) >> | { >> | #if[rw] >> | super(mark, pos, lim, cap); >> | address = db.address() + off; >> | #if[byte] >> | cleaner = null; >> | #end[byte] >> | att = db; >> | #else[rw] >> | super(db, mark, pos, lim, cap, off); >> | this.isReadOnly = true; >> | #end[rw] >> | } >> >> The key part is the assignment to the att member. If I understand >> this correctly, it is needed to keep the backing object alive during >> the lifetime of this buffer. > > Correct. > >> However, it causes the creation of a >> long chain of buffer objects. With -Xmx100m or so, the following test >> will OOM fairly quickly for this reason: >> >> | volatile ByteBuffer buffer; >> | ? >> | buffer = ByteBuffer.allocateDirect(16384); >> | while (true) { >> | buffer = buffer.duplicate(); >> | } > > Well spotted! This bug has been lurking there for sixteen years. > >> >> I wonder if it would be possible to change the setting of the att >> member to this instead: >> >> | if (db.att == null) { >> | att = db; >> | } else { >> | att = db.att; >> | } >> >> This would only keep the object alive which actually owns the backing >> storage, as if Buffer::slice had been invoked on it directly. > > Your suggested fix looks fine. > > - Mark From xu.y.yin at oracle.com Fri Jul 27 07:42:57 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Fri, 27 Jul 2018 15:42:57 +0800 Subject: [12] RFR 8200151: Add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ In-Reply-To: References: <9FD6703A-49E8-41D8-B527-8D0D09726EB0@oracle.com> Message-ID: <8961A4DD-5119-46BD-B2B4-A3162302CAF7@oracle.com> Hi, Vyom Thank a lot for your review and comments, inline and update new webrev as below, thanks http://cr.openjdk.java.net/~xyin/8200151/webrev.01/ > On 26 Jul 2018, at 5:27 PM, vyom tewari wrote: > > Hi Chris, > > Thanks for writing the tests overall code looks good to me, below are few minor comments. > > 1`-> Fix tag order, my Netbeans always complains :) . Fixed, thanks > > 2-> you make AuthRecursiveBase class as public but i can not see it is being used outside, i will suggest you to give the default access if possible. Sure, changed it to default access now > > 3-> AuthTrue.java, change the message as follows > > "Got exception as expected : " -> "Got expected exception: ?; Fixed > > > 4-> PortUnreachable.java , any specific reason for 1000ms or you just choose random value ? Please don't hard-code this specific value create a constant and use it . If possible put some comment why we choose 1 second, this will help in debugging if this test fails intermittently in future. Sure, I created a constant ?THRESHOLD' for it, 1000ms will be used as a threshold value for this test, normally the request should fail very quick (in a few ms), but consider to different platform and test machine performance, we used an acceptable threshold here, some comments added to it in the code too, thanks > > 5-> Timeout.java, do you really need to set the env using "DNSTestUtils.initEnv" ? > > I see, this test is not using the DNSServer and other environments variables which were set by "DNSTestUtils.initEnv()" or i am missing something. Actually, it?s indeed not necessary, just set very few default value such as ?Context.INITIAL_CONTEXT_FACTORY? in the test code should be fine, but I may still want to keep the test consistency, then we could have better maintainability for those tests. Sorry, I forgot to refactor this test with testbase (actually, its not necessary too, but guess the consistent style and shared base/methods will make the test easy to read and maintain, refactored this test and PortUnreachable.java now) Regards, Chris > > Thanks, > Vyom > > On Wednesday 25 July 2018 02:41 PM, Chris Yin wrote: >> Please review the changes to add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ in OpenJDK, due to known issue 7164518, PortUnreachable.java will be problem list for 'macosx-all', thanks >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8200151 >> webrev: http://cr.openjdk.java.net/~xyin/8200151/webrev.00/ >> >> Regards, >> Chris > From fw at deneb.enyo.de Fri Jul 27 08:09:24 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Fri, 27 Jul 2018 10:09:24 +0200 Subject: Long chains created by direct Buffer::slice In-Reply-To: <029E054A-7238-4271-AAC5-7904E86DB3D6@oracle.com> (Paul Sandoz's message of "Thu, 26 Jul 2018 17:02:18 -0700") References: <87o9f1n653.fsf@mid.deneb.enyo.de> <20180720111559.849741094@eggemoggin.niobe.net> <029E054A-7238-4271-AAC5-7904E86DB3D6@oracle.com> Message-ID: <87tvolksln.fsf@mid.deneb.enyo.de> * Paul Sandoz: > I created this issue: > > https://bugs.openjdk.java.net/browse/JDK-8208362 Thanks. > The suggested fix requires a tweak though since an instance of a > DirectBuffer interface is passed. This is required because views > over direct ByteBuffers can be created. Noticed that as well. > diff -r 448cd909c9e2 src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template > --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Jul 26 11:53:59 2018 -0700 > +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Jul 26 16:46:20 2018 -0700 > @@ -194,7 +194,8 @@ > #if[byte] > cleaner = null; > #end[byte] > - att = db; > + Object attachment = db.attachment(); > + att = (attachment == null ? db : attachment); This is essentially what I put through jtreg (jdk_core), without any obvious issues, but I had not time yet to file a bug and create a webrev. The parenthesis seem unnecessary. If this is an official JDK coding style, it is not widely used. From xu.y.yin at oracle.com Fri Jul 27 08:42:09 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Fri, 27 Jul 2018 16:42:09 +0800 Subject: [12] RFR 8208363: test/jdk/java/lang/Package/PackageFromManifest.java missing module dependencies declaration Message-ID: <4977474F-3B24-43E8-B6EB-1B9984C1DB6B@oracle.com> Please review below minor change to add missing module dependencies declaration, thanks bug: https://bugs.openjdk.java.net/browse/JDK-8208363 Changes: diff -r 65fc31d1042b test/jdk/java/lang/Package/PackageFromManifest.java --- a/test/jdk/java/lang/Package/PackageFromManifest.java Thu Jul 26 16:22:58 2018 -0700 +++ b/test/jdk/java/lang/Package/PackageFromManifest.java Fri Jul 27 15:46:34 2018 +0800 @@ -29,6 +29,7 @@ * same package if multiple jars). Then verify package versioning info * @library /lib/testlibrary * @library /test/lib + * @modules jdk.compiler * @run main PackageFromManifest setup test * @run main PackageFromManifest runJar test1.jar * @run main PackageFromManifest runJar test1.jar test2.jar foo.Foo1 Regards, Chris From Joerg.Buchberger at pruftechnik.com Fri Jul 27 11:59:04 2018 From: Joerg.Buchberger at pruftechnik.com (Buchberger, Joerg) Date: Fri, 27 Jul 2018 11:59:04 +0000 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> Message-ID: Thanks for the info - I was not aware of these possibilities in javapackager. @Kevin: just fyi, this would be also a very useful feature for the company I work for and for our projects @Scott: please, let me know, how to achieve this with javapackager Cheers J?rg -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Scott Palmer Sent: Donnerstag, 26. Juli 2018 16:39 To: Kevin Rushforth Cc: core-libs-dev at openjdk.java.net Subject: Re: Draft JEP proposal: JDK-8200758: Packaging Tool "The input to jpackager includes: a Java runtime image, and a Java application in one of several formats..." Will it be possible to NOT include the JRE, but specify instead a pre-existing location for the JRE? As an example use-case consider an office productivity suite where there are separate installers for a word processor and a spreadsheet application. These applications are independent and can be installed in any combination (word processor only, both, spreadsheet only). However they are part of the same versioned application suite and have been developed and tested with a particular JRE. Only a single JRE needs to be installed. The applications can share it. This is not the same as using a system-wide JRE (is that even supported for Java 11 and beyond?). But a shared private JRE controlled by the application developer. This is similar but not the same as the proposed "Multiple launchers" feature (if time allows), as the shared JRE could be used by different software packages. In many cases the JRE is a very large part of the software installation, both in terms of the size of the distributed installer package and the storage requirements of the installed application. JRE sharing can help with this. I'm thinking that eventually we could get to the point where developers could treat the JRE as a versioned dependency, also covering the case of customized JRE images. I don't propose that this jpackager tool be involved in creating or distributing such JRE images, only that it supports running applications using a pre-installed JRE where the location can be determined at installer build time or perhaps install time. This was possible with the javapackager tool. Scott > On Jul 25, 2018, at 7:12 PM, Kevin Rushforth wrote: > > Thank you to all who provided feedback. I have updated the draft JEP [1], and I think I have incorporated most of the feedback I received. Specifically, I have reorganized and reworded a few things for clarity, added '.exe' and '.app in a .dmg' native package format to the list of features, and added the service bundler (daemon) feature to the "if we have time" list (at the top of that list). > > Please let me know if I missed an important point. I hope to submit this JEP in the next week or two. > > -- Kevin > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8200758&d=DwIFAg&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=KlUfgj63empIAez-vk164MMdqh0UIH4raiiK6AKkZmE&s=_R35pRHxi2Ndq3uYY8hHyYuFLWB2IGONK0xIZ_D7GfU&e= > > > On 5/30/2018 5:10 PM, Kevin Rushforth wrote: >> I would like to propose the following Draft JEP [1] for discussion. >> >> JDK-8200758: Packaging Tool >> >> This is intended as a JDK-scope replacement for the existing javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK releases), and was delivered as part of the JavaFX build. The javapackager tool has been removed from Oracle JDK 11 along with the removal of JavaFX. >> >> Comments on this JEP are welcome. It is currently not targeted for a specific release, but we are aiming for JDK 12. >> >> -- Kevin >> >> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8200758&d=DwIFAg&c=uD3W7j5M6i1jDeSybgeVwm110GaiTFmxRW_bPSUkfEI&r=iA565f2Lw9W7rluKs5jkpPnslpNKVsvq0dJJKhVEy_Q&m=KlUfgj63empIAez-vk164MMdqh0UIH4raiiK6AKkZmE&s=_R35pRHxi2Ndq3uYY8hHyYuFLWB2IGONK0xIZ_D7GfU&e= >> > From kevin.rushforth at oracle.com Fri Jul 27 13:13:56 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Fri, 27 Jul 2018 06:13:56 -0700 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> Message-ID: <4927ecfb-7dca-3b13-8726-5b23ae3a4064@oracle.com> > Will it be possible to NOT include the JRE, but specify instead a pre-existing location for the JRE? This does seem like an interesting use case. As you say, it is similar in many ways to both the Multiple Launchers and system JRE, but not quite the same as either. The mechanism to manage and locate these shared-but-private JREs seems like the most challenging part. We can add it to the "if there is time" list of features, but I don't know how feasible it is for the first version. Andy or Alexey can comment as to whether the current prototype has done anything that would make this difficult. -- Kevin On 7/26/2018 7:38 AM, Scott Palmer wrote: > "The input to jpackager includes: a Java runtime image, and a Java application in one of several formats..." > > Will it be possible to NOT include the JRE, but specify instead a pre-existing location for the JRE? > > As an example use-case consider an office productivity suite where there are separate installers for a word processor and a spreadsheet application. These applications are independent and can be installed in any combination (word processor only, both, spreadsheet only). However they are part of the same versioned application suite and have been developed and tested with a particular JRE. Only a single JRE needs to be installed. The applications can share it. This is not the same as using a system-wide JRE (is that even supported for Java 11 and beyond?). But a shared private JRE controlled by the application developer. > > This is similar but not the same as the proposed "Multiple launchers" feature (if time allows), as the shared JRE could be used by different software packages. > > In many cases the JRE is a very large part of the software installation, both in terms of the size of the distributed installer package and the storage requirements of the installed application. JRE sharing can help with this. > > I'm thinking that eventually we could get to the point where developers could treat the JRE as a versioned dependency, also covering the case of customized JRE images. I don't propose that this jpackager tool be involved in creating or distributing such JRE images, only that it supports running applications using a pre-installed JRE where the location can be determined at installer build time or perhaps install time. > > This was possible with the javapackager tool. > > Scott > >> On Jul 25, 2018, at 7:12 PM, Kevin Rushforth wrote: >> >> Thank you to all who provided feedback. I have updated the draft JEP [1], and I think I have incorporated most of the feedback I received. Specifically, I have reorganized and reworded a few things for clarity, added '.exe' and '.app in a .dmg' native package format to the list of features, and added the service bundler (daemon) feature to the "if we have time" list (at the top of that list). >> >> Please let me know if I missed an important point. I hope to submit this JEP in the next week or two. >> >> -- Kevin >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >> >> >> On 5/30/2018 5:10 PM, Kevin Rushforth wrote: >>> I would like to propose the following Draft JEP [1] for discussion. >>> >>> JDK-8200758: Packaging Tool >>> >>> This is intended as a JDK-scope replacement for the existing javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK releases), and was delivered as part of the JavaFX build. The javapackager tool has been removed from Oracle JDK 11 along with the removal of JavaFX. >>> >>> Comments on this JEP are welcome. It is currently not targeted for a specific release, but we are aiming for JDK 12. >>> >>> -- Kevin >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>> From fw at deneb.enyo.de Thu Jul 26 19:05:22 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Thu, 26 Jul 2018 21:05:22 +0200 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: (Andrew Dinn's message of "Mon, 23 Jul 2018 12:01:17 +0100") References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <87o9f1lfjd.fsf@mid.deneb.enyo.de> Message-ID: <87tvolu8b1.fsf@mid.deneb.enyo.de> * Andrew Dinn: >> + // TODO - remove the following defines >> + // for testing we need to define the extra MAP_XXX flags used for >> + // persistent memory. they will eventually get defined by the >> + // system and included via sys/mman.h >> >> Do you really want to change implementation behavior based on which >> glibc headers (or kernel headers) were used to build the JDK? It's >> likely more appropriate to define the constants yourself if they are >> not available. There is some architecture variance for the MAP_* >> constants, but in case of MAP_SHARED_VALIDATE, even HPPA has the same >> definition. > > No, I guess I don't really want to change implementation behaviour based > on which glibc headers (or kernel headers) were used to build the JDK. > So, I think I need to modify this so the constants are always defined by > FileChannelImpl.c > > I have followed the Intel libpmem code in defining the values for these > constants. Can you provide details of the 'architecture variance' you > refer to above? Just a quick note on that: Some of the constants in vary between the Linux architectures for historic reasons. However, for MAP_SHARED_VALIDATE, the constant is consistent across all architectures supported by glibc and in the mainline kernel. You could add something like this to be on the safe side: #ifdef MAP_SHARED_VALIDATE #if MAP_SHARED_VALIDATE != 3 #error Unexpected value for MAP_SHARED_VALIDATE #endif #else /* !MAP_SHARED_VALIDATE */ #define MAP_SHARED_VALIDATE 3 #endif But I doubt that this is required for this constant. From adinn at redhat.com Fri Jul 27 07:39:19 2018 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 27 Jul 2018 08:39:19 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <87tvolu8b1.fsf@mid.deneb.enyo.de> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <87o9f1lfjd.fsf@mid.deneb.enyo.de> <87tvolu8b1.fsf@mid.deneb.enyo.de> Message-ID: <7c703beb-9c19-52c4-25bc-949915ce5ca2@redhat.com> On 26/07/18 20:05, Florian Weimer wrote: > * Andrew Dinn: > >>> + // TODO - remove the following defines >>> + // for testing we need to define the extra MAP_XXX flags used for >>> + // persistent memory. they will eventually get defined by the >>> + // system and included via sys/mman.h >>> >>> Do you really want to change implementation behavior based on which >>> glibc headers (or kernel headers) were used to build the JDK? It's >>> likely more appropriate to define the constants yourself if they are >>> not available. There is some architecture variance for the MAP_* >>> constants, but in case of MAP_SHARED_VALIDATE, even HPPA has the same >>> definition. >> >> No, I guess I don't really want to change implementation behaviour based >> on which glibc headers (or kernel headers) were used to build the JDK. >> So, I think I need to modify this so the constants are always defined by >> FileChannelImpl.c >> >> I have followed the Intel libpmem code in defining the values for these >> constants. Can you provide details of the 'architecture variance' you >> refer to above? > > Just a quick note on that: Some of the constants in vary > between the Linux architectures for historic reasons. However, for > MAP_SHARED_VALIDATE, the constant is consistent across all > architectures supported by glibc and in the mainline kernel. > > You could add something like this to be on the safe side: > > #ifdef MAP_SHARED_VALIDATE > #if MAP_SHARED_VALIDATE != 3 > #error Unexpected value for MAP_SHARED_VALIDATE > #endif > #else /* !MAP_SHARED_VALIDATE */ > #define MAP_SHARED_VALIDATE 3 > #endif > > But I doubt that this is required for this constant. Thanks for the update, Florian. I agree that the above tweak is unnecessary at present -- given that the current code is targeted at Linux on x86_64 and AArch64 where a change is extremely unlikely. That decision may need revisiting if/when the implementation is extended to other ports. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From swpalmer at gmail.com Fri Jul 27 14:07:30 2018 From: swpalmer at gmail.com (Scott Palmer) Date: Fri, 27 Jul 2018 10:07:30 -0400 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> Message-ID: With the existing javapackager there are some limitations. On Windows I only use it to go as far as creating the application image. I then run the WiX commands in my Gradle build script to finish creating an installer that has an added custom component used to tweak the .cfg file. (Creating just the application image is something that should still be possible with the new tool.) On Linux the full install package can be built directly with javapackager. The idea is to set the packager to use the system JRE. This creates an application image with the .cfg file configured with no JRE path ("app.runtime=?) and no JRE files bundled. For Linux builds, I customize the .spec file used by the packager (in verbose mode the packager tells you where it put the generated file .spec file so you can customize it). I add a simple command to modify the application?s .cfg file with a simple search/replace of the app.runtime= line. E.g. for MyApplication which installs in /opt: # Tweak .cfg to point to JRE if it is not already specified. E.g.: change ?app.runtime=" to "app.runtime=/usr/java/latest" sed -e 's/^app\.runtime=$/app\.runtime=\/usr\/java\/latest/' %{_sourcedir}/MyApplication/app/MyApplication.cfg > %{buildroot}/opt/MyApplication/app/MyApplication.cfg On Windows, I take advantage of the fact that the .cfg file follows the convention of a Windows .ini file and MSI/WiX has a mechanism to add or modify .ini file entries. This WiX code added to some component, where the jre64FolderId is refers to the install location of the private JRE, and appDirId refers the the ?app? sub-folder of the application image. My Gradle build script sets some of these variables: In both cases I use a separate module for the private JRE that can be shared among other applications. The JRE installs to a common company folder in a versioned sub-folder so we can have multiple private JREs as we migrate products to newer JREs, but only ever one copy of any given version of the JRE. My concern with the new tool is that removal of support for a ?system JRE? would make it impossible to build a package that doesn't contain a JRE, bloating the sizes of both the installer and installed image footprint (since applications can?t share a common JRE easily). Having the applications run on the same JRE may also have benefits related to class data sharing. Scott > On Jul 27, 2018, at 7:59 AM, Buchberger, Joerg wrote: > > Thanks for the info - I was not aware of these possibilities in javapackager. > > @Kevin: just fyi, this would be also a very useful feature for the company I work for and for our projects > > @Scott: please, let me know, how to achieve this with javapackager > > Cheers > J?rg > > > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net ] On Behalf Of Scott Palmer > Sent: Donnerstag, 26. Juli 2018 16:39 > To: Kevin Rushforth > > Cc: core-libs-dev at openjdk.java.net > Subject: Re: Draft JEP proposal: JDK-8200758: Packaging Tool > > "The input to jpackager includes: a Java runtime image, and a Java application in one of several formats..." > > Will it be possible to NOT include the JRE, but specify instead a pre-existing location for the JRE? > > > This was possible with the javapackager tool. > > Scott From andy.herrick at oracle.com Fri Jul 27 14:14:36 2018 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 27 Jul 2018 10:14:36 -0400 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <4927ecfb-7dca-3b13-8726-5b23ae3a4064@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> <4927ecfb-7dca-3b13-8726-5b23ae3a4064@oracle.com> Message-ID: I don't see why this isn't feasible, and will file such an enhancement request, but should be possible to deliver a suite of apps in one bundle.? This is the third 'stretch goal' : "Multiple launchers (enables a suite of applications to be bundled in a single self-contained application package)" /Andy On 7/27/2018 9:13 AM, Kevin Rushforth wrote: > > Will it be possible to NOT include the JRE, but specify instead a > pre-existing location for the JRE? > > This does seem like an interesting use case. As you say, it is similar > in many ways to both the Multiple Launchers and system JRE, but not > quite the same as either. The mechanism to manage and locate these > shared-but-private JREs seems like the most challenging part. We can > add it to the "if there is time" list of features, but I don't know > how feasible it is for the first version. Andy or Alexey can comment > as to whether the current prototype has done anything that would make > this difficult. > > -- Kevin > > > On 7/26/2018 7:38 AM, Scott Palmer wrote: >> "The input to jpackager includes: a Java runtime image, and a Java >> application in one of several formats..." >> >> Will it be possible to NOT include the JRE, but specify instead a >> pre-existing location for the JRE? >> >> As an example use-case consider an office productivity suite where >> there are separate installers for a word processor and a spreadsheet >> application.? These applications are independent and can be installed >> in any combination (word processor only, both, spreadsheet only).? >> However they are part of the same versioned application suite and >> have been developed and tested with a particular JRE.? Only a single >> JRE needs to be installed.? The applications can share it.? This is >> not the same as using a system-wide JRE (is that even supported for >> Java 11 and beyond?).? But a shared private JRE controlled by the >> application developer. >> >> This is similar but not the same as the proposed "Multiple launchers" >> feature (if time allows), as the shared JRE could be used by >> different software packages. >> >> In many cases the JRE is a very large part of the software >> installation, both in terms of the size of the distributed installer >> package and the storage requirements of the installed application.? >> JRE sharing can help with this. >> >> I'm thinking that eventually we could get to the point where >> developers could treat the JRE as a versioned dependency, also >> covering the case of customized JRE images.? I don't propose that >> this jpackager tool be involved in creating or distributing such JRE >> images, only that it supports running applications using a >> pre-installed JRE where the location can be determined at installer >> build time or perhaps install time. >> >> This was possible with the javapackager tool. >> >> Scott >> >>> On Jul 25, 2018, at 7:12 PM, Kevin Rushforth >>> wrote: >>> >>> Thank you to all who provided feedback. I have updated the draft JEP >>> [1], and I think I have incorporated most of the feedback I >>> received. Specifically, I have reorganized and reworded a few things >>> for clarity, added '.exe' and '.app in a .dmg' native package format >>> to the list of features, and added the service bundler (daemon) >>> feature to the "if we have time" list (at the top of that list). >>> >>> Please let me know if I missed an important point. I hope to submit >>> this JEP in the next week or two. >>> >>> -- Kevin >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>> >>> >>> On 5/30/2018 5:10 PM, Kevin Rushforth wrote: >>>> I would like to propose the following Draft JEP [1] for discussion. >>>> >>>> JDK-8200758: Packaging Tool >>>> >>>> This is intended as a JDK-scope replacement for the existing >>>> javapackager tool that ships with Oracle JDK 10 (and earlier Oracle >>>> JDK releases), and was delivered as part of the JavaFX build. The >>>> javapackager tool has been removed from Oracle JDK 11 along with >>>> the removal of JavaFX. >>>> >>>> Comments on this JEP are welcome. It is currently not targeted for >>>> a specific release, but we are aiming for JDK 12. >>>> >>>> -- Kevin >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>>> > From bsrbnd at gmail.com Fri Jul 27 14:16:30 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 27 Jul 2018 16:16:30 +0200 Subject: [12] (AIX) 8207744: Clean up inconsistent use of opendir/closedir versus opendir64/closedir64 In-Reply-To: References: Message-ID: Hi Brian, On 26 July 2018 at 22:13, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8207744 > http://cr.openjdk.java.net/~bpb/8207744/webrev.00/ > > This change simply adds > > #if defined(_AIX) > #define DIR DIR64 > #define opendir opendir64 > #define closedir closedir64 > #endif > > to each of > > src/java.base/share/native/libjli/wildcard.c src/java.base/unix/native/libjava/ProcessHandleImpl_unix.c src/java.base/unix/native/libjava/TimeZone_md.c > > If someone who builds OpenJDK on AIX would try this out that would be great as I can only verify that it does not cause problems on the platforms that we support (currently in progress). > > Thanks, > > Brian I cannot experiment on AIX myself but as mentioned in [1], shouldn't this also/only be necessary in 'OperatingSystemImpl.c' & 'TimeZone_md.c'? I guess the problem on AIX is that 'readdir64' needs 'openddir64/closedir64/DIR64' while these methods don't exist and are de-facto not necessary on Linux (at least with glibc 2.26 64-bit). The other files you suggest to fix seem to use 'readdir' so I believe 'opendir/closedir' are well deserved here. What do you think? Thanks, Bernard [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054439.html From swpalmer at gmail.com Fri Jul 27 14:36:05 2018 From: swpalmer at gmail.com (Scott Palmer) Date: Fri, 27 Jul 2018 10:36:05 -0400 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> <7d908e0f-6ad4-a379-2786-6e2f952a1e30@oracle.com> <86C0E9BC-E87B-4AD4-A0D7-96326F0FD9FD@gmail.com> <4927ecfb-7dca-3b13-8726-5b23ae3a4064@oracle.com> Message-ID: <6750302D-F2F7-4C88-B181-3345D5A3C597@gmail.com> Right, I explicitly want to make the distinction between this and the ?Multiple launchers? stretch goal. I want to use the "multiple launchers? as well. One way that I have used multiple launchers (on Linux, I don?t think I could get it to work on Windows) is to have one launcher be a service/daemon and a second launcher be a configuration utility for the service. I have some cases where multiple services are installed, each can be independent, but normally they are part of several different product suites. The total product itself can share a JRE, that?s where I want the option to specify a private, shared JRE. The JRE often makes up the majority of the application?s size. This can come down with jlink of course, but because of the nature of our product - with many plugins that are installed after the fact, we need a full JRE as we can?t anticipate what modules the plugins will need. Scott > On Jul 27, 2018, at 10:14 AM, Andy Herrick wrote: > > I don't see why this isn't feasible, and will file such an enhancement request, but should be possible to deliver a suite of apps in one bundle. This is the third 'stretch goal' : "Multiple launchers (enables a suite of applications to be bundled in a single self-contained application package)" > > /Andy > > > On 7/27/2018 9:13 AM, Kevin Rushforth wrote: >> > Will it be possible to NOT include the JRE, but specify instead a pre-existing location for the JRE? >> >> This does seem like an interesting use case. As you say, it is similar in many ways to both the Multiple Launchers and system JRE, but not quite the same as either. The mechanism to manage and locate these shared-but-private JREs seems like the most challenging part. We can add it to the "if there is time" list of features, but I don't know how feasible it is for the first version. Andy or Alexey can comment as to whether the current prototype has done anything that would make this difficult. >> >> -- Kevin >> >> >> On 7/26/2018 7:38 AM, Scott Palmer wrote: >>> "The input to jpackager includes: a Java runtime image, and a Java application in one of several formats..." >>> >>> Will it be possible to NOT include the JRE, but specify instead a pre-existing location for the JRE? >>> >>> As an example use-case consider an office productivity suite where there are separate installers for a word processor and a spreadsheet application. These applications are independent and can be installed in any combination (word processor only, both, spreadsheet only). However they are part of the same versioned application suite and have been developed and tested with a particular JRE. Only a single JRE needs to be installed. The applications can share it. This is not the same as using a system-wide JRE (is that even supported for Java 11 and beyond?). But a shared private JRE controlled by the application developer. >>> >>> This is similar but not the same as the proposed "Multiple launchers" feature (if time allows), as the shared JRE could be used by different software packages. >>> >>> In many cases the JRE is a very large part of the software installation, both in terms of the size of the distributed installer package and the storage requirements of the installed application. JRE sharing can help with this. >>> >>> I'm thinking that eventually we could get to the point where developers could treat the JRE as a versioned dependency, also covering the case of customized JRE images. I don't propose that this jpackager tool be involved in creating or distributing such JRE images, only that it supports running applications using a pre-installed JRE where the location can be determined at installer build time or perhaps install time. >>> >>> This was possible with the javapackager tool. >>> >>> Scott >>> >>>> On Jul 25, 2018, at 7:12 PM, Kevin Rushforth wrote: >>>> >>>> Thank you to all who provided feedback. I have updated the draft JEP [1], and I think I have incorporated most of the feedback I received. Specifically, I have reorganized and reworded a few things for clarity, added '.exe' and '.app in a .dmg' native package format to the list of features, and added the service bundler (daemon) feature to the "if we have time" list (at the top of that list). >>>> >>>> Please let me know if I missed an important point. I hope to submit this JEP in the next week or two. >>>> >>>> -- Kevin >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>>> >>>> >>>> On 5/30/2018 5:10 PM, Kevin Rushforth wrote: >>>>> I would like to propose the following Draft JEP [1] for discussion. >>>>> >>>>> JDK-8200758: Packaging Tool >>>>> >>>>> This is intended as a JDK-scope replacement for the existing javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK releases), and was delivered as part of the JavaFX build. The javapackager tool has been removed from Oracle JDK 11 along with the removal of JavaFX. >>>>> >>>>> Comments on this JEP are welcome. It is currently not targeted for a specific release, but we are aiming for JDK 12. >>>>> >>>>> -- Kevin >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8200758 >>>>> >> > From lance.andersen at oracle.com Fri Jul 27 15:57:38 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 27 Jul 2018 11:57:38 -0400 Subject: [12] RFR 8208363: test/jdk/java/lang/Package/PackageFromManifest.java missing module dependencies declaration In-Reply-To: <4977474F-3B24-43E8-B6EB-1B9984C1DB6B@oracle.com> References: <4977474F-3B24-43E8-B6EB-1B9984C1DB6B@oracle.com> Message-ID: Hi Chris, This the change is fine > On Jul 27, 2018, at 4:42 AM, Chris Yin wrote: > > Please review below minor change to add missing module dependencies declaration, thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8208363 > > Changes: > > diff -r 65fc31d1042b test/jdk/java/lang/Package/PackageFromManifest.java > --- a/test/jdk/java/lang/Package/PackageFromManifest.java Thu Jul 26 16:22:58 2018 -0700 > +++ b/test/jdk/java/lang/Package/PackageFromManifest.java Fri Jul 27 15:46:34 2018 +0800 > @@ -29,6 +29,7 @@ > * same package if multiple jars). Then verify package versioning info > * @library /lib/testlibrary > * @library /test/lib > + * @modules jdk.compiler > * @run main PackageFromManifest setup test > * @run main PackageFromManifest runJar test1.jar > * @run main PackageFromManifest runJar test1.jar test2.jar foo.Foo1 > > > Regards, > Chris Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From mandy.chung at oracle.com Fri Jul 27 16:02:29 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 27 Jul 2018 09:02:29 -0700 Subject: [12] RFR 8208363: test/jdk/java/lang/Package/PackageFromManifest.java missing module dependencies declaration In-Reply-To: <4977474F-3B24-43E8-B6EB-1B9984C1DB6B@oracle.com> References: <4977474F-3B24-43E8-B6EB-1B9984C1DB6B@oracle.com> Message-ID: Looks fine. This test uses JarUtils instead of jar tool and so it does not depend on jdk.jar. Mandy On 7/27/18 1:42 AM, Chris Yin wrote: > Please review below minor change to add missing module dependencies declaration, thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8208363 > > Changes: > > diff -r 65fc31d1042b test/jdk/java/lang/Package/PackageFromManifest.java > --- a/test/jdk/java/lang/Package/PackageFromManifest.java Thu Jul 26 16:22:58 2018 -0700 > +++ b/test/jdk/java/lang/Package/PackageFromManifest.java Fri Jul 27 15:46:34 2018 +0800 > @@ -29,6 +29,7 @@ > * same package if multiple jars). Then verify package versioning info > * @library /lib/testlibrary > * @library /test/lib > + * @modules jdk.compiler > * @run main PackageFromManifest setup test > * @run main PackageFromManifest runJar test1.jar > * @run main PackageFromManifest runJar test1.jar test2.jar foo.Foo1 > > > Regards, > Chris > From paul.sandoz at oracle.com Fri Jul 27 19:35:56 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 27 Jul 2018 12:35:56 -0700 Subject: Long chains created by direct Buffer::slice In-Reply-To: <87tvolksln.fsf@mid.deneb.enyo.de> References: <87o9f1n653.fsf@mid.deneb.enyo.de> <20180720111559.849741094@eggemoggin.niobe.net> <029E054A-7238-4271-AAC5-7904E86DB3D6@oracle.com> <87tvolksln.fsf@mid.deneb.enyo.de> Message-ID: <04231173-6899-4A4E-BF91-2ACA9AC6B5D4@oracle.com> > On Jul 27, 2018, at 1:09 AM, Florian Weimer wrote: > > * Paul Sandoz: > >> I created this issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8208362 > > Thanks. > >> The suggested fix requires a tweak though since an instance of a >> DirectBuffer interface is passed. This is required because views >> over direct ByteBuffers can be created. > > Noticed that as well. > >> diff -r 448cd909c9e2 src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Jul 26 11:53:59 2018 -0700 >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template Thu Jul 26 16:46:20 2018 -0700 >> @@ -194,7 +194,8 @@ >> #if[byte] >> cleaner = null; >> #end[byte] >> - att = db; >> + Object attachment = db.attachment(); >> + att = (attachment == null ? db : attachment); > > This is essentially what I put through jtreg (jdk_core), without any > obvious issues, Thanks for testing. > but I had not time yet to file a bug and create a > webrev. Ah, my apologies, i did not realize you had author status. > > The parenthesis seem unnecessary. If this is an official JDK coding > style, it is not widely used. It is unusual but I chose to keep consistent with the style in the source. Paul. From brian.burkhalter at oracle.com Fri Jul 27 20:09:11 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 27 Jul 2018 13:09:11 -0700 Subject: [12] (AIX) 8207744: Clean up inconsistent use of opendir/closedir versus opendir64/closedir64 In-Reply-To: References: Message-ID: <0F8E94F5-CD1D-45DC-88A5-89E0960405E9@oracle.com> Hi Bernard, I think you are correct. Webrev updated accordingly: http://cr.openjdk.java.net/~bpb/8207744/webrev.01/ I know you cannot do it, but a build of this on AIX by someone would be appreciated (not urgent). Thanks, Brian On Jul 27, 2018, at 7:16 AM, B. Blaser wrote: > I cannot experiment on AIX myself but as mentioned in [1], shouldn't > this also/only be necessary in 'OperatingSystemImpl.c' & > 'TimeZone_md.c'? > I guess the problem on AIX is that 'readdir64' needs > 'openddir64/closedir64/DIR64' while these methods don't exist and are > de-facto not necessary on Linux (at least with glibc 2.26 64-bit). > The other files you suggest to fix seem to use 'readdir' so I believe > 'opendir/closedir' are well deserved here. > > What do you think? > > Thanks, > Bernard > > > [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054439.html From bsrbnd at gmail.com Sat Jul 28 14:10:55 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sat, 28 Jul 2018 16:10:55 +0200 Subject: [12] (AIX) 8207744: Clean up inconsistent use of opendir/closedir versus opendir64/closedir64 In-Reply-To: <0F8E94F5-CD1D-45DC-88A5-89E0960405E9@oracle.com> References: <0F8E94F5-CD1D-45DC-88A5-89E0960405E9@oracle.com> Message-ID: I checked Linux [1], Solaris [2], MacOS [3], FreeBSD [4] and AIX [5] docs and I tried to summarize 64-bit directory usage as follows: * [1] & [2] seem to use 'opendir/closedir/DIR' in conjunction with 'readdir64' * [3] seems to be based on [4] which doesn't provide any specific 64-bit interface for directory usage * [5] seems to require 'opendir64/closedir64/DIR64' to use 'readdir64' Then, I found the following 'readdir64' usages in the JDK: $ find ./src/ -type f -print | xargs grep "readdir64" ./src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c:#define readdir64 readdir ./src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c: ptr = readdir64(dirp); ./src/java.base/unix/native/libjava/TimeZone_md.c:#define readdir64 readdir ./src/java.base/unix/native/libjava/TimeZone_md.c: while ((dp = readdir64(dirp)) != NULL) { ./src/java.base/unix/native/libjava/childproc.c: #define readdir64 readdir ./src/java.base/unix/native/libjava/childproc.c: /* We use readdir64 instead of readdir to work around Solaris bug ./src/java.base/unix/native/libjava/childproc.c: while ((dirp = readdir64(dp)) != NULL) { ./src/java.base/unix/native/libjava/UnixFileSystem_md.c: #define readdir64 readdir ./src/java.base/unix/native/libjava/UnixFileSystem_md.c: while ((ptr = readdir64(dir)) != NULL) { ./src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c: #define readdir64 readdir ./src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c: while ((dentp = readdir64(dirp)) != NULL) { In these files, only 'TimeZone_md.c' & 'OperatingSystemImpl.c' seem to be missing '#define opendir opendir64' (etc...) for AIX. So, I guess I would do (blindly) something more like the patch below. Do we agree? Bernard [1] https://www.gnu.org/software/libc/manual/html_node/Reading_002fClosing-Directory.html [2] https://docs.oracle.com/cd/E36784_01/html/E36883/lf64-5.html [3] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/OSX_Technology_Overview/SystemTechnology/SystemTechnology.html [4] https://www.freebsd.org/cgi/man.cgi?query=opendir&sektion=3 [5] https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.basetrf1/opendir.htm diff -r b0fcf59be391 src/java.base/unix/native/libjava/TimeZone_md.c --- a/src/java.base/unix/native/libjava/TimeZone_md.c Fri Jul 20 14:48:41 2018 -0700 +++ b/src/java.base/unix/native/libjava/TimeZone_md.c Sat Jul 28 15:24:10 2018 +0200 @@ -55,6 +55,12 @@ #define readdir64 readdir #endif +#if defined(_AIX) + #define DIR DIR64 + #define opendir opendir64 + #define closedir closedir64 +#endif + #if !defined(__solaris__) || defined(__sparcv9) || defined(amd64) #define fileopen fopen #define filegets fgets diff -r b0fcf59be391 src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Fri Jul 20 14:48:41 2018 -0700 +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Sat Jul 28 15:24:10 2018 +0200 @@ -79,6 +79,12 @@ #define readdir64 readdir #endif +#if defined(_AIX) + #define DIR DIR64 + #define opendir opendir64 + #define closedir closedir64 +#endif + // true = get available swap in bytes // false = get total swap in bytes static jlong get_total_or_available_swap_space_size(JNIEnv* env, jboolean available) { On 27 July 2018 at 22:09, Brian Burkhalter wrote: > Hi Bernard, > > I think you are correct. Webrev updated accordingly: > > http://cr.openjdk.java.net/~bpb/8207744/webrev.01/ > > I know you cannot do it, but a build of this on AIX by someone would be > appreciated (not urgent). > > Thanks, > > Brian > > On Jul 27, 2018, at 7:16 AM, B. Blaser wrote: > > I cannot experiment on AIX myself but as mentioned in [1], shouldn't > this also/only be necessary in 'OperatingSystemImpl.c' & > 'TimeZone_md.c'? > I guess the problem on AIX is that 'readdir64' needs > 'openddir64/closedir64/DIR64' while these methods don't exist and are > de-facto not necessary on Linux (at least with glibc 2.26 64-bit). > The other files you suggest to fix seem to use 'readdir' so I believe > 'opendir/closedir' are well deserved here. > > What do you think? > > Thanks, > Bernard > > > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054439.html > > From ecki at zusammenkunft.net Sat Jul 28 20:42:47 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Sat, 28 Jul 2018 22:42:47 +0200 Subject: List Windows FileStores misses named mounts Message-ID: <5b5cd547.1c69fb81.80661.cd96@mx.google.com> Hello, I noticed that fs.getRootDirectories() or fs.getFileStores() on Windows only list the Windows drives, but not other disks mounted on a path. When I do the following I actually do get a FileStore for a Name-mounted FS: FileSystem fs = FileSystems.getDefault(); Iterable rootStores = fs.getFileStores(); printStores(rootStores); Path disk = Paths.get("C:", "temp", "disk").toAbsolutePath(); // mountpoint FileSystemProvider fsp = FileSystemProvider.installedProviders().get(0); FileStore fss = fsp.getFileStore(disk); printStore(fss) this is the result on my Windows (USB+Virtual Drive Name mounted) root dir C:\ root dir D:\ (C:) (NTFS) 17.658,68 MB 243.303,05 MB rw r?=false cd?=false vsn=N/A on C:\ MYLINUXLIVE (D:) (FAT32) 226,73 MB 957,50 MB rw r?=true cd?=false vsn=N/A on D:\ -- vdisk (C:\temp\disk) (NTFS) 3,27 MB 7,00 MB rw r?=false cd?=false vsn=N/A on C:\temp\disk Code: https://gist.github.com/ecki/da76c0e724e947ebb77a05094dac1f5c Is there a reason why named Mounts are not listed and is there a cleaner way (without going through the FSP SPI) to get to its FileStore Also what I noticed is that the ?root? member in WindowsFileSystem is the identifying part but there is no API (other than parsing .toString()) to get to it? Gruss Bernd -- http://bernd.eckenfels.net From stuart.marks at oracle.com Sun Jul 29 00:11:11 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Sat, 28 Jul 2018 17:11:11 -0700 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: Message-ID: Hi, I finally got some time to get my head around this. Conceptually, I like the idea of a List that's stored in a circular array, like ArrayDeque. The best way to manifest this in the API isn't obvious though. I filed the bug as "retrofit ArrayDeque to implement List" but of course it doesn't have to be this way. I also think we want to reuse an existing implementation as much as possible. There's already too much duplication between ArrayList and ArrayDeque; we don't want a third similar implementation that will need to be maintained in parallel. To Martin's points: * Adding List-like methods to ArrayDeque without it implementing List is indeed odd, but not necessarily fatal. It does seem to raise the question "Isn't there a better way?" though. * I don't want to have to add null support if at all possible, for the reasons Martin mentioned. Also, it's an implementation and maintenance headache. Of course the implementations like ArrayList and HashMap are proof that it can be done. But each time HashMap has been enhanced significantly, there's been a bug tail where null checking was missed in certain cases. * Growth policy. The shared null array setup for ArrayList was added when we observed that a significant number of ArrayLists are created with the default constructor and then never populated. Allocating the backing array lazily resulted in a considerable space savings. I think this would be a good idea to do for ArrayDeque, but this is somewhat orthogonal to the current "retrofit List" discussion. * Regarding nestmates, I don't think the use of nestmates has any advantage or disadvantage over package-level access between top-level classes in the same package. I think we should decide what we want the API to look like first, and then figure out how to factor things so that we can get the code sharing and coupling that we want. We might not be forced into contorting the API in order to get better sharing/coupling, but let's wait to cross that bridge if we come to it. -- Alex, I'm not sure where your current proposal stands. Have you sent an update since the head of the thread? If you had included something as an attachment, it has likely been stripped. Thanks, s'marks On 7/23/18 8:46 PM, Martin Buchholz wrote: > (As usual, I don't have enough time to devote to this at the?moment) > > I sort of like the idea of adding all of those List methods to ArrayDeque, but > it's __weird__ for java code to do that, and remove(int) is a problem when you > have ArrayDeque, so it will probably end up rejected. > > --- > > Similarly, the idea of an ArrayDeque subclass that accepts nulls will be > unpopular - null elements have fallen out of favor. > > While|Deque|implementations are not strictly required to prohibit the insertion > of null elements, they are strongly encouraged to do so. Users of > any|Deque|implementations that do allow null elements are strongly > encouraged/not/to take advantage of the ability to insert nulls. This is so > because|null|is used as a special return value by various methods to indicate > that the deque is empty. > > --- > > It makes some sense for ArrayDeque's growth policy to be very similar to > ArrayList's - that should be done as an independent change.? Are there any > lessons to be learned from ArrayList's experience?? Is the world filled with > empty ArrayDeques that could share a common backing array? > > --- > > I do like the idea of having ArrayDeque's List-implementing subclass be a > nestmate of ArrayDeque, to share implementation details and to help discovery > and naming.? I'm not thrilled with reusing "List" in ArrayDeque.List but I don't > have a great alternative to suggest. > > > On Mon, Jul 16, 2018 at 10:36 AM, Alex Foster > wrote: > > Hi again, > > I updated ArrayDeque with your last idea (adding all list methods but not > implementing List) and added a subclass ArrayDeque.List which overrides > equals and hashcode and implements List. There is also a subclass > ArrayDeque.WithNulls that accepts null elements. ArrayDeque has removeAt(int > index) instead of remove(index) to avoid overloading remove(Object). > > I also added shared empty arrays similar to Arraylist, and a reallocate > method which can do the same things as trimToSize and ensure capacity in > Arraylist. It also allows you to trim to a specific capacity other than the > size or skip trimming if the capacity is within a specified distance of the > target capacity. > > Also the bulk add methods call collection.toArray, then check the array for > illegal elements, then add the array, which means that a collection could > keep the array it returns from toArray and modify it from another thread > after it has been checked but before it has been added which could lead to > illegal elements being added to the ArrayDeque. We could maybe avoid this by > cloning the array or checking the elements after adding them but I'm not > sure if it's worth it... > > What do you think? > > I also changed the WhiteBox test a bit: > > --- a/test/jdk/java/util/ArrayDeque/WhiteBox.java > +++ b/test/jdk/java/util/ArrayDeque/WhiteBox.java > @@ -88,7 +88,10 @@ > > ???? @Test > ???? public void defaultConstructor() { > -??????? checkCapacity(new ArrayDeque(), 16); > +??????? ArrayDeque d = new ArrayDeque(); > +??????? d.add(new Object()); > +??????? d.clear(); > +??????? checkCapacity(d, 16); > ???? } > > ???? @Test > @@ -131,7 +134,7 @@ > ???????????? if (rnd.nextBoolean()) d.add(99); > ???????????? ArrayDeque clone = serialClone(d); > ???????????? assertInvariants(clone); > -??????????? assertNotSame(elements(d), elements(clone)); > +??????????? assertTrue(d.isEmpty() || elements(d) != elements(clone)); > ???????????? assertEquals(d, clone); > ???????? } > ???? } > > Alex > > From alexfoster at hotmail.ca Sun Jul 29 06:05:59 2018 From: alexfoster at hotmail.ca (Alex Foster) Date: Sun, 29 Jul 2018 06:05:59 +0000 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: , Message-ID: Hi, Here's my current proposal: https://pastebin.com/EABgwLYS Alex ________________________________ From: Stuart Marks Sent: July 28, 2018 8:11 PM To: Martin Buchholz; Alex Foster Cc: Doug Lea; core-libs-dev at openjdk.java.net Subject: Re: 8143850: retrofit ArrayDeque to implement List Hi, I finally got some time to get my head around this. Conceptually, I like the idea of a List that's stored in a circular array, like ArrayDeque. The best way to manifest this in the API isn't obvious though. I filed the bug as "retrofit ArrayDeque to implement List" but of course it doesn't have to be this way. I also think we want to reuse an existing implementation as much as possible. There's already too much duplication between ArrayList and ArrayDeque; we don't want a third similar implementation that will need to be maintained in parallel. To Martin's points: * Adding List-like methods to ArrayDeque without it implementing List is indeed odd, but not necessarily fatal. It does seem to raise the question "Isn't there a better way?" though. * I don't want to have to add null support if at all possible, for the reasons Martin mentioned. Also, it's an implementation and maintenance headache. Of course the implementations like ArrayList and HashMap are proof that it can be done. But each time HashMap has been enhanced significantly, there's been a bug tail where null checking was missed in certain cases. * Growth policy. The shared null array setup for ArrayList was added when we observed that a significant number of ArrayLists are created with the default constructor and then never populated. Allocating the backing array lazily resulted in a considerable space savings. I think this would be a good idea to do for ArrayDeque, but this is somewhat orthogonal to the current "retrofit List" discussion. * Regarding nestmates, I don't think the use of nestmates has any advantage or disadvantage over package-level access between top-level classes in the same package. I think we should decide what we want the API to look like first, and then figure out how to factor things so that we can get the code sharing and coupling that we want. We might not be forced into contorting the API in order to get better sharing/coupling, but let's wait to cross that bridge if we come to it. -- Alex, I'm not sure where your current proposal stands. Have you sent an update since the head of the thread? If you had included something as an attachment, it has likely been stripped. Thanks, s'marks On 7/23/18 8:46 PM, Martin Buchholz wrote: > (As usual, I don't have enough time to devote to this at the moment) > > I sort of like the idea of adding all of those List methods to ArrayDeque, but > it's __weird__ for java code to do that, and remove(int) is a problem when you > have ArrayDeque, so it will probably end up rejected. > > --- > > Similarly, the idea of an ArrayDeque subclass that accepts nulls will be > unpopular - null elements have fallen out of favor. > > While|Deque|implementations are not strictly required to prohibit the insertion > of null elements, they are strongly encouraged to do so. Users of > any|Deque|implementations that do allow null elements are strongly > encouraged/not/to take advantage of the ability to insert nulls. This is so > because|null|is used as a special return value by various methods to indicate > that the deque is empty. > > --- > > It makes some sense for ArrayDeque's growth policy to be very similar to > ArrayList's - that should be done as an independent change. Are there any > lessons to be learned from ArrayList's experience? Is the world filled with > empty ArrayDeques that could share a common backing array? > > --- > > I do like the idea of having ArrayDeque's List-implementing subclass be a > nestmate of ArrayDeque, to share implementation details and to help discovery > and naming. I'm not thrilled with reusing "List" in ArrayDeque.List but I don't > have a great alternative to suggest. > > > On Mon, Jul 16, 2018 at 10:36 AM, Alex Foster > wrote: > > Hi again, > > I updated ArrayDeque with your last idea (adding all list methods but not > implementing List) and added a subclass ArrayDeque.List which overrides > equals and hashcode and implements List. There is also a subclass > ArrayDeque.WithNulls that accepts null elements. ArrayDeque has removeAt(int > index) instead of remove(index) to avoid overloading remove(Object). > > I also added shared empty arrays similar to Arraylist, and a reallocate > method which can do the same things as trimToSize and ensure capacity in > Arraylist. It also allows you to trim to a specific capacity other than the > size or skip trimming if the capacity is within a specified distance of the > target capacity. > > Also the bulk add methods call collection.toArray, then check the array for > illegal elements, then add the array, which means that a collection could > keep the array it returns from toArray and modify it from another thread > after it has been checked but before it has been added which could lead to > illegal elements being added to the ArrayDeque. We could maybe avoid this by > cloning the array or checking the elements after adding them but I'm not > sure if it's worth it... > > What do you think? > > I also changed the WhiteBox test a bit: > > --- a/test/jdk/java/util/ArrayDeque/WhiteBox.java > +++ b/test/jdk/java/util/ArrayDeque/WhiteBox.java > @@ -88,7 +88,10 @@ > > @Test > public void defaultConstructor() { > - checkCapacity(new ArrayDeque(), 16); > + ArrayDeque d = new ArrayDeque(); > + d.add(new Object()); > + d.clear(); > + checkCapacity(d, 16); > } > > @Test > @@ -131,7 +134,7 @@ > if (rnd.nextBoolean()) d.add(99); > ArrayDeque clone = serialClone(d); > assertInvariants(clone); > - assertNotSame(elements(d), elements(clone)); > + assertTrue(d.isEmpty() || elements(d) != elements(clone)); > assertEquals(d, clone); > } > } > > Alex > > From Alan.Bateman at oracle.com Sun Jul 29 06:49:28 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 29 Jul 2018 07:49:28 +0100 Subject: List Windows FileStores misses named mounts In-Reply-To: <5b5cd547.1c69fb81.80661.cd96@mx.google.com> References: <5b5cd547.1c69fb81.80661.cd96@mx.google.com> Message-ID: On 28/07/2018 21:42, Bernd Eckenfels wrote: > > Is there a reason why named Mounts are not listed and is there a cleaner way (without going through the FSP SPI) to get to its FileStore > The implementation only enumerates volumes that are associated with a drive letter. It's something that doesn't come up very often so didn't make it to the top of anyone's list. Files.getFileStore is the API that I think you didn't find, no need to use the SPI directly. -Alan. From xu.y.yin at oracle.com Mon Jul 30 00:59:15 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 30 Jul 2018 08:59:15 +0800 Subject: [12] RFR 8208363: test/jdk/java/lang/Package/PackageFromManifest.java missing module dependencies declaration In-Reply-To: References: <4977474F-3B24-43E8-B6EB-1B9984C1DB6B@oracle.com> Message-ID: <5C1D0240-4547-456B-A666-54D67F0D56AD@oracle.com> Thank you, Lance Regards, Chris > On 27 Jul 2018, at 11:57 PM, Lance Andersen wrote: > > Hi Chris, > > This the change is fine >> On Jul 27, 2018, at 4:42 AM, Chris Yin > wrote: >> >> Please review below minor change to add missing module dependencies declaration, thanks >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8208363 > >> >> Changes: >> >> diff -r 65fc31d1042b test/jdk/java/lang/Package/PackageFromManifest.java >> --- a/test/jdk/java/lang/Package/PackageFromManifest.java Thu Jul 26 16:22:58 2018 -0700 >> +++ b/test/jdk/java/lang/Package/PackageFromManifest.java Fri Jul 27 15:46:34 2018 +0800 >> @@ -29,6 +29,7 @@ >> * same package if multiple jars). Then verify package versioning info >> * @library /lib/testlibrary >> * @library /test/lib >> + * @modules jdk.compiler >> * @run main PackageFromManifest setup test >> * @run main PackageFromManifest runJar test1.jar >> * @run main PackageFromManifest runJar test1.jar test2.jar foo.Foo1 >> >> >> Regards, >> Chris > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From xu.y.yin at oracle.com Mon Jul 30 01:01:07 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 30 Jul 2018 09:01:07 +0800 Subject: [12] RFR 8208363: test/jdk/java/lang/Package/PackageFromManifest.java missing module dependencies declaration In-Reply-To: References: <4977474F-3B24-43E8-B6EB-1B9984C1DB6B@oracle.com> Message-ID: Thank you, Mandy Regards, Chris > On 28 Jul 2018, at 12:02 AM, mandy chung wrote: > > Looks fine. This test uses JarUtils instead of jar tool and so > it does not depend on jdk.jar. > > Mandy > > On 7/27/18 1:42 AM, Chris Yin wrote: >> Please review below minor change to add missing module dependencies declaration, thanks >> bug: https://bugs.openjdk.java.net/browse/JDK-8208363 Changes: >> diff -r 65fc31d1042b test/jdk/java/lang/Package/PackageFromManifest.java >> --- a/test/jdk/java/lang/Package/PackageFromManifest.java Thu Jul 26 16:22:58 2018 -0700 >> +++ b/test/jdk/java/lang/Package/PackageFromManifest.java Fri Jul 27 15:46:34 2018 +0800 >> @@ -29,6 +29,7 @@ >> * same package if multiple jars). Then verify package versioning info >> * @library /lib/testlibrary >> * @library /test/lib >> + * @modules jdk.compiler >> * @run main PackageFromManifest setup test >> * @run main PackageFromManifest runJar test1.jar >> * @run main PackageFromManifest runJar test1.jar test2.jar foo.Foo1 >> Regards, >> Chris From xu.y.yin at oracle.com Mon Jul 30 03:11:10 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 30 Jul 2018 11:11:10 +0800 Subject: [12] RFR 8208279: Add 8 JNDI tests to com/sun/jndi/dns/EnvTests/ In-Reply-To: <0EFABB87-51B9-4454-85DF-33C49C7BF805@oracle.com> References: <0EFABB87-51B9-4454-85DF-33C49C7BF805@oracle.com> Message-ID: <2ED52B01-8332-4BC4-966C-0C3DF85897B2@oracle.com> Please find the new webrev as below, it addressed some similar issues which mentioned in review comments from another thread RFR 8200151, thanks webrev: http://cr.openjdk.java.net/~xyin/8208279/webrev.01/ Regards, Chris > On 26 Jul 2018, at 3:37 PM, Chris Yin wrote: > > Please review the changes to add another 8 JNDI tests to com/sun/jndi/dns/EnvTests/ in OpenJDK, thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8208279 > webrev: http://cr.openjdk.java.net/~xyin/8208279/webrev.00/ > > Regards, > Chris From xu.y.yin at oracle.com Mon Jul 30 08:38:43 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 30 Jul 2018 16:38:43 +0800 Subject: [12] RFR 8208483: Add 5 JNDI tests to com/sun/jndi/dns/FactoryTests/ Message-ID: Please review the changes to add 5 JNDI tests to com/sun/jndi/dns/FactoryTests/ in OpenJDK, thanks bug: https://bugs.openjdk.java.net/browse/JDK-8208483 webrev: http://cr.openjdk.java.net/~xyin/8208483/webrev.00/ Regards, Chris From vyom.tewari at oracle.com Mon Jul 30 09:06:36 2018 From: vyom.tewari at oracle.com (vyom tewari) Date: Mon, 30 Jul 2018 14:36:36 +0530 Subject: [12] RFR 8200151: Add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ In-Reply-To: <8961A4DD-5119-46BD-B2B4-A3162302CAF7@oracle.com> References: <9FD6703A-49E8-41D8-B527-8D0D09726EB0@oracle.com> <8961A4DD-5119-46BD-B2B4-A3162302CAF7@oracle.com> Message-ID: <7fc8c142-1047-84b3-a009-d06e39dc0e92@oracle.com> Hi Chris, Latest code looks good to me. Thanks, Vyom On Friday 27 July 2018 01:12 PM, Chris Yin wrote: > Hi, Vyom > > Thank a lot for your review and comments, inline and update new webrev > as below, thanks > > http://cr.openjdk.java.net/~xyin/8200151/webrev.01/ > > > >> On 26 Jul 2018, at 5:27 PM, vyom tewari > > wrote: >> >> Hi Chris, >> >> Thanks for writing the tests overall code looks good to me, below are >> few minor comments. >> >> 1`-> Fix tag order, my Netbeans always complains :) . > > Fixed, thanks > >> >> 2->? you make AuthRecursiveBase class as public but i can not see it >> is being used outside,? i will suggest you to give the default access >> if possible. > > Sure, changed it to default access now > >> >> 3-> AuthTrue.java, change the message as follows >> >> "Got exception as expected : " -> "Got expected exception: ?; > > Fixed > >> >> >> 4-> PortUnreachable.java , any specific reason for 1000ms or you just >> choose random value ? Please don't hard-code this specific value? >> create a? constant and use it . If possible put some comment why we >> choose 1 second, this will help in debugging if this test fails >> intermittently in future. > > Sure, I created a constant ?THRESHOLD' for it, 1000ms will be used as > a threshold value for this test, normally the request should fail very > quick (in a few ms), but consider to different platform and test > machine performance, we used an acceptable threshold here, some > comments added to it in the code too, thanks > >> >> 5-> Timeout.java, do you really need to set the env using >> "DNSTestUtils.initEnv" ? >> >> I see, this test is not using the DNSServer and other environments >> variables which were set by "DNSTestUtils.initEnv()" or i am missing >> something. > > Actually, it?s indeed not necessary, just set very few default value > such as ?Context.INITIAL_CONTEXT_FACTORY? in the test code should be > fine, but I may still want to keep the test consistency, then we could > have better maintainability for those tests. Sorry, I forgot to > refactor this test with testbase (actually, its not necessary too, but > guess the consistent style and shared base/methods will make the test > easy to read and maintain, refactored this test and > PortUnreachable.java now) > > Regards, > Chris > >> >> Thanks, >> Vyom >> >> On Wednesday 25 July 2018 02:41 PM, Chris Yin wrote: >>> Please review the changes to add 8 JNDI tests to >>> com/sun/jndi/dns/ConfigTests/ in OpenJDK, due to known issue >>> 7164518, PortUnreachable.java will be problem list for 'macosx-all', >>> thanks >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8200151 >>> >>> webrev: http://cr.openjdk.java.net/~xyin/8200151/webrev.00/ >>> >>> >> > >>> >>> Regards, >>> Chris >> > From xu.y.yin at oracle.com Mon Jul 30 09:08:34 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 30 Jul 2018 17:08:34 +0800 Subject: [12] RFR 8200151: Add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ In-Reply-To: <7fc8c142-1047-84b3-a009-d06e39dc0e92@oracle.com> References: <9FD6703A-49E8-41D8-B527-8D0D09726EB0@oracle.com> <8961A4DD-5119-46BD-B2B4-A3162302CAF7@oracle.com> <7fc8c142-1047-84b3-a009-d06e39dc0e92@oracle.com> Message-ID: Thank you, Vyom Regards, Chris > On 30 Jul 2018, at 5:06 PM, vyom tewari wrote: > > Hi Chris, > > Latest code looks good to me. > > Thanks, > > Vyom > > On Friday 27 July 2018 01:12 PM, Chris Yin wrote: >> Hi, Vyom >> >> Thank a lot for your review and comments, inline and update new webrev as below, thanks >> >> http://cr.openjdk.java.net/~xyin/8200151/webrev.01/ >> >> >>> On 26 Jul 2018, at 5:27 PM, vyom tewari > wrote: >>> >>> Hi Chris, >>> >>> Thanks for writing the tests overall code looks good to me, below are few minor comments. >>> >>> 1`-> Fix tag order, my Netbeans always complains :) . >> >> Fixed, thanks >> >>> >>> 2-> you make AuthRecursiveBase class as public but i can not see it is being used outside, i will suggest you to give the default access if possible. >> >> Sure, changed it to default access now >> >>> >>> 3-> AuthTrue.java, change the message as follows >>> >>> "Got exception as expected : " -> "Got expected exception: ?; >> >> Fixed >> >>> >>> >>> 4-> PortUnreachable.java , any specific reason for 1000ms or you just choose random value ? Please don't hard-code this specific value create a constant and use it . If possible put some comment why we choose 1 second, this will help in debugging if this test fails intermittently in future. >> >> Sure, I created a constant ?THRESHOLD' for it, 1000ms will be used as a threshold value for this test, normally the request should fail very quick (in a few ms), but consider to different platform and test machine performance, we used an acceptable threshold here, some comments added to it in the code too, thanks >> >>> >>> 5-> Timeout.java, do you really need to set the env using "DNSTestUtils.initEnv" ? >>> >>> I see, this test is not using the DNSServer and other environments variables which were set by "DNSTestUtils.initEnv()" or i am missing something. >> >> Actually, it?s indeed not necessary, just set very few default value such as ?Context.INITIAL_CONTEXT_FACTORY? in the test code should be fine, but I may still want to keep the test consistency, then we could have better maintainability for those tests. Sorry, I forgot to refactor this test with testbase (actually, its not necessary too, but guess the consistent style and shared base/methods will make the test easy to read and maintain, refactored this test and PortUnreachable.java now) >> >> Regards, >> Chris >> >>> >>> Thanks, >>> Vyom >>> >>> On Wednesday 25 July 2018 02:41 PM, Chris Yin wrote: >>>> Please review the changes to add 8 JNDI tests to com/sun/jndi/dns/ConfigTests/ in OpenJDK, due to known issue 7164518, PortUnreachable.java will be problem list for 'macosx-all', thanks >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8200151 > >>>> webrev: http://cr.openjdk.java.net/~xyin/8200151/webrev.00/ > >>>> >>>> Regards, >>>> Chris >>> >> > From enasser at in.ibm.com Mon Jul 30 15:47:59 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Mon, 30 Jul 2018 15:47:59 +0000 Subject: Add x-IBM-1129 charset In-Reply-To: <5B59FF21.8010808@oracle.com> References: <5B59FF21.8010808@oracle.com> Message-ID: Thank you Sherman for reviewing the webrev and approving the changes. Can you please create a bug to proceed further. Regarding the other two charsets, IBM964 and IBM33722, we will analyze further to see whether it can be better implemented. Regards, Nasser Ebrahim From: Xueming Shen To: core-libs-dev at openjdk.java.net Date: 07/26/2018 10:36 PM Subject: Re: Add x-IBM-1129 charset Sent by: "core-libs-dev" The change looks fine. Btw, do you guy have better implementation for these two charsets? These two were left untouched when I did the reimplementation back to 6/7. Did not have time to figure out its mapping table (especially the compatibility concern when dealing with the mapping hard-coded inside the source and those maps from cdc repository). This was the main reason I did not move the SimpleEUCEncoder into the sun.nio.cs.DoubleByte.java. -Sherman On 5/19/18, 12:56 AM, Nasser Ebrahim wrote: > Hello, > > With the following three bugs, all the default locale charsets except two > (Vi_VN.IBM-1129& ja_JP.IBM-eucJP) are fixed for AIX platform. > > - JDK-8201540: [AIX] Extend the set of supported charsets in java.base > - JDK-8202329: Codepage mappings for IBM-943 and Big5 (aix) > - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053050.html > : [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs > [bug not yet opened]. > > For those fixed charsets, the charsets were existing in the extended > charsets (jdk.charsets) and they were not working with default locale > charset as it did not exist in the standard charset (java.base). The > charsets correspond to the two pending locale (Vi_VN.IBM-1129& > ja_JP.IBM-eucJP) does not exist in the jdk. They need to be added to the > extended charsets before adding to stdcs on AIX platform. > > Here, am including the patch to fix the charset IBM-1129 for the locale > Vi_VN.IBM-1129. We are working on the other missing charset (for > ja_JP.IBM-eucJP) which will be contributed in some time. > > The webrev of the fix is available at > http://cr.openjdk.java.net/~aleonard/IBM1129/webrev.00/ > > Kindly request you to open a bug and review the fix. Please let me know if > you have any questions. > > Thank you, > Nasser Ebrahim > From maurizio.cimadamore at oracle.com Mon Jul 30 17:50:52 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 30 Jul 2018 18:50:52 +0100 Subject: RFR 8208524: IntelliJ support broken since 2018.2 Message-ID: <6612fe8f-5dea-1f0f-df6f-f76572c0535b@oracle.com> Hi, support for IntelliJ projects (both langtools specific and JDK-wide) is broken by a fix in the latest IntelliJ 2018.2, which prevents parsing of ant files inside the .idea configuration folder. The solution is to copy the ant file into some sub-folder of the build folder (I've called it .idea-support for now), which fixes the issue. I have more changes in the pipeline to improve the IntelliJ support, including one which fixes the -o flag (e.g. JDK-8180794) - that's to say that I know that this patch adds more assumptions on the location of the build folder, but we have plans to remove such assumptions in due course. In the short term, priority is to keep support working. Webrev: http://cr.openjdk.java.net/~mcimadamore/8208524/ Cheers Maurizio From brian.burkhalter at oracle.com Mon Jul 30 20:35:32 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 30 Jul 2018 13:35:32 -0700 Subject: [12] (AIX) 8207744: Clean up inconsistent use of opendir/closedir versus opendir64/closedir64 In-Reply-To: References: <0F8E94F5-CD1D-45DC-88A5-89E0960405E9@oracle.com> Message-ID: Hi Bernard, Yes we agree. I believe I already had that change to those two files in the revised (.01) version of the patch which I posted on Friday: http://cr.openjdk.java.net/~bpb/8207744/webrev.01/index.html Thanks, Brian On Jul 28, 2018, at 7:10 AM, B. Blaser wrote: > In these files, only 'TimeZone_md.c' & 'OperatingSystemImpl.c' seem to > be missing '#define opendir opendir64' (etc...) for AIX. > > So, I guess I would do (blindly) something more like the patch below. > > Do we agree? From felix.yang at oracle.com Tue Jul 31 06:16:14 2018 From: felix.yang at oracle.com (Felix Yang) Date: Tue, 31 Jul 2018 14:16:14 +0800 Subject: RFR8194230, jdk/internal/jrtfs/remote/RemoteRuntimeImageTest.java fails with NPE Message-ID: <73b18b56-5266-4274-f05a-75fcb87eef44@oracle.com> Hi all, ??? please review a patch to improve the checking on the settings. Originally it will throw NPE, if specified path is invalid. Bug: ??? https://bugs.openjdk.java.net/browse/JDK-8194230 Webrev: ??? http://cr.openjdk.java.net/~xiaofeya/8194230/webrev.00/ Thanks, Felix From xu.y.yin at oracle.com Tue Jul 31 06:39:04 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 31 Jul 2018 14:39:04 +0800 Subject: [12] RFR 8208542: Add 4 JNDI tests to com/sun/jndi/dns/ListTests/ Message-ID: <86430F5C-B7FD-4C6C-87C2-477FD8EC8704@oracle.com> Please review the changes to add 4 JNDI tests to com/sun/jndi/dns/ListTests/ in OpenJDK, thanks bug: https://bugs.openjdk.java.net/browse/JDK-8208542 webrev: http://cr.openjdk.java.net/~xyin/8208542/webrev.00/ Regards, Chris From matthias.baesken at sap.com Tue Jul 31 10:28:01 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 31 Jul 2018 10:28:01 +0000 Subject: [RFR] 8205525 : Improve exception messages during manifest parsing of jar archives In-Reply-To: <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> References: <0cee5b67a5d1476d8318837b770de382@sap.com> <446574fd-4a70-fad8-3bd1-c43d0e374b92@oracle.com> <34b096660ca44a4cb3f674198cd51d35@sap.com> <036f88e423b743bbb68c73e2a59486d4@sap.com> <943590a6-9adc-7cb2-937b-4fbd8203cd8b@oracle.com> <6cfa4b0266de4fdf85d6c959c450043c@sap.com> <00ab9b6f-f1c2-19b1-eccf-62b49794682f@oracle.com> <60acb2950d9147e5adc8e246de7d7b08@sap.com> <4ab295a8-a61e-efa2-cad8-cf177613057e@oracle.com> <73BAE390-FD57-45CB-A137-D1917B48B9B1@oracle.com> Message-ID: <252b3b00f4e94b8796256675ce2571b1@sap.com> Hello , looks like the generalization of the `includeInExceptions` security property is now in jdk/jdk after "8207846: Generalize the jdk.net.includeInExceptions security property" was added, great news and thanks to Chris for driving this ! I use this security property now as well , and updated the change : http://cr.openjdk.java.net/~mbaesken/webrevs/8205525.3/ I updated the CSR as well : https://bugs.openjdk.java.net/browse/JDK-8207768 Best regards, Matthias > -----Original Message----- > From: Chris Hegarty > Sent: Donnerstag, 19. Juli 2018 14:54 > To: Alan Bateman ; Baesken, Matthias > > Cc: core-libs-dev at openjdk.java.net; Lindenmaier, Goetz > > Subject: Re: [RFR] 8205525 : Improve exception messages during manifest > parsing of jar archives > > > > On 19 Jul 2018, at 11:46, Alan Bateman > wrote: > > > > On 19/07/2018 09:07, Baesken, Matthias wrote: > >> Hello, in the meantime I prepared a CSR : > >> > >> https://bugs.openjdk.java.net/browse/JDK-8207768 > >> > >> > >>> jdk.includeInExceptions expands the scope. That might be okay but we > >>> will need to re-visit jdk.net.includeInExceptions and also move the > >>> support to somewhere in jdk.internal so that it can be used by other > >>> code in java.base. > >> Is there some support code for " jdk.net.includeInExceptions " or do > you just mean the name of the property ? > >> > > Chris is right that it's a bit premature to submit a CSR while the question on > whether to rename the existing security property is on the table. I have no > objection to doing that. > > I filed the following issue to generalize the `includeInExceptions` security > property: > https://bugs.openjdk.java.net/browse/JDK-8207846 > > I would like to resolve 8207846 first, then 8205525 can propose to add the > new argument value, `jarPath`. > > -Chris. From adinn at redhat.com Tue Jul 31 11:01:09 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 31 Jul 2018 12:01:09 +0100 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <8e1fe763-c631-1060-e714-9f9b9481cfe2@redhat.com> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <8e1fe763-c631-1060-e714-9f9b9481cfe2@redhat.com> Message-ID: Well, given the lack of any further input I am left wondering, JEP neophyte that I am, whether: i) said happy lacuna implies that it is appropriate to submit this JEP (as prompted by both the process blurb provided in JEP 1 and the accordingly labelled button in the JEP JIRA interface) or ii) the which unfortunate hiatus in commentary indicates simply that I have failed to engage with the relevant worthies of this parish Also, Iris (privately) mentioned something about "CSRs that add/modify public APIs in the "java.*" modules". Does that mean there is more paperwork to come? Now or later? Advice on how to progress would be very welcome. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From nishit.jain at oracle.com Tue Jul 31 11:27:40 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Tue, 31 Jul 2018 16:57:40 +0530 Subject: [12] RFR 8208560: ChoiceFormat class has unused constants needs cleanup Message-ID: <606a0a30-895c-bc38-35d2-cb92b94cd348@oracle.com> Hi, Please review this cleanup fix for 8208560 Bug: https://bugs.openjdk.java.net/browse/JDK-8208560 Webrev: http://cr.openjdk.java.net/~nishjain/8208560/webrev.00/ Cause and Fix: Changes made for JDK-8021322 has left some unused static fields, not sure why these fields have package private access, but they are not getting used elsewhere in the package. Removed them from ChoiceFormat class. Regards, Nishit Jain From thomas.stuefe at gmail.com Tue Jul 31 13:36:48 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 31 Jul 2018 15:36:48 +0200 Subject: RFR 8208524: IntelliJ support broken since 2018.2 In-Reply-To: <6612fe8f-5dea-1f0f-df6f-f76572c0535b@oracle.com> References: <6612fe8f-5dea-1f0f-df6f-f76572c0535b@oracle.com> Message-ID: Hi Maurizio, thank you for fixing this. I tested it and it works. I did not dive deeply into the issue, but your changes look like they do what you claim they do. Thanks, Thomas On Mon, Jul 30, 2018 at 7:50 PM, Maurizio Cimadamore wrote: > Hi, > support for IntelliJ projects (both langtools specific and JDK-wide) is > broken by a fix in the latest IntelliJ 2018.2, which prevents parsing of ant > files inside the .idea configuration folder. > > The solution is to copy the ant file into some sub-folder of the build > folder (I've called it .idea-support for now), which fixes the issue. > > I have more changes in the pipeline to improve the IntelliJ support, > including one which fixes the -o flag (e.g. JDK-8180794) - that's to say > that I know that this patch adds more assumptions on the location of the > build folder, but we have plans to remove such assumptions in due course. In > the short term, priority is to keep support working. > > Webrev: > > http://cr.openjdk.java.net/~mcimadamore/8208524/ > > Cheers > Maurizio > From roger.riggs at oracle.com Tue Jul 31 14:57:49 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 31 Jul 2018 07:57:49 -0700 Subject: [12] RFR 8208560: ChoiceFormat class has unused constants needs cleanup In-Reply-To: <606a0a30-895c-bc38-35d2-cb92b94cd348@oracle.com> References: <606a0a30-895c-bc38-35d2-cb92b94cd348@oracle.com> Message-ID: <6c265bee-b2e0-f594-e233-fcc4b9376bda@oracle.com> Looks fine. On 7/31/18 4:27 AM, Nishit Jain wrote: > Hi, > > Please review this cleanup fix for 8208560 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8208560 > Webrev: http://cr.openjdk.java.net/~nishjain/8208560/webrev.00/ > > Cause and Fix: Changes made for JDK-8021322 has left some unused > static fields, not sure why these fields have package private access, > but they are not getting used elsewhere in the package. Removed them > from ChoiceFormat class. > > Regards, > Nishit Jain From maurizio.cimadamore at oracle.com Tue Jul 31 15:08:45 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 31 Jul 2018 16:08:45 +0100 Subject: RFR 8208524: IntelliJ support broken since 2018.2 In-Reply-To: References: <6612fe8f-5dea-1f0f-df6f-f76572c0535b@oracle.com> Message-ID: <572571e6-f150-9c24-7397-ce04f4d30314@oracle.com> Thanks for testing and reviewing Thomas! P.S. I'll be pushing shortly another RFR in code tools for the jtreg plugin source code, which is also broken by the latest IJ update. Cheers Maurizio On 31/07/18 14:36, Thomas St?fe wrote: > Hi Maurizio, > > thank you for fixing this. I tested it and it works. I did not dive > deeply into the issue, but your changes look like they do what you > claim they do. > > Thanks, Thomas > > On Mon, Jul 30, 2018 at 7:50 PM, Maurizio Cimadamore > wrote: >> Hi, >> support for IntelliJ projects (both langtools specific and JDK-wide) is >> broken by a fix in the latest IntelliJ 2018.2, which prevents parsing of ant >> files inside the .idea configuration folder. >> >> The solution is to copy the ant file into some sub-folder of the build >> folder (I've called it .idea-support for now), which fixes the issue. >> >> I have more changes in the pipeline to improve the IntelliJ support, >> including one which fixes the -o flag (e.g. JDK-8180794) - that's to say >> that I know that this patch adds more assumptions on the location of the >> build folder, but we have plans to remove such assumptions in due course. In >> the short term, priority is to keep support working. >> >> Webrev: >> >> http://cr.openjdk.java.net/~mcimadamore/8208524/ >> >> Cheers >> Maurizio >> From roger.riggs at oracle.com Tue Jul 31 15:24:16 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 31 Jul 2018 08:24:16 -0700 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <8e1fe763-c631-1060-e714-9f9b9481cfe2@redhat.com> Message-ID: <65f95e06-a70b-7d8c-e62a-4ade23ddbbba@oracle.com> Hi Andrew, The JEP is clear on the concept and has many more details than are necessary to submit it.? Supplying so much detail at the earliest stage invites more discussion about the implementation than the concept. Usually, the details of API and implementation are added as the JEP progresses. By the time it is implemented and complete, the JEP description will be fully up to date and concrete. Regards, Roger On 7/31/18 4:01 AM, Andrew Dinn wrote: > Well, given the lack of any further input I am left wondering, JEP > neophyte that I am, whether: > > i) said happy lacuna implies that it is appropriate to submit this JEP > (as prompted by both the process blurb provided in JEP 1 and the > accordingly labelled button in the JEP JIRA interface) > > or > > ii) the which unfortunate hiatus in commentary indicates simply that I > have failed to engage with the relevant worthies of this parish > > Also, Iris (privately) mentioned something about "CSRs that add/modify > public APIs in the "java.*" modules". Does that mean there is more > paperwork to come? Now or later? > > Advice on how to progress would be very welcome. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From naoto.sato at oracle.com Tue Jul 31 16:06:58 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 31 Jul 2018 09:06:58 -0700 Subject: [12] RFR 8208560: ChoiceFormat class has unused constants needs cleanup In-Reply-To: <6c265bee-b2e0-f594-e233-fcc4b9376bda@oracle.com> References: <606a0a30-895c-bc38-35d2-cb92b94cd348@oracle.com> <6c265bee-b2e0-f594-e233-fcc4b9376bda@oracle.com> Message-ID: <64c869a9-1b97-aab0-8d74-44d06def010b@oracle.com> +1 Naoto On 7/31/18 7:57 AM, Roger Riggs wrote: > Looks fine. > > On 7/31/18 4:27 AM, Nishit Jain wrote: >> Hi, >> >> Please review this cleanup fix for 8208560 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8208560 >> Webrev: http://cr.openjdk.java.net/~nishjain/8208560/webrev.00/ >> >> Cause and Fix: Changes made for JDK-8021322 has left some unused >> static fields, not sure why these fields have package private access, >> but they are not getting used elsewhere in the package. Removed them >> from ChoiceFormat class. >> >> Regards, >> Nishit Jain > From stuart.marks at oracle.com Tue Jul 31 18:40:22 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 31 Jul 2018 11:40:22 -0700 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: Message-ID: <6f2ec600-288f-8cea-9ba9-4748c3e8ad5d@oracle.com> OK, great. It looks like a heavily modified version of ArrayDeque.java. This makes it a bit difficult to understand what the changes are. Would you mind if I rehosted this to cr.openjdk.java.net and generated a webrev for it? Thanks, s'marks On 7/28/18 11:05 PM, Alex Foster wrote: > > Hi, > > > Here's my current proposal: https://pastebin.com/EABgwLYS > > > > Alex > > > > -------------------------------------------------------------------------------- > *From:* Stuart Marks > *Sent:* July 28, 2018 8:11 PM > *To:* Martin Buchholz; Alex Foster > *Cc:* Doug Lea; core-libs-dev at openjdk.java.net > *Subject:* Re: 8143850: retrofit ArrayDeque to implement List > Hi, I finally got some time to get my head around this. > > Conceptually, I like the idea of a List that's stored in a circular array, like > ArrayDeque. The best way to manifest this in the API isn't obvious though. I > filed the bug as "retrofit ArrayDeque to implement List" but of course it > doesn't have to be this way. > > I also think we want to reuse an existing implementation as much as possible. > There's already too much duplication between ArrayList and ArrayDeque; we don't > want a third similar implementation that will need to be maintained in parallel. > > To Martin's points: > > * Adding List-like methods to ArrayDeque without it implementing List is indeed > odd, but not necessarily fatal. It does seem to raise the question "Isn't there > a better way?" though. > > * I don't want to have to add null support if at all possible, for the reasons > Martin mentioned. Also, it's an implementation and maintenance headache. Of > course the implementations like ArrayList and HashMap are proof that it can be > done. But each time HashMap has been enhanced significantly, there's been a bug > tail where null checking was missed in certain cases. > > * Growth policy. The shared null array setup for ArrayList was added when we > observed that a significant number of ArrayLists are created with the default > constructor and then never populated. Allocating the backing array lazily > resulted in a considerable space savings. I think this would be a good idea to > do for ArrayDeque, but this is somewhat orthogonal to the current "retrofit > List" discussion. > > * Regarding nestmates, I don't think the use of nestmates has any advantage or > disadvantage over package-level access between top-level classes in the same > package. I think we should decide what we want the API to look like first, and > then figure out how to factor things so that we can get the code sharing and > coupling that we want. We might not be forced into contorting the API in order > to get better sharing/coupling, but let's wait to cross that bridge if we come > to it. > > -- > > Alex, I'm not sure where your current proposal stands. Have you sent an update > since the head of the thread? If you had included something as an attachment, it > has likely been stripped. > > Thanks, > > s'marks > > > > > > On 7/23/18 8:46 PM, Martin Buchholz wrote: > > (As usual, I don't have enough time to devote to this at the?moment) > > > > I sort of like the idea of adding all of those List methods to ArrayDeque, but > > it's __weird__ for java code to do that, and remove(int) is a problem when you > > have ArrayDeque, so it will probably end up rejected. > > > > --- > > > > Similarly, the idea of an ArrayDeque subclass that accepts nulls will be > > unpopular - null elements have fallen out of favor. > > > > While|Deque|implementations are not strictly required to prohibit the insertion > > of null elements, they are strongly encouraged to do so. Users of > > any|Deque|implementations that do allow null elements are strongly > > encouraged/not/to take advantage of the ability to insert nulls. This is so > > because|null|is used as a special return value by various methods to indicate > > that the deque is empty. > > > > --- > > > > It makes some sense for ArrayDeque's growth policy to be very similar to > > ArrayList's - that should be done as an independent change.? Are there any > > lessons to be learned from ArrayList's experience?? Is the world filled with > > empty ArrayDeques that could share a common backing array? > > > > --- > > > > I do like the idea of having ArrayDeque's List-implementing subclass be a > > nestmate of ArrayDeque, to share implementation details and to help discovery > > and naming.? I'm not thrilled with reusing "List" in ArrayDeque.List but I > don't > > have a great alternative to suggest. > > > > > > On Mon, Jul 16, 2018 at 10:36 AM, Alex Foster > > wrote: > > > >???? Hi again, > > > >???? I updated ArrayDeque with your last idea (adding all list methods but not > >???? implementing List) and added a subclass ArrayDeque.List which overrides > >???? equals and hashcode and implements List. There is also a subclass > >???? ArrayDeque.WithNulls that accepts null elements. ArrayDeque has removeAt(int > >???? index) instead of remove(index) to avoid overloading remove(Object). > > > >???? I also added shared empty arrays similar to Arraylist, and a reallocate > >???? method which can do the same things as trimToSize and ensure capacity in > >???? Arraylist. It also allows you to trim to a specific capacity other than the > >???? size or skip trimming if the capacity is within a specified distance of the > >???? target capacity. > > > >???? Also the bulk add methods call collection.toArray, then check the array for > >???? illegal elements, then add the array, which means that a collection could > >???? keep the array it returns from toArray and modify it from another thread > >???? after it has been checked but before it has been added which could lead to > >???? illegal elements being added to the ArrayDeque. We could maybe avoid this by > >???? cloning the array or checking the elements after adding them but I'm not > >???? sure if it's worth it... > > > >???? What do you think? > > > >???? I also changed the WhiteBox test a bit: > > > >???? --- a/test/jdk/java/util/ArrayDeque/WhiteBox.java > >???? +++ b/test/jdk/java/util/ArrayDeque/WhiteBox.java > >???? @@ -88,7 +88,10 @@ > > > >????? ???? @Test > >????? ???? public void defaultConstructor() { > >???? -??????? checkCapacity(new ArrayDeque(), 16); > >???? +??????? ArrayDeque d = new ArrayDeque(); > >???? +??????? d.add(new Object()); > >???? +??????? d.clear(); > >???? +??????? checkCapacity(d, 16); > >????? ???? } > > > >????? ???? @Test > >???? @@ -131,7 +134,7 @@ > >????? ???????????? if (rnd.nextBoolean()) d.add(99); > >????? ???????????? ArrayDeque clone = serialClone(d); > >????? ???????????? assertInvariants(clone); > >???? -??????????? assertNotSame(elements(d), elements(clone)); > >???? +??????????? assertTrue(d.isEmpty() || elements(d) != elements(clone)); > >????? ???????????? assertEquals(d, clone); > >????? ???????? } > >????? ???? } > > > >???? Alex > > > > From brian.burkhalter at oracle.com Tue Jul 31 18:47:57 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 31 Jul 2018 11:47:57 -0700 Subject: [PING] Re: [12] (AIX) 8207744: Clean up inconsistent use of opendir/closedir versus opendir64/closedir64 In-Reply-To: References: <0F8E94F5-CD1D-45DC-88A5-89E0960405E9@oracle.com> Message-ID: <944A18E9-D6E3-494D-9C94-875845507E33@oracle.com> Hi, Does anyone have any further comments on this? Is there anyone willing to volunteer to build and test it on AIX? Thanks, Brian On Jul 30, 2018, at 1:35 PM, Brian Burkhalter wrote: > Hi Bernard, > > Yes we agree. I believe I already had that change to those two files in the revised (.01) version of the patch which I posted on Friday: > > http://cr.openjdk.java.net/~bpb/8207744/webrev.01/index.html > > Thanks, > > Brian > > On Jul 28, 2018, at 7:10 AM, B. Blaser wrote: > >> In these files, only 'TimeZone_md.c' & 'OperatingSystemImpl.c' seem to >> be missing '#define opendir opendir64' (etc...) for AIX. >> >> So, I guess I would do (blindly) something more like the patch below. >> >> Do we agree? From alexfoster at hotmail.ca Tue Jul 31 19:08:45 2018 From: alexfoster at hotmail.ca (Alex Foster) Date: Tue, 31 Jul 2018 19:08:45 +0000 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: <6f2ec600-288f-8cea-9ba9-4748c3e8ad5d@oracle.com> References: , <6f2ec600-288f-8cea-9ba9-4748c3e8ad5d@oracle.com> Message-ID: Sure, I don't mind. I would have done that myself but I don't think I have access to upload to cr.openjdk.java.net. Alex ________________________________ From: Stuart Marks Sent: July 31, 2018 2:40 PM To: Alex Foster Cc: core-libs-dev at openjdk.java.net Subject: Re: 8143850: retrofit ArrayDeque to implement List OK, great. It looks like a heavily modified version of ArrayDeque.java. This makes it a bit difficult to understand what the changes are. Would you mind if I rehosted this to cr.openjdk.java.net and generated a webrev for it? Thanks, s'marks On 7/28/18 11:05 PM, Alex Foster wrote: Hi, Here's my current proposal: https://pastebin.com/EABgwLYS Alex ________________________________ From: Stuart Marks Sent: July 28, 2018 8:11 PM To: Martin Buchholz; Alex Foster Cc: Doug Lea; core-libs-dev at openjdk.java.net Subject: Re: 8143850: retrofit ArrayDeque to implement List Hi, I finally got some time to get my head around this. Conceptually, I like the idea of a List that's stored in a circular array, like ArrayDeque. The best way to manifest this in the API isn't obvious though. I filed the bug as "retrofit ArrayDeque to implement List" but of course it doesn't have to be this way. I also think we want to reuse an existing implementation as much as possible. There's already too much duplication between ArrayList and ArrayDeque; we don't want a third similar implementation that will need to be maintained in parallel. To Martin's points: * Adding List-like methods to ArrayDeque without it implementing List is indeed odd, but not necessarily fatal. It does seem to raise the question "Isn't there a better way?" though. * I don't want to have to add null support if at all possible, for the reasons Martin mentioned. Also, it's an implementation and maintenance headache. Of course the implementations like ArrayList and HashMap are proof that it can be done. But each time HashMap has been enhanced significantly, there's been a bug tail where null checking was missed in certain cases. * Growth policy. The shared null array setup for ArrayList was added when we observed that a significant number of ArrayLists are created with the default constructor and then never populated. Allocating the backing array lazily resulted in a considerable space savings. I think this would be a good idea to do for ArrayDeque, but this is somewhat orthogonal to the current "retrofit List" discussion. * Regarding nestmates, I don't think the use of nestmates has any advantage or disadvantage over package-level access between top-level classes in the same package. I think we should decide what we want the API to look like first, and then figure out how to factor things so that we can get the code sharing and coupling that we want. We might not be forced into contorting the API in order to get better sharing/coupling, but let's wait to cross that bridge if we come to it. -- Alex, I'm not sure where your current proposal stands. Have you sent an update since the head of the thread? If you had included something as an attachment, it has likely been stripped. Thanks, s'marks On 7/23/18 8:46 PM, Martin Buchholz wrote: > (As usual, I don't have enough time to devote to this at the moment) > > I sort of like the idea of adding all of those List methods to ArrayDeque, but > it's __weird__ for java code to do that, and remove(int) is a problem when you > have ArrayDeque, so it will probably end up rejected. > > --- > > Similarly, the idea of an ArrayDeque subclass that accepts nulls will be > unpopular - null elements have fallen out of favor. > > While|Deque|implementations are not strictly required to prohibit the insertion > of null elements, they are strongly encouraged to do so. Users of > any|Deque|implementations that do allow null elements are strongly > encouraged/not/to take advantage of the ability to insert nulls. This is so > because|null|is used as a special return value by various methods to indicate > that the deque is empty. > > --- > > It makes some sense for ArrayDeque's growth policy to be very similar to > ArrayList's - that should be done as an independent change. Are there any > lessons to be learned from ArrayList's experience? Is the world filled with > empty ArrayDeques that could share a common backing array? > > --- > > I do like the idea of having ArrayDeque's List-implementing subclass be a > nestmate of ArrayDeque, to share implementation details and to help discovery > and naming. I'm not thrilled with reusing "List" in ArrayDeque.List but I don't > have a great alternative to suggest. > > > On Mon, Jul 16, 2018 at 10:36 AM, Alex Foster > > wrote: > > Hi again, > > I updated ArrayDeque with your last idea (adding all list methods but not > implementing List) and added a subclass ArrayDeque.List which overrides > equals and hashcode and implements List. There is also a subclass > ArrayDeque.WithNulls that accepts null elements. ArrayDeque has removeAt(int > index) instead of remove(index) to avoid overloading remove(Object). > > I also added shared empty arrays similar to Arraylist, and a reallocate > method which can do the same things as trimToSize and ensure capacity in > Arraylist. It also allows you to trim to a specific capacity other than the > size or skip trimming if the capacity is within a specified distance of the > target capacity. > > Also the bulk add methods call collection.toArray, then check the array for > illegal elements, then add the array, which means that a collection could > keep the array it returns from toArray and modify it from another thread > after it has been checked but before it has been added which could lead to > illegal elements being added to the ArrayDeque. We could maybe avoid this by > cloning the array or checking the elements after adding them but I'm not > sure if it's worth it... > > What do you think? > > I also changed the WhiteBox test a bit: > > --- a/test/jdk/java/util/ArrayDeque/WhiteBox.java > +++ b/test/jdk/java/util/ArrayDeque/WhiteBox.java > @@ -88,7 +88,10 @@ > > @Test > public void defaultConstructor() { > - checkCapacity(new ArrayDeque(), 16); > + ArrayDeque d = new ArrayDeque(); > + d.add(new Object()); > + d.clear(); > + checkCapacity(d, 16); > } > > @Test > @@ -131,7 +134,7 @@ > if (rnd.nextBoolean()) d.add(99); > ArrayDeque clone = serialClone(d); > assertInvariants(clone); > - assertNotSame(elements(d), elements(clone)); > + assertTrue(d.isEmpty() || elements(d) != elements(clone)); > assertEquals(d, clone); > } > } > > Alex > > From patrick at reini.net Tue Jul 31 19:10:53 2018 From: patrick at reini.net (Patrick Reinhart) Date: Tue, 31 Jul 2018 21:10:53 +0200 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: <6f2ec600-288f-8cea-9ba9-4748c3e8ad5d@oracle.com> Message-ID: Hi Alex, I can do that for you .... -Patrick Am 31.07.2018 um 21:08 schrieb Alex Foster: > Sure, I don't mind. I would have done that myself but I don't think I have access to upload to cr.openjdk.java.net. > > > Alex > > ________________________________ > From: Stuart Marks > Sent: July 31, 2018 2:40 PM > To: Alex Foster > Cc: core-libs-dev at openjdk.java.net > Subject: Re: 8143850: retrofit ArrayDeque to implement List > > > OK, great. > > > It looks like a heavily modified version of ArrayDeque.java. This makes it a bit difficult to understand what the changes are. Would you mind if I rehosted this to cr.openjdk.java.net and generated a webrev for it? > > > Thanks, > > > s'marks > > On 7/28/18 11:05 PM, Alex Foster wrote: > > Hi, > > > Here's my current proposal: https://pastebin.com/EABgwLYS > > > Alex > > > ________________________________ > From: Stuart Marks > Sent: July 28, 2018 8:11 PM > To: Martin Buchholz; Alex Foster > Cc: Doug Lea; core-libs-dev at openjdk.java.net > Subject: Re: 8143850: retrofit ArrayDeque to implement List > > Hi, I finally got some time to get my head around this. > > Conceptually, I like the idea of a List that's stored in a circular array, like > ArrayDeque. The best way to manifest this in the API isn't obvious though. I > filed the bug as "retrofit ArrayDeque to implement List" but of course it > doesn't have to be this way. > > I also think we want to reuse an existing implementation as much as possible. > There's already too much duplication between ArrayList and ArrayDeque; we don't > want a third similar implementation that will need to be maintained in parallel. > > To Martin's points: > > * Adding List-like methods to ArrayDeque without it implementing List is indeed > odd, but not necessarily fatal. It does seem to raise the question "Isn't there > a better way?" though. > > * I don't want to have to add null support if at all possible, for the reasons > Martin mentioned. Also, it's an implementation and maintenance headache. Of > course the implementations like ArrayList and HashMap are proof that it can be > done. But each time HashMap has been enhanced significantly, there's been a bug > tail where null checking was missed in certain cases. > > * Growth policy. The shared null array setup for ArrayList was added when we > observed that a significant number of ArrayLists are created with the default > constructor and then never populated. Allocating the backing array lazily > resulted in a considerable space savings. I think this would be a good idea to > do for ArrayDeque, but this is somewhat orthogonal to the current "retrofit > List" discussion. > > * Regarding nestmates, I don't think the use of nestmates has any advantage or > disadvantage over package-level access between top-level classes in the same > package. I think we should decide what we want the API to look like first, and > then figure out how to factor things so that we can get the code sharing and > coupling that we want. We might not be forced into contorting the API in order > to get better sharing/coupling, but let's wait to cross that bridge if we come > to it. > > -- > > Alex, I'm not sure where your current proposal stands. Have you sent an update > since the head of the thread? If you had included something as an attachment, it > has likely been stripped. > > Thanks, > > s'marks > > > > > > On 7/23/18 8:46 PM, Martin Buchholz wrote: >> (As usual, I don't have enough time to devote to this at the moment) >> >> I sort of like the idea of adding all of those List methods to ArrayDeque, but >> it's __weird__ for java code to do that, and remove(int) is a problem when you >> have ArrayDeque, so it will probably end up rejected. >> >> --- >> >> Similarly, the idea of an ArrayDeque subclass that accepts nulls will be >> unpopular - null elements have fallen out of favor. >> >> While|Deque|implementations are not strictly required to prohibit the insertion >> of null elements, they are strongly encouraged to do so. Users of >> any|Deque|implementations that do allow null elements are strongly >> encouraged/not/to take advantage of the ability to insert nulls. This is so >> because|null|is used as a special return value by various methods to indicate >> that the deque is empty. >> >> --- >> >> It makes some sense for ArrayDeque's growth policy to be very similar to >> ArrayList's - that should be done as an independent change. Are there any >> lessons to be learned from ArrayList's experience? Is the world filled with >> empty ArrayDeques that could share a common backing array? >> >> --- >> >> I do like the idea of having ArrayDeque's List-implementing subclass be a >> nestmate of ArrayDeque, to share implementation details and to help discovery >> and naming. I'm not thrilled with reusing "List" in ArrayDeque.List but I don't >> have a great alternative to suggest. >> >> >> On Mon, Jul 16, 2018 at 10:36 AM, Alex Foster >> > wrote: >> >> Hi again, >> >> I updated ArrayDeque with your last idea (adding all list methods but not >> implementing List) and added a subclass ArrayDeque.List which overrides >> equals and hashcode and implements List. There is also a subclass >> ArrayDeque.WithNulls that accepts null elements. ArrayDeque has removeAt(int >> index) instead of remove(index) to avoid overloading remove(Object). >> >> I also added shared empty arrays similar to Arraylist, and a reallocate >> method which can do the same things as trimToSize and ensure capacity in >> Arraylist. It also allows you to trim to a specific capacity other than the >> size or skip trimming if the capacity is within a specified distance of the >> target capacity. >> >> Also the bulk add methods call collection.toArray, then check the array for >> illegal elements, then add the array, which means that a collection could >> keep the array it returns from toArray and modify it from another thread >> after it has been checked but before it has been added which could lead to >> illegal elements being added to the ArrayDeque. We could maybe avoid this by >> cloning the array or checking the elements after adding them but I'm not >> sure if it's worth it... >> >> What do you think? >> >> I also changed the WhiteBox test a bit: >> >> --- a/test/jdk/java/util/ArrayDeque/WhiteBox.java >> +++ b/test/jdk/java/util/ArrayDeque/WhiteBox.java >> @@ -88,7 +88,10 @@ >> >> @Test >> public void defaultConstructor() { >> - checkCapacity(new ArrayDeque(), 16); >> + ArrayDeque d = new ArrayDeque(); >> + d.add(new Object()); >> + d.clear(); >> + checkCapacity(d, 16); >> } >> >> @Test >> @@ -131,7 +134,7 @@ >> if (rnd.nextBoolean()) d.add(99); >> ArrayDeque clone = serialClone(d); >> assertInvariants(clone); >> - assertNotSame(elements(d), elements(clone)); >> + assertTrue(d.isEmpty() || elements(d) != elements(clone)); >> assertEquals(d, clone); >> } >> } >> >> Alex >> >> From patrick at reini.net Tue Jul 31 19:21:17 2018 From: patrick at reini.net (Patrick Reinhart) Date: Tue, 31 Jul 2018 21:21:17 +0200 Subject: 8143850: retrofit ArrayDeque to implement List In-Reply-To: References: <6f2ec600-288f-8cea-9ba9-4748c3e8ad5d@oracle.com> Message-ID: <7e10860f-2619-df70-4fdb-7dbc60a429ee@reini.net> Am 31.07.2018 um 21:10 schrieb Patrick Reinhart: > Hi Alex, > > I can do that for you .... > > -Patrick > Here it is: http://cr.openjdk.java.net/~reinhapa/reviews/8143850/webrev -Patrick From Sergey.Bylokhov at oracle.com Tue Jul 31 23:39:59 2018 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Tue, 31 Jul 2018 16:39:59 -0700 Subject: [12] Review Request: 8205537 Drop of sun.applet package Message-ID: Hello. Please review the fix for jdk11. Bug: https://bugs.openjdk.java.net/browse/JDK-8205537 Webrev: http://cr.openjdk.java.net/~serb/8205537/webrev.00/ sun.applet is an internal package contained some code related to implementation of applets and appletviewer. Some of its classes were dropped already: JDK-8204454, JDK-8203308. Now there are only 3 classes, all related to the applet security and we can drop them as well. In the fix this package was removed, and the tests were updated to not use it. I am not sure how "ClassnameCharTest.java" is useful after applets removal, but since this test used subclass of the AppletClassLoader, I have copied some code from the removed class to the test. -- Best regards, Sergey. From Sergey.Bylokhov at oracle.com Tue Jul 31 23:55:31 2018 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Tue, 31 Jul 2018 16:55:31 -0700 Subject: [12] Review Request: 8205537 Drop of sun.applet package In-Reply-To: References: Message-ID: <862c094a-1d53-fca6-9a7b-123dbd45af0e@oracle.com> On 31/07/2018 16:39, Sergey Bylokhov wrote: > Hello. > Please review the fix for jdk11. typo, I meant jdk12. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8205537 > Webrev: http://cr.openjdk.java.net/~serb/8205537/webrev.00/ > > sun.applet is an internal package contained some code related to > implementation of applets and appletviewer. Some of its classes were > dropped already: JDK-8204454, JDK-8203308. Now there are only 3 classes, > all related to the applet security and we can drop them as well. > > In the fix this package was removed, and the tests were updated to not > use it. I am not sure how "ClassnameCharTest.java" is useful after > applets removal, but since this test used subclass of the > AppletClassLoader, I have copied some code from the removed class to the > test. > -- Best regards, Sergey.