From alex.buckley at oracle.com Sat Aug 1 00:05:43 2020 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 31 Jul 2020 17:05:43 -0700 Subject: Preview APIs in the Java Platform In-Reply-To: <952661d4-3f1d-2a05-a178-e11b37141cf7@oracle.com> References: <952661d4-3f1d-2a05-a178-e11b37141cf7@oracle.com> Message-ID: <0da465e7-82ff-a2c7-a95e-7529fc0d957f@oracle.com> On 3/3/2020 1:15 PM, Alex Buckley wrote: > Ultimately, though, the best way to provoke feedback on a feature is to > ship it in the GA binary of a JDK feature release. This approach has > worked well for preview language features, where the Java community has > accepted the idea of non-final features that are disabled by default and > can thus be changed in response to feedback. Ideally, we want a way to > ship highly-evolved but non-final APIs in a JDK feature release, without > distorting the API by relocating its packages and modules, and without > misleading developers about its status. I am pleased to report that http://openjdk.java.net/jeps/12 has been updated to define preview APIs as the third kind of preview feature, alongside preview language features and preview VM features. The connection between preview language features and preview APIs is strong. We now have preview language features that allow ordinary code to declare new types (e.g., record classes, sealed classes), which means that not only do we have to flag where a declaration relies on a non-final feature (`record Point(..) {..}`), but we also have to flag where code uses the declaration (`new Point(..)`), since it relies indirectly on a non-final feature too. JEP 12 always suggested this double-sided flagging, but it emerges more clearly once preview APIs are in the story -- using your own record class `Point` is akin to using a preview API in java.*. The use of preview language features, types declared using preview language features, and preview APIs is discussed in a unified way in the section "Use of Preview Features". Other topics of interest are the four-way categorization of preview APIs (with special rules to support use of "reflective" preview APIs -- also not new, but clearer now), and the impact of preview APIs in java.lang. javac's support for preview APIs is on track for JDK 16. The CSR (https://bugs.openjdk.java.net/browse/JDK-8250769) includes a concise overview of what happens when you use each kind of preview feature. Alex From david.holmes at oracle.com Mon Aug 3 01:52:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Aug 2020 11:52:59 +1000 Subject: Copyright Issues : JDK15 - GPLv2 is present but Classpath exception is missing In-Reply-To: References: <772e0556-28b7-aa9e-0f28-aa76330d576d@oracle.com> <2fc3326890cdeffdf2a71f4004e0d159e0bd361b.camel@redhat.com>

Message-ID: <273b263d-b0ea-aa92-c0e3-efc3e495236e@oracle.com> On 29/07/2020 7:25 pm, Archana Nogriya wrote: > Thank David for confirming. > > Yes agree with you they are either hotspot or build tool related. > > and I will wait for the fix for > src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java, > as you mentioned :) I have filed: https://bugs.openjdk.java.net/browse/JDK-8250929 Cheers, David > Thanks again. > > > Many Thanks & Regards > Archana Nogriya > Java Runtimes Development and Project Manager > IBM Hursley > IBM United Kingdom Ltd > internet email: archana.nogriya at uk.ibm.com > > > > > From: David Holmes > To: Severin Gehwolf , Archana Nogriya > , jdk-dev at openjdk.java.net > Cc: Andrew Leonard , Simon Rushton > > Date: 28/07/2020 14:51 > Subject: [EXTERNAL] Re: Copyright Issues : JDK15 - GPLv2 is present but > Classpath exception is missing > ------------------------------------------------------------------------ > > > > On 28/07/2020 11:44 pm, Severin Gehwolf wrote: >> On Tue, 2020-07-28 at 23:40 +1000, David Holmes wrote: >>> Hi, >>> >>> On 28/07/2020 10:07 pm, Archana Nogriya wrote: >>>> Hi, >>>> >>>> Please can someone help to fix these copyright issues in JDK15. >>>> We found these issues in our internal scan. >>> >>> Why should these files require the classpath exception? They are not >>> classes you would link against. >> [...] >> >>>> src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java: >>>> GPLv2 is present but Classpath exception is missing >> >> ---^ >> >> This one probably is? > > Yes I'll give you that one :) > > David > >> Thanks, >> Severin >> > > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From GopalaKrishna.Konakanchi at arcserve.com Tue Aug 4 11:08:29 2020 From: GopalaKrishna.Konakanchi at arcserve.com (Konakanchi, Gopala Krishna V N.) Date: Tue, 4 Aug 2020 11:08:29 +0000 Subject: Shortcut files created in openjdk installation Message-ID: Hi, When we have installed open-jdk 1.8.0 on Red hat Linux instance ( AWS ec2-instance - red hat version 8.2 ), some shortcut files were created. Installed using yum. We have faced some issues due to the creation of shortcut files instead of complete file. Observation: When clicked on the shortcut file, the file with full content is opened in the same location. ( Were shortcut file is placed ). Can you please help to let us know why the short cut files are getting created. Best Regards, Gopala Krishna. From sgehwolf at redhat.com Tue Aug 4 11:38:46 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 04 Aug 2020 13:38:46 +0200 Subject: Shortcut files created in openjdk installation In-Reply-To: References: Message-ID: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> On Tue, 2020-08-04 at 11:08 +0000, Konakanchi, Gopala Krishna V N. wrote: > Hi, > When we have installed open-jdk 1.8.0 on Red hat Linux instance ( AWS ec2-instance - red hat version 8.2 ), some shortcut files were created. > Installed using yum. We have faced some issues due to the creation of shortcut files instead of complete file. This seems to be the wrong list for this issue. jdk-dev list is about development of OpenJDK. Your issue seems to be downstream packaging related. Please open a bug with the vendor which provides the RPM you've installed via yum. Thanks, Severin From GopalaKrishna.Konakanchi at arcserve.com Tue Aug 4 14:22:45 2020 From: GopalaKrishna.Konakanchi at arcserve.com (Konakanchi, Gopala Krishna V N.) Date: Tue, 4 Aug 2020 14:22:45 +0000 Subject: Shortcut files created in openjdk installation In-Reply-To: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> References: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> Message-ID: Hi Gehwolf, Can you please let me know the vendor you have referred in the below mail?. As it was suggested to contact jdk-dev, I contacted. Please find the attached mail for the earlier conversation happened. Best Regards, Gopala Krishna. -----Original Message----- From: Severin Gehwolf Sent: Tuesday, August 4, 2020 5:09 PM To: Konakanchi, Gopala Krishna V N. ; jdk-dev at openjdk.java.net Subject: Re: Shortcut files created in openjdk installation On Tue, 2020-08-04 at 11:08 +0000, Konakanchi, Gopala Krishna V N. wrote: > Hi, > When we have installed open-jdk 1.8.0 on Red hat Linux instance ( AWS ec2-instance - red hat version 8.2 ), some shortcut files were created. > Installed using yum. We have faced some issues due to the creation of shortcut files instead of complete file. This seems to be the wrong list for this issue. jdk-dev list is about development of OpenJDK. Your issue seems to be downstream packaging related. Please open a bug with the vendor which provides the RPM you've installed via yum. Thanks, Severin -------------- next part -------------- An embedded message was scrubbed... From: Maurizio Cimadamore Subject: Re: Your message to ide-support-dev awaits moderator approval Date: Tue, 4 Aug 2020 10:52:36 +0000 Size: 11592 URL: From joe.darcy at oracle.com Tue Aug 4 16:14:54 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Tue, 4 Aug 2020 09:14:54 -0700 Subject: Shortcut files created in openjdk installation In-Reply-To: References: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> Message-ID: <00db5a4d-8549-6a5f-91ff-62a47d770bcf@oracle.com> Hello Gopala, Expanding Gehwolf's point, the aliases under openjdk.java.net are not general no-cost Java support channels and they should not be treated as such. Cheers, -Joe On 8/4/2020 7:22 AM, Konakanchi, Gopala Krishna V N. wrote: > Hi Gehwolf, > Can you please let me know the vendor you have referred in the below mail?. > As it was suggested to contact jdk-dev, I contacted. > Please find the attached mail for the earlier conversation happened. > > > Best Regards, > Gopala Krishna. > > From shade at redhat.com Wed Aug 5 10:35:47 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 5 Aug 2020 12:35:47 +0200 Subject: RFC (round 1), JEP draft: Low-level Object layout introspection methods Message-ID: Hi, I would like to solicit early feedback on "Low-level Object layout introspection methods" JEP: https://openjdk.java.net/jeps/8249196 It crosscuts core libraries and Hotspot, and therefore jdk-dev@ seems to be the mailing list to use for it. The work seems to be at the point where we understand the initial design and implementation constraints. Those are both reflected in JEP text (where possible), and in the prototype implementation (webrevs and builds are linked in JEP itself). The API itself is not cast in stone, and open for discussion. JEP text hopefully makes a good argument for each of new methods. Strong (dis)agreements about the existence of particular methods are good to have during this review, so we could swing the JEP direction early. The implementation itself is not the focus of this discussion. If you find implementation bugs/quirks, send me a direct email. Ditto if you have a drive-by comment and do not want to participate in more discussions. The API initial performance characteristics look good and briefly mentioned in JEP text. Robert Stupp had provided the experimental JAMM branch that hooks to new API, and it seems to considerably improve accuracy and performance. I also did a few touch-ups in JOL to bring the performance baseline for comparisons closer, and JOL is still quite far away performance-wise. (Logistics: This review does _not_ require prompt review, as many folks -- including myself -- are having vacations at this time. I shall be able to sift through replies and do JEP/implementation adjustments some time next week.) -- Thanks, -Aleksey From Alan.Bateman at oracle.com Wed Aug 5 12:03:01 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 5 Aug 2020 13:03:01 +0100 Subject: RFC (round 1), JEP draft: Low-level Object layout introspection methods In-Reply-To: References: Message-ID: On 05/08/2020 11:35, Aleksey Shipilev wrote: > Hi, > > I would like to solicit early feedback on "Low-level Object layout introspection methods" JEP: > https://openjdk.java.net/jeps/8249196 > > It crosscuts core libraries and Hotspot, and therefore jdk-dev@ seems to be the mailing list to use > for it. > > The work seems to be at the point where we understand the initial design and implementation > constraints. Those are both reflected in JEP text (where possible), and in the prototype > implementation (webrevs and builds are linked in JEP itself). > > The API itself is not cast in stone, and open for discussion. JEP text hopefully makes a good > argument for each of new methods. Strong (dis)agreements about the existence of particular methods > are good to have during this review, so we could swing the JEP direction early. > I think the consequences of exposing layout and address information in the standard API needs discussion. Although the intention seems to be for benign introspection by tools, and maybe a small number of super advanced libraries, I could imagine addressOf and fieldOffsetOf being misused. I don't have a good sense on how layout will be change in Project Valhalla with specialized generics to know if these APIs even make sense (I see you have a section already on the challenges with inline types). As you note in the JEP, the Instrumentation and JVM TI APIs are for tool agents, not libraries. The getObjectSIze method was meant for profilers and other tooling that wanted to estimate memory usage. Those APIs have not been updated since Java 5 and probably need to be modernized so that might be useful to separate out for its own investigation. You list "heap dumps" as a non-alternative. The reason that the built-in heap dump generates VM-independent HPROF format is because that was the only format that tooling could consume at the time (Java 6). That is something that could be separated out too as it might be an interesting project to re-visit this area, maybe even augment the heap dump with something that would help heap dump analyzer get a better sense of the memory usage. I see in the alternatives that you don't want to pursue moving JOL in the JDK. I think you could expand a bit more on this as I don't think "backports" or "support previous JDKs" are relevant, meaning I don't think it would need to be constrained by needing to work across different JDK builds/versions. Maybe this alternative is about more diagnostic options to print layout information, maybe it doesn't need a separate tool. I'm sure you will get lots of other feedback on the draft JEP. My main concern is implications of putting these low-level APIs into the java.* API, whether it creates attractive nuisance, and/or will they be regrets in the long term as layouts get more sophisticated. -Alan. From mark.reinhold at oracle.com Fri Aug 7 15:44:13 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 7 Aug 2020 08:44:13 -0700 (PDT) Subject: JDK 15: First Release Candidate Message-ID: <20200807154413.CB26D3BD844@eggemoggin.niobe.net> Per the JDK 15 schedule [1], we are now in the Release Candidate phase. The stabilization repository, jdk/jdk15, is open for P1 bug fixes per the JDK Release Process (JEP 3) [2]. All changes require approval via the Fix-Request Process [3]. If you?re responsible for any of the bugs on the RC candidate-bug list [4] then please see JEP 3 for guidance on how to handle them. There are no unresolved P1 bugs in build 35, so that is our first JDK 15 Release Candidate. Binaries available here, as usual: https://jdk.java.net/15 - Mark [1] https://openjdk.java.net/projects/jdk/15/#Schedule [2] https://openjdk.java.net/jeps/3 [3] https://openjdk.java.net/jeps/3#Fix-Request-Process [4] https://j.mp/jdk-rc From GopalaKrishna.Konakanchi at arcserve.com Fri Aug 7 16:46:09 2020 From: GopalaKrishna.Konakanchi at arcserve.com (Konakanchi, Gopala Krishna V N.) Date: Fri, 7 Aug 2020 16:46:09 +0000 Subject: Shortcut files created in openjdk installation In-Reply-To: <00db5a4d-8549-6a5f-91ff-62a47d770bcf@oracle.com> References: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> <00db5a4d-8549-6a5f-91ff-62a47d770bcf@oracle.com> Message-ID: Thank you Joe. Can you please help to suggest right vendor / point of contact we have to reach for this issue. Best Regards, Gopala Krishna. -----Original Message----- From: Joe Darcy Sent: Tuesday, August 4, 2020 9:45 PM To: Konakanchi, Gopala Krishna V N. ; Severin Gehwolf ; jdk-dev at openjdk.java.net Subject: Re: Shortcut files created in openjdk installation Hello Gopala, Expanding Gehwolf's point, the aliases under openjdk.java.net are not general no-cost Java support channels and they should not be treated as such. Cheers, -Joe On 8/4/2020 7:22 AM, Konakanchi, Gopala Krishna V N. wrote: > Hi Gehwolf, > Can you please let me know the vendor you have referred in the below mail?. > As it was suggested to contact jdk-dev, I contacted. > Please find the attached mail for the earlier conversation happened. > > > Best Regards, > Gopala Krishna. > > From joe.darcy at oracle.com Fri Aug 7 19:31:37 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 7 Aug 2020 12:31:37 -0700 Subject: Shortcut files created in openjdk installation In-Reply-To: References: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> <00db5a4d-8549-6a5f-91ff-62a47d770bcf@oracle.com> Message-ID: <1e710dec-2105-548e-d624-2f5ab061fa08@oracle.com> Hello Gopala, I suggest talking to the party you get your JDK from. Cheers, -Joe On 8/7/2020 9:46 AM, Konakanchi, Gopala Krishna V N. wrote: > Thank you Joe. > Can you please help to suggest right vendor / point of contact we have to reach for this issue. > > Best Regards, > Gopala Krishna. > > -----Original Message----- > From: Joe Darcy > Sent: Tuesday, August 4, 2020 9:45 PM > To: Konakanchi, Gopala Krishna V N. ; Severin Gehwolf ; jdk-dev at openjdk.java.net > Subject: Re: Shortcut files created in openjdk installation > > Hello Gopala, > > Expanding Gehwolf's point, the aliases under openjdk.java.net are not general no-cost Java support channels and they should not be treated as such. > > Cheers, > > -Joe > > On 8/4/2020 7:22 AM, Konakanchi, Gopala Krishna V N. wrote: >> Hi Gehwolf, >> Can you please let me know the vendor you have referred in the below mail?. >> As it was suggested to contact jdk-dev, I contacted. >> Please find the attached mail for the earlier conversation happened. >> >> >> Best Regards, >> Gopala Krishna. >> >> From GopalaKrishna.Konakanchi at arcserve.com Fri Aug 7 19:36:15 2020 From: GopalaKrishna.Konakanchi at arcserve.com (Konakanchi, Gopala Krishna V N.) Date: Fri, 7 Aug 2020 19:36:15 +0000 Subject: Shortcut files created in openjdk installation In-Reply-To: <1e710dec-2105-548e-d624-2f5ab061fa08@oracle.com> References: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> <00db5a4d-8549-6a5f-91ff-62a47d770bcf@oracle.com> <1e710dec-2105-548e-d624-2f5ab061fa08@oracle.com> Message-ID: Thank you Joe for the quick reply. We have tried by using yum command: sudo yum install java-1.8.0-openjdk-devel in the ec-2 red hat machine. As its open-jdk I thought open jdk is the point of contact we have reached you. Can you please let me know whether we need to reach out red hat / other vendor for this. Best Regards, Gopala Krishna. -----Original Message----- From: Joe Darcy Sent: Saturday, August 8, 2020 1:02 AM To: Konakanchi, Gopala Krishna V N. ; Severin Gehwolf Cc: jdk-dev at openjdk.java.net Subject: Re: Shortcut files created in openjdk installation Hello Gopala, I suggest talking to the party you get your JDK from. Cheers, -Joe On 8/7/2020 9:46 AM, Konakanchi, Gopala Krishna V N. wrote: > Thank you Joe. > Can you please help to suggest right vendor / point of contact we have to reach for this issue. > > Best Regards, > Gopala Krishna. > > -----Original Message----- > From: Joe Darcy > Sent: Tuesday, August 4, 2020 9:45 PM > To: Konakanchi, Gopala Krishna V N. ; Severin Gehwolf ; jdk-dev at openjdk.java.net > Subject: Re: Shortcut files created in openjdk installation > > Hello Gopala, > > Expanding Gehwolf's point, the aliases under openjdk.java.net are not general no-cost Java support channels and they should not be treated as such. > > Cheers, > > -Joe > > On 8/4/2020 7:22 AM, Konakanchi, Gopala Krishna V N. wrote: >> Hi Gehwolf, >> Can you please let me know the vendor you have referred in the below mail?. >> As it was suggested to contact jdk-dev, I contacted. >> Please find the attached mail for the earlier conversation happened. >> >> >> Best Regards, >> Gopala Krishna. >> >> From joe.darcy at oracle.com Fri Aug 7 19:50:06 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 7 Aug 2020 12:50:06 -0700 Subject: Shortcut files created in openjdk installation In-Reply-To: References: <8240f438cb7e8ce26bf41ba1f3d265a0fc8c7327.camel@redhat.com> <00db5a4d-8549-6a5f-91ff-62a47d770bcf@oracle.com> <1e710dec-2105-548e-d624-2f5ab061fa08@oracle.com> Message-ID: <2995760e-715d-f207-7384-67571fcfcd58@oracle.com> If you are looking for support, you should contact whichever company you have a JDK support contract with; there are a variable of alternatives. This will be my last message on this thread; cheers, -Joe On 8/7/2020 12:36 PM, Konakanchi, Gopala Krishna V N. wrote: > Thank you Joe for the quick reply. > We have tried by using yum command: sudo yum install java-1.8.0-openjdk-devel in the ec-2 red hat machine. > As its open-jdk I thought open jdk is the point of contact we have reached you. > Can you please let me know whether we need to reach out red hat / other vendor for this. > > > Best Regards, > Gopala Krishna. > > -----Original Message----- > From: Joe Darcy > Sent: Saturday, August 8, 2020 1:02 AM > To: Konakanchi, Gopala Krishna V N. ; Severin Gehwolf > Cc: jdk-dev at openjdk.java.net > Subject: Re: Shortcut files created in openjdk installation > > Hello Gopala, > > I suggest talking to the party you get your JDK from. > > Cheers, > > -Joe > > On 8/7/2020 9:46 AM, Konakanchi, Gopala Krishna V N. wrote: >> Thank you Joe. >> Can you please help to suggest right vendor / point of contact we have to reach for this issue. >> >> Best Regards, >> Gopala Krishna. >> >> -----Original Message----- >> From: Joe Darcy >> Sent: Tuesday, August 4, 2020 9:45 PM >> To: Konakanchi, Gopala Krishna V N. ; Severin Gehwolf ; jdk-dev at openjdk.java.net >> Subject: Re: Shortcut files created in openjdk installation >> >> Hello Gopala, >> >> Expanding Gehwolf's point, the aliases under openjdk.java.net are not general no-cost Java support channels and they should not be treated as such. >> >> Cheers, >> >> -Joe >> >> On 8/4/2020 7:22 AM, Konakanchi, Gopala Krishna V N. wrote: >>> Hi Gehwolf, >>> Can you please let me know the vendor you have referred in the below mail?. >>> As it was suggested to contact jdk-dev, I contacted. >>> Please find the attached mail for the earlier conversation happened. >>> >>> >>> Best Regards, >>> Gopala Krishna. >>> >>> From brian.goetz at oracle.com Fri Aug 7 19:53:28 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 7 Aug 2020 15:53:28 -0400 Subject: RFC (round 1), JEP draft: Low-level Object layout introspection methods In-Reply-To: References: Message-ID: > I would like to solicit early feedback on "Low-level Object layout > introspection methods" JEP: > https://openjdk.java.net/jeps/8249196 I enjoyed reading this JEP draft; clearly a lot of thought has gone into the technical details.? I find much to like, and some to dislike, in this proposal. First, the good news.? I am sympathetic to the needs that certain libraries have to assess memory usage of objects, such as for cache eviction.? While to some level this is an impossible problem (how do we know whether an object reference buried in a graph is aliased or not?), providing users with better ways of measuring cost than simply counting cache entries is a noble cause, and the work you've done on `estimateSize` definitely is a step forward. I am also sympathetic for the desire to better understand layout, for purposes of optimizing footprint ("cool, I can stash this field in the alignment shadow") and cache efficiency ("how do I get these fields so they are more likely on the same cache line.")? While such feedback can never be relied on 100%, I agree it can be useful to have this information so that one can make more accurate estimates of cost. Now, the bad news: I have deep objections to several of the sub-features in this JEP (unfortunately some of them even overlap with the second point above.)? Specifically, I have deep misgivings about exposing field offsets, and deeper misgivings about exposing object addresses. For context, remember that we are in the middle of a deliberate, decade-long transition away from Unsafe.? We knew that we can't just turn off Unsafe immediately, but we also knew we had to wean people from Unsafe -- and not only from the specific class, but from some of the concepts.? And of course we can't do so without providing good-enough replacements for at least some of the use cases, so it will necessarily take time.? (Reasonable people can reasonably disagree on the meaning of "good enough" and which use cases, but that's a separate discussion.) The down payment on this plan was VarHandles; a key goal here was obviate the need for using Unsafe to access data in the Java heap. Naturally it took some time to flesh out the feature set, shake out the performance, and adapt the JDK code to use it, but the goal is clear -- there should be no excuse for using Unsafe to access the Java heap at all.?? And we're? well on the way. People still use Unsafe to access off-heap memory, but we're working on a plan there too -- the Panama foreign memory access API. This is less mature than VarHandles, but the goal is similar; by the time we're done, there should be no excuse to use Unsafe for access to memory at all, ever, because there will be better, safer, supported APIs that do the same thing with comparable performance. (Similarly, Panama has a notion of Layout, and it is possible that, over time, it might be possible to get a Layout for a Java object, and access it through the safer Panama APIs.) The very model that Unsafe assumes for on-heap data (and therefore encourages users to assume) -- that a field offset is a fixed thing that can be relied on -- undermines the VMs ability to optimize.? As just one example, the VM could dynamically redo layout based on profiling data (putting frequently-accessed-together fields so they are in the same cache line) and rewrite existing instances during GC.? In a world where even one user can have Unsafe, then _no one_ can ever have this optimization.? This seems a bad trade! Unfortunately the proposal to include field offset and address information in the API grabs the ball and runs not only to the opposite goal line, but into the next stadium -- by elevating this model to "something you rely on at your own risk" to "something we promise Java will be constrained by until the end of time."? No thank you!? And it seems especially bad to pour fresh concrete on "no one can have this ever" when we're five years into jackhammering away the old dam. In fact, the next methods I would like to _remove_ from Unsafe are the get-field-offset and get-object-address methods.?? You might reply you are enabling this goal, but that would be mistaking the letter for the spirit: I want people to stop assuming that fields HAVE an offset, or that Java objects HAVE an address.? And, what else would someone do with the address and offset, other than provide it to Unsafe (or to the native equivalent)?? This is propping up the wrong model, and runs counter to the long-term direction we've been on for years, and are already halfway there on. So, while there are some things in this JEP that seem quite cool and which I enthusiastically support, there are some fundamental assumptions -- that field offsets and object addresses should be accessible to Java code -- that I disagree with. Setting aside philosophy and stewardship for a moment, I think what's really going on is that this JEP is trying to serve two different audiences: those that are happy to have estimates and/or use the information for making approximate decisions, and those who are interested in low-level hackery for accessing the Java heap, rather than through `getfield` and `putfield`.? I think if you were to split this into two JEPs, you would find enthusiastic support for the first, and strong resistance to the second. To the extent that there is overlap between these groups (e.g., fieldOffsetOf is useful to offline tools that may be used to estimate layouts, as well as online users who would be tempted to believe that this is really a field offset), some creativity will be required to keep the first from becoming an attractive nuisance to the second. So, my feedback is "glass half full" -- I'm happy to see the JEP that provides estimates and such.? My recommendation is to focus on that half for now. Cheers, -Brian From christian.tornqvist at oracle.com Mon Aug 10 21:42:42 2020 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Mon, 10 Aug 2020 14:42:42 -0700 Subject: jdk/submit repository reset Message-ID: <8878D6A6-DC58-4D1A-BC04-3AED51C046FA@oracle.com> Hello, The jdk/submit repository was reset. A new repository was created today with no branches except from the default one. http://hg.openjdk.java.net/jdk/submit In order to continue using the submit repo please make a fresh clone and run ?hg out? before push. Also an archive of the old jdk/submit was created https://hg.openjdk.java.net/jdk/submit-archive-2020-08-10 Please continue using the system and report any issues/ask questions related to the jdk/submit repo using ops at openjdk.java.net. Thanks, Christian From shade at redhat.com Tue Aug 11 07:28:00 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 11 Aug 2020 09:28:00 +0200 Subject: jdk/submit repository reset In-Reply-To: <8878D6A6-DC58-4D1A-BC04-3AED51C046FA@oracle.com> References: <8878D6A6-DC58-4D1A-BC04-3AED51C046FA@oracle.com> Message-ID: On 8/10/20 11:42 PM, Christian Tornqvist wrote: > The jdk/submit repository was reset. > A new repository was created today with no branches except from the default one. > > http://hg.openjdk.java.net/jdk/submit As usual, re-cloned workspace bundle is regenerated here: https://builds.shipilev.net/workspaces/jdk-submit.tar.xz -- Thanks, -Aleksey From shade at redhat.com Tue Aug 11 10:22:24 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 11 Aug 2020 12:22:24 +0200 Subject: RFC (round 1), JEP draft: Low-level Object layout introspection methods In-Reply-To: References:

Message-ID: <787faac2-f156-5da1-4b82-e9318d25d6da@redhat.com> Hi Alan, Thanks for taking a look! On 8/5/20 2:03 PM, Alan Bateman wrote: > On 05/08/2020 11:35, Aleksey Shipilev wrote: > As you note in the JEP, the Instrumentation and JVM TI APIs are for tool > agents, not libraries. The getObjectSIze method was meant for profilers > and other tooling that wanted to estimate memory usage. Those APIs have > not been updated since Java 5 and probably need to be modernized so that > might be useful to separate out for its own investigation. You list > "heap dumps" as a non-alternative. The reason that the built-in heap > dump generates VM-independent HPROF format is because that was the only > format that tooling could consume at the time (Java 6). That is > something that could be separated out too as it might be an interesting > project to re-visit this area, maybe even augment the heap dump with > something that would help heap dump analyzer get a better sense of the > memory usage. Yes, maybe. I don't want to creep the scope of this work, though. > I see in the alternatives that you don't want to pursue moving JOL in > the JDK. I think you could expand a bit more on this as I don't think > "backports" or "support previous JDKs" are relevant, meaning I don't > think it would need to be constrained by needing to work across > different JDK builds/versions. Maybe this alternative is about more > diagnostic options to print layout information, maybe it doesn't need a > separate tool. I don't quite see how moving JOL to JDK helps. I added that alternative after your initial review, so please help me out here? If we move the JOL in its current form, then all the new APIs would be accessible through JOL. In other words, it would be the same API with extra steps. Which means all arguments about exposing offsets, addresses and apply there as well. If we are talking about moving JOL and then disabling some of its features (e.g. getting offsets and addresses), this is not _really_ moving JOL, but rather a shallow version of it. My argument about backports and supporting previous JDK still stands, and here is why. Currently, JOL supports JDK 6..15. Every little bug that is found in JOL is automatically "released" for every supported JDK version. If JOL is inside JDK, that means those bugfixes would need to be backported to previous JDK releases. Basically, that asks JOL maintainers (well, me) to maintain several forked versions of JOL. You can counter that with saying that only the latest version matters, but this is where we would disagree philosophically. > I'm sure you will get lots of other feedback on the draft JEP. My main > concern is implications of putting these low-level APIs into the java.* > API, whether it creates attractive nuisance, and/or will they be regrets > in the long term as layouts get more sophisticated. Quite. I believe JEP calls that out in Risks and Assumptions? I think the weasel-wording the specification text is necessary to prevent _the informational APIs_ from blocking future VM work. I was trying to capture that in many places in the text. Is that still not transparent? More answers in reply to Brian... -- Thanks, -Aleksey From shade at redhat.com Tue Aug 11 10:22:35 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 11 Aug 2020 12:22:35 +0200 Subject: RFC (round 1), JEP draft: Low-level Object layout introspection methods In-Reply-To: References:

Message-ID: <6a9abead-1bc6-336d-b11c-6e6e9ab8983f@redhat.com> Hi Brian, Thanks for taking a look! I think we need to get one thing very straight: these are best-effort informational APIs. They are not required to work 100% reliably. They are expected to provide reasonable answers where possible, until something else gets in the way, including disabled feature flags, corner cases where current and future optimizations turn the answers ambiguous, future work where layouts change on the fly, etc. I tried to be clear about this in JEP text, does that intent register when you read the text? On 8/7/20 9:53 PM, Brian Goetz wrote: > People still use Unsafe to access off-heap memory, but we're working on a plan there too -- the > Panama foreign memory access API. This is less mature than VarHandles, but the goal is similar; by > the time we're done, there should be no excuse to use Unsafe for access to memory at all, ever, > because there will be better, safer, supported APIs that do the same thing with comparable > performance.? (Similarly, Panama has a notion of Layout, and it is possible that, over time, it > might be possible to get a Layout for a Java object, and access it through the safer Panama APIs.)? OK, so let's look a bit forward. Assume Panama does the layout for Java objects. Does it mean that current APIs, that is the MemoryLayout.{bitOffset|byteOffset}: https://hg.openjdk.java.net/jdk/jdk/file/60612063f75a/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryLayout.java#l332 ...apply to Java fields as well? This looks similar to what proposed fieldOffsetOf does. This seems to run counter to the idea that "fields do not have offsets", and that must mean Panama should never do this for heap objects? > The very model that Unsafe assumes for on-heap data (and therefore encourages users to assume) -- > that a field offset is a fixed thing that can be relied on -- undermines the VMs ability to > optimize.? As just one example, the VM could dynamically redo layout based on profiling data > (putting frequently-accessed-together fields so they are in the same cache line) and rewrite > existing instances during GC.? In a world where even one user can have Unsafe, then _no one_ can > ever have this optimization.? This seems a bad trade! Yes, I agree with this part. > Unfortunately the proposal to include field offset and address information in the API grabs the > ball and runs not only to the opposite goal line, but into the next stadium -- by elevating this > model to "something you rely on at your own risk" to "something we promise Java will be > constrained by until the end of time." No thank you! And it seems especially bad to pour fresh > concrete on "no one can have this ever" when we're five years into jackhammering away the old > dam. But I disagree with this part. I don't think the existence of the best-effort informational APIs undermine the future work. If anything comes up that goes against those APIs, then APIs are told to get off our collective lawn, and start replying "don't know". But until that (and even if that) happens, it seems odd to deprive people from getting useful best-effort answers today. > In fact, the next methods I would like to _remove_ from Unsafe are the get-field-offset and > get-object-address methods.?? You might reply you are enabling this goal, but that would be > mistaking the letter for the spirit: I want people to stop assuming that fields HAVE an offset, or > that Java objects HAVE an address.? And, what else would someone do with the address and offset, > other than provide it to Unsafe (or to the native equivalent)?? Before I address the core of this argument below, let me mention there is the answer to that question in JEP text: JOL would introspect the internal and external object layout; Java code would test that padding really works; etc. It would be a mistake to paint Unsafe as the only use. > So, while there are some things in this JEP that seem quite cool and which I enthusiastically > support, there are some fundamental assumptions -- that field offsets and object addresses should be > accessible to Java code -- that I disagree with. I agree that from the pure Java standpoint, language objects/fields are more or less the platonic ideals of themselves. Nothing is really known about them, except the limited set of their expected behaviors. But we cannot be oblivious that implementation details also matter. I would argue that object/field sizes are also such an implementation detail. Yet, we seem to agree that object/field sizes are good to know. So it follows that the problem here is not object/field sizes being the implementation details. We are able to put fundamental/idealistic assumptions on the side, if we really want it. A funny example of such the implementation details concession is not even JVMTI/Instrumentation, but the core libraries themselves: https://docs.oracle.com/javase/8/docs/api/java/lang/Integer.html#SIZE I guess the real question is where to put something that goes beyond the "pure Java" and provides the implementation details. So it could avoid the impression of telling "pure Java" facts? I put prototype methods in Runtime, because that looked like one of the least "pure Java" facilities. I'd be happy to learn some other place where such API can be put :) > Setting aside philosophy and stewardship for a moment, I think what's really going on is that this > JEP is trying to serve two different audiences: those that are happy to have estimates and/or use > the information for making approximate decisions, and those who are interested in low-level hackery > for accessing the Java heap, rather than through `getfield` and `putfield`.? I think if you were to > split this into two JEPs, you would find enthusiastic support for the first, and strong resistance > to the second.? With my JEP author hat on: JEP caters for low-level introspection use case, such as JOL and JAMM. It does not intend to cater for "low-level hackery for accessing the Java heap", and it tries to specifically make the argument that new APIs do not allow this low-level hackery beyond what is possible today and in a foreseeable future. I can see that the JEP can be split in "non-controversial" (sizeOf, deepSizeOf) and "controversial" (addressOf, fieldOffsetOf) parts. For the process overhead reasons, I'd push forward with just one JEP at the moment, though. > To the extent that there is overlap between these groups (e.g., fieldOffsetOf is useful to offline > tools that may be used to estimate layouts, as well as online users who would be tempted to believe > that this is really a field offset), some creativity will be required to keep the first from > becoming an attractive nuisance to the second. I am open for suggestions :) One of the reasons the prototype has RuntimeAddressOf and RuntimeOffsetsOf JVM flags (as much as I hate having another JVM option) is to consider if they can be disabled by default, so online users would have to explicitly opt-in for using these facilities: https://builds.shipilev.net/patch-openjdk-jdk-jep-8249196/src/hotspot/share/runtime/globals.hpp.sdiff.html I do believe the Java API is useful, to enable Java-only libraries like JOL. Otherwise, the argument could be made to make fieldOffsetOf-like data available by turning PrintFieldLayout into product VM flag, and then let JOL parse the output of forked JVM. That's tedious, but doable. addressOf cannot be done like that, because it requires passing the object reference somehow. > So, my feedback is "glass half full" -- I'm happy to see the JEP that provides estimates and such.? > My recommendation is to focus on that half for now. Let me ask the specific question: which APIs do you like, and which do you dislike? I gather you like: public static long sizeOf(Object obj); public static long deepSizeOf(Object obj); public static long deepSizeOf(Object obj, ToLongFunction