From zoltan.jose at gmail.com Wed Apr 1 06:33:48 2020 From: zoltan.jose at gmail.com (Timmy Jose) Date: Wed, 1 Apr 2020 12:03:48 +0530 Subject: Call for Discussion: New Project: Developer's Guide In-Reply-To: <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> References: <62FA7D5A-5D25-4617-853F-3B0BC09CE154@oracle.com> <0D9DE337-FD2D-41ED-BA1C-832ACF2D1988@gmail.com> <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> Message-ID: I think that this is a brilliant idea. The existing developer guide is indeed very outdated, and an up-to-date, easy-to-locate guide is definitely required. It would make things so much easier for people new to the process such as myself. If possible, I would also like to be part of this initiative - maybe it would also help me get onboarded myself. Best, Timmy On Tue, Mar 31, 2020 at 5:01 AM Jesper Wilhelmsson < jesper.wilhelmsson at oracle.com> wrote: > Hi Mohammed. > > I'm happy to hear that you are interested in this project. The next step > is the vote that has been initiated now. All Members of the OpenJDK have a > vote and if they approve to creating the project it will be created. Once > the project is in place we can start discussing the future. > > Thanks, > /Jesper > > > On 29 Mar 2020, at 11:43, Mohammed Al-Moayed > wrote: > > > > Hi All > > > > I am interested to be part of this. > > > > Please, let me know next steps > > > > Regards > > Mohammed Almo > > > >> On Mar 12, 2020, at 2:51 AM, Jesper Wilhelmsson < > jesper.wilhelmsson at oracle.com> wrote: > >> > >> ?Hi. > >> > >> I would like to discuss the possible creation of the Developer's Guide > Project > >> with myself, Jesper Wilhelmsson, as the lead and the Core-Libs, > Compiler, and > >> Hotspot groups as the sponsoring Groups. > >> > >> The initial goal of the project would be to create up-to-date > guidelines for > >> OpenJDK development and contributions. The intent is to update the > existing > >> OpenJDK Developer's Guide [1] which hasn't been updated since 2012. > Parts of > >> the existing guide are in need of updates while other parts are yet to > be > >> written. The initial source code for this project will be based on the > current > >> OpenJDK Developer's Guide. > >> > >> In more recent years some process related information has been > published on the > >> OpenJDK wiki [2], but this information was never reviewed or approved > by the > >> community. Even though large parts of this information may be accurate, > it needs > >> to go through proper review before it can be seen as official > guidelines for > >> OpenJDK development. > >> > >> Once the initial goal has been reached the project will transition into > a > >> maintenance project to keep the OpenJDK Developer's Guide up to date. > >> > >> The OpenJDK Developer's Guide is intended to contain tutorial style > texts that > >> give examples and step-by-step directions for common parts of the > development > >> process. Strict rules for OpenJDK development are better suited to a > process > >> JEP. The development of such a JEP is outside of the scope of this > project but > >> will be developed as part of a separate effort in parallel. > >> > >> Initial committers of the Developer's Guide Project include Mark > Reinhold, Iris > >> Clark, and Jesper Wilhelmsson. Additional committers may be identified > through > >> this discussion. > >> > >> The project will host at least one mailing list, > guide-dev at openjdk.java.net, > >> for discussions and reviews of changes. > >> > >> Thanks, > >> /Jesper > >> > >> [1] http://openjdk.java.net/guide/ > >> [2] https://wiki.openjdk.java.net/ > >> > > From hufeng1987 at gmail.com Wed Apr 1 08:38:25 2020 From: hufeng1987 at gmail.com (Netroby) Date: Wed, 1 Apr 2020 16:38:25 +0800 Subject: Call for Discussion: New Project: Developer's Guide In-Reply-To: References: <62FA7D5A-5D25-4617-853F-3B0BC09CE154@oracle.com> <0D9DE337-FD2D-41ED-BA1C-832ACF2D1988@gmail.com> <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> Message-ID: How about create a git repository, using some document tools, like facebook/docusaurus. Then every one can easy contribute to the guide, and add I18n translate. Appreciate your time. ---------------------------- Netroby Timmy Jose ?2020?4?1??? ??2:37??? > > I think that this is a brilliant idea. The existing developer guide is > indeed very outdated, and an up-to-date, easy-to-locate guide is definitely > required. It would make things so much easier for people new to the process > such as myself. If possible, I would also like to be part of this > initiative - maybe it would also help > me get onboarded myself. > > Best, > > Timmy > > On Tue, Mar 31, 2020 at 5:01 AM Jesper Wilhelmsson < > jesper.wilhelmsson at oracle.com> wrote: > > > Hi Mohammed. > > > > I'm happy to hear that you are interested in this project. The next step > > is the vote that has been initiated now. All Members of the OpenJDK have a > > vote and if they approve to creating the project it will be created. Once > > the project is in place we can start discussing the future. > > > > Thanks, > > /Jesper > > > > > On 29 Mar 2020, at 11:43, Mohammed Al-Moayed > > wrote: > > > > > > Hi All > > > > > > I am interested to be part of this. > > > > > > Please, let me know next steps > > > > > > Regards > > > Mohammed Almo > > > > > >> On Mar 12, 2020, at 2:51 AM, Jesper Wilhelmsson < > > jesper.wilhelmsson at oracle.com> wrote: > > >> > > >> ?Hi. > > >> > > >> I would like to discuss the possible creation of the Developer's Guide > > Project > > >> with myself, Jesper Wilhelmsson, as the lead and the Core-Libs, > > Compiler, and > > >> Hotspot groups as the sponsoring Groups. > > >> > > >> The initial goal of the project would be to create up-to-date > > guidelines for > > >> OpenJDK development and contributions. The intent is to update the > > existing > > >> OpenJDK Developer's Guide [1] which hasn't been updated since 2012. > > Parts of > > >> the existing guide are in need of updates while other parts are yet to > > be > > >> written. The initial source code for this project will be based on the > > current > > >> OpenJDK Developer's Guide. > > >> > > >> In more recent years some process related information has been > > published on the > > >> OpenJDK wiki [2], but this information was never reviewed or approved > > by the > > >> community. Even though large parts of this information may be accurate, > > it needs > > >> to go through proper review before it can be seen as official > > guidelines for > > >> OpenJDK development. > > >> > > >> Once the initial goal has been reached the project will transition into > > a > > >> maintenance project to keep the OpenJDK Developer's Guide up to date. > > >> > > >> The OpenJDK Developer's Guide is intended to contain tutorial style > > texts that > > >> give examples and step-by-step directions for common parts of the > > development > > >> process. Strict rules for OpenJDK development are better suited to a > > process > > >> JEP. The development of such a JEP is outside of the scope of this > > project but > > >> will be developed as part of a separate effort in parallel. > > >> > > >> Initial committers of the Developer's Guide Project include Mark > > Reinhold, Iris > > >> Clark, and Jesper Wilhelmsson. Additional committers may be identified > > through > > >> this discussion. > > >> > > >> The project will host at least one mailing list, > > guide-dev at openjdk.java.net, > > >> for discussions and reviews of changes. > > >> > > >> Thanks, > > >> /Jesper > > >> > > >> [1] http://openjdk.java.net/guide/ > > >> [2] https://wiki.openjdk.java.net/ > > >> > > > > From magnus.ihse.bursie at oracle.com Wed Apr 1 10:08:13 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 1 Apr 2020 12:08:13 +0200 Subject: CFV: New Project: Developer's Guide In-Reply-To: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> References: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> Message-ID: <5b46a78f-d624-1835-d47e-9f52f02f23dc@oracle.com> Vote: yes /Magnus On 2020-03-30 23:35, Jesper Wilhelmsson wrote: > I hereby propose the creation of the Developer's Guide Project with myself, > Jesper Wilhelmsson, as the lead and the Core Libraries, Compiler, and HotSpot > Groups as the sponsoring Groups. > > The suggested project has been out for discussion on discuss at openjdk.java.net > for two weeks [1]. It was well-received and I haven't noted any concerns with > creating the project in that discussion. > > The initial goal of the project would be to create up-to-date guidelines for > OpenJDK development and contributions. The intent is to update the existing > OpenJDK Developer's Guide [2] which hasn't been updated since 2012. Parts of > the existing guide are in need of updates while other parts are yet to be > written. The initial source code for this project will be based on the current > OpenJDK Developer's Guide. > > In more recent years some process related information has been published on the > OpenJDK wiki [3], but this information was never reviewed or approved by the > community. Even though large parts of this information may be accurate, it > needs to go through proper review before it can be seen as official guidelines > for OpenJDK development. > > Once the initial goal has been reached the project will transition into a > maintenance project to keep the OpenJDK Developer's Guide up to date. > > The OpenJDK Developer's Guide is intended to contain tutorial style texts that > give examples and step-by-step directions for common parts of the development > process. Strict rules for OpenJDK development are better suited to a process > JEP. The development of such a JEP is outside of the scope of this project but > will be developed as part of a separate effort in parallel. > > I (Jesper Wilhelmsson) am part of the Java Platform Group at Oracle and have > been working with virtual machine development since 2001. My initial focus was > on memory management and garbage collection but over the last few years I've > been more involved in gatekeeping and process questions around the OpenJDK. > I authored several of the guides available on the OpenJDK HotSpot wiki and I > regularly give talks on how to contribute to the OpenJDK. > > Initial committers of the Developer's Guide Project include Mark Reinhold (mr), > Iris Clark (iris), and Jesper Wilhelmsson (jwilhelm). A complete list will be > provided to the registrar at the successful conclusion of this CFV. > > The project will host at least one mailing list, guide-dev at openjdk.java.net, > for discussions and reviews of changes. > > Votes are due by 5pm PDT on Tuesday April 14 2020. > > Only current OpenJDK Members [4] are eligible to vote on this motion. > Votes must be cast in the open on the discuss list. Replying to this message > is sufficient if your mail program honors the Reply-To header. > > For Lazy Consensus voting instructions, see [5]. > > Thanks, > /Jesper > > [1] https://mail.openjdk.java.net/pipermail/discuss/2020-March/005298.html > [2] http://openjdk.java.net/guide/ > [3] https://wiki.openjdk.java.net/ > [4] http://openjdk.java.net/census#members > [5] http://openjdk.java.net/projects/#new-project-vote > From jesper.wilhelmsson at oracle.com Wed Apr 1 16:38:32 2020 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 1 Apr 2020 18:38:32 +0200 Subject: Call for Discussion: New Project: Developer's Guide In-Reply-To: References: <62FA7D5A-5D25-4617-853F-3B0BC09CE154@oracle.com> <0D9DE337-FD2D-41ED-BA1C-832ACF2D1988@gmail.com> <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> Message-ID: <6AD965BE-8C69-4333-A47F-7DC7AA0B603B@oracle.com> Hi Netroby. As the project will be an OpenJDK project we will use the same infrastructure as all other OpenJDK projects. Contributions will be handled in the same fashion as in other OpenJDK projects through reviews and pushes made by Contributors in the project. There are some best practices in place already for how to handle markdown and documentation in OpenJDK so we'll likely follow that as well. Cheers, /Jesper > On 1 Apr 2020, at 10:38, Netroby wrote: > > How about create a git repository, using some document tools, like > facebook/docusaurus. > Then every one can easy contribute to the guide, and add I18n translate. > > Appreciate your time. > ---------------------------- > Netroby > > Timmy Jose ?2020?4?1??? ??2:37??? >> >> I think that this is a brilliant idea. The existing developer guide is >> indeed very outdated, and an up-to-date, easy-to-locate guide is definitely >> required. It would make things so much easier for people new to the process >> such as myself. If possible, I would also like to be part of this >> initiative - maybe it would also help >> me get onboarded myself. >> >> Best, >> >> Timmy >> >> On Tue, Mar 31, 2020 at 5:01 AM Jesper Wilhelmsson < >> jesper.wilhelmsson at oracle.com> wrote: >> >>> Hi Mohammed. >>> >>> I'm happy to hear that you are interested in this project. The next step >>> is the vote that has been initiated now. All Members of the OpenJDK have a >>> vote and if they approve to creating the project it will be created. Once >>> the project is in place we can start discussing the future. >>> >>> Thanks, >>> /Jesper >>> >>>> On 29 Mar 2020, at 11:43, Mohammed Al-Moayed >>> wrote: >>>> >>>> Hi All >>>> >>>> I am interested to be part of this. >>>> >>>> Please, let me know next steps >>>> >>>> Regards >>>> Mohammed Almo >>>> >>>>> On Mar 12, 2020, at 2:51 AM, Jesper Wilhelmsson < >>> jesper.wilhelmsson at oracle.com> wrote: >>>>> >>>>> ?Hi. >>>>> >>>>> I would like to discuss the possible creation of the Developer's Guide >>> Project >>>>> with myself, Jesper Wilhelmsson, as the lead and the Core-Libs, >>> Compiler, and >>>>> Hotspot groups as the sponsoring Groups. >>>>> >>>>> The initial goal of the project would be to create up-to-date >>> guidelines for >>>>> OpenJDK development and contributions. The intent is to update the >>> existing >>>>> OpenJDK Developer's Guide [1] which hasn't been updated since 2012. >>> Parts of >>>>> the existing guide are in need of updates while other parts are yet to >>> be >>>>> written. The initial source code for this project will be based on the >>> current >>>>> OpenJDK Developer's Guide. >>>>> >>>>> In more recent years some process related information has been >>> published on the >>>>> OpenJDK wiki [2], but this information was never reviewed or approved >>> by the >>>>> community. Even though large parts of this information may be accurate, >>> it needs >>>>> to go through proper review before it can be seen as official >>> guidelines for >>>>> OpenJDK development. >>>>> >>>>> Once the initial goal has been reached the project will transition into >>> a >>>>> maintenance project to keep the OpenJDK Developer's Guide up to date. >>>>> >>>>> The OpenJDK Developer's Guide is intended to contain tutorial style >>> texts that >>>>> give examples and step-by-step directions for common parts of the >>> development >>>>> process. Strict rules for OpenJDK development are better suited to a >>> process >>>>> JEP. The development of such a JEP is outside of the scope of this >>> project but >>>>> will be developed as part of a separate effort in parallel. >>>>> >>>>> Initial committers of the Developer's Guide Project include Mark >>> Reinhold, Iris >>>>> Clark, and Jesper Wilhelmsson. Additional committers may be identified >>> through >>>>> this discussion. >>>>> >>>>> The project will host at least one mailing list, >>> guide-dev at openjdk.java.net, >>>>> for discussions and reviews of changes. >>>>> >>>>> Thanks, >>>>> /Jesper >>>>> >>>>> [1] http://openjdk.java.net/guide/ >>>>> [2] https://wiki.openjdk.java.net/ >>>>> >>> >>> From jesper.wilhelmsson at oracle.com Wed Apr 1 16:41:34 2020 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 1 Apr 2020 18:41:34 +0200 Subject: Call for Discussion: New Project: Developer's Guide In-Reply-To: References: <62FA7D5A-5D25-4617-853F-3B0BC09CE154@oracle.com> <0D9DE337-FD2D-41ED-BA1C-832ACF2D1988@gmail.com> <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> Message-ID: Hi Timmy. To be part of the project just join the guide-dev email list once it has been created :-) Looking forward to see you there! /Jesper > On 1 Apr 2020, at 08:33, Timmy Jose wrote: > > I think that this is a brilliant idea. The existing developer guide is indeed very outdated, and an up-to-date, easy-to-locate guide is definitely required. It would make things so much easier for people new to the process such as myself. If possible, I would also like to be part of this initiative - maybe it would also help > me get onboarded myself. > > Best, > > Timmy > > On Tue, Mar 31, 2020 at 5:01 AM Jesper Wilhelmsson > wrote: > Hi Mohammed. > > I'm happy to hear that you are interested in this project. The next step is the vote that has been initiated now. All Members of the OpenJDK have a vote and if they approve to creating the project it will be created. Once the project is in place we can start discussing the future. > > Thanks, > /Jesper > > > On 29 Mar 2020, at 11:43, Mohammed Al-Moayed > wrote: > > > > Hi All > > > > I am interested to be part of this. > > > > Please, let me know next steps > > > > Regards > > Mohammed Almo > > > >> On Mar 12, 2020, at 2:51 AM, Jesper Wilhelmsson > wrote: > >> > >> ?Hi. > >> > >> I would like to discuss the possible creation of the Developer's Guide Project > >> with myself, Jesper Wilhelmsson, as the lead and the Core-Libs, Compiler, and > >> Hotspot groups as the sponsoring Groups. > >> > >> The initial goal of the project would be to create up-to-date guidelines for > >> OpenJDK development and contributions. The intent is to update the existing > >> OpenJDK Developer's Guide [1] which hasn't been updated since 2012. Parts of > >> the existing guide are in need of updates while other parts are yet to be > >> written. The initial source code for this project will be based on the current > >> OpenJDK Developer's Guide. > >> > >> In more recent years some process related information has been published on the > >> OpenJDK wiki [2], but this information was never reviewed or approved by the > >> community. Even though large parts of this information may be accurate, it needs > >> to go through proper review before it can be seen as official guidelines for > >> OpenJDK development. > >> > >> Once the initial goal has been reached the project will transition into a > >> maintenance project to keep the OpenJDK Developer's Guide up to date. > >> > >> The OpenJDK Developer's Guide is intended to contain tutorial style texts that > >> give examples and step-by-step directions for common parts of the development > >> process. Strict rules for OpenJDK development are better suited to a process > >> JEP. The development of such a JEP is outside of the scope of this project but > >> will be developed as part of a separate effort in parallel. > >> > >> Initial committers of the Developer's Guide Project include Mark Reinhold, Iris > >> Clark, and Jesper Wilhelmsson. Additional committers may be identified through > >> this discussion. > >> > >> The project will host at least one mailing list, guide-dev at openjdk.java.net , > >> for discussions and reviews of changes. > >> > >> Thanks, > >> /Jesper > >> > >> [1] http://openjdk.java.net/guide/ > >> [2] https://wiki.openjdk.java.net/ > >> > From zoltan.jose at gmail.com Wed Apr 1 17:33:13 2020 From: zoltan.jose at gmail.com (Timmy Jose) Date: Wed, 1 Apr 2020 23:03:13 +0530 Subject: Call for Discussion: New Project: Developer's Guide In-Reply-To: References: <62FA7D5A-5D25-4617-853F-3B0BC09CE154@oracle.com> <0D9DE337-FD2D-41ED-BA1C-832ACF2D1988@gmail.com> <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> Message-ID: Hey Jesper, That sounds good. Will do! :-) Cheers, Timmy On Wed, 1 Apr, 2020, 10:13 PM Jesper Wilhelmsson, < jesper.wilhelmsson at oracle.com> wrote: > Hi Timmy. > > To be part of the project just join the guide-dev email list once it has > been created :-) > Looking forward to see you there! > /Jesper > > On 1 Apr 2020, at 08:33, Timmy Jose wrote: > > I think that this is a brilliant idea. The existing developer guide is > indeed very outdated, and an up-to-date, easy-to-locate guide is definitely > required. It would make things so much easier for people new to the process > such as myself. If possible, I would also like to be part of this > initiative - maybe it would also help > me get onboarded myself. > > Best, > > Timmy > > On Tue, Mar 31, 2020 at 5:01 AM Jesper Wilhelmsson < > jesper.wilhelmsson at oracle.com> wrote: > >> Hi Mohammed. >> >> I'm happy to hear that you are interested in this project. The next step >> is the vote that has been initiated now. All Members of the OpenJDK have a >> vote and if they approve to creating the project it will be created. Once >> the project is in place we can start discussing the future. >> >> Thanks, >> /Jesper >> >> > On 29 Mar 2020, at 11:43, Mohammed Al-Moayed >> wrote: >> > >> > Hi All >> > >> > I am interested to be part of this. >> > >> > Please, let me know next steps >> > >> > Regards >> > Mohammed Almo >> > >> >> On Mar 12, 2020, at 2:51 AM, Jesper Wilhelmsson < >> jesper.wilhelmsson at oracle.com> wrote: >> >> >> >> ?Hi. >> >> >> >> I would like to discuss the possible creation of the Developer's Guide >> Project >> >> with myself, Jesper Wilhelmsson, as the lead and the Core-Libs, >> Compiler, and >> >> Hotspot groups as the sponsoring Groups. >> >> >> >> The initial goal of the project would be to create up-to-date >> guidelines for >> >> OpenJDK development and contributions. The intent is to update the >> existing >> >> OpenJDK Developer's Guide [1] which hasn't been updated since 2012. >> Parts of >> >> the existing guide are in need of updates while other parts are yet to >> be >> >> written. The initial source code for this project will be based on the >> current >> >> OpenJDK Developer's Guide. >> >> >> >> In more recent years some process related information has been >> published on the >> >> OpenJDK wiki [2], but this information was never reviewed or approved >> by the >> >> community. Even though large parts of this information may be >> accurate, it needs >> >> to go through proper review before it can be seen as official >> guidelines for >> >> OpenJDK development. >> >> >> >> Once the initial goal has been reached the project will transition >> into a >> >> maintenance project to keep the OpenJDK Developer's Guide up to date. >> >> >> >> The OpenJDK Developer's Guide is intended to contain tutorial style >> texts that >> >> give examples and step-by-step directions for common parts of the >> development >> >> process. Strict rules for OpenJDK development are better suited to a >> process >> >> JEP. The development of such a JEP is outside of the scope of this >> project but >> >> will be developed as part of a separate effort in parallel. >> >> >> >> Initial committers of the Developer's Guide Project include Mark >> Reinhold, Iris >> >> Clark, and Jesper Wilhelmsson. Additional committers may be identified >> through >> >> this discussion. >> >> >> >> The project will host at least one mailing list, >> guide-dev at openjdk.java.net, >> >> for discussions and reviews of changes. >> >> >> >> Thanks, >> >> /Jesper >> >> >> >> [1] http://openjdk.java.net/guide/ >> >> [2] https://wiki.openjdk.java.net/ >> >> >> >> > From bsrbnd at gmail.com Wed Apr 1 18:32:42 2020 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 1 Apr 2020 20:32:42 +0200 Subject: Enhancing expressions with mutability? In-Reply-To: <6a4113cb-5e51-4406-cf26-d24d8474f257@oracle.com> References: <9238fa7d-3035-6f10-7334-3772df712584@oracle.com> <3C71B207-24D0-429F-80A9-EB9C6E2AC381@oracle.com> <6a4113cb-5e51-4406-cf26-d24d8474f257@oracle.com> Message-ID: On Mon, 23 Mar 2020 at 20:33, Maurizio Cimadamore wrote: > > > I'm planning to employ experimentally to solve some > > tricky performance issues in javac's type inference solving system, > > see [3]. > > With respect to implement type inference using these algorithms, I feel > less sanguine. So, here is a concrete experiment: https://bugs.openjdk.java.net/secure/attachment/87565/learning.patch Quoting from my comment on JBS, it uses a parallel genetic algorithm to solve all contextual inference variables at once requiring only a small subset of propagation steps which reduces JDK-8152289 example's compilation time to a couple of seconds! Note that this heuristic is dedicated to similar examples and the usual inference algorithm is still used if the former doesn't converge rapidly enough. The current design is absolutely not affected as this implementation additionally uses very expressive mutable expressions like "Term gene = (`able).link(`adn, `undetvars.get(i), `type());" employing the quotation operator `. It simply requires a boot JDK (14) including the patch minus inference changes in Attr, Infer & InferenceContext. Still not convinced? Ciao, Bernard From iris.clark at oracle.com Wed Apr 1 18:41:29 2020 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 1 Apr 2020 11:41:29 -0700 (PDT) Subject: CFV: New Project: Developer's Guide In-Reply-To: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> References: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> Message-ID: <1c0b0363-d05e-45ca-aaa5-be16e1bd12ac@default> Vote: yes Thanks, Iris From maurizio.cimadamore at oracle.com Wed Apr 1 20:59:30 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 1 Apr 2020 21:59:30 +0100 Subject: Enhancing expressions with mutability? In-Reply-To: References: <9238fa7d-3035-6f10-7334-3772df712584@oracle.com> <3C71B207-24D0-429F-80A9-EB9C6E2AC381@oracle.com> <6a4113cb-5e51-4406-cf26-d24d8474f257@oracle.com> Message-ID: <069958fe-9d79-3482-d771-58a319eb5e4f@oracle.com> On 01/04/2020 19:32, B. Blaser wrote: > On Mon, 23 Mar 2020 at 20:33, Maurizio Cimadamore > wrote: >>> I'm planning to employ experimentally to solve some >>> tricky performance issues in javac's type inference solving system, >>> see [3]. >> With respect to implement type inference using these algorithms, I feel >> less sanguine. > So, here is a concrete experiment: > > https://bugs.openjdk.java.net/secure/attachment/87565/learning.patch > > Quoting from my comment on JBS, it uses a parallel genetic algorithm > to solve all contextual inference variables at once requiring only a > small subset of propagation steps which reduces JDK-8152289 example's > compilation time to a couple of seconds! Note that this heuristic is > dedicated to similar examples and the usual inference algorithm is > still used if the former doesn't converge rapidly enough. > > The current design is absolutely not affected as this implementation > additionally uses very expressive mutable expressions like "Term gene > = (`able).link(`adn, `undetvars.get(i), `type());" employing the > quotation operator `. It simply requires a boot JDK (14) including the > patch minus inference changes in Attr, Infer & InferenceContext. > > Still not convinced? Sadly yes :-) I mentioned a number of issues, performances aside, in my previous email; error recovery seems particularly nasty with an approach such as this, but more generally, having two inference engines opens up a whole new can of worms, where now you have two implementations to maintain and keep in sync, and there are many more things that can go wrong - and testing becomes similarly much harder. What if the two algorithms don't agree? Might we end up in situations where, depending on how much time one engine takes to converge, we get an error or a failure because a different solver is used? This is the kind of problems that worry me. In my eyes, 8152289 is "just" a performance bug, and one that we know how to solve (*), but we did not get enough time to work on it. Also, as far as I know, the pathological cases described in that bug are the only thing left for us to solve performance-wise (e.g. when speaking about programs which keep running through inference for minutes). Coming up with a new inference engine to solve these classes of problem is, in my opinion, the wrong way to go about things. It might have been a nice validation for your mutable expression experiment, but we can't consider that as a candidate fix for JDK-8152289, sorry. Maurizio (*) The crux of the issue in that bug is detecting that many inference variables are just "duplicates" of each other, so they are going to get exactly same bounds, and be resolved in exactly the same way. Vicente, Dan and I have brainstormed months ago about ways to take an inference context, minimize it aggressively, solve the minimized context, the project resolutions back to the original context. We already applied a similar techniques in cases like a(b(c(d .... ) And that worked really well. But the logic we have implemented has some limitations and, currently, it doesn't scale to cases where hundreds of tvars are added to the _same_ inference context. But this is fixable, with some work. Maurizio > > Ciao, > Bernard From mikael.vidstedt at oracle.com Thu Apr 2 01:33:49 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 1 Apr 2020 18:33:49 -0700 Subject: CFV: New Project: Developer's Guide In-Reply-To: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> References: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> Message-ID: Vote: yes Cheers, Mikael > On Mar 30, 2020, at 2:35 PM, Jesper Wilhelmsson wrote: > > I hereby propose the creation of the Developer's Guide Project with myself, > Jesper Wilhelmsson, as the lead and the Core Libraries, Compiler, and HotSpot > Groups as the sponsoring Groups. > > The suggested project has been out for discussion on discuss at openjdk.java.net > for two weeks [1]. It was well-received and I haven't noted any concerns with > creating the project in that discussion. > > The initial goal of the project would be to create up-to-date guidelines for > OpenJDK development and contributions. The intent is to update the existing > OpenJDK Developer's Guide [2] which hasn't been updated since 2012. Parts of > the existing guide are in need of updates while other parts are yet to be > written. The initial source code for this project will be based on the current > OpenJDK Developer's Guide. > > In more recent years some process related information has been published on the > OpenJDK wiki [3], but this information was never reviewed or approved by the > community. Even though large parts of this information may be accurate, it > needs to go through proper review before it can be seen as official guidelines > for OpenJDK development. > > Once the initial goal has been reached the project will transition into a > maintenance project to keep the OpenJDK Developer's Guide up to date. > > The OpenJDK Developer's Guide is intended to contain tutorial style texts that > give examples and step-by-step directions for common parts of the development > process. Strict rules for OpenJDK development are better suited to a process > JEP. The development of such a JEP is outside of the scope of this project but > will be developed as part of a separate effort in parallel. > > I (Jesper Wilhelmsson) am part of the Java Platform Group at Oracle and have > been working with virtual machine development since 2001. My initial focus was > on memory management and garbage collection but over the last few years I've > been more involved in gatekeeping and process questions around the OpenJDK. > I authored several of the guides available on the OpenJDK HotSpot wiki and I > regularly give talks on how to contribute to the OpenJDK. > > Initial committers of the Developer's Guide Project include Mark Reinhold (mr), > Iris Clark (iris), and Jesper Wilhelmsson (jwilhelm). A complete list will be > provided to the registrar at the successful conclusion of this CFV. > > The project will host at least one mailing list, guide-dev at openjdk.java.net, > for discussions and reviews of changes. > > Votes are due by 5pm PDT on Tuesday April 14 2020. > > Only current OpenJDK Members [4] are eligible to vote on this motion. > Votes must be cast in the open on the discuss list. Replying to this message > is sufficient if your mail program honors the Reply-To header. > > For Lazy Consensus voting instructions, see [5]. > > Thanks, > /Jesper > > [1] https://mail.openjdk.java.net/pipermail/discuss/2020-March/005298.html > [2] http://openjdk.java.net/guide/ > [3] https://wiki.openjdk.java.net/ > [4] http://openjdk.java.net/census#members > [5] http://openjdk.java.net/projects/#new-project-vote > From thomas.stuefe at gmail.com Thu Apr 2 06:34:51 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 2 Apr 2020 08:34:51 +0200 Subject: Project Detroit Sources? Message-ID: Greetings, I was wondering, with the impending removal of Nashorn from JDK 15, what the technical reasons were for scrapping project Detroit? It seemed such a fine idea. Also, would it be possible to get access to the sources somewhere - the repository at https://hg.openjdk.java.net/detroit seems to be empty. Thanks, Thomas From eric.vergnaud at wanadoo.fr Thu Apr 2 13:29:03 2020 From: eric.vergnaud at wanadoo.fr (Eric Vergnaud) Date: Thu, 2 Apr 2020 21:29:03 +0800 Subject: Project Detroit Sources? In-Reply-To: References: Message-ID: Hi, Yes totally support the concept. Please make Detroit OSS rather than trash it, then the OSS community can make it an add-on to the jdk once Nashorn is dropped. Whilst GraalVM is probably a great option for low footprint languages, Detroit is a much more solid approach for JavaScript i.e. runs JS on the reference implementation. Envoy? de mon iPhone > Le 2 avr. 2020 ? 14:35, Thomas St?fe a ?crit : > > ?Greetings, > > I was wondering, with the impending removal of Nashorn from JDK 15, what > the technical reasons were for scrapping project Detroit? It seemed such a > fine idea. > > Also, would it be possible to get access to the sources somewhere - the > repository at https://hg.openjdk.java.net/detroit seems to be empty. > > Thanks, Thomas From org.openjdk at io7m.com Thu Apr 2 15:56:28 2020 From: org.openjdk at io7m.com (Mark Raynsford) Date: Thu, 2 Apr 2020 15:56:28 +0000 Subject: Project Detroit Sources? In-Reply-To: References: Message-ID: <20200402155628.4669966b@sunflower.int.arc7.info> On 2020-04-02T21:29:03 +0800 Eric Vergnaud wrote: > Hi, > Yes totally support the concept. > Please make Detroit OSS rather than trash it, then the OSS community can make it an add-on to the jdk once Nashorn is dropped. > Whilst GraalVM is probably a great option for low footprint languages, Detroit is a much more solid approach for JavaScript i.e. runs JS on the reference implementation. GraalJS also runs on the reference implementation, just at reduced performance compared to running it on a Graal VM: "As GraalVM JavaScript is a Java application, it is possible to execute it on a stock Java VM like OpenJDK. When executed without the GraalVM Compiler as optimizing compiler, performance of GraalVM JavaScript will be significantly worse. While the JIT compilers available on stock JVMs can execute and JIT-compile the GraalVM JavaScript codebase, they cannot optimize it to its full performance potential. This document describes how to run GraalVM JavaScript on stock Java VMs, and shows how you can use the GraalVm Compiler as JIT compiler to guarantee best possible performance." https://github.com/graalvm/graaljs/blob/master/docs/user/RunOnJDK.md I think GraalJS also has the advantage that it supports ES6 and newer. -- Mark Raynsford | http://www.io7m.com From eric.vergnaud at wanadoo.fr Thu Apr 2 17:03:54 2020 From: eric.vergnaud at wanadoo.fr (Eric Vergnaud) Date: Fri, 3 Apr 2020 01:03:54 +0800 Subject: Project Detroit Sources? In-Reply-To: <20200402155628.4669966b@sunflower.int.arc7.info> References: <20200402155628.4669966b@sunflower.int.arc7.info> Message-ID: <3796C0CF-414B-42E6-90CF-45ADE74FD798@wanadoo.fr> The reference implementation I was referring to is V8. Envoy? de mon iPhone > Le 2 avr. 2020 ? 23:56, Mark Raynsford a ?crit : > > ?On 2020-04-02T21:29:03 +0800 > Eric Vergnaud wrote: > >> Hi, >> Yes totally support the concept. >> Please make Detroit OSS rather than trash it, then the OSS community can make it an add-on to the jdk once Nashorn is dropped. >> Whilst GraalVM is probably a great option for low footprint languages, Detroit is a much more solid approach for JavaScript i.e. runs JS on the reference implementation. > > GraalJS also runs on the reference implementation, just at reduced > performance compared to running it on a Graal VM: > > "As GraalVM JavaScript is a Java application, it is possible to > execute it on a stock Java VM like OpenJDK. When executed without the > GraalVM Compiler as optimizing compiler, performance of GraalVM > JavaScript will be significantly worse. While the JIT compilers > available on stock JVMs can execute and JIT-compile the GraalVM > JavaScript codebase, they cannot optimize it to its full performance > potential. This document describes how to run GraalVM JavaScript on > stock Java VMs, and shows how you can use the GraalVm Compiler as JIT > compiler to guarantee best possible performance." > > https://github.com/graalvm/graaljs/blob/master/docs/user/RunOnJDK.md > > I think GraalJS also has the advantage that it supports ES6 and newer. > > -- > Mark Raynsford | http://www.io7m.com From erik.helin at oracle.com Fri Apr 3 06:32:12 2020 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 3 Apr 2020 08:32:12 +0200 Subject: CFV: New Project: Developer's Guide In-Reply-To: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> References: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> Message-ID: <6ec91a1f-e5aa-dcc9-4fb4-f53d5df1f4da@oracle.com> Vote: yes Thanks, Erik On 3/30/20 11:35 PM, Jesper Wilhelmsson wrote: > I hereby propose the creation of the Developer's Guide Project with myself, > Jesper Wilhelmsson, as the lead and the Core Libraries, Compiler, and HotSpot > Groups as the sponsoring Groups. > > The suggested project has been out for discussion on discuss at openjdk.java.net > for two weeks [1]. It was well-received and I haven't noted any concerns with > creating the project in that discussion. > > The initial goal of the project would be to create up-to-date guidelines for > OpenJDK development and contributions. The intent is to update the existing > OpenJDK Developer's Guide [2] which hasn't been updated since 2012. Parts of > the existing guide are in need of updates while other parts are yet to be > written. The initial source code for this project will be based on the current > OpenJDK Developer's Guide. > > In more recent years some process related information has been published on the > OpenJDK wiki [3], but this information was never reviewed or approved by the > community. Even though large parts of this information may be accurate, it > needs to go through proper review before it can be seen as official guidelines > for OpenJDK development. > > Once the initial goal has been reached the project will transition into a > maintenance project to keep the OpenJDK Developer's Guide up to date. > > The OpenJDK Developer's Guide is intended to contain tutorial style texts that > give examples and step-by-step directions for common parts of the development > process. Strict rules for OpenJDK development are better suited to a process > JEP. The development of such a JEP is outside of the scope of this project but > will be developed as part of a separate effort in parallel. > > I (Jesper Wilhelmsson) am part of the Java Platform Group at Oracle and have > been working with virtual machine development since 2001. My initial focus was > on memory management and garbage collection but over the last few years I've > been more involved in gatekeeping and process questions around the OpenJDK. > I authored several of the guides available on the OpenJDK HotSpot wiki and I > regularly give talks on how to contribute to the OpenJDK. > > Initial committers of the Developer's Guide Project include Mark Reinhold (mr), > Iris Clark (iris), and Jesper Wilhelmsson (jwilhelm). A complete list will be > provided to the registrar at the successful conclusion of this CFV. > > The project will host at least one mailing list, guide-dev at openjdk.java.net, > for discussions and reviews of changes. > > Votes are due by 5pm PDT on Tuesday April 14 2020. > > Only current OpenJDK Members [4] are eligible to vote on this motion. > Votes must be cast in the open on the discuss list. Replying to this message > is sufficient if your mail program honors the Reply-To header. > > For Lazy Consensus voting instructions, see [5]. > > Thanks, > /Jesper > > [1] https://mail.openjdk.java.net/pipermail/discuss/2020-March/005298.html > [2] http://openjdk.java.net/guide/ > [3] https://wiki.openjdk.java.net/ > [4] http://openjdk.java.net/census#members > [5] http://openjdk.java.net/projects/#new-project-vote > From rob.mckenna at oracle.com Fri Apr 3 13:11:27 2020 From: rob.mckenna at oracle.com (Robert McKenna) Date: Fri, 3 Apr 2020 14:11:27 +0100 Subject: CFV: New Project: Developer's Guide In-Reply-To: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> References: <96139BAE-FB33-44F1-ADA6-04E6068A842D@oracle.com> Message-ID: <5f09600b-e1c7-3168-f022-d21a57010114@oracle.com> Vote: yes ??? -Rob On 30/03/2020 22:35, Jesper Wilhelmsson wrote: > I hereby propose the creation of the Developer's Guide Project with myself, > Jesper Wilhelmsson, as the lead and the Core Libraries, Compiler, and HotSpot > Groups as the sponsoring Groups. > > The suggested project has been out for discussion on discuss at openjdk.java.net > for two weeks [1]. It was well-received and I haven't noted any concerns with > creating the project in that discussion. > > The initial goal of the project would be to create up-to-date guidelines for > OpenJDK development and contributions. The intent is to update the existing > OpenJDK Developer's Guide [2] which hasn't been updated since 2012. Parts of > the existing guide are in need of updates while other parts are yet to be > written. The initial source code for this project will be based on the current > OpenJDK Developer's Guide. > > In more recent years some process related information has been published on the > OpenJDK wiki [3], but this information was never reviewed or approved by the > community. Even though large parts of this information may be accurate, it > needs to go through proper review before it can be seen as official guidelines > for OpenJDK development. > > Once the initial goal has been reached the project will transition into a > maintenance project to keep the OpenJDK Developer's Guide up to date. > > The OpenJDK Developer's Guide is intended to contain tutorial style texts that > give examples and step-by-step directions for common parts of the development > process. Strict rules for OpenJDK development are better suited to a process > JEP. The development of such a JEP is outside of the scope of this project but > will be developed as part of a separate effort in parallel. > > I (Jesper Wilhelmsson) am part of the Java Platform Group at Oracle and have > been working with virtual machine development since 2001. My initial focus was > on memory management and garbage collection but over the last few years I've > been more involved in gatekeeping and process questions around the OpenJDK. > I authored several of the guides available on the OpenJDK HotSpot wiki and I > regularly give talks on how to contribute to the OpenJDK. > > Initial committers of the Developer's Guide Project include Mark Reinhold (mr), > Iris Clark (iris), and Jesper Wilhelmsson (jwilhelm). A complete list will be > provided to the registrar at the successful conclusion of this CFV. > > The project will host at least one mailing list, guide-dev at openjdk.java.net, > for discussions and reviews of changes. > > Votes are due by 5pm PDT on Tuesday April 14 2020. > > Only current OpenJDK Members [4] are eligible to vote on this motion. > Votes must be cast in the open on the discuss list. Replying to this message > is sufficient if your mail program honors the Reply-To header. > > For Lazy Consensus voting instructions, see [5]. > > Thanks, > /Jesper > > [1] https://mail.openjdk.java.net/pipermail/discuss/2020-March/005298.html > [2] http://openjdk.java.net/guide/ > [3] https://wiki.openjdk.java.net/ > [4] http://openjdk.java.net/census#members > [5] http://openjdk.java.net/projects/#new-project-vote > From andru at cs.cornell.edu Fri Apr 3 14:55:29 2020 From: andru at cs.cornell.edu (Andrew Myers) Date: Fri, 3 Apr 2020 10:55:29 -0400 Subject: Java's strings, UTF-8 etc In-Reply-To: <20200319233527.wmuzlfit7nb4hx4l@eye.freedbms.net> References: <20200319233527.wmuzlfit7nb4hx4l@eye.freedbms.net> Message-ID: Afundamental problem with UTF-8 strings is the String API itself. String relies on being able to use integer indices as iterators and then using charAt() to select characters. A UTF-8 string should be a different abstraction that does not encourage programmers to do inefficient random access into strings, and also is able to return characters larger than 16 bits in size -- probably int/Integer rather than char/Character. -- Andrew Zenaan Harkness wrote: > > Hi Brian, in case it is of interest, I did an exploration over a week > or two, and wrote up that journey here: > > https://zenaan.github.io/zen/javadoc/zen/lang/string.html > > See also of course: > > https://github.com/zenaan/zen > > And for reference, see also: > > https://mail.openjdk.java.net/pipermail/discuss/2016-November/004065.html > > https://mail.openjdk.java.net/pipermail/discuss/2016-November/004070.html > > https://mail.openjdk.java.net/pipermail/discuss/2016-November/004072.html > > > (I was hoping to one day do a proof of concept for byte array backed > UTF-8 strings in Java, but have not returned to this little project - > in any case, I do believe this would be a fundamental improvement to > the core of Java.) > > Best regards, > Zenaan From bsrbnd at gmail.com Sun Apr 5 18:25:25 2020 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 5 Apr 2020 20:25:25 +0200 Subject: Enhancing expressions with mutability? In-Reply-To: <069958fe-9d79-3482-d771-58a319eb5e4f@oracle.com> References: <9238fa7d-3035-6f10-7334-3772df712584@oracle.com> <3C71B207-24D0-429F-80A9-EB9C6E2AC381@oracle.com> <6a4113cb-5e51-4406-cf26-d24d8474f257@oracle.com> <069958fe-9d79-3482-d771-58a319eb5e4f@oracle.com> Message-ID: On Wed, 1 Apr 2020 at 22:59, Maurizio Cimadamore wrote: > > On 01/04/2020 19:32, B. Blaser wrote: > > So, here is a concrete experiment: > > > > https://bugs.openjdk.java.net/secure/attachment/87565/learning.patch > > > > Quoting from my comment on JBS, it uses a parallel genetic algorithm > > to solve all contextual inference variables at once requiring only a > > small subset of propagation steps which reduces JDK-8152289 example's > > compilation time to a couple of seconds! Note that this heuristic is > > dedicated to similar examples and the usual inference algorithm is > > still used if the former doesn't converge rapidly enough. > > > > The current design is absolutely not affected as this implementation > > additionally uses very expressive mutable expressions like "Term gene > > = (`able).link(`adn, `undetvars.get(i), `type());" employing the > > quotation operator `. It simply requires a boot JDK (14) including the > > patch minus inference changes in Attr, Infer & InferenceContext. > > > > Still not convinced? > > Sadly yes :-) Paradoxically, I agree with you, a few questions still remain... > I mentioned a number of issues, performances aside, in my previous > email; error recovery seems particularly nasty with an approach such as > this, Yes, this is the most problematic issue but I'm rather confident that if a solution exists the genetic algorithm will converge rapidly enough (less than 10 generations) otherwise we can consider that an error occurred. Of course, this would have to be confirmed with many measures but as far as I know, * either the context has few inference variables (1 to 4) and a clever choice of the initial population might simply perform an exhaustive combinatorial search using multiple threads, * or the context has potentially many variables which are most of the time exact copies of themselves resulting in very low liberty-degrees and thus a very fast convergence too. Of course, you might argue that this genetic engine might fail to infer valid expressions but I believe this is inherent to inference algorithms in general. Recall JDK-8219318, you said yourself that the current engine might be "out of gas" in some situations, and if I remember well, this unresolved issue comes from a weakness in propagation rules which seems not so trivial to be solved... and although the current genetic prototype isn't evolved enough to resolve it yet, I'm rather confident that it might do that intrinsically with some more maturation! > but more generally, having two inference engines opens up a whole > new can of worms, where now you have two implementations to maintain and > keep in sync, and there are many more things that can go wrong - and > testing becomes similarly much harder. What if the two algorithms don't > agree? Might we end up in situations where, depending on how much time > one engine takes to converge, we get an error or a failure because a > different solver is used? This is the kind of problems that worry me. That's right, we cannot live with two different engines. The current prototype is only a first partial attempt, but the goal of this experiment would be to finish with a complete genetic inference engine to be compared with the existing more formal one. > In my eyes, 8152289 is "just" a performance bug, and one that we know > how to solve (*), but we did not get enough time to work on it. Also, as > far as I know, the pathological cases described in that bug are the only > thing left for us to solve performance-wise (e.g. when speaking about > programs which keep running through inference for minutes). Coming up > with a new inference engine to solve these classes of problem is, in my > opinion, the wrong way to go about things. It might have been a nice > validation for your mutable expression experiment, Sure, it is ;-) > but we can't consider > that as a candidate fix for JDK-8152289, sorry. Not as is, of course, as discussed above. Bernard > Maurizio From maurizio.cimadamore at oracle.com Mon Apr 6 10:33:49 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 6 Apr 2020 11:33:49 +0100 Subject: Enhancing expressions with mutability? In-Reply-To: References: <9238fa7d-3035-6f10-7334-3772df712584@oracle.com> <3C71B207-24D0-429F-80A9-EB9C6E2AC381@oracle.com> <6a4113cb-5e51-4406-cf26-d24d8474f257@oracle.com> <069958fe-9d79-3482-d771-58a319eb5e4f@oracle.com> Message-ID: On 05/04/2020 19:25, B. Blaser wrote: > On Wed, 1 Apr 2020 at 22:59, Maurizio Cimadamore > wrote: >> On 01/04/2020 19:32, B. Blaser wrote: >>> So, here is a concrete experiment: >>> >>> https://bugs.openjdk.java.net/secure/attachment/87565/learning.patch >>> >>> Quoting from my comment on JBS, it uses a parallel genetic algorithm >>> to solve all contextual inference variables at once requiring only a >>> small subset of propagation steps which reduces JDK-8152289 example's >>> compilation time to a couple of seconds! Note that this heuristic is >>> dedicated to similar examples and the usual inference algorithm is >>> still used if the former doesn't converge rapidly enough. >>> >>> The current design is absolutely not affected as this implementation >>> additionally uses very expressive mutable expressions like "Term gene >>> = (`able).link(`adn, `undetvars.get(i), `type());" employing the >>> quotation operator `. It simply requires a boot JDK (14) including the >>> patch minus inference changes in Attr, Infer & InferenceContext. >>> >>> Still not convinced? >> Sadly yes :-) > Paradoxically, I agree with you, a few questions still remain... > >> I mentioned a number of issues, performances aside, in my previous >> email; error recovery seems particularly nasty with an approach such as >> this, > Yes, this is the most problematic issue but I'm rather confident that > if a solution exists the genetic algorithm will converge rapidly > enough (less than 10 generations) otherwise we can consider that an > error occurred. Of course, this would have to be confirmed with many > measures but as far as I know, > * either the context has few inference variables (1 to 4) and a clever > choice of the initial population might simply perform an exhaustive > combinatorial search using multiple threads, > * or the context has potentially many variables which are most of the > time exact copies of themselves resulting in very low liberty-degrees > and thus a very fast convergence too. > > Of course, you might argue that this genetic engine might fail to > infer valid expressions but I believe this is inherent to inference > algorithms in general. Recall JDK-8219318, you said yourself that the > current engine might be "out of gas" in some situations, Let me underline all the things that raise my concern level with your message: * " I'm rather confident that if a solution exists the genetic algorithm will converge rapidly enough (less than 10 generations) otherwise we can consider that an error occurred" - pretty sure that this statement alone is unjustifiable from a JLS perspective, where subtyping chains can be arbitrarily long and complex * "... either the context has few inference variables (1 to 4)" - in such cases the existing inference engine is uber fast, why would you want to improve it? * "combinatorial search using multiple threads" - and... even more complexity. * "or the context has potentially many variables which are most of the time exact copies of themselves" - another assumption which holds for the case you are looking at, but doesn't hold in general. When working with collectors I've seen many cases of inference context which, after normalization (e.g. equivalent type variables dropped) were still in the 10-15 mark. In any case I think we're talking past each other here, and using the term inference engine for different purposes. When I said in my comment to JDK-8219318 that the engine was running out of gas I meant the engine "as specified in the JLS". In fact, this thread seems to forget that, no matter what the impl will do, it will have to be backed up by whatever text is in the JLS - each pass and each fail will have to be explained in terms of normative text in the spec. So, popping back, JDK-8219318 and JDK-8152289 are two _very_ different issues; the former is essentially tied to how inference works in the spec; the latter is an implementation performance issue. I already talked at length as to why I wouldn't like the javac codebase to evolve and embrace some kind of bi-modal genetic inference engine in order to speed up performances. I think that, by extension, by skepticism also applies (in much greater way) to _change the language specifications_ to allow for such changes (e.g. to support JDK-8219318). Popping back even more; we started this thread by discussing your proposal for mutable expressions (although this list shouldn't be used for proposing language changes, but, fine) - both Paul and I replied connecting the work you did with some of the stuff we have in the oven. From there we descended into some kind of inference engine war, which I don't think was the point of your original communication? Inference engine changes have a lot of strings attached to them; it's not about the code and, to a degree, not even about performance, as long as it's acceptable - which I think it is, it's primarily about maintenance, and conformance with the spec. Invariably issues will pop up, and stuff will break, and when that happens, the more similar the spec is to the implementation, the better. So, in my view, the way to move the inference code forward is to make it _more in sync_ with the JLS, so that concepts are implemented in a more 1-1 way (which wasn't possible at the outset in Java 8 as we had to be compatible with pre Java 8 inference - this requirement might go away at some point, so we might be able to cleanup some more). Maurizio From amnojeeuw at gmail.com Mon Apr 6 11:01:34 2020 From: amnojeeuw at gmail.com (AmnoJeeuw) Date: Mon, 6 Apr 2020 07:01:34 -0400 Subject: Call for Discussion: New Project: Developer's Guide In-Reply-To: References: <62FA7D5A-5D25-4617-853F-3B0BC09CE154@oracle.com> <0D9DE337-FD2D-41ED-BA1C-832ACF2D1988@gmail.com> <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> Message-ID: <8ab88c41-3aeb-e70d-5d80-8d53e0a0f09f@gmail.com> I might not be participating as a writer, but I will definitely follow its development either by just reading the the new written files or by making suggestions on what has been presented. I just don't fee right clicking on the send button without saying... OUTSTANDING IDEA On 2020-04-01 12:41 p.m., Jesper Wilhelmsson wrote: > Hi Timmy. > > To be part of the project just join the guide-dev email list once it has been created :-) > Looking forward to see you there! > /Jesper > >> On 1 Apr 2020, at 08:33, Timmy Jose wrote: >> >> I think that this is a brilliant idea. The existing developer guide is indeed very outdated, and an up-to-date, easy-to-locate guide is definitely required. It would make things so much easier for people new to the process such as myself. If possible, I would also like to be part of this initiative - maybe it would also help >> me get onboarded myself. >> >> Best, >> >> Timmy >> >> On Tue, Mar 31, 2020 at 5:01 AM Jesper Wilhelmsson > wrote: >> Hi Mohammed. >> >> I'm happy to hear that you are interested in this project. The next step is the vote that has been initiated now. All Members of the OpenJDK have a vote and if they approve to creating the project it will be created. Once the project is in place we can start discussing the future. >> >> Thanks, >> /Jesper >> >>> On 29 Mar 2020, at 11:43, Mohammed Al-Moayed > wrote: >>> >>> Hi All >>> >>> I am interested to be part of this. >>> >>> Please, let me know next steps >>> >>> Regards >>> Mohammed Almo >>> >>>> On Mar 12, 2020, at 2:51 AM, Jesper Wilhelmsson > wrote: >>>> >>>> ?Hi. >>>> >>>> I would like to discuss the possible creation of the Developer's Guide Project >>>> with myself, Jesper Wilhelmsson, as the lead and the Core-Libs, Compiler, and >>>> Hotspot groups as the sponsoring Groups. >>>> >>>> The initial goal of the project would be to create up-to-date guidelines for >>>> OpenJDK development and contributions. The intent is to update the existing >>>> OpenJDK Developer's Guide [1] which hasn't been updated since 2012. Parts of >>>> the existing guide are in need of updates while other parts are yet to be >>>> written. The initial source code for this project will be based on the current >>>> OpenJDK Developer's Guide. >>>> >>>> In more recent years some process related information has been published on the >>>> OpenJDK wiki [2], but this information was never reviewed or approved by the >>>> community. Even though large parts of this information may be accurate, it needs >>>> to go through proper review before it can be seen as official guidelines for >>>> OpenJDK development. >>>> >>>> Once the initial goal has been reached the project will transition into a >>>> maintenance project to keep the OpenJDK Developer's Guide up to date. >>>> >>>> The OpenJDK Developer's Guide is intended to contain tutorial style texts that >>>> give examples and step-by-step directions for common parts of the development >>>> process. Strict rules for OpenJDK development are better suited to a process >>>> JEP. The development of such a JEP is outside of the scope of this project but >>>> will be developed as part of a separate effort in parallel. >>>> >>>> Initial committers of the Developer's Guide Project include Mark Reinhold, Iris >>>> Clark, and Jesper Wilhelmsson. Additional committers may be identified through >>>> this discussion. >>>> >>>> The project will host at least one mailing list, guide-dev at openjdk.java.net , >>>> for discussions and reviews of changes. >>>> >>>> Thanks, >>>> /Jesper >>>> >>>> [1] http://openjdk.java.net/guide/ >>>> [2] https://wiki.openjdk.java.net/ >>>> -- This email has been checked for viruses by AVG. https://www.avg.com From bsrbnd at gmail.com Mon Apr 6 12:25:31 2020 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 6 Apr 2020 14:25:31 +0200 Subject: Enhancing expressions with mutability? In-Reply-To: References: <9238fa7d-3035-6f10-7334-3772df712584@oracle.com> <3C71B207-24D0-429F-80A9-EB9C6E2AC381@oracle.com> <6a4113cb-5e51-4406-cf26-d24d8474f257@oracle.com> <069958fe-9d79-3482-d771-58a319eb5e4f@oracle.com> Message-ID: Thanks for your detailed explanation, Maurizio. On Mon, 6 Apr 2020 at 12:34, Maurizio Cimadamore wrote: > > Popping back even more; we started this thread by discussing your > proposal for mutable expressions (although this list shouldn't be used > for proposing language changes, but, fine) - both Paul and I replied > connecting the work you did with some of the stuff we have in the oven. > > From there we descended into some kind of inference engine war, which I > don't think was the point of your original communication? Of course, my initial intention was only to suggest enhancing expressions with mutability which I tried to illustrate with a concrete prototype using the quotation operator and connecting it to Lisp's S-Expressions. Then, speaking about potential applications, we deviate to inference engine alternative implementations and their potential abilities to solve some known "pathological" issues. However, these were just raw ideas I wanted to experiment with some prototype which I shared in the hope that it might be useful. Finally, finding the right list to discuss about this wasn't an easy task and I thought "discuss" (or maybe "compiler-dev") were appropriate since such proposals may lead to new project creation but you're welcome to forward it to the right list or to ask me to do so if necessary. Cheers, Bernard From jesper.wilhelmsson at oracle.com Mon Apr 6 16:09:01 2020 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 6 Apr 2020 18:09:01 +0200 Subject: Call for Discussion: New Project: Developer's Guide In-Reply-To: <8ab88c41-3aeb-e70d-5d80-8d53e0a0f09f@gmail.com> References: <62FA7D5A-5D25-4617-853F-3B0BC09CE154@oracle.com> <0D9DE337-FD2D-41ED-BA1C-832ACF2D1988@gmail.com> <0C45825A-821F-4C9A-A88D-2F5EAC406F01@oracle.com> <8ab88c41-3aeb-e70d-5d80-8d53e0a0f09f@gmail.com> Message-ID: <276429DE-269C-47AA-B4A1-BF703F2E3BDB@oracle.com> Thank you for your support! /Jesper > On 6 Apr 2020, at 13:01, AmnoJeeuw wrote: > > I might not be participating as a writer, but I will definitely follow its development either by just reading the the new written files or by making suggestions on what has been presented. > > I just don't fee right clicking on the send button without saying... OUTSTANDING IDEA > > On 2020-04-01 12:41 p.m., Jesper Wilhelmsson wrote: >> Hi Timmy. >> >> To be part of the project just join the guide-dev email list once it has been created :-) >> Looking forward to see you there! >> /Jesper >> >>> On 1 Apr 2020, at 08:33, Timmy Jose wrote: >>> >>> I think that this is a brilliant idea. The existing developer guide is indeed very outdated, and an up-to-date, easy-to-locate guide is definitely required. It would make things so much easier for people new to the process such as myself. If possible, I would also like to be part of this initiative - maybe it would also help >>> me get onboarded myself. >>> >>> Best, >>> >>> Timmy >>> >>> On Tue, Mar 31, 2020 at 5:01 AM Jesper Wilhelmsson > wrote: >>> Hi Mohammed. >>> >>> I'm happy to hear that you are interested in this project. The next step is the vote that has been initiated now. All Members of the OpenJDK have a vote and if they approve to creating the project it will be created. Once the project is in place we can start discussing the future. >>> >>> Thanks, >>> /Jesper >>> >>>> On 29 Mar 2020, at 11:43, Mohammed Al-Moayed > wrote: >>>> >>>> Hi All >>>> >>>> I am interested to be part of this. >>>> >>>> Please, let me know next steps >>>> >>>> Regards >>>> Mohammed Almo >>>> >>>>> On Mar 12, 2020, at 2:51 AM, Jesper Wilhelmsson > wrote: >>>>> >>>>> ?Hi. >>>>> >>>>> I would like to discuss the possible creation of the Developer's Guide Project >>>>> with myself, Jesper Wilhelmsson, as the lead and the Core-Libs, Compiler, and >>>>> Hotspot groups as the sponsoring Groups. >>>>> >>>>> The initial goal of the project would be to create up-to-date guidelines for >>>>> OpenJDK development and contributions. The intent is to update the existing >>>>> OpenJDK Developer's Guide [1] which hasn't been updated since 2012. Parts of >>>>> the existing guide are in need of updates while other parts are yet to be >>>>> written. The initial source code for this project will be based on the current >>>>> OpenJDK Developer's Guide. >>>>> >>>>> In more recent years some process related information has been published on the >>>>> OpenJDK wiki [2], but this information was never reviewed or approved by the >>>>> community. Even though large parts of this information may be accurate, it needs >>>>> to go through proper review before it can be seen as official guidelines for >>>>> OpenJDK development. >>>>> >>>>> Once the initial goal has been reached the project will transition into a >>>>> maintenance project to keep the OpenJDK Developer's Guide up to date. >>>>> >>>>> The OpenJDK Developer's Guide is intended to contain tutorial style texts that >>>>> give examples and step-by-step directions for common parts of the development >>>>> process. Strict rules for OpenJDK development are better suited to a process >>>>> JEP. The development of such a JEP is outside of the scope of this project but >>>>> will be developed as part of a separate effort in parallel. >>>>> >>>>> Initial committers of the Developer's Guide Project include Mark Reinhold, Iris >>>>> Clark, and Jesper Wilhelmsson. Additional committers may be identified through >>>>> this discussion. >>>>> >>>>> The project will host at least one mailing list, guide-dev at openjdk.java.net , >>>>> for discussions and reviews of changes. >>>>> >>>>> Thanks, >>>>> /Jesper >>>>> >>>>> [1] http://openjdk.java.net/guide/ >>>>> [2] https://wiki.openjdk.java.net/ >>>>> > > -- > This email has been checked for viruses by AVG. > https://www.avg.com > From info at j-kuhn.de Wed Apr 15 13:35:38 2020 From: info at j-kuhn.de (Johannes Kuhn) Date: Wed, 15 Apr 2020 15:35:38 +0200 Subject: Some Classes with a public void close() don't implement AutoCloseable Message-ID: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> Hi, After failing to wrap a XMLStreamReader in a try-with-resources after discovering it's close() method, I thought about checking what other classes have a public void close() method in the JDK but don't implement AutoCloseable. For this reason I wrote a small program that enumerates all classes present in in my jrt image with a public void close() method that does not directly or indirectly implement AutoCloseable: https://gist.github.com/DasBrain/8d50e02e35654870e2c2c00bf3f79954 (Output & Code, Code requires JDK 14 with preview features and ASM) As I am not an expert in any of those areas, I don't know if that was just an oversight or intended. The XMLStreamReaderImpl will just set a flag to indicate that this instance can be reused, with a comment //xxx: Check what this function is intended to do. The most likely candidate for adding AutoCloseable is javax.naming.Context in my opinion. Not sure about java.util.logging.Handler. As far as I know those are closed by the Logger and closed when reloading the configuration for example. Anyway, thought I share this. We could maybe add a few implements AutoCloseable or add some comments why they don't implement that. Some IDEs rely on resource leak warnings on the interface, so adding the interface can be double edged sword. With best regards, Johannes Kuhn Output: com/sun/jdi/connect/spi/Connection com/sun/jndi/dns/BaseNameClassPairEnumeration com/sun/jndi/dns/DnsClient com/sun/jndi/dns/DnsContext com/sun/jndi/dns/Resolver com/sun/jndi/ldap/AbstractLdapNamingEnumeration com/sun/jndi/ldap/LdapCtx com/sun/jndi/ldap/LdapReferralContext com/sun/jndi/ldap/LdapSchemaCtx com/sun/jndi/ldap/ext/StartTlsResponseImpl com/sun/jndi/rmi/registry/BindingEnumeration com/sun/jndi/rmi/registry/NameClassPairEnumeration com/sun/jndi/rmi/registry/RegistryContext com/sun/jndi/toolkit/dir/ContextEnumerator com/sun/jndi/toolkit/dir/HierMemDirCtx com/sun/jndi/toolkit/dir/HierMemDirCtx$BaseFlatNames com/sun/jndi/toolkit/dir/LazySearchEnumerationImpl com/sun/jndi/toolkit/url/GenericURLContext com/sun/media/sound/AudioFloatFormatConverter$AudioFloatInputStreamChannelMixer com/sun/media/sound/AudioFloatFormatConverter$AudioFloatInputStreamResampler com/sun/media/sound/AudioFloatInputStream com/sun/media/sound/AudioFloatInputStream$BytaArrayAudioFloatInputStream com/sun/media/sound/AudioFloatInputStream$DirectAudioFloatInputStream com/sun/media/sound/ModelAbstractOscillator com/sun/media/sound/ModelDirector com/sun/media/sound/ModelOscillatorStream com/sun/media/sound/ModelStandardDirector com/sun/media/sound/ModelStandardIndexedDirector com/sun/media/sound/RIFFWriter$RandomAccessByteWriter com/sun/media/sound/RIFFWriter$RandomAccessFileWriter com/sun/media/sound/RIFFWriter$RandomAccessWriter com/sun/media/sound/SoftAbstractResampler$ModelAbstractResamplerStream com/sun/media/sound/SoftMainMixer com/sun/media/sound/SoftMixingDataLine$AudioFloatInputStreamResampler com/sun/media/sound/SoftMixingMainMixer com/sun/media/sound/SoftMixingSourceDataLine$NonBlockingFloatInputStream com/sun/naming/internal/VersionHelper$InputStreamEnumeration com/sun/org/apache/xerces/internal/impl/XMLStreamFilterImpl com/sun/org/apache/xerces/internal/impl/XMLStreamReaderImpl com/sun/org/apache/xerces/internal/xinclude/XIncludeTextReader com/sun/org/apache/xml/internal/serializer/EmptySerializer com/sun/org/apache/xml/internal/serializer/SerializationHandler com/sun/org/apache/xml/internal/serializer/SerializerBase com/sun/org/apache/xml/internal/serializer/ToHTMLSAXHandler com/sun/org/apache/xml/internal/serializer/ToUnknownStream com/sun/org/apache/xml/internal/serializer/WriterChain com/sun/tools/javac/api/JavacTaskPool$ReusableContext$ReusableJavaCompiler com/sun/tools/javac/file/JavacFileManager$1 com/sun/tools/javac/file/JavacFileManager$ArchiveContainer com/sun/tools/javac/file/JavacFileManager$Container com/sun/tools/javac/file/JavacFileManager$DirectoryContainer com/sun/tools/javac/file/JavacFileManager$JRTImageContainer com/sun/tools/javac/file/Locations com/sun/tools/javac/main/JavaCompiler com/sun/tools/javac/processing/JavacProcessingEnvironment$DiscoveredProcessors com/sun/tools/javac/processing/JavacProcessingEnvironment$ServiceIterator com/sun/tools/jdi/SharedMemoryConnection com/sun/tools/jdi/SocketConnection com/sun/tools/sjavac/PubApiExtractor com/sun/xml/internal/stream/Entity$ScannedEntity com/sun/xml/internal/stream/XMLEventReaderImpl com/sun/xml/internal/stream/writers/XMLDOMWriterImpl com/sun/xml/internal/stream/writers/XMLEventWriterImpl com/sun/xml/internal/stream/writers/XMLStreamWriterImpl java/awt/SplashScreen java/util/logging/ConsoleHandler java/util/logging/FileHandler java/util/logging/Handler java/util/logging/MemoryHandler java/util/logging/SocketHandler java/util/logging/StreamHandler javax/naming/Context javax/naming/InitialContext javax/naming/NamingEnumeration javax/naming/directory/BasicAttribute$ValuesEnumImpl javax/naming/directory/BasicAttributes$AttrEnumImpl javax/naming/directory/BasicAttributes$IDEnumImpl javax/naming/ldap/StartTlsResponse javax/naming/spi/ContinuationContext javax/smartcardio/CardChannel javax/sql/PooledConnection javax/swing/ProgressMonitor javax/swing/text/rtf/RTFReader$AttributeTrackingDestination javax/swing/text/rtf/RTFReader$ColortblDestination javax/swing/text/rtf/RTFReader$Destination javax/swing/text/rtf/RTFReader$DiscardingDestination javax/swing/text/rtf/RTFReader$FonttblDestination javax/swing/text/rtf/RTFReader$StylesheetDestination javax/swing/text/rtf/RTFReader$StylesheetDestination$StyleDefiningDestination javax/swing/text/rtf/RTFReader$TextHandlingDestination javax/xml/stream/XMLEventReader javax/xml/stream/XMLEventWriter javax/xml/stream/XMLStreamReader javax/xml/stream/XMLStreamWriter javax/xml/stream/util/EventReaderDelegate javax/xml/stream/util/StreamReaderDelegate jdk/internal/jrtfs/ExplodedImage jdk/internal/util/xml/XMLStreamWriter jdk/internal/util/xml/impl/XMLStreamWriterImpl jdk/jfr/internal/consumer/ChunkParser jdk/jshell/SourceCodeAnalysisImpl jdk/tools/jlink/internal/Archive jdk/tools/jlink/internal/DirArchive jdk/tools/jlink/internal/JarArchive jdk/tools/jlink/internal/JmodArchive org/graalvm/compiler/code/CompilationResult org/graalvm/compiler/code/DataSection org/graalvm/compiler/debug/DiagnosticsOutputDirectory org/w3c/dom/html/HTMLDocument sun/awt/image/ImageDecoder sun/jvm/hotspot/debugger/DataSource sun/jvm/hotspot/debugger/InputLexer sun/jvm/hotspot/debugger/MappedByteBufferDataSource sun/jvm/hotspot/debugger/RandomAccessFileDataSource sun/jvm/hotspot/debugger/posix/AddressDataSource sun/jvm/hotspot/debugger/posix/elf/ELFFile sun/jvm/hotspot/debugger/posix/elf/ELFFileParser$ELFFileImpl sun/jvm/hotspot/debugger/win32/coff/COFFFile sun/jvm/hotspot/debugger/win32/coff/COFFFileParser$COFFFileImpl sun/jvm/hotspot/debugger/windbg/AddressDataSource sun/jvm/hotspot/debugger/windbg/DLL sun/net/ProgressSource sun/net/httpserver/ExchangeImpl sun/net/www/URLConnection sun/rmi/log/ReliableLog sun/rmi/runtime/Log$InternalStreamHandler sun/rmi/transport/Connection sun/rmi/transport/tcp/TCPConnection sun/security/smartcardio/ChannelImpl sun/tools/java/ClassPath From forax at univ-mlv.fr Wed Apr 15 15:53:14 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 15 Apr 2020 17:53:14 +0200 (CEST) Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> Message-ID: <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> interface FakeAutoCloseable extends AutoCloseable { public void close(); } // suppress exception XMLReader xmlReader = ... try(FakeAutoCloseable __ = xmlReader::close) { ... } R?mi ----- Mail original ----- > De: "Johannes Kuhn" > ?: "discuss" > Envoy?: Mercredi 15 Avril 2020 15:35:38 > Objet: Some Classes with a public void close() don't implement AutoCloseable > Hi, > > After failing to wrap a XMLStreamReader in a try-with-resources after > discovering it's close() method, > I thought about checking what other classes have a public void close() > method in the JDK but don't implement AutoCloseable. > > For this reason I wrote a small program that enumerates all classes > present in in my jrt image with a public void close() method that does not > directly or indirectly implement AutoCloseable: > https://gist.github.com/DasBrain/8d50e02e35654870e2c2c00bf3f79954 > (Output & Code, Code requires JDK 14 with preview features and ASM) > > As I am not an expert in any of those areas, I don't know if that was > just an oversight or intended. > > The XMLStreamReaderImpl will just set a flag to indicate that this > instance can be reused, with a comment //xxx: Check what this function > is intended to do. > The most likely candidate for adding AutoCloseable is > javax.naming.Context in my opinion. > Not sure about java.util.logging.Handler. As far as I know those are > closed by the Logger and closed when reloading the configuration for > example. > > Anyway, thought I share this. We could maybe add a few implements > AutoCloseable or add some comments why they don't implement that. > Some IDEs rely on resource leak warnings on the interface, so adding the > interface can be double edged sword. > > With best regards, > Johannes Kuhn > > Output: > com/sun/jdi/connect/spi/Connection > com/sun/jndi/dns/BaseNameClassPairEnumeration > com/sun/jndi/dns/DnsClient > com/sun/jndi/dns/DnsContext > com/sun/jndi/dns/Resolver > com/sun/jndi/ldap/AbstractLdapNamingEnumeration > com/sun/jndi/ldap/LdapCtx > com/sun/jndi/ldap/LdapReferralContext > com/sun/jndi/ldap/LdapSchemaCtx > com/sun/jndi/ldap/ext/StartTlsResponseImpl > com/sun/jndi/rmi/registry/BindingEnumeration > com/sun/jndi/rmi/registry/NameClassPairEnumeration > com/sun/jndi/rmi/registry/RegistryContext > com/sun/jndi/toolkit/dir/ContextEnumerator > com/sun/jndi/toolkit/dir/HierMemDirCtx > com/sun/jndi/toolkit/dir/HierMemDirCtx$BaseFlatNames > com/sun/jndi/toolkit/dir/LazySearchEnumerationImpl > com/sun/jndi/toolkit/url/GenericURLContext > com/sun/media/sound/AudioFloatFormatConverter$AudioFloatInputStreamChannelMixer > com/sun/media/sound/AudioFloatFormatConverter$AudioFloatInputStreamResampler > com/sun/media/sound/AudioFloatInputStream > com/sun/media/sound/AudioFloatInputStream$BytaArrayAudioFloatInputStream > com/sun/media/sound/AudioFloatInputStream$DirectAudioFloatInputStream > com/sun/media/sound/ModelAbstractOscillator > com/sun/media/sound/ModelDirector > com/sun/media/sound/ModelOscillatorStream > com/sun/media/sound/ModelStandardDirector > com/sun/media/sound/ModelStandardIndexedDirector > com/sun/media/sound/RIFFWriter$RandomAccessByteWriter > com/sun/media/sound/RIFFWriter$RandomAccessFileWriter > com/sun/media/sound/RIFFWriter$RandomAccessWriter > com/sun/media/sound/SoftAbstractResampler$ModelAbstractResamplerStream > com/sun/media/sound/SoftMainMixer > com/sun/media/sound/SoftMixingDataLine$AudioFloatInputStreamResampler > com/sun/media/sound/SoftMixingMainMixer > com/sun/media/sound/SoftMixingSourceDataLine$NonBlockingFloatInputStream > com/sun/naming/internal/VersionHelper$InputStreamEnumeration > com/sun/org/apache/xerces/internal/impl/XMLStreamFilterImpl > com/sun/org/apache/xerces/internal/impl/XMLStreamReaderImpl > com/sun/org/apache/xerces/internal/xinclude/XIncludeTextReader > com/sun/org/apache/xml/internal/serializer/EmptySerializer > com/sun/org/apache/xml/internal/serializer/SerializationHandler > com/sun/org/apache/xml/internal/serializer/SerializerBase > com/sun/org/apache/xml/internal/serializer/ToHTMLSAXHandler > com/sun/org/apache/xml/internal/serializer/ToUnknownStream > com/sun/org/apache/xml/internal/serializer/WriterChain > com/sun/tools/javac/api/JavacTaskPool$ReusableContext$ReusableJavaCompiler > com/sun/tools/javac/file/JavacFileManager$1 > com/sun/tools/javac/file/JavacFileManager$ArchiveContainer > com/sun/tools/javac/file/JavacFileManager$Container > com/sun/tools/javac/file/JavacFileManager$DirectoryContainer > com/sun/tools/javac/file/JavacFileManager$JRTImageContainer > com/sun/tools/javac/file/Locations > com/sun/tools/javac/main/JavaCompiler > com/sun/tools/javac/processing/JavacProcessingEnvironment$DiscoveredProcessors > com/sun/tools/javac/processing/JavacProcessingEnvironment$ServiceIterator > com/sun/tools/jdi/SharedMemoryConnection > com/sun/tools/jdi/SocketConnection > com/sun/tools/sjavac/PubApiExtractor > com/sun/xml/internal/stream/Entity$ScannedEntity > com/sun/xml/internal/stream/XMLEventReaderImpl > com/sun/xml/internal/stream/writers/XMLDOMWriterImpl > com/sun/xml/internal/stream/writers/XMLEventWriterImpl > com/sun/xml/internal/stream/writers/XMLStreamWriterImpl > java/awt/SplashScreen > java/util/logging/ConsoleHandler > java/util/logging/FileHandler > java/util/logging/Handler > java/util/logging/MemoryHandler > java/util/logging/SocketHandler > java/util/logging/StreamHandler > javax/naming/Context > javax/naming/InitialContext > javax/naming/NamingEnumeration > javax/naming/directory/BasicAttribute$ValuesEnumImpl > javax/naming/directory/BasicAttributes$AttrEnumImpl > javax/naming/directory/BasicAttributes$IDEnumImpl > javax/naming/ldap/StartTlsResponse > javax/naming/spi/ContinuationContext > javax/smartcardio/CardChannel > javax/sql/PooledConnection > javax/swing/ProgressMonitor > javax/swing/text/rtf/RTFReader$AttributeTrackingDestination > javax/swing/text/rtf/RTFReader$ColortblDestination > javax/swing/text/rtf/RTFReader$Destination > javax/swing/text/rtf/RTFReader$DiscardingDestination > javax/swing/text/rtf/RTFReader$FonttblDestination > javax/swing/text/rtf/RTFReader$StylesheetDestination > javax/swing/text/rtf/RTFReader$StylesheetDestination$StyleDefiningDestination > javax/swing/text/rtf/RTFReader$TextHandlingDestination > javax/xml/stream/XMLEventReader > javax/xml/stream/XMLEventWriter > javax/xml/stream/XMLStreamReader > javax/xml/stream/XMLStreamWriter > javax/xml/stream/util/EventReaderDelegate > javax/xml/stream/util/StreamReaderDelegate > jdk/internal/jrtfs/ExplodedImage > jdk/internal/util/xml/XMLStreamWriter > jdk/internal/util/xml/impl/XMLStreamWriterImpl > jdk/jfr/internal/consumer/ChunkParser > jdk/jshell/SourceCodeAnalysisImpl > jdk/tools/jlink/internal/Archive > jdk/tools/jlink/internal/DirArchive > jdk/tools/jlink/internal/JarArchive > jdk/tools/jlink/internal/JmodArchive > org/graalvm/compiler/code/CompilationResult > org/graalvm/compiler/code/DataSection > org/graalvm/compiler/debug/DiagnosticsOutputDirectory > org/w3c/dom/html/HTMLDocument > sun/awt/image/ImageDecoder > sun/jvm/hotspot/debugger/DataSource > sun/jvm/hotspot/debugger/InputLexer > sun/jvm/hotspot/debugger/MappedByteBufferDataSource > sun/jvm/hotspot/debugger/RandomAccessFileDataSource > sun/jvm/hotspot/debugger/posix/AddressDataSource > sun/jvm/hotspot/debugger/posix/elf/ELFFile > sun/jvm/hotspot/debugger/posix/elf/ELFFileParser$ELFFileImpl > sun/jvm/hotspot/debugger/win32/coff/COFFFile > sun/jvm/hotspot/debugger/win32/coff/COFFFileParser$COFFFileImpl > sun/jvm/hotspot/debugger/windbg/AddressDataSource > sun/jvm/hotspot/debugger/windbg/DLL > sun/net/ProgressSource > sun/net/httpserver/ExchangeImpl > sun/net/www/URLConnection > sun/rmi/log/ReliableLog > sun/rmi/runtime/Log$InternalStreamHandler > sun/rmi/transport/Connection > sun/rmi/transport/tcp/TCPConnection > sun/security/smartcardio/ChannelImpl > sun/tools/java/ClassPath From david.holmes at oracle.com Wed Apr 15 23:22:47 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Apr 2020 09:22:47 +1000 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> Message-ID: <528ba5b1-71c7-591b-8b61-9dfb43a5e410@oracle.com> Hi please take this to core-libs-dev at openjdk.java.net. Thanks, David On 15/04/2020 11:35 pm, Johannes Kuhn wrote: > Hi, > > After failing to wrap a XMLStreamReader in a try-with-resources after > discovering it's close() method, > I thought about checking what other classes have a public void close() > method in the JDK but don't implement AutoCloseable. > > For this reason I wrote a small program that enumerates all classes > present in in my jrt image with a public void close() method that does not > directly or indirectly implement AutoCloseable: > https://gist.github.com/DasBrain/8d50e02e35654870e2c2c00bf3f79954 > (Output & Code, Code requires JDK 14 with preview features and ASM) > > As I am not an expert in any of those areas, I don't know if that was > just an oversight or intended. > > The XMLStreamReaderImpl will just set a flag to indicate that this > instance can be reused, with a comment //xxx: Check what this function > is intended to do. > The most likely candidate for adding AutoCloseable is > javax.naming.Context in my opinion. > Not sure about java.util.logging.Handler. As far as I know those are > closed by the Logger and closed when reloading the configuration for > example. > > Anyway, thought I share this. We could maybe add a few implements > AutoCloseable or add some comments why they don't implement that. > Some IDEs rely on resource leak warnings on the interface, so adding the > interface can be double edged sword. > > With best regards, > Johannes Kuhn > > Output: > com/sun/jdi/connect/spi/Connection > com/sun/jndi/dns/BaseNameClassPairEnumeration > com/sun/jndi/dns/DnsClient > com/sun/jndi/dns/DnsContext > com/sun/jndi/dns/Resolver > com/sun/jndi/ldap/AbstractLdapNamingEnumeration > com/sun/jndi/ldap/LdapCtx > com/sun/jndi/ldap/LdapReferralContext > com/sun/jndi/ldap/LdapSchemaCtx > com/sun/jndi/ldap/ext/StartTlsResponseImpl > com/sun/jndi/rmi/registry/BindingEnumeration > com/sun/jndi/rmi/registry/NameClassPairEnumeration > com/sun/jndi/rmi/registry/RegistryContext > com/sun/jndi/toolkit/dir/ContextEnumerator > com/sun/jndi/toolkit/dir/HierMemDirCtx > com/sun/jndi/toolkit/dir/HierMemDirCtx$BaseFlatNames > com/sun/jndi/toolkit/dir/LazySearchEnumerationImpl > com/sun/jndi/toolkit/url/GenericURLContext > com/sun/media/sound/AudioFloatFormatConverter$AudioFloatInputStreamChannelMixer > > com/sun/media/sound/AudioFloatFormatConverter$AudioFloatInputStreamResampler > > com/sun/media/sound/AudioFloatInputStream > com/sun/media/sound/AudioFloatInputStream$BytaArrayAudioFloatInputStream > com/sun/media/sound/AudioFloatInputStream$DirectAudioFloatInputStream > com/sun/media/sound/ModelAbstractOscillator > com/sun/media/sound/ModelDirector > com/sun/media/sound/ModelOscillatorStream > com/sun/media/sound/ModelStandardDirector > com/sun/media/sound/ModelStandardIndexedDirector > com/sun/media/sound/RIFFWriter$RandomAccessByteWriter > com/sun/media/sound/RIFFWriter$RandomAccessFileWriter > com/sun/media/sound/RIFFWriter$RandomAccessWriter > com/sun/media/sound/SoftAbstractResampler$ModelAbstractResamplerStream > com/sun/media/sound/SoftMainMixer > com/sun/media/sound/SoftMixingDataLine$AudioFloatInputStreamResampler > com/sun/media/sound/SoftMixingMainMixer > com/sun/media/sound/SoftMixingSourceDataLine$NonBlockingFloatInputStream > com/sun/naming/internal/VersionHelper$InputStreamEnumeration > com/sun/org/apache/xerces/internal/impl/XMLStreamFilterImpl > com/sun/org/apache/xerces/internal/impl/XMLStreamReaderImpl > com/sun/org/apache/xerces/internal/xinclude/XIncludeTextReader > com/sun/org/apache/xml/internal/serializer/EmptySerializer > com/sun/org/apache/xml/internal/serializer/SerializationHandler > com/sun/org/apache/xml/internal/serializer/SerializerBase > com/sun/org/apache/xml/internal/serializer/ToHTMLSAXHandler > com/sun/org/apache/xml/internal/serializer/ToUnknownStream > com/sun/org/apache/xml/internal/serializer/WriterChain > com/sun/tools/javac/api/JavacTaskPool$ReusableContext$ReusableJavaCompiler > com/sun/tools/javac/file/JavacFileManager$1 > com/sun/tools/javac/file/JavacFileManager$ArchiveContainer > com/sun/tools/javac/file/JavacFileManager$Container > com/sun/tools/javac/file/JavacFileManager$DirectoryContainer > com/sun/tools/javac/file/JavacFileManager$JRTImageContainer > com/sun/tools/javac/file/Locations > com/sun/tools/javac/main/JavaCompiler > com/sun/tools/javac/processing/JavacProcessingEnvironment$DiscoveredProcessors > > com/sun/tools/javac/processing/JavacProcessingEnvironment$ServiceIterator > com/sun/tools/jdi/SharedMemoryConnection > com/sun/tools/jdi/SocketConnection > com/sun/tools/sjavac/PubApiExtractor > com/sun/xml/internal/stream/Entity$ScannedEntity > com/sun/xml/internal/stream/XMLEventReaderImpl > com/sun/xml/internal/stream/writers/XMLDOMWriterImpl > com/sun/xml/internal/stream/writers/XMLEventWriterImpl > com/sun/xml/internal/stream/writers/XMLStreamWriterImpl > java/awt/SplashScreen > java/util/logging/ConsoleHandler > java/util/logging/FileHandler > java/util/logging/Handler > java/util/logging/MemoryHandler > java/util/logging/SocketHandler > java/util/logging/StreamHandler > javax/naming/Context > javax/naming/InitialContext > javax/naming/NamingEnumeration > javax/naming/directory/BasicAttribute$ValuesEnumImpl > javax/naming/directory/BasicAttributes$AttrEnumImpl > javax/naming/directory/BasicAttributes$IDEnumImpl > javax/naming/ldap/StartTlsResponse > javax/naming/spi/ContinuationContext > javax/smartcardio/CardChannel > javax/sql/PooledConnection > javax/swing/ProgressMonitor > javax/swing/text/rtf/RTFReader$AttributeTrackingDestination > javax/swing/text/rtf/RTFReader$ColortblDestination > javax/swing/text/rtf/RTFReader$Destination > javax/swing/text/rtf/RTFReader$DiscardingDestination > javax/swing/text/rtf/RTFReader$FonttblDestination > javax/swing/text/rtf/RTFReader$StylesheetDestination > javax/swing/text/rtf/RTFReader$StylesheetDestination$StyleDefiningDestination > > javax/swing/text/rtf/RTFReader$TextHandlingDestination > javax/xml/stream/XMLEventReader > javax/xml/stream/XMLEventWriter > javax/xml/stream/XMLStreamReader > javax/xml/stream/XMLStreamWriter > javax/xml/stream/util/EventReaderDelegate > javax/xml/stream/util/StreamReaderDelegate > jdk/internal/jrtfs/ExplodedImage > jdk/internal/util/xml/XMLStreamWriter > jdk/internal/util/xml/impl/XMLStreamWriterImpl > jdk/jfr/internal/consumer/ChunkParser > jdk/jshell/SourceCodeAnalysisImpl > jdk/tools/jlink/internal/Archive > jdk/tools/jlink/internal/DirArchive > jdk/tools/jlink/internal/JarArchive > jdk/tools/jlink/internal/JmodArchive > org/graalvm/compiler/code/CompilationResult > org/graalvm/compiler/code/DataSection > org/graalvm/compiler/debug/DiagnosticsOutputDirectory > org/w3c/dom/html/HTMLDocument > sun/awt/image/ImageDecoder > sun/jvm/hotspot/debugger/DataSource > sun/jvm/hotspot/debugger/InputLexer > sun/jvm/hotspot/debugger/MappedByteBufferDataSource > sun/jvm/hotspot/debugger/RandomAccessFileDataSource > sun/jvm/hotspot/debugger/posix/AddressDataSource > sun/jvm/hotspot/debugger/posix/elf/ELFFile > sun/jvm/hotspot/debugger/posix/elf/ELFFileParser$ELFFileImpl > sun/jvm/hotspot/debugger/win32/coff/COFFFile > sun/jvm/hotspot/debugger/win32/coff/COFFFileParser$COFFFileImpl > sun/jvm/hotspot/debugger/windbg/AddressDataSource > sun/jvm/hotspot/debugger/windbg/DLL > sun/net/ProgressSource > sun/net/httpserver/ExchangeImpl > sun/net/www/URLConnection > sun/rmi/log/ReliableLog > sun/rmi/runtime/Log$InternalStreamHandler > sun/rmi/transport/Connection > sun/rmi/transport/tcp/TCPConnection > sun/security/smartcardio/ChannelImpl > sun/tools/java/ClassPath > From magnus.ihse.bursie at oracle.com Thu Apr 16 15:02:45 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 16 Apr 2020 17:02:45 +0200 Subject: Revival of JEP 198 (Light-Weight JSON API)? Message-ID: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> The recent removal of Nashorn has indicated several places were javascript were used in the JDK mainly due to it's built-in support for parsing JSON. While there exists several third party JSON libraries, the lack of a JSON parser included in the JDK is becoming more and more apparent. I guess it might be common for other users as well, to have utilized Nashorn and Javascript for it's JSON abilities. Even if we now deprive these users of Javascript, we need not deprive them of JSON handling ability. Fortunately, there already exists a JEP (JEP 198: Light-Weight JSON API) [1] that proposes this. Unfortunately, to my knowledge, this is nowhere closer to being delivered in the JDK than it was when it was created in 2014. It's 2020. I believe the inclusion of a standard JSON library in the JDK is long overdue. This is not a technically challenging JEP. It's more a question of "just doing it". Of course it's important to get the API right, but once again, the problem space is small, and there are several other successful JSON libraries out there that can serve as good inspiration. /Magnus [1] https://openjdk.java.net/jeps/198 From forax at univ-mlv.fr Thu Apr 16 15:41:55 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 16 Apr 2020 17:41:55 +0200 (CEST) Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> Message-ID: <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> Yes, go for it ! As far as i remember, - there was a human issue, why creating a JDK version when what you do is standardize the library "foo" - some technical challenges around providing a stream API, interoperable with java.util.stream or not. R?mi ----- Mail original ----- > De: "Magnus Ihse Bursie" > ?: "discuss" > Envoy?: Jeudi 16 Avril 2020 17:02:45 > Objet: Revival of JEP 198 (Light-Weight JSON API)? > The recent removal of Nashorn has indicated several places were > javascript were used in the JDK mainly due to it's built-in support for > parsing JSON. While there exists several third party JSON libraries, the > lack of a JSON parser included in the JDK is becoming more and more > apparent. I guess it might be common for other users as well, to have > utilized Nashorn and Javascript for it's JSON abilities. Even if we now > deprive these users of Javascript, we need not deprive them of JSON > handling ability. > > Fortunately, there already exists a JEP (JEP 198: Light-Weight JSON API) > [1] that proposes this. Unfortunately, to my knowledge, this is nowhere > closer to being delivered in the JDK than it was when it was created in > 2014. > > It's 2020. I believe the inclusion of a standard JSON library in the JDK > is long overdue. This is not a technically challenging JEP. It's more a > question of "just doing it". Of course it's important to get the API > right, but once again, the problem space is small, and there are several > other successful JSON libraries out there that can serve as good > inspiration. > > /Magnus > > [1] https://openjdk.java.net/jeps/198 From brian.goetz at oracle.com Thu Apr 16 15:53:45 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 16 Apr 2020 11:53:45 -0400 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> Message-ID: <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> And, of course, take full advantage of Valhalla when it?s there. > On Apr 16, 2020, at 11:53 AM, Brian Goetz wrote: > > Also, a JSON parser should be designed so that, when the full pattern matching story is delivered, it works perfectly with pattern matching :) This is a difficult constraint as it involves keeping all the possible directions for pattern matching in your head, and also designing something that can work now with the language we have and trivially migrate to the language we?ll have tomorrow. This is no simple task! > >> On Apr 16, 2020, at 11:41 AM, Remi Forax wrote: >> >> Yes, go for it ! >> >> As far as i remember, >> - there was a human issue, why creating a JDK version when what you do is standardize the library "foo" >> - some technical challenges around providing a stream API, interoperable with java.util.stream or not. >> >> R?mi >> >> ----- Mail original ----- >>> De: "Magnus Ihse Bursie" >>> ?: "discuss" >>> Envoy?: Jeudi 16 Avril 2020 17:02:45 >>> Objet: Revival of JEP 198 (Light-Weight JSON API)? >> >>> The recent removal of Nashorn has indicated several places were >>> javascript were used in the JDK mainly due to it's built-in support for >>> parsing JSON. While there exists several third party JSON libraries, the >>> lack of a JSON parser included in the JDK is becoming more and more >>> apparent. I guess it might be common for other users as well, to have >>> utilized Nashorn and Javascript for it's JSON abilities. Even if we now >>> deprive these users of Javascript, we need not deprive them of JSON >>> handling ability. >>> >>> Fortunately, there already exists a JEP (JEP 198: Light-Weight JSON API) >>> [1] that proposes this. Unfortunately, to my knowledge, this is nowhere >>> closer to being delivered in the JDK than it was when it was created in >>> 2014. >>> >>> It's 2020. I believe the inclusion of a standard JSON library in the JDK >>> is long overdue. This is not a technically challenging JEP. It's more a >>> question of "just doing it". Of course it's important to get the API >>> right, but once again, the problem space is small, and there are several >>> other successful JSON libraries out there that can serve as good >>> inspiration. >>> >>> /Magnus >>> >>> [1] https://openjdk.java.net/jeps/198 > From brian.goetz at oracle.com Thu Apr 16 15:53:25 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 16 Apr 2020 11:53:25 -0400 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> Message-ID: <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> Also, a JSON parser should be designed so that, when the full pattern matching story is delivered, it works perfectly with pattern matching :) This is a difficult constraint as it involves keeping all the possible directions for pattern matching in your head, and also designing something that can work now with the language we have and trivially migrate to the language we?ll have tomorrow. This is no simple task! > On Apr 16, 2020, at 11:41 AM, Remi Forax wrote: > > Yes, go for it ! > > As far as i remember, > - there was a human issue, why creating a JDK version when what you do is standardize the library "foo" > - some technical challenges around providing a stream API, interoperable with java.util.stream or not. > > R?mi > > ----- Mail original ----- >> De: "Magnus Ihse Bursie" >> ?: "discuss" >> Envoy?: Jeudi 16 Avril 2020 17:02:45 >> Objet: Revival of JEP 198 (Light-Weight JSON API)? > >> The recent removal of Nashorn has indicated several places were >> javascript were used in the JDK mainly due to it's built-in support for >> parsing JSON. While there exists several third party JSON libraries, the >> lack of a JSON parser included in the JDK is becoming more and more >> apparent. I guess it might be common for other users as well, to have >> utilized Nashorn and Javascript for it's JSON abilities. Even if we now >> deprive these users of Javascript, we need not deprive them of JSON >> handling ability. >> >> Fortunately, there already exists a JEP (JEP 198: Light-Weight JSON API) >> [1] that proposes this. Unfortunately, to my knowledge, this is nowhere >> closer to being delivered in the JDK than it was when it was created in >> 2014. >> >> It's 2020. I believe the inclusion of a standard JSON library in the JDK >> is long overdue. This is not a technically challenging JEP. It's more a >> question of "just doing it". Of course it's important to get the API >> right, but once again, the problem space is small, and there are several >> other successful JSON libraries out there that can serve as good >> inspiration. >> >> /Magnus >> >> [1] https://openjdk.java.net/jeps/198 From larry.cable at oracle.com Thu Apr 16 16:28:15 2020 From: larry.cable at oracle.com (Laurence Cable) Date: Thu, 16 Apr 2020 09:28:15 -0700 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> Message-ID: <868590c3-aa87-9271-675a-eef435481866@oracle.com> +1 I have recently seen a few places where code is generating JSON in an ad-hoc fashion ... the JDK should have a well integrated API for parsing and generation... On 4/16/20 8:02 AM, Magnus Ihse Bursie wrote: > The recent removal of Nashorn has indicated several places were > javascript were used in the JDK mainly due to it's built-in support > for parsing JSON. While there exists several third party JSON > libraries, the lack of a JSON parser included in the JDK is becoming > more and more apparent. I guess it might be common for other users as > well, to have utilized Nashorn and Javascript for it's JSON abilities. > Even if we now deprive these users of Javascript, we need not deprive > them of JSON handling ability. > > Fortunately, there already exists a JEP (JEP 198: Light-Weight JSON > API) [1] that proposes this. Unfortunately, to my knowledge, this is > nowhere closer to being delivered in the JDK than it was when it was > created in 2014. > > It's 2020. I believe the inclusion of a standard JSON library in the > JDK is long overdue. This is not a technically challenging JEP. It's > more a question of "just doing it". Of course it's important to get > the API right, but once again, the problem space is small, and there > are several other successful JSON libraries out there that can serve > as good inspiration. > > /Magnus > > [1] https://openjdk.java.net/jeps/198 > > From john.r.rose at oracle.com Thu Apr 16 21:31:49 2020 From: john.r.rose at oracle.com (John Rose) Date: Thu, 16 Apr 2020 14:31:49 -0700 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> Message-ID: <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> Nice. I tried to find a way to make this a little more generic and came up with an API which handles arbitrary exceptions and is packaged as a method instead of a special idiom. There?s a natural-enough method called `forceTryable` that could be created. http://cr.openjdk.java.net/~jrose/jdk/ForceTryableDemo.java https://bugs.openjdk.java.net/browse/JDK-8243015 (There are further generalizations of this, if you look at it as a ?named cast method? for a functional interface type related to AutoCloseable. See my comment on JDK-8243015.) ? John On Apr 15, 2020, at 8:53 AM, Remi Forax wrote: > > interface FakeAutoCloseable extends AutoCloseable { public void close(); } // suppress exception > > XMLReader xmlReader = ... > try(FakeAutoCloseable __ = xmlReader::close) { > ? > From forax at univ-mlv.fr Thu Apr 16 23:00:33 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 17 Apr 2020 01:00:33 +0200 (CEST) Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> Message-ID: <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> A couple of points, - it can be fixed by making TWR using a functional type instead of a nominal one by supporting a syntax like try(expr) { }. But this is in the same realm of ideas as allowing the enhanced for loop to use a an Iterator::iterator, so it has been rule out has a step too far by the lambda EG. - until lambdas are retrofitted as inline types, i'm worry people will use forceTryable with a ReentrantLock, try(var __ = forceTryable(lock::unlock)) { ... } because most of the time, the lambda will not be allocated but not always, sometimes it may still be found escaping by the JITs. - my preferred fix is, as you already, know to remove the concept of checked exceptions from the language instead of spending time running in circles around them. R?mi ----- Mail original ----- > De: "John Rose" > ?: "Remi Forax" > Cc: "Johannes Kuhn" , "discuss" > Envoy?: Jeudi 16 Avril 2020 23:31:49 > Objet: Re: Some Classes with a public void close() don't implement AutoCloseable > Nice. I tried to find a way to make this a little more generic > and came up with an API which handles arbitrary exceptions > and is packaged as a method instead of a special idiom. > There?s a natural-enough method called `forceTryable` that > could be created. > > http://cr.openjdk.java.net/~jrose/jdk/ForceTryableDemo.java > https://bugs.openjdk.java.net/browse/JDK-8243015 > > (There are further generalizations of this, if you look at it as > a ?named cast method? for a functional interface type related > to AutoCloseable. See my comment on JDK-8243015.) > > ? John > > On Apr 15, 2020, at 8:53 AM, Remi Forax wrote: >> >> interface FakeAutoCloseable extends AutoCloseable { public void close(); } // >> suppress exception >> >> XMLReader xmlReader = ... >> try(FakeAutoCloseable __ = xmlReader::close) { >> ? From fw at deneb.enyo.de Fri Apr 17 16:35:07 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Fri, 17 Apr 2020 18:35:07 +0200 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> (forax's message of "Fri, 17 Apr 2020 01:00:33 +0200 (CEST)") References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> Message-ID: <87imhyax50.fsf@mid.deneb.enyo.de> * forax: > - until lambdas are retrofitted as inline types, i'm worry people > will use forceTryable with a ReentrantLock, try(var __ = > forceTryable(lock::unlock)) { ... } because most of the time, the > lambda will not be allocated but not always, sometimes it may still > be found escaping by the JITs. The JDK itself routinely does something very similar when registering cleaners. Is this lack of robustness in the presence of OOMEs a quality-of-implementation issue? Or less or more important? From john.r.rose at oracle.com Fri Apr 17 21:52:46 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 17 Apr 2020 14:52:46 -0700 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <87imhyax50.fsf@mid.deneb.enyo.de> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> <87imhyax50.fsf@mid.deneb.enyo.de> Message-ID: <7DAE8438-E208-4D94-97CD-93989EBEBE96@oracle.com> On Apr 17, 2020, at 9:35 AM, Florian Weimer wrote: > > * forax: > >> - until lambdas are retrofitted as inline types, i'm worry people >> will use forceTryable with a ReentrantLock, try(var __ = >> forceTryable(lock::unlock)) { ... } because most of the time, the >> lambda will not be allocated but not always, sometimes it may still >> be found escaping by the JITs. > > The JDK itself routinely does something very similar when registering > cleaners. Is this lack of robustness in the presence of OOMEs a > quality-of-implementation issue? Or less or more important? Turning this around in my mind, it seems like the root cause of this non-robustness is a minor mis-feature of TWR. TWR says to itself, ?I only have one job, and that is to call close?. That?s fine, but that means that means that the object to be closed must be created *and* opened before TWR takes control. If the closing operation is mediated by an adapter, the adapter will almost certainly be created *after* the original (unadapted) object is created and opened. This is the point where an OOME could bollix things up. To defend against such failures, TWR would have to define a distinct point where the object is opened. This point would be a precise state transition immediately before entry into the block covered by catches and the `finally` block. Because TWR has the reasonable simplification that it only cares about closing?the opening is up to the coder?it cannot help precisely define the opening point; the opening point happens ?some time before? the TWR gets to work. That ?some time before? happens either textually before the TWR, or inside the TWR?s head expression. If the actual logical open succeeds, but then there is an adapter step in the TWR?s head expression which causes an exception to be thrown, then we have a problem we could call a ?lost close?. The TWR seems to promise that its head action will *always* be undone by an auto-close, but that promise has fine print; there are small gaps between the logical open and the actual finalization guarantees of the TWR. (Apologies if this has already been discussed to death. I was only a tangential part of the design discussions where the problem of lost closes would have been covered. I?m sure the problem of lost closes was covered, but I?m not sure how deep was the argument that dismissed the problem. It seems to me that the lore about lost closes has grown years after TWR was designed. For example, we have had to build JVM features to assist?partially?with the problem.) There are possible workarounds for lost closes, which ask the user of the TWR block to place an ?open? expression or statement explicitly somewhere in the TWR: try (var locker = new Locker()) { locker.open(); // explicit open statement ensures all setup happens first } or: try (var __ = new Locker() .open() // explicit open expression ensures all setup happens first ) { } The problem with these workarounds is they require a lot of cooperation from the user. Which means they are error-prone. Which means designers of abstractions for the head expressions of TWR avoid making such requirements on end users, and so place the logical open operation deep inside the TWR head expression (abstracted away from the end user?s control). I am going out on a limb saying TWR has a ?mis-feature? because the focus of TWR on close is obviously a reasonable Java-like simplification of TWR design. Culturally, we go simple when we can; there seemed at the time to be little or no downside to omitting consideration of `open` and focusing only on `close`. It?s arguably only a mis-feature because programmers do not think about ?close? operations as something completely separate from ?open? operations; they think in terms of balanced pairs, which bracket transactional code. TWR purports to help programmers ?balance the brackets? but it doesn?t help them precisely manage the opening brackets, just the closing ones. To the extent that this leads programmers into bracketing mismatches problems, that?s a mis-feature, despite TWR?s claim to do its one job flawlessly; the problem is that TWR in practice ought to help the programmer with both brackets, not just one. How might this mis-feature be corrected? Maybe it can?t. Maybe there?s no way to deal with lost closes other than educating programmers to avoid allocations or other computations in the head of the TWR, at least allocations after the logical open. (Those are the ones which might cause an OOME or SOE that would lead to a lost close.) But (and this is why I?m going on at length here) maybe there is a way to give the user more help avoiding lost closes. It seems that if TWR were willing to define open operations separately, some progress could be made. Suppose the object at the head of a TWR implemented an optional new interface `java.lang.AutoOpenable`. In such cases, the TWR would call a nullary `open` operation on that object, immediately before entering the block covered by the `finally` clause that calls `close`. Would this help? Yes, it would, because the designer of the abstraction at the head of the TWR could take care to do all argument validation and storage allocation *without* doing the actual logical open or allocate the resource that needs to be closed (or seize the lock). The evaluation of the head expression(s) of the TWR would (as before) not be under any finally clause or catches. The new feature here would be that the designer of the abstraction (at the head of the TWR) would know that the expression creating the abstraction would do all necessary storage allocation, adapter creation, argument validation, and (if necessary) stack banging, *before* the final critical opening step. It would not be up to the end user; it would be more under the control of the abstraction, to sequence the preparatory steps *before* the logical open. Anyway, I think this discussion may have shaken loose a new idea (at least new to me) for dealing with a truly vexed problem. I filed https://bugs.openjdk.java.net/browse/JDK-8243098 to track. ? John From dmytro.sheyko.jdk at gmail.com Sat Apr 18 07:01:37 2020 From: dmytro.sheyko.jdk at gmail.com (dmytro sheyko) Date: Sat, 18 Apr 2020 10:01:37 +0300 Subject: Fwd: Type-parameterized complement to Object.equals(Object) In-Reply-To: References: Message-ID: Hello, I need a sponsor for following JDK change. Summary ------- Provide means for type safe comparison for equality by introducing type-parameterized interface `java.lang.Equable` with method `boolean equ(T)`. (The actual names of the interface and the method is subject to discussion.) Goals ----- The primary goal is to avoid programmer mistakes when values of different and incomparable types are compared using `equals` method, which always returns `false` in such cases according to its contract. Non-Goals --------- Custom hashers and equators for Collection Framework are out of the scope of this proposal. Motivation ---------- This is quite common approach to use `equals` method to compare value for equality. However, it may happen that the values have different incomparable types and, therefore, the result of such a comparison is doomed to `false`. For example, instances of `java.lang.Integer` and `java.lang.Long` will never be equal even though they represent the same numerical value. I.e. this code Long l = 42L; Integer i = 42; if (l.equals(i)) { // always false ... } else { ... } is very likely a mistake because the else-branch will never be taken. The same applies to attempts to compare among themselves instances of `java.lang.Short`, `java.lang.String`, `java.util.Date`, `java.util.List`, `java.util.Set`, `java.util.Map` etc. The reason why such erroneous comparisons remain unnoticed is the fact that the `equals` method accepts any value. One way to make mistakes with such comparisons more visible it to use `compareTo` method. Long l = 42L; Integer i = 42; if (l.compareTo(i) == 0) { // compilation error ... } else { ... } However this approach has some drawbacks: * It's counter-intuitive and unnecessarily verbose. It's not clear at glance why `compareTo` is used instead of `equals`. * It can be inefficient because there are cases when an attempt to figure out whether given value is less or greater that the other value requires more effort than an attempt to figure out whether the values are just not equal. For example, when we compare strings and have noticed that the strings have different length we can stop here and do not scan them in order to figure out, which one is greater than the other. Another example is the class that represents rational number as a reduced fraction. To make sure that the rational numbers are equal, we can just compare their numerators and denominators. But in order to find out which one is greater than the other we can't avoid arithmetic operations (multiplications). * It's not applicable to the classes that do not implement `java.lang.Comparable`. For example, collections, complex numbers, points in 2D or 3D space etc. Therefore it's proposed to introduce interface `Equable` in order to explicitly mark comparable classes. public interface Equable { boolean equ(T o); } Let `Comparable` extend it and provide default method for `equ`. public interface Comparable<> extends Equable { int compareTo(T o); default boolean equ(T o) { return compareTo(o) == 0; } } Provide efficient implementation of `equ` method for some JDK classes, at least those that are in `java.lang` package. Also it would be desiable to let `java.util.List`, `java.util.Set`, `java.util.Map` and `java.util.Map.Entry` to extend `Equable` interface as well (but without extending `Comparable`). === I am sending a set of patches. I split the whole change into several logical parts. * equable-0.patch contain `java.lang.Equable`, `java.lang.Comparable`, `java.util.Objects.equ` * equable-1.patch contain `j.l.Boolean`, `j.l.Byte`, `j.l.Character`, `j.l.Double`, `j.l.Enum`, `j.l.Float`, `j.l.Integer`, `j.l.Long`, `j.l.Short` and `j.l.String` * equable-2.patch contain `j.u.Optional`, `j.u.OptionalDouble`, `j.u.OptionalInt`, `j.u.OptionalLong` * equable-3.patch contain `j.u.List`, `j.u.AbstractList`, `j.u.Set`, `j.u.AbstractSet`, `j.u.Map`, `j.u.AbstractMap` * equable-4.patch is a tricky one. It contains `com.sun.tools.jdi.EventSetImpl`. Previously this class extended `java.lang.ArrayList` and implemented `java.lang.Set` through `EventSet`. Therefore it appeared that it implemented both `java.util.List` and `java.util.Set` at the same time. This is not right, because these interfaces have different contracts regarding equality. So I had to change it by extending `java.lang.AbstractSet`. Thank you, Dmytro From john.r.rose at oracle.com Sat Apr 18 07:53:58 2020 From: john.r.rose at oracle.com (John Rose) Date: Sat, 18 Apr 2020 00:53:58 -0700 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> Message-ID: <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> Representing a JSON tree as a tree of Valhalla values (yes, they can come in trees!) is very much worth investigating IMO. The nearest approximation before Valhalla would be to design a JSON node model which consists only of (or mainly of) value-based classes. Or, perhaps Valhalla-style cursors (iterators which say x=x.next() instead of x.next()) can express a walk over a mutable tree, whose structure is hidden (and thus can be materialized lazily or streamily). The nearest approximation to that would be an iterator-style API, where one object updates itself in place. One other thought: Sometimes that parsing and unparsing logic for a syntax like JSON or XML can be expressed as operating on a linear event-stream, instead of a nested tree model. You can build the latter on top of the former, and event streams have a simpler object structure than tree-like models. It might be safer to start with a parser/unparser API that runs on event streams, and then build node models (with cursors, pattern matching, etc.) on top later when the tools for those appear. ?Not that I?m longing to program an event-based API, but it?s better than nothing, and not lost effort either. > On Apr 16, 2020, at 8:53 AM, Brian Goetz wrote: > > And, of course, take full advantage of Valhalla when it?s there. > >> On Apr 16, 2020, at 11:53 AM, Brian Goetz wrote: >> >> Also, a JSON parser should be designed so that, when the full pattern matching story is delivered, it works perfectly with pattern matching :) This is a difficult constraint as it involves keeping all the possible directions for pattern matching in your head, and also designing something that can work now with the language we have and trivially migrate to the language we?ll have tomorrow. This is no simple task! >> >>> On Apr 16, 2020, at 11:41 AM, Remi Forax wrote: >>> >>> Yes, go for it ! >>> >>> As far as i remember, >>> - there was a human issue, why creating a JDK version when what you do is standardize the library "foo" >>> - some technical challenges around providing a stream API, interoperable with java.util.stream or not. >>> >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "Magnus Ihse Bursie" >>>> ?: "discuss" >>>> Envoy?: Jeudi 16 Avril 2020 17:02:45 >>>> Objet: Revival of JEP 198 (Light-Weight JSON API)? >>> >>>> The recent removal of Nashorn has indicated several places were >>>> javascript were used in the JDK mainly due to it's built-in support for >>>> parsing JSON. While there exists several third party JSON libraries, the >>>> lack of a JSON parser included in the JDK is becoming more and more >>>> apparent. I guess it might be common for other users as well, to have >>>> utilized Nashorn and Javascript for it's JSON abilities. Even if we now >>>> deprive these users of Javascript, we need not deprive them of JSON >>>> handling ability. >>>> >>>> Fortunately, there already exists a JEP (JEP 198: Light-Weight JSON API) >>>> [1] that proposes this. Unfortunately, to my knowledge, this is nowhere >>>> closer to being delivered in the JDK than it was when it was created in >>>> 2014. >>>> >>>> It's 2020. I believe the inclusion of a standard JSON library in the JDK >>>> is long overdue. This is not a technically challenging JEP. It's more a >>>> question of "just doing it". Of course it's important to get the API >>>> right, but once again, the problem space is small, and there are several >>>> other successful JSON libraries out there that can serve as good >>>> inspiration. >>>> >>>> /Magnus >>>> >>>> [1] https://openjdk.java.net/jeps/198 >> > From forax at univ-mlv.fr Sat Apr 18 11:25:28 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 18 Apr 2020 13:25:28 +0200 (CEST) Subject: Type-parameterized complement to Object.equals(Object) In-Reply-To: References: Message-ID: <1711306028.1264107.1587209128145.JavaMail.zimbra@u-pem.fr> Hi Dmytro, this was discussed at length at the time Scala was the new kid, the solution you propose for Java does not work because you can not implement the same interface with different type arguments. Let say i've a class Vehicle and a subclass Car, using Equable, i will want to write class Vehicle implements Equable { ... } class Car extends Vehicle implements Equable { ... } The other good reason to not do any changes on equality now is that we may revisit how equality works if we introduce operator overloading on inline type as part of Valhalla. You can also note that the status quo is not as bad as you may think because all IDEs provide an inspection/analysis to detect such case. regards, R?mi ----- Mail original ----- > De: "dmytro sheyko" > ?: "discuss" > Envoy?: Samedi 18 Avril 2020 09:01:37 > Objet: Fwd: Type-parameterized complement to Object.equals(Object) > Hello, > > I need a sponsor for following JDK change. > > Summary > ------- > > Provide means for type safe comparison for equality by introducing > type-parameterized interface `java.lang.Equable` with method `boolean > equ(T)`. > (The actual names of the interface and the method is subject to discussion.) > > > Goals > ----- > > The primary goal is to avoid programmer mistakes when values of different > and > incomparable types are compared using `equals` method, which always returns > `false` > in such cases according to its contract. > > > Non-Goals > --------- > > Custom hashers and equators for Collection Framework are out of the scope > of this proposal. > > > Motivation > ---------- > > This is quite common approach to use `equals` method to compare value for > equality. > However, it may happen that the values have different incomparable types > and, therefore, > the result of such a comparison is doomed to `false`. For example, > instances of > `java.lang.Integer` and `java.lang.Long` will never be equal even though > they represent > the same numerical value. I.e. this code > > Long l = 42L; > Integer i = 42; > if (l.equals(i)) { // always false > ... > } else { > ... > } > > is very likely a mistake because the else-branch will never be taken. The > same applies to attempts > to compare among themselves instances of `java.lang.Short`, > `java.lang.String`, `java.util.Date`, > `java.util.List`, `java.util.Set`, `java.util.Map` etc. The reason why such > erroneous comparisons > remain unnoticed is the fact that the `equals` method accepts any value. > One way to make mistakes > with such comparisons more visible it to use `compareTo` method. > > Long l = 42L; > Integer i = 42; > if (l.compareTo(i) == 0) { // compilation error > ... > } else { > ... > } > > However this approach has some drawbacks: > * It's counter-intuitive and unnecessarily verbose. It's not clear at > glance why `compareTo` > is used instead of `equals`. > * It can be inefficient because there are cases when an attempt to figure > out whether given value > is less or greater that the other value requires more effort than an > attempt to figure out > whether the values are just not equal. For example, when we compare > strings and have noticed > that the strings have different length we can stop here and do not scan > them in order to > figure out, which one is greater than the other. Another example is the > class that represents > rational number as a reduced fraction. To make sure that the rational > numbers are equal, we can > just compare their numerators and denominators. But in order to find out > which one is greater > than the other we can't avoid arithmetic operations (multiplications). > * It's not applicable to the classes that do not implement > `java.lang.Comparable`. For example, > collections, complex numbers, points in 2D or 3D space etc. > > Therefore it's proposed to introduce interface `Equable` in order to > explicitly mark comparable classes. > > public interface Equable { > boolean equ(T o); > } > > Let `Comparable` extend it and provide default method for `equ`. > > public interface Comparable<> extends Equable { > int compareTo(T o); > > default boolean equ(T o) { > return compareTo(o) == 0; > } > } > > Provide efficient implementation of `equ` method for some JDK classes, at > least those that are in > `java.lang` package. Also it would be desiable to let `java.util.List`, > `java.util.Set`, > `java.util.Map` and `java.util.Map.Entry` to extend `Equable` interface as > well (but without > extending `Comparable`). > > === > > I am sending a set of patches. I split the whole change into several > logical parts. > > * equable-0.patch contain `java.lang.Equable`, `java.lang.Comparable`, > `java.util.Objects.equ` > > * equable-1.patch contain `j.l.Boolean`, `j.l.Byte`, `j.l.Character`, > `j.l.Double`, `j.l.Enum`, > `j.l.Float`, `j.l.Integer`, `j.l.Long`, `j.l.Short` and `j.l.String` > > * equable-2.patch contain `j.u.Optional`, `j.u.OptionalDouble`, > `j.u.OptionalInt`, > `j.u.OptionalLong` > > * equable-3.patch contain `j.u.List`, `j.u.AbstractList`, `j.u.Set`, > `j.u.AbstractSet`, `j.u.Map`, > `j.u.AbstractMap` > > * equable-4.patch is a tricky one. It contains > `com.sun.tools.jdi.EventSetImpl`. Previously this class > extended `java.lang.ArrayList` and implemented `java.lang.Set` through > `EventSet`. Therefore it appeared > that it implemented both `java.util.List` and `java.util.Set` at the same > time. This is not right, > because these interfaces have different contracts regarding equality. So > I had to change it by extending > `java.lang.AbstractSet`. > > Thank you, > Dmytro From dmytro.sheyko.jdk at gmail.com Sat Apr 18 14:40:39 2020 From: dmytro.sheyko.jdk at gmail.com (dmytro sheyko) Date: Sat, 18 Apr 2020 17:40:39 +0300 Subject: Type-parameterized complement to Object.equals(Object) In-Reply-To: <1711306028.1264107.1587209128145.JavaMail.zimbra@u-pem.fr> References: <1711306028.1264107.1587209128145.JavaMail.zimbra@u-pem.fr> Message-ID: Thank you for your response, R?mi. > The solution you propose for Java does not work because you can not implement the same interface with different type arguments. > Let say i've a class Vehicle and a subclass Car, using Equable, i will want to write > class Vehicle implements Equable { ... } > class Car extends Vehicle implements Equable { ... } In this particular case `Car` can perfectly remain `Equable`, there is not need to make it `Equable`. Even though instance of `Car` can never be equal to instance of `Vehicle`, it's still possible to assign instance of `Car` to variable of type `Vehicle` and when compiler see comparison between `Car` and `Vehicle`, there is a chance that they are equal. In this case `equ` works at least not worse than `equals`. The benefit of `equ` becomes more apparent when `Car` or `Vehicle` is compared with something totally unrelated. You are right that it's impossible to implement the same interface with different type arguments. This can be an obstacle for classes that implement, let's say, both `java.util.Set` and `java.util.List` at the same time (assuming that List extends Equable> and Set extends Equable>). But I would consider this case rather as abnormal because it's impossible to implement the contract for equality (`equals`+`hashCode`) for `List` and `Set` simultaneously not even taking `Equable` into account. The `Equable` will just reveal such problems. If there will such an example, when it's completely sane and legal to implement both `List` and `Set`, there is a plan B. The interfaces such as `List` and `Set` will not extend `Equable<...>` directly, but provide `Equable<...>` object instead. E.g. interface Set extends Collection { : Equable> asEquableSet(); : } interface List extends Collection { : Equable> asEquableList(); : } Although I doubt there will such an example. After all the argument that it's impossible to implement the same interface with different type arguments is also applicable to `Comparable`, but it does not prevent it to exist. The `Equable` is just more general variant of `Comparable`. > You can also note that the status quo is not as bad as you may think because all IDEs provide an inspection/analysis to detect such case. Right, IDEs sometimes smart with analysis, however I am not sure that they are able to detect all such cases simply because they do not have enough information to make right decision. Marking classes/interfaces with `Equable` is the way to provide this information to the compiler or IDE. By the way, I tried following example with IntelliJ IDEA static class A { @Override public boolean equals(Object o) { return o instanceof A; } } static class B extends A {} static class C extends A {} if (new B().equals(new C())) { System.out.println("!"); } And it reports "'equals()' between objects of inconvertible types 'B' and 'C'", which is definitely a false alarm (and luckily an ignorable warning). Regards, Dmytro On Sat, Apr 18, 2020 at 2:25 PM Remi Forax wrote: > Hi Dmytro, > this was discussed at length at the time Scala was the new kid, > the solution you propose for Java does not work because you can not > implement the same interface with different type arguments. > > Let say i've a class Vehicle and a subclass Car, using Equable, i will > want to write > class Vehicle implements Equable { ... } > class Car extends Vehicle implements Equable { ... } > > The other good reason to not do any changes on equality now is that we may > revisit how equality works if we introduce operator overloading on inline > type as part of Valhalla. > > You can also note that the status quo is not as bad as you may think > because all IDEs provide an inspection/analysis to detect such case. > > regards, > R?mi > > ----- Mail original ----- > > De: "dmytro sheyko" > > ?: "discuss" > > Envoy?: Samedi 18 Avril 2020 09:01:37 > > Objet: Fwd: Type-parameterized complement to Object.equals(Object) > > > Hello, > > > > I need a sponsor for following JDK change. > > > > Summary > > ------- > > > > Provide means for type safe comparison for equality by introducing > > type-parameterized interface `java.lang.Equable` with method `boolean > > equ(T)`. > > (The actual names of the interface and the method is subject to > discussion.) > > > > > > Goals > > ----- > > > > The primary goal is to avoid programmer mistakes when values of different > > and > > incomparable types are compared using `equals` method, which always > returns > > `false` > > in such cases according to its contract. > > > > > > Non-Goals > > --------- > > > > Custom hashers and equators for Collection Framework are out of the scope > > of this proposal. > > > > > > Motivation > > ---------- > > > > This is quite common approach to use `equals` method to compare value for > > equality. > > However, it may happen that the values have different incomparable types > > and, therefore, > > the result of such a comparison is doomed to `false`. For example, > > instances of > > `java.lang.Integer` and `java.lang.Long` will never be equal even though > > they represent > > the same numerical value. I.e. this code > > > > Long l = 42L; > > Integer i = 42; > > if (l.equals(i)) { // always false > > ... > > } else { > > ... > > } > > > > is very likely a mistake because the else-branch will never be taken. The > > same applies to attempts > > to compare among themselves instances of `java.lang.Short`, > > `java.lang.String`, `java.util.Date`, > > `java.util.List`, `java.util.Set`, `java.util.Map` etc. The reason why > such > > erroneous comparisons > > remain unnoticed is the fact that the `equals` method accepts any value. > > One way to make mistakes > > with such comparisons more visible it to use `compareTo` method. > > > > Long l = 42L; > > Integer i = 42; > > if (l.compareTo(i) == 0) { // compilation error > > ... > > } else { > > ... > > } > > > > However this approach has some drawbacks: > > * It's counter-intuitive and unnecessarily verbose. It's not clear at > > glance why `compareTo` > > is used instead of `equals`. > > * It can be inefficient because there are cases when an attempt to figure > > out whether given value > > is less or greater that the other value requires more effort than an > > attempt to figure out > > whether the values are just not equal. For example, when we compare > > strings and have noticed > > that the strings have different length we can stop here and do not scan > > them in order to > > figure out, which one is greater than the other. Another example is the > > class that represents > > rational number as a reduced fraction. To make sure that the rational > > numbers are equal, we can > > just compare their numerators and denominators. But in order to find out > > which one is greater > > than the other we can't avoid arithmetic operations (multiplications). > > * It's not applicable to the classes that do not implement > > `java.lang.Comparable`. For example, > > collections, complex numbers, points in 2D or 3D space etc. > > > > Therefore it's proposed to introduce interface `Equable` in order to > > explicitly mark comparable classes. > > > > public interface Equable { > > boolean equ(T o); > > } > > > > Let `Comparable` extend it and provide default method for `equ`. > > > > public interface Comparable<> extends Equable { > > int compareTo(T o); > > > > default boolean equ(T o) { > > return compareTo(o) == 0; > > } > > } > > > > Provide efficient implementation of `equ` method for some JDK classes, at > > least those that are in > > `java.lang` package. Also it would be desiable to let `java.util.List`, > > `java.util.Set`, > > `java.util.Map` and `java.util.Map.Entry` to extend `Equable` interface > as > > well (but without > > extending `Comparable`). > > > > === > > > > I am sending a set of patches. I split the whole change into several > > logical parts. > > > > * equable-0.patch contain `java.lang.Equable`, `java.lang.Comparable`, > > `java.util.Objects.equ` > > > > * equable-1.patch contain `j.l.Boolean`, `j.l.Byte`, `j.l.Character`, > > `j.l.Double`, `j.l.Enum`, > > `j.l.Float`, `j.l.Integer`, `j.l.Long`, `j.l.Short` and `j.l.String` > > > > * equable-2.patch contain `j.u.Optional`, `j.u.OptionalDouble`, > > `j.u.OptionalInt`, > > `j.u.OptionalLong` > > > > * equable-3.patch contain `j.u.List`, `j.u.AbstractList`, `j.u.Set`, > > `j.u.AbstractSet`, `j.u.Map`, > > `j.u.AbstractMap` > > > > * equable-4.patch is a tricky one. It contains > > `com.sun.tools.jdi.EventSetImpl`. Previously this class > > extended `java.lang.ArrayList` and implemented `java.lang.Set` through > > `EventSet`. Therefore it appeared > > that it implemented both `java.util.List` and `java.util.Set` at the > same > > time. This is not right, > > because these interfaces have different contracts regarding equality. So > > I had to change it by extending > > `java.lang.AbstractSet`. > > > > Thank you, > > Dmytro > From fw at deneb.enyo.de Sat Apr 18 15:18:08 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 18 Apr 2020 17:18:08 +0200 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <7DAE8438-E208-4D94-97CD-93989EBEBE96@oracle.com> (John Rose's message of "Fri, 17 Apr 2020 14:52:46 -0700") References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> <87imhyax50.fsf@mid.deneb.enyo.de> <7DAE8438-E208-4D94-97CD-93989EBEBE96@oracle.com> Message-ID: <87d0847rgv.fsf@mid.deneb.enyo.de> * John Rose: > On Apr 17, 2020, at 9:35 AM, Florian Weimer wrote: >> >> * forax: >> >>> - until lambdas are retrofitted as inline types, i'm worry people >>> will use forceTryable with a ReentrantLock, try(var __ = >>> forceTryable(lock::unlock)) { ... } because most of the time, the >>> lambda will not be allocated but not always, sometimes it may still >>> be found escaping by the JITs. >> >> The JDK itself routinely does something very similar when registering >> cleaners. Is this lack of robustness in the presence of OOMEs a >> quality-of-implementation issue? Or less or more important? > > Turning this around in my mind, it seems like the root cause of > this non-robustness is a minor mis-feature of TWR. TWR says > to itself, ?I only have one job, and that is to call close?. That?s > fine, but that means that means that the object to be closed > must be created *and* opened before TWR takes control. > If the closing operation is mediated by an adapter, the adapter > will almost certainly be created *after* the original (unadapted) > object is created and opened. This is the point where an OOME > could bollix things up. To defend against such failures, TWR > would have to define a distinct point where the object is opened. > This point would be a precise state transition immediately before > entry into the block covered by catches and the `finally` block. I think these issues were discussed and are the reason why try-with-resources requires the introduction of a new variable, to make clearer that there is no ownership in the evaluation of the expression. I guess this probably was onl partially successful, due to the wrapping of streams and code like this: try (PrintStream ps = new PrintStream(new FileOutputStream (filename))) { doCertReq(alias, sigAlgName, ps); } It should have been: try (InputStream in = new FileOutputStream (filename); PrintStream ps = new PrintStream(in)) { doCertReq(alias, sigAlgName, ps); } > How might this mis-feature be corrected? Maybe it can?t. > Maybe there?s no way to deal with lost closes other than > educating programmers to avoid allocations or other > computations in the head of the TWR, at least allocations > after the logical open. (Those are the ones which might cause > an OOME or SOE that would lead to a lost close.) In the streams example above, the consequences of the wrong style may not be so bad because (at least in OpenJDK 8), garbage collection eventually cleans up the temporary resource leak. With the unlock example from the proposed function, the effect would be permanent, though. However, with the move from finalize() to Cleaners, the OpenJDK itself has given up on avoiding permanent resource leaks due to OOME at inopportune moments, I think. > But (and this is why I?m going on at length here) maybe there > is a way to give the user more help avoiding lost closes. > It seems that if TWR were willing to define open operations > separately, some progress could be made. Suppose the object > at the head of a TWR implemented an optional new interface > `java.lang.AutoOpenable`. In such cases, the TWR would > call a nullary `open` operation on that object, immediately > before entering the block covered by the `finally` clause > that calls `close`. Would this help? Yes, it would, because > the designer of the abstraction at the head of the TWR could > take care to do all argument validation and storage allocation > *without* doing the actual logical open or allocate the resource > that needs to be closed (or seize the lock). The evaluation > of the head expression(s) of the TWR would (as before) not > be under any finally clause or catches. The new feature here > would be that the designer of the abstraction (at the head > of the TWR) would know that the expression creating > the abstraction would do all necessary storage allocation, > adapter creation, argument validation, and (if necessary) > stack banging, *before* the final critical opening step. > It would not be up to the end user; it would be more under > the control of the abstraction, to sequence the preparatory > steps *before* the logical open. Is this proposal similar to context managers for Python's with statement? I don't think this helps with the chaining problem or the cleaners issue. Maybe those issues are unrelated, after all. From brian.goetz at oracle.com Sat Apr 18 15:31:48 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 18 Apr 2020 11:31:48 -0400 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <87d0847rgv.fsf@mid.deneb.enyo.de> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> <87imhyax50.fsf@mid.deneb.enyo.de> <7DAE8438-E208-4D94-97CD-93989EBEBE96@oracle.com> <87d0847rgv.fsf@mid.deneb.enyo.de> Message-ID: <20027AAB-3781-4BA4-B432-A4DC7327A4FE@oracle.com> > I think these issues were discussed and are the reason why > try-with-resources requires the introduction of a new variable, to > make clearer that there is no ownership in the evaluation of the > expression. > > I guess this probably was onl partially successful, due to the > wrapping of streams and code like this: These discussions illustrate the tensions inherent in proposing ?pain reduction? features like TWR. On the one hand, TWR addresses a real pain point; on the other, it was knowingly limited. The arguments pro and con write themselves: ?this is addressing a real problem?, ?don?t let the perfect be the enemy of the good?, etc; while these might in fact be compelling arguments in any given situation, the fact remains that, should these features succeed in their essential purpose, their limitations will become immediately constraining. (Similar complaints could be made of the for-each loop, in that it offers no way to access the index (for array loops) or iterator (for Iterable loops.) And the limitations were known at the time the feature was being proposed. So here we are, now trying to go back and fix something that was considered and rejected at the time, because we have since discovered that we were wrong when we said ?we?d be happy if we could just solve X.? And fixing it is harder now, because we have to work within the constraints of a feature that was deliberately designed to be limited (whereas, when the feature was being designed, more things were possible.) Without commenting on the merits of the current round of proposals, let me just remind everyone that nailing epicyclical bags on the side of an incomplete feature will not make it complete?. From eric.vergnaud at wanadoo.fr Sat Apr 18 15:52:37 2020 From: eric.vergnaud at wanadoo.fr (Eric Vergnaud) Date: Sat, 18 Apr 2020 23:52:37 +0800 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <20027AAB-3781-4BA4-B432-A4DC7327A4FE@oracle.com> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> <87imhyax50.fsf@mid.deneb.enyo.de> <7DAE8438-E208-4D94-97CD-93989EBEBE96@oracle.com> <87d0847rgv.fsf@mid.deneb.enyo.de> <20027AAB-3781-4BA4-B432-A4DC7327A4FE@oracle.com> Message-ID: But without changing the existing behavior, it could be enhanced? In en experimental language called Prompto, there is a concept of Resource which must implement open() and close(). The runtime calls open() and close() as part of the TWR equivalent construct. This enables elegant explicit constructs: try(res = new MyResource("some locator")) { l1 = readLine(res); l2 = readLine(res); ... } and implicit ones: lines = readAll(new MyResource("some locator")) (readLine and readAll are Prompto builtins which only accept Resouce as parameters) you can only instantiate a resource within an explicit TWR header or and implicit TWR construct. (so: new MyResource() outside a TWR header or implicit TWR throws a syntax error) this provides much better guarantees than AutoCloseable How about adding something like: public interface AutoOpenable { T open(); } If TWR accepted both AutoOpenable and AutoCloseable, then there would be no break the current behavior, but it would become possible to implement the now desired behavior? (appreciate this would require some additional thinking for existing AutoCloseable classes) Eric > Le 18 avr. 2020 ? 23:31, Brian Goetz a ?crit : > >> I think these issues were discussed and are the reason why >> try-with-resources requires the introduction of a new variable, to >> make clearer that there is no ownership in the evaluation of the >> expression. >> >> I guess this probably was onl partially successful, due to the >> wrapping of streams and code like this: > > These discussions illustrate the tensions inherent in proposing ?pain reduction? features like TWR. On the one hand, TWR addresses a real pain point; on the other, it was knowingly limited. The arguments pro and con write themselves: ?this is addressing a real problem?, ?don?t let the perfect be the enemy of the good?, etc; while these might in fact be compelling arguments in any given situation, the fact remains that, should these features succeed in their essential purpose, their limitations will become immediately constraining. (Similar complaints could be made of the for-each loop, in that it offers no way to access the index (for array loops) or iterator (for Iterable loops.) And the limitations were known at the time the feature was being proposed. > > So here we are, now trying to go back and fix something that was considered and rejected at the time, because we have since discovered that we were wrong when we said ?we?d be happy if we could just solve X.? And fixing it is harder now, because we have to work within the constraints of a feature that was deliberately designed to be limited (whereas, when the feature was being designed, more things were possible.) > > Without commenting on the merits of the current round of proposals, let me just remind everyone that nailing epicyclical bags on the side of an incomplete feature will not make it complete?. From john.r.rose at oracle.com Sat Apr 18 18:27:52 2020 From: john.r.rose at oracle.com (John Rose) Date: Sat, 18 Apr 2020 11:27:52 -0700 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: References: Message-ID: <5F06C3C3-078F-4249-89CC-A6F3F537A329@oracle.com> On Apr 18, 2020, at 8:54 AM, Eric Vergnaud wrote: > > If TWR accepted both AutoOpenable and AutoCloseable, then there would be no break the current behavior, but it would become possible to implement the now desired behavior? > (appreciate this would require some additional thinking for existing AutoCloseable classes) Yes, that?s the opening motion of a long and searching conversation. Getting it wrong leads to Brian?s ?epicyclical bags?, turning a simply-flawed design into a complexly-flawed one. Getting it right is what Mark calls ?selective sedimentation?, that carefully matches good ideas from other languages to Java?s peculiarities without breaking backward compatibility. It takes much time and effort to seal these deals. FTR my precedents here are C++ RAII and Scheme?s dynamic-wind, which are only starting points. I haven?t looked closely at Python or Mesa or Modula3, which are probably relevant, since they are system-building languages with objects and some CF abstraction. Common Lisp and Smalltalk make the same compromises as today?s TWR in Java. (Sorry; never heard of Prompto, but I do know that with care you can build compositional open/close resources out of objects.) One hard part is coaxing Java?s object-oriented types to host yet another control flow abstraction, even a small one; AutoCloseable is a very carefully balanced OO design (look at its exception handling). It?s very hard to write CF abstractions so they compose in the OO style. (Yes, it?s easy to propose hooks as we are discussing but hard to make them hang together cleanly.) Just pointing out a precedent, or a proposed ?hook?, or a surprising mis-feature, as we are doing here, is about 2% of the effort. Brian is right to mention for-each, which is an even trickier problem. There are uncertain futures which may be relevant to TWR extensions: Loom?s continuations are currently under the covers, but might need a way to interact with re-entrant blocks (that?s part of dynamic-wind). We know that Java may need new abstraction tools to model parts of Java?s built-in language, to unify primitives with values?it is possible that these tools, once built, may provide the best approach to some TWR 2.0 which is decisively better, rather than a bolt-on. Or, maybe after running some corpus experiments and getting several architect types on EGs to think for a few weeks about it, something like what we are discussing here might turn out to be an economical move. The actual hook I?m suggesting here is not the interesting part of TWR?s technical debt to me?such hooks were surely discussed originally and rejected for reasons Brian suggested. The interesting part (to me) is the practical results of the deliberate simplification of TWR, which I have ventured to call (in hindsight *only*) a mis-feature, because (only in hindsight) we have realized that it doesn?t sufficiently encourage exactly-correct placement of the opening bracket that TWR will close, *in a minority of use cases*. Normally we don?t design for a minority of use cases, but we might reconsider if those use cases are not otherwise controllable by the user. What?s the minority use case here? Well, it?s the need (by some users) to clearly and exactly separate argument checking and setup (including low-level effects on JVM stack and heap!) from the actual commit-to-open (in some lock abstractions), leading to a hard-to-solve ?lost close? problem that was only a vague theory when TWR was designed. Now we are clumsily patching around some of these problems at the JVM level (JDK-8046936) and I?m beginning to think there?s a more comprehensive and reliable solution that involves help from the user, via an amendment to TWR. Anyway, this conversation may elucidate some technical debt or investment opportunity concerning TWR but executing on it (even with a small ?tweak?) will require time and effort, far outside the scope of an email thread. Most of us know this, but it bears repeating, because it?s easy to forget in the heat of the creative moment. A good exposition of how small changes can be made to Java is this by Joe Darcy for Project Coin: https://wiki.openjdk.java.net/display/Coin/Main HTH! ? John From john.r.rose at oracle.com Sat Apr 18 19:18:55 2020 From: john.r.rose at oracle.com (John Rose) Date: Sat, 18 Apr 2020 12:18:55 -0700 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <87d0847rgv.fsf@mid.deneb.enyo.de> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> <87imhyax50.fsf@mid.deneb.enyo.de> <7DAE8438-E208-4D94-97CD-93989EBEBE96@oracle.com> <87d0847rgv.fsf@mid.deneb.enyo.de> Message-ID: On Apr 18, 2020, at 8:18 AM, Florian Weimer wrote: > > * John Rose: > >> On Apr 17, 2020, at 9:35 AM, Florian Weimer wrote: >>> >>> * forax: >>> >>>> - until lambdas are retrofitted as inline types, i'm worry people >>>> will use forceTryable with a ReentrantLock, try(var __ = >>>> forceTryable(lock::unlock)) { ... } because most of the time, the >>>> lambda will not be allocated but not always, sometimes it may still >>>> be found escaping by the JITs. >>> >>> The JDK itself routinely does something very similar when registering >>> cleaners. Is this lack of robustness in the presence of OOMEs a >>> quality-of-implementation issue? Or less or more important? >> >> Turning this around in my mind, it seems like the root cause of >> this non-robustness is a minor mis-feature of TWR. TWR says >> to itself, ?I only have one job, and that is to call close?. That?s >> fine, but that means that means that the object to be closed >> must be created *and* opened before TWR takes control. >> If the closing operation is mediated by an adapter, the adapter >> will almost certainly be created *after* the original (unadapted) >> object is created and opened. This is the point where an OOME >> could bollix things up. To defend against such failures, TWR >> would have to define a distinct point where the object is opened. >> This point would be a precise state transition immediately before >> entry into the block covered by catches and the `finally` block. > > I think these issues were discussed and are the reason why > try-with-resources requires the introduction of a new variable, to > make clearer that there is no ownership in the evaluation of the > expression. The new variable requirement is reasonable, but as you know it only amounts to a ?nudge? in the right direction. The ?var? feature, and perhaps pattern matching, make it easier to work around that nudge. And there?s a deeper problem with this attempt to nail down the life cycle of the TWR resource to a named var. Unlike C++, Java vars don?t have a life cycle with a constructor bound to the occurrence of the var; they just have a pre-history in whatever expression produced the initial value of the var. So the ?stuff? that creates the var of a TWR has no particular relation to the structure of the TWR. My suggestion about an AutoOpenable attempts to nail down the part of the ?stuff? which really and necessarily pertains to the TWR?s job, which is the specific (sometimes atomic) operation that must be reversed by AutoCloseable::close. > I guess this probably was onl partially successful, due to the > wrapping of streams and code like this: > > try (PrintStream ps = new PrintStream(new FileOutputStream > (filename))) { > doCertReq(alias, sigAlgName, ps); > } > > It should have been: > > try (InputStream in = new FileOutputStream (filename); > PrintStream ps = new PrintStream(in)) { > doCertReq(alias, sigAlgName, ps); > } > >> How might this mis-feature be corrected? Maybe it can?t. >> Maybe there?s no way to deal with lost closes other than >> educating programmers to avoid allocations or other >> computations in the head of the TWR, at least allocations >> after the logical open. (Those are the ones which might cause >> an OOME or SOE that would lead to a lost close.) > > In the streams example above, the consequences of the wrong style may > not be so bad because (at least in OpenJDK 8), garbage collection > eventually cleans up the temporary resource leak. GC is great; it lets us simplify everything. But there?s a cost, which is that the much-ignored GC can sometimes create very rare side effects. (Same point for the virtually-unlimited JVM control stack.) The corner case I?m worrying at here is what happens when a very rare GC side effect causes a lost close. The solution I?m groping for is a clearer signal from the source code about which operation is the Point of No Return when the close must *not* be lost; currently the JVM & JIT only see a pile of ?stuff? including possible GC side effects, leading up to the actual try/finally underneath the TWR. To actually collect benefit from my groped-for solution, the libraries that define things like the streams in your example would have to be retrofitted to ?speak AutoOpenable?, and (even worse) users might well need to change their code to use new API points. (Thought experiment: What if most streams implemented AutoOpenable? Then they would distinguish ?pre-open? from ?really open? states, where today their constructors return them in fully open states, ready and raring to go. How would this new ?pre-open? state work? For compatibility it would have to automagically transition to ?fully open? the first time any side-effect was applied to the stream. Surprises in store for the user! But when just chaining streams together, they would conspire to stay in the pre-open state, in case a TWR was driving the bus. Then the TWR would say, ?we are open for business? and all of them would then allocate resources, with all due attention to fractional intermediate states needing cleanup. In non-TWR use, the first ?hard touch? on the outermost stream would cause this resource allocation to happen. Is it all worth it? I don?t know; a thought experiment cannot resolve this question IMO.) > With the unlock example from the proposed function, the effect would > be permanent, though. > > However, with the move from finalize() to Cleaners, the OpenJDK itself > has given up on avoiding permanent resource leaks due to OOME at > inopportune moments, I think. Yes, cleaners affect the balance here. In the future, if we invest more heavily in thread-confined data (see Panama), then missed closes might also be swept up when a thread dies. OTOH, thread-confined abstractions rely heavily on TWR, so there?s possibly more motivation to enhance TWR reliability. > But (and this is why I?m going on at length here) maybe there >> is a way to give the user more help avoiding lost closes. >> It seems that if TWR were willing to define open operations >> separately, some progress could be made. Suppose the object >> at the head of a TWR implemented an optional new interface >> `java.lang.AutoOpenable`. In such cases, the TWR would >> call a nullary `open` operation on that object, immediately >> before entering the block covered by the `finally` clause >> that calls `close`. Would this help? Yes, it would, because >> the designer of the abstraction at the head of the TWR could >> take care to do all argument validation and storage allocation >> *without* doing the actual logical open or allocate the resource >> that needs to be closed (or seize the lock). The evaluation >> of the head expression(s) of the TWR would (as before) not >> be under any finally clause or catches. The new feature here >> would be that the designer of the abstraction (at the head >> of the TWR) would know that the expression creating >> the abstraction would do all necessary storage allocation, >> adapter creation, argument validation, and (if necessary) >> stack banging, *before* the final critical opening step. >> It would not be up to the end user; it would be more under >> the control of the abstraction, to sequence the preparatory >> steps *before* the logical open. > > Is this proposal similar to context managers for Python's with > statement? Yes, because it distinguishes __init__ from __enter__, rather than putting their operations into the same ?pile of stuff?. The state diagram for a TWR resource today is: created-and-opened =close=> closed To the extent that we really, really need to distinguish a separate ?open? event that exactly matches the ?close? event, we would benefit from a TWR which supported this life cycle: created =open=> opened =close=> closed The use cases that support such an additional distinction are involve worryingly low-level side effects like OOME and SOE which break through otherwise-intact abstractions, which (apart from OOME and SOE) would seem only to require the two-state model instead of the three-state model. > I don't think this helps with the chaining problem or the cleaners > issue. Maybe those issues are unrelated, after all. I think it *would* help with some chaining problems, as I sketched above. It?s not surprising: If you add a new O-O entry point that reifies a new state in the state diagram, then the various objects can get busy and properly model that new state, by means of appropriate overrides to the new O-O entry point, and refactorings to their interior state. If you compose object A on top of object B, as var a = new A(new B()), then the composite *can* (if both classes agree) implement two states instead of one. I assume Eric?s language (Prompto) does something like this, and so can Python. With exquisite care, the transition to the second state (opened) can avoid even low-level side effects like OOME and SOE, because all such effects can be covered by previous actions performed for the first state (created). This can be ensured by cooperating objects A and B even with encapsulation of B relative to A and vice versa, and can scale to any number of objects A, B, C? Here?s one more important point, and I?m done for now: Even if the end-user carelessly loads any number of side-effect-ful items into the TWR head expression, none of those side effects interacts with the TWR proper (this is an intentional feature of TWR today). So TWR is robust about mis-placement of ?piles of stuff? in the TWR head expression, with the exception of the chaining problem, when the user should have called out several ?close points? to the TWR. But, if there?s a new state transition (AutoOpenable::open) then the semantics of the ?open? operation, like those of the ?close? operation, are not under the control of the end-user, but rather of the library writer, who can more carefully take responsibility to account for an exact state transition (open) that matches exactly to the final state transition (close). The library abstraction controls atomicity and bracket matching, instead of the vagaries of user code. Rather than requiring named variables to ?hint? at the close points, the object itself, and its abstraction over possible sub-objects, controls both the opening and closing points. I think that may be enough to justify the cost of an upgrade to TWR, *if* the high cost would in fact be justified at all, which it might not. I?m done here, for now. Got other stuff I need to worry about. I hope an appropriate EG can take this up when a convenient time rolls around. ? John From info at j-kuhn.de Sat Apr 18 20:49:07 2020 From: info at j-kuhn.de (Johannes Kuhn) Date: Sat, 18 Apr 2020 22:49:07 +0200 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> Message-ID: <64bfce07-73bd-fa34-37b6-780645cef85f@j-kuhn.de> On 17-Apr-20 1:00, forax at univ-mlv.fr wrote: > A couple of points, > - it can be fixed by making TWR using a functional type instead of a nominal one > by supporting a syntax like try(expr) { }. But this is in the same realm of ideas as allowing the enhanced for loop to use a an Iterator::iterator, so it has been rule out has a step too far by the lambda EG. > - until lambdas are retrofitted as inline types, i'm worry people will use forceTryable with a ReentrantLock, > try(var __ = forceTryable(lock::unlock)) { ... } > because most of the time, the lambda will not be allocated but not always, sometimes it may still be found escaping by the JITs. > - my preferred fix is, as you already, know to remove the concept of checked exceptions from the language instead of spending time running in circles around them. > > R?mi While this can not be implemented outside of ReenetrantLock, ReenetrantLock could do it: private final AutoCloseable unlocker = this::unlock; public AutoCloseable autoLock() { ??? this.lock(); ?? return unlocker; } The type should be narrowed down so it doesn't declare to throw Exception, but you get the idea. Also, I don't quite understand how inline types would get rid of the allocation. The type declared in the class file (with the TWR) is the interface, so the inline type might need to be widened. From my understanding, if and when the widening happens is subject to the JIT. The benefit (again, to my understanding) is that widened inline values are indistinguishable from other widened inline values with the same contents, so the JIT can defer the widening as much to it's liking. Or not. And the widening includes an allocation & copy of the data from stack to heap. Or did I miss something obvious? (And how does the interpreter deal with all that?) Johannes > ----- Mail original ----- >> De: "John Rose" >> ?: "Remi Forax" >> Cc: "Johannes Kuhn" , "discuss" >> Envoy?: Jeudi 16 Avril 2020 23:31:49 >> Objet: Re: Some Classes with a public void close() don't implement AutoCloseable >> Nice. I tried to find a way to make this a little more generic >> and came up with an API which handles arbitrary exceptions >> and is packaged as a method instead of a special idiom. >> There?s a natural-enough method called `forceTryable` that >> could be created. >> >> http://cr.openjdk.java.net/~jrose/jdk/ForceTryableDemo.java >> https://bugs.openjdk.java.net/browse/JDK-8243015 >> >> (There are further generalizations of this, if you look at it as >> a ?named cast method? for a functional interface type related >> to AutoCloseable. See my comment on JDK-8243015.) >> >> ? John >> >> On Apr 15, 2020, at 8:53 AM, Remi Forax wrote: >>> interface FakeAutoCloseable extends AutoCloseable { public void close(); } // >>> suppress exception >>> >>> XMLReader xmlReader = ... >>> try(FakeAutoCloseable __ = xmlReader::close) { >>> ? From forax at univ-mlv.fr Sun Apr 19 16:55:49 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Sun, 19 Apr 2020 18:55:49 +0200 (CEST) Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <64bfce07-73bd-fa34-37b6-780645cef85f@j-kuhn.de> References: <7a5b5a60-7b42-6654-ba90-671419b1f19c@j-kuhn.de> <2070181537.1220515.1586965994923.JavaMail.zimbra@u-pem.fr> <70B22129-3460-495B-9D2F-ECE067F538F9@oracle.com> <806212363.503727.1587078033980.JavaMail.zimbra@u-pem.fr> <64bfce07-73bd-fa34-37b6-780645cef85f@j-kuhn.de> Message-ID: <36345316.1668923.1587315349257.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Johannes Kuhn" > ?: "Remi Forax" > Cc: "discuss" > Envoy?: Samedi 18 Avril 2020 22:49:07 > Objet: Re: Some Classes with a public void close() don't implement AutoCloseable > On 17-Apr-20 1:00, forax at univ-mlv.fr wrote: >> A couple of points, >> - it can be fixed by making TWR using a functional type instead of a nominal one >> by supporting a syntax like try(expr) { }. But this is in the same realm of >> ideas as allowing the enhanced for loop to use a an Iterator::iterator, so it >> has been rule out has a step too far by the lambda EG. >> - until lambdas are retrofitted as inline types, i'm worry people will use >> forceTryable with a ReentrantLock, >> try(var __ = forceTryable(lock::unlock)) { ... } >> because most of the time, the lambda will not be allocated but not always, >> sometimes it may still be found escaping by the JITs. >> - my preferred fix is, as you already, know to remove the concept of checked >> exceptions from the language instead of spending time running in circles around >> them. >> >> R?mi > Hi Johannes, > While this can not be implemented outside of ReenetrantLock, > ReenetrantLock could do it: > > private final AutoCloseable unlocker = this::unlock; > public AutoCloseable autoLock() { > ??? this.lock(); > ?? return unlocker; > } > > The type should be narrowed down so it doesn't declare to throw > Exception, but you get the idea. yes, this solution has the advantage that if a lock is constructed then this::unlock has been correctly allocated, but it means that even if the application don't use autoLock, you pay an allocation per lock. You can fix that issue with the code below public AutoCloseable autoLock() { AutoCloseable unlocker = this::unlock; // allocation before taking the lock ! this.lock(); return unlocker; } > > Also, I don't quite understand how inline types would get rid of the > allocation. > The type declared in the class file (with the TWR) is the interface, so > the inline type might need to be widened. > From my understanding, if and when the widening happens is subject to > the JIT. > The benefit (again, to my understanding) is that widened inline values > are indistinguishable from other widened inline values with the same > contents, > so the JIT can defer the widening as much to it's liking. Or not. And > the widening includes an allocation & copy of the data from stack to heap. > Or did I miss something obvious? (And how does the interpreter deal with > all that?) yes, it depends if the code is JITed or not. If the code is JITed, given that the lambda creation and the last usage (the call to close()) are in the same method, if nobody is stupidly using the lambda inside the TWR body (that why i have used __ as identifier), any JITs will use the lambda real type and not the interface, so no allocation ! > > Johannes R?mi > >> ----- Mail original ----- >>> De: "John Rose" >>> ?: "Remi Forax" >>> Cc: "Johannes Kuhn" , "discuss" >>> Envoy?: Jeudi 16 Avril 2020 23:31:49 >>> Objet: Re: Some Classes with a public void close() don't implement AutoCloseable >>> Nice. I tried to find a way to make this a little more generic >>> and came up with an API which handles arbitrary exceptions >>> and is packaged as a method instead of a special idiom. >>> There?s a natural-enough method called `forceTryable` that >>> could be created. >>> >>> http://cr.openjdk.java.net/~jrose/jdk/ForceTryableDemo.java >>> https://bugs.openjdk.java.net/browse/JDK-8243015 >>> >>> (There are further generalizations of this, if you look at it as >>> a ?named cast method? for a functional interface type related >>> to AutoCloseable. See my comment on JDK-8243015.) >>> >>> ? John >>> >>> On Apr 15, 2020, at 8:53 AM, Remi Forax wrote: >>>> interface FakeAutoCloseable extends AutoCloseable { public void close(); } // >>>> suppress exception >>>> >>>> XMLReader xmlReader = ... >>>> try(FakeAutoCloseable __ = xmlReader::close) { > >>> ? From dmytro.sheyko.jdk at gmail.com Sun Apr 19 17:17:22 2020 From: dmytro.sheyko.jdk at gmail.com (dmytro sheyko) Date: Sun, 19 Apr 2020 20:17:22 +0300 Subject: Type-parameterized complement to Object.equals(Object) In-Reply-To: <17BD2167-2B94-456E-A001-7A06C5284AB3@oracle.com> References: <17BD2167-2B94-456E-A001-7A06C5284AB3@oracle.com> Message-ID: Yes, I agree with you. And thank you for pointing this out. Sorry for being impudent, I thought that the more detailed the proposal would be, the more attention it can attract. Please consider my email as an invitation to discussion in order to build a consensus. As R?mi Forax pointed out, IDEs provide an inspection/analysis to detect incorrect usage of equals. However their analysis is based on assumption that instances of different classes can't be equal to each other and therefore they typically check whether one class is inheritor of the other. Thus NetBeans, Eclipse and IDEA complain if I try to compare `java.sql.Time` and `java.sql.Date`. NetBeans and Eclipse also complain if I try to compare collections e.g. `java.util.TreeMap` with `java.util.LinkedHashMap` or `java.util.ArrayList` with `java.util.LinkedList`. IDEA complain only if I try to compare collections parameterized with different types, e.g. `java.util.ArrayList` with `java.util.LinkedList`. This looks suspect indeed, but honestly they can be equal - they can be both empty. SpotBugs complains on an attempt to compare `java.util.AbstractMap.SimpleEntry` with `java.util.AbstractMap.SimpleImmutableEntry`. Eclipse and NetBeans do the same, but IDEA doesn't. Perhaps IDEA has list of exceptions in order to avoid false positives, while Eclipse and NetBeans do not. And SpotBugs does more sophisticated analysis, but sometimes it also makes mistakes. Nevertheless the assumption that instances of different classes can't be equal to each other is wrong. But this is the best thing they can do. They do not have information which different classes can be comparable to each other. The fact that various tools try to solve this problem and the fact that the result of their work is not always satisfactory means that there is a problem. The suggested `Equable` interface do not detect incorrect usage of equals. It attempts to solve the problem from another direction. Well, in some cases (but definitely not in all cases) method `equals` is overridden this way: class ExampleA { @Override public boolean equals(Object o) { if (o instanceof ExampleA) { Example that = (ExampleA) o; return <> } return false; } } The part of code <> can be extracted to separate method. Let's call it `equ` (it's often named as `strictEquals`, `contentEquals`, `isEqualTo` etc). class ExampleA { @Override public boolean equals(Object o) { if (o instanceof ExampleA) { Example that = (ExampleA) o; return equ(that); } return false; } public boolean equ(ExampleA that) { return <> } } The inheritors of ExampleA may override the behavior, but not the equality property. Thus if we have two values of classes ExampleB and ExampleC and these classes have common base class ExampleA, it makes sense to compare these values. And the fact that we extracted `equ` method plays important role here. If we use it, this becomes clear that the values are comparable. Please note that if we use plain old `equals`, this is not so clear because `equals` is overly permissive. Moreover IDE inspection may report a false positive here. Also if we have two values of classes ExampleB and ExampleC and the one of the classes has parent ExampleA and the other does not, it does not make sense to compare these values. It would be a compilation error if we try to use `equ` method. And finally if we have two values of classes ExampleB and ExampleC and none of the the classes have parent like ExampleA, we can't make a decision, the values may be comparable or may be not. And here the only thing we can use is `equals` and rely on IDE inspections. In this case the IDE inspection whether one class is inheritor of the other looks reasonable. Returning to ExampleA class, we can extract `Equable` interface in order to make the pattern more universal class ExampleA extends Equable { @Override public boolean equals(Object o) { if (o instanceof ExampleA) { Example that = (ExampleA) o; return equ(that); } return false; } @Override public boolean equ(ExampleA that) { return <> } } This will let us write an analog if `Object.equals` method public static > boolean equ(A a, B b) { return (a == b) || (a != null ? (b != null && a.equ(b)) : b == null); } So with `Equable` interface we can mark the class (or interface) that is the root of hierarchy of mutually comparable classes. Alternatively we can use annotation for this purpose, however in this case we can only rely on IDE inspections. Also `Equable.equ` has some small bonus. It lets implementors of `Comparable` classes to provide efficient alternative to `compareTo() == 0`, especially when `equals` and `compareTo` are inconsistent. Of course, it does not make sense to make all classes that override `equals` `Equable`. If `equals` is overridden as class ExampleZ { @Override public boolean equals(Object o) { if (o == null) return false; if (getClass() == o.getClass()) { Example that = (ExampleZ) o; return <> } return false; } } where all subclasses are comparable only with itself, maybe there are no benefits. Your comments are welcome. Thank you, Dmytro On Sat, Apr 18, 2020 at 6:20 PM Brian Goetz wrote: > Without commenting on the merits of the proposal, let me point out that > asking for a sponsor at this point is skipping many important steps. > > I would recommend you focus first on *building consensus* that (a) there > is a problem, (b) you have identified the correct problem, (c) it is a > problem that needs to be solved, (d) it is problem that needs to be solved > in the JDK, (e) that you have evaluated the possible solutions and analyzed > their strengths and weaknesses, and (f) that the correct solution has been > identified. (There are more steps, but you get the idea.) > > On Apr 17, 2020, at 7:44 AM, dmytro sheyko > wrote: > > I need a sponsor for following JDK change. > > > From forax at univ-mlv.fr Sun Apr 19 17:24:14 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 19 Apr 2020 19:24:14 +0200 (CEST) Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <5F06C3C3-078F-4249-89CC-A6F3F537A329@oracle.com> References: <5F06C3C3-078F-4249-89CC-A6F3F537A329@oracle.com> Message-ID: <681016925.1670961.1587317054455.JavaMail.zimbra@u-pem.fr> TWR really means try-with-IO-resources. Java IO objects (InputStream/Channel etc) uses a 2 transitions model instead of a 3 transitions model, something i've always found smart. Instead of new -> open -> close, it's new/open -> close. The object creation and the opening of the resource are concomitant. Obviously you have exception (URLConnection) and all the buffered streams [0] but the idea is that the resource object is first allocated and in the constructor the resource is allocated so you have no issue with OutOfMemoryError. TWR is tailored to follow that model, that why it's not a good construct for locks (new -> lock -> unlock) or any objects that have a 3 transition model. If a some point we want to introduce a construct for the 3 transition model, instead of trying to patch TWR, it may better to consider it as its own construct and see it can be retrofit it as TWR in a second step. regards, R?mi [0] java.nio.file.Files alswo introduce in Java 7 provides methods carefully designed to create and open buffered streams in one call. ----- Mail original ----- > De: "John Rose" > ?: "Eric Vergnaud" , "Florian Weimer" , "Brian Goetz" > > Cc: "discuss" > Envoy?: Samedi 18 Avril 2020 20:27:52 > Objet: Re: Some Classes with a public void close() don't implement AutoCloseable > On Apr 18, 2020, at 8:54 AM, Eric Vergnaud wrote: >> >> If TWR accepted both AutoOpenable and AutoCloseable, then there would be no >> break the current behavior, but it would become possible to implement the now >> desired behavior? >> (appreciate this would require some additional thinking for existing >> AutoCloseable classes) > > Yes, that?s the opening motion of a long and searching conversation. Getting it > wrong leads to Brian?s ?epicyclical bags?, turning a simply-flawed design into > a complexly-flawed one. Getting it right is what Mark calls ?selective > sedimentation?, that carefully matches good ideas from other languages to > Java?s peculiarities without breaking backward compatibility. It takes much > time and effort to seal these deals. > > FTR my precedents here are C++ RAII and Scheme?s dynamic-wind, which are only > starting points. I haven?t looked closely at Python or Mesa or Modula3, which > are probably relevant, since they are system-building languages with objects > and some CF abstraction. Common Lisp and Smalltalk make the same compromises > as today?s TWR in Java. (Sorry; never heard of Prompto, but I do know that with > care you can build compositional open/close resources out of objects.) > > One hard part is coaxing Java?s object-oriented types to host yet another > control flow abstraction, even a small one; AutoCloseable is a very carefully > balanced OO design (look at its exception handling). It?s very hard to write > CF abstractions so they compose in the OO style. (Yes, it?s easy to propose > hooks as we are discussing but hard to make them hang together cleanly.) Just > pointing out a precedent, or a proposed ?hook?, or a surprising mis-feature, as > we are doing here, is about 2% of the effort. > > Brian is right to mention for-each, which is an even trickier problem. There are > uncertain futures which may be relevant to TWR extensions: Loom?s > continuations are currently under the covers, but might need a way to interact > with re-entrant blocks (that?s part of dynamic-wind). We know that Java may > need new abstraction tools to model parts of Java?s built-in language, to unify > primitives with values?it is possible that these tools, once built, may provide > the best approach to some TWR 2.0 which is decisively better, rather than a > bolt-on. Or, maybe after running some corpus experiments and getting several > architect types on EGs to think for a few weeks about it, something like what > we are discussing here might turn out to be an economical move. > > The actual hook I?m suggesting here is not the interesting part of TWR?s > technical debt to me?such hooks were surely discussed originally and rejected > for reasons Brian suggested. The interesting part (to me) is the practical > results of the deliberate simplification of TWR, which I have ventured to call > (in hindsight *only*) a mis-feature, because (only in hindsight) we have > realized that it doesn?t sufficiently encourage exactly-correct placement of > the opening bracket that TWR will close, *in a minority of use cases*. > Normally we don?t design for a minority of use cases, but we might reconsider > if those use cases are not otherwise controllable by the user. > > What?s the minority use case here? Well, it?s the need (by some users) to > clearly and exactly separate argument checking and setup (including low-level > effects on JVM stack and heap!) from the actual commit-to-open (in some lock > abstractions), leading to a hard-to-solve ?lost close? problem that was only a > vague theory when TWR was designed. Now we are clumsily patching around some > of these problems at the JVM level (JDK-8046936) and I?m beginning to think > there?s a more comprehensive and reliable solution that involves help from the > user, via an amendment to TWR. > > Anyway, this conversation may elucidate some technical debt or investment > opportunity concerning TWR but executing on it (even with a small ?tweak?) will > require time and effort, far outside the scope of an email thread. Most of us > know this, but it bears repeating, because it?s easy to forget in the heat of > the creative moment. > > A good exposition of how small changes can be made to Java is this by Joe Darcy > for Project Coin: > https://wiki.openjdk.java.net/display/Coin/Main > > HTH! > > ? John From eric.vergnaud at wanadoo.fr Mon Apr 20 01:13:49 2020 From: eric.vergnaud at wanadoo.fr (Eric Vergnaud) Date: Mon, 20 Apr 2020 09:13:49 +0800 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <681016925.1670961.1587317054455.JavaMail.zimbra@u-pem.fr> References: <5F06C3C3-078F-4249-89CC-A6F3F537A329@oracle.com> <681016925.1670961.1587317054455.JavaMail.zimbra@u-pem.fr> Message-ID: I don?t disagree with the analysis, but isn?t the problem precisely that IO objects can be instantiated (and thus opened) outside a TWR? An option could be to disallow instantiation (of AutoOpenable) outside a TWR header, and disallow close() anywhere. It?s probably too much of a breaking change to apply this to existing IO classes, but creating a new package (which would internally leverage the existing IO doesn?t sound like a big task. > Le 20 avr. 2020 ? 01:24, Remi Forax a ?crit : > > TWR really means try-with-IO-resources. > > Java IO objects (InputStream/Channel etc) uses a 2 transitions model instead of a 3 transitions model, something i've always found smart. > Instead of new -> open -> close, it's new/open -> close. The object creation and the opening of the resource are concomitant. > Obviously you have exception (URLConnection) and all the buffered streams [0] but the idea is that the resource object is first allocated and in the constructor the resource is allocated so you have no issue with OutOfMemoryError. > > TWR is tailored to follow that model, that why it's not a good construct for locks (new -> lock -> unlock) or any objects that have a 3 transition model. > > If a some point we want to introduce a construct for the 3 transition model, instead of trying to patch TWR, it may better to consider it as its own construct and see it can be retrofit it as TWR in a second step. > > regards, > R?mi > > [0] java.nio.file.Files alswo introduce in Java 7 provides methods carefully designed to create and open buffered streams in one call. > > ----- Mail original ----- >> De: "John Rose" >> ?: "Eric Vergnaud" , "Florian Weimer" , "Brian Goetz" >> >> Cc: "discuss" >> Envoy?: Samedi 18 Avril 2020 20:27:52 >> Objet: Re: Some Classes with a public void close() don't implement AutoCloseable > >> On Apr 18, 2020, at 8:54 AM, Eric Vergnaud wrote: >>> >>> If TWR accepted both AutoOpenable and AutoCloseable, then there would be no >>> break the current behavior, but it would become possible to implement the now >>> desired behavior? >>> (appreciate this would require some additional thinking for existing >>> AutoCloseable classes) >> >> Yes, that?s the opening motion of a long and searching conversation. Getting it >> wrong leads to Brian?s ?epicyclical bags?, turning a simply-flawed design into >> a complexly-flawed one. Getting it right is what Mark calls ?selective >> sedimentation?, that carefully matches good ideas from other languages to >> Java?s peculiarities without breaking backward compatibility. It takes much >> time and effort to seal these deals. >> >> FTR my precedents here are C++ RAII and Scheme?s dynamic-wind, which are only >> starting points. I haven?t looked closely at Python or Mesa or Modula3, which >> are probably relevant, since they are system-building languages with objects >> and some CF abstraction. Common Lisp and Smalltalk make the same compromises >> as today?s TWR in Java. (Sorry; never heard of Prompto, but I do know that with >> care you can build compositional open/close resources out of objects.) >> >> One hard part is coaxing Java?s object-oriented types to host yet another >> control flow abstraction, even a small one; AutoCloseable is a very carefully >> balanced OO design (look at its exception handling). It?s very hard to write >> CF abstractions so they compose in the OO style. (Yes, it?s easy to propose >> hooks as we are discussing but hard to make them hang together cleanly.) Just >> pointing out a precedent, or a proposed ?hook?, or a surprising mis-feature, as >> we are doing here, is about 2% of the effort. >> >> Brian is right to mention for-each, which is an even trickier problem. There are >> uncertain futures which may be relevant to TWR extensions: Loom?s >> continuations are currently under the covers, but might need a way to interact >> with re-entrant blocks (that?s part of dynamic-wind). We know that Java may >> need new abstraction tools to model parts of Java?s built-in language, to unify >> primitives with values?it is possible that these tools, once built, may provide >> the best approach to some TWR 2.0 which is decisively better, rather than a >> bolt-on. Or, maybe after running some corpus experiments and getting several >> architect types on EGs to think for a few weeks about it, something like what >> we are discussing here might turn out to be an economical move. >> >> The actual hook I?m suggesting here is not the interesting part of TWR?s >> technical debt to me?such hooks were surely discussed originally and rejected >> for reasons Brian suggested. The interesting part (to me) is the practical >> results of the deliberate simplification of TWR, which I have ventured to call >> (in hindsight *only*) a mis-feature, because (only in hindsight) we have >> realized that it doesn?t sufficiently encourage exactly-correct placement of >> the opening bracket that TWR will close, *in a minority of use cases*. >> Normally we don?t design for a minority of use cases, but we might reconsider >> if those use cases are not otherwise controllable by the user. >> >> What?s the minority use case here? Well, it?s the need (by some users) to >> clearly and exactly separate argument checking and setup (including low-level >> effects on JVM stack and heap!) from the actual commit-to-open (in some lock >> abstractions), leading to a hard-to-solve ?lost close? problem that was only a >> vague theory when TWR was designed. Now we are clumsily patching around some >> of these problems at the JVM level (JDK-8046936) and I?m beginning to think >> there?s a more comprehensive and reliable solution that involves help from the >> user, via an amendment to TWR. >> >> Anyway, this conversation may elucidate some technical debt or investment >> opportunity concerning TWR but executing on it (even with a small ?tweak?) will >> require time and effort, far outside the scope of an email thread. Most of us >> know this, but it bears repeating, because it?s easy to forget in the heat of >> the creative moment. >> >> A good exposition of how small changes can be made to Java is this by Joe Darcy >> for Project Coin: >> https://wiki.openjdk.java.net/display/Coin/Main >> >> HTH! >> >> ? John From brian.goetz at oracle.com Mon Apr 20 15:36:58 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 20 Apr 2020 11:36:58 -0400 Subject: Some Classes with a public void close() don't implement AutoCloseable In-Reply-To: <681016925.1670961.1587317054455.JavaMail.zimbra@u-pem.fr> References: <5F06C3C3-078F-4249-89CC-A6F3F537A329@oracle.com> <681016925.1670961.1587317054455.JavaMail.zimbra@u-pem.fr> Message-ID: Slight correction; it's 2 vs 3 states, so 1 vs 2 transitions :) Counting state transitions is a nice way to characterize the resource-management behavior of APIs. We can put a little more structure on this characterization: ?- monotonic 3-state, such as `Thread` where it goes from created to "opened" to "closed", and never goes back (there are IO-oriented versions of this too); ?- "loopy" 3-state, such as locks, where you create a lock, and the normal case is that the lock then cycles between locked and unlocked. There is a design tension between the monotonic 3-state entities and the 2-state entities; OO purists will turn their noses up at the 2-state model used by `FileInputStream`, because side effects in constructors is Bad.? On the other hand, the 2-state model seems a quite practical compromise, in that the number of error conditions the user has to deal with is reduced (and error handling is always the hardest part of such APIs.) Some of the 3-state entities, locks especially, are really only 2 1/2 states, though; in many situations, the lock either is long-lived (such as a static lock) or has its own lifecycle that is managed by GC.? Which means that for the local purposes of TWR with a known-reachable lock, there are still only 2 states. But TWR wants to manage the _entire_ lifecycle of the AC object. So I think your first statement -- that TWR was designed mostly with the needs of IO resources in mind -- is accurate, if sad. On 4/19/2020 1:24 PM, Remi Forax wrote: > TWR really means try-with-IO-resources. > > Java IO objects (InputStream/Channel etc) uses a 2 transitions model instead of a 3 transitions model, something i've always found smart. > Instead of new -> open -> close, it's new/open -> close. The object creation and the opening of the resource are concomitant. > Obviously you have exception (URLConnection) and all the buffered streams [0] but the idea is that the resource object is first allocated and in the constructor the resource is allocated so you have no issue with OutOfMemoryError. > > TWR is tailored to follow that model, that why it's not a good construct for locks (new -> lock -> unlock) or any objects that have a 3 transition model. > > If a some point we want to introduce a construct for the 3 transition model, instead of trying to patch TWR, it may better to consider it as its own construct and see it can be retrofit it as TWR in a second step. > > regards, > R?mi > > [0] java.nio.file.Files alswo introduce in Java 7 provides methods carefully designed to create and open buffered streams in one call. > > ----- Mail original ----- >> De: "John Rose" >> ?: "Eric Vergnaud" , "Florian Weimer" , "Brian Goetz" >> >> Cc: "discuss" >> Envoy?: Samedi 18 Avril 2020 20:27:52 >> Objet: Re: Some Classes with a public void close() don't implement AutoCloseable >> On Apr 18, 2020, at 8:54 AM, Eric Vergnaud wrote: >>> If TWR accepted both AutoOpenable and AutoCloseable, then there would be no >>> break the current behavior, but it would become possible to implement the now >>> desired behavior? >>> (appreciate this would require some additional thinking for existing >>> AutoCloseable classes) >> Yes, that?s the opening motion of a long and searching conversation. Getting it >> wrong leads to Brian?s ?epicyclical bags?, turning a simply-flawed design into >> a complexly-flawed one. Getting it right is what Mark calls ?selective >> sedimentation?, that carefully matches good ideas from other languages to >> Java?s peculiarities without breaking backward compatibility. It takes much >> time and effort to seal these deals. >> >> FTR my precedents here are C++ RAII and Scheme?s dynamic-wind, which are only >> starting points. I haven?t looked closely at Python or Mesa or Modula3, which >> are probably relevant, since they are system-building languages with objects >> and some CF abstraction. Common Lisp and Smalltalk make the same compromises >> as today?s TWR in Java. (Sorry; never heard of Prompto, but I do know that with >> care you can build compositional open/close resources out of objects.) >> >> One hard part is coaxing Java?s object-oriented types to host yet another >> control flow abstraction, even a small one; AutoCloseable is a very carefully >> balanced OO design (look at its exception handling). It?s very hard to write >> CF abstractions so they compose in the OO style. (Yes, it?s easy to propose >> hooks as we are discussing but hard to make them hang together cleanly.) Just >> pointing out a precedent, or a proposed ?hook?, or a surprising mis-feature, as >> we are doing here, is about 2% of the effort. >> >> Brian is right to mention for-each, which is an even trickier problem. There are >> uncertain futures which may be relevant to TWR extensions: Loom?s >> continuations are currently under the covers, but might need a way to interact >> with re-entrant blocks (that?s part of dynamic-wind). We know that Java may >> need new abstraction tools to model parts of Java?s built-in language, to unify >> primitives with values?it is possible that these tools, once built, may provide >> the best approach to some TWR 2.0 which is decisively better, rather than a >> bolt-on. Or, maybe after running some corpus experiments and getting several >> architect types on EGs to think for a few weeks about it, something like what >> we are discussing here might turn out to be an economical move. >> >> The actual hook I?m suggesting here is not the interesting part of TWR?s >> technical debt to me?such hooks were surely discussed originally and rejected >> for reasons Brian suggested. The interesting part (to me) is the practical >> results of the deliberate simplification of TWR, which I have ventured to call >> (in hindsight *only*) a mis-feature, because (only in hindsight) we have >> realized that it doesn?t sufficiently encourage exactly-correct placement of >> the opening bracket that TWR will close, *in a minority of use cases*. >> Normally we don?t design for a minority of use cases, but we might reconsider >> if those use cases are not otherwise controllable by the user. >> >> What?s the minority use case here? Well, it?s the need (by some users) to >> clearly and exactly separate argument checking and setup (including low-level >> effects on JVM stack and heap!) from the actual commit-to-open (in some lock >> abstractions), leading to a hard-to-solve ?lost close? problem that was only a >> vague theory when TWR was designed. Now we are clumsily patching around some >> of these problems at the JVM level (JDK-8046936) and I?m beginning to think >> there?s a more comprehensive and reliable solution that involves help from the >> user, via an amendment to TWR. >> >> Anyway, this conversation may elucidate some technical debt or investment >> opportunity concerning TWR but executing on it (even with a small ?tweak?) will >> require time and effort, far outside the scope of an email thread. Most of us >> know this, but it bears repeating, because it?s easy to forget in the heat of >> the creative moment. >> >> A good exposition of how small changes can be made to Java is this by Joe Darcy >> for Project Coin: >> https://wiki.openjdk.java.net/display/Coin/Main >> >> HTH! >> >> ? John From forax at univ-mlv.fr Thu Apr 23 18:05:09 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 23 Apr 2020 20:05:09 +0200 (CEST) Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> Message-ID: <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "John Rose" > ?: "Brian Goetz" > Cc: "Remi Forax" , "discuss" > Envoy?: Samedi 18 Avril 2020 09:53:58 > Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? > Representing a JSON tree as a tree of Valhalla values (yes, they > can come in trees!) is very much worth investigating IMO. The nearest > approximation before Valhalla would be to design a JSON node model > which consists only of (or mainly of) value-based classes. Or, perhaps > Valhalla-style cursors (iterators which say x=x.next() instead of x.next()) > can express a walk over a mutable tree, whose structure is hidden (and > thus can be materialized lazily or streamily). The nearest approximation > to that would be an iterator-style API, where one object updates itself > in place. > > One other thought: Sometimes that parsing and unparsing logic > for a syntax like JSON or XML can be expressed as operating on a > linear event-stream, instead of a nested tree model. You can build > the latter on top of the former, and event streams have a simpler > object structure than tree-like models. It might be safer to start > with a parser/unparser API that runs on event streams, and then > build node models (with cursors, pattern matching, etc.) on top > later when the tools for those appear. ?Not that I?m longing > to program an event-based API, but it?s better than nothing, and > not lost effort either. It's what the usual stream APIs do, at least what jackson and the json api from Java EE does. The stream is just a tokenization of the text format [1, 3, { "foo": null } ] -> START_ARRAY NUMBER NUMBER START_OBJECT FIELD_NAME NULL END_OBJECT END_ARRAY >From that you can construct an API like ASM when you call a method visit for each value token (NUMBER, STRING, NULL, TRUE, etc) and a method visit that ask for a sub visitor for START_ARRAY and START_OBJECT. The next step, providing a tree/node API is harder and i think stupid mostly because JSON NUMBER is not typed so you have no idea if you should represent a number (as an int, a long, a float, a double, a BigInteger or a DigDecimal*, ??). You can introduce your own JsonNumber has an inline type but you will incompatible with any existing Java values. It's better to have a type representation that provide the type the user want for a number. That why most APIs do not provide a generic tree API but a data binding API that takes a tree of types (a bean/record/interface tree) and a stream of JSON tokens and construct the corresponding Java representation (creating instances Bean/Record/Proxy). This tree of type can also be seen as a tokenization of the type informations (START_OBJECT_TYPE, START_ARRAY_TYPE, NUMBER_TYPE, etc). Annotations on the tree of types are used to derive an adapter (or several with the concept of jackson views) to the visitor that will filter/map the field name and the value before creating the instances. So IMO, a tree of instance of Valhalla inline types, a tree of nodes, is not useful but inline types can still be useful to simplify the visitor interface (instead of visitNumberInt/visitNumberLong/visitNumberDouble, etc, have a sole visitNumber(JsonNumber) with the inline type only existing on the stack to transfer the information from the reader to the deserializer (and vice-versa). regards, R?mi * ECMA 404 doesn't define a size for the numbers ! From cay.horstmann at gmail.com Thu Apr 23 18:53:02 2020 From: cay.horstmann at gmail.com (Cay Horstmann) Date: Thu, 23 Apr 2020 20:53:02 +0200 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> Message-ID: <8faeedab-14b5-ec6d-133b-9721f385e74e@gmail.com> A JavaScript number is a double-precision floating-point number. JSON cannot currently serialize BigInt. https://www.ecma-international.org/ecma-262/#sec-json.stringify https://www.ecma-international.org/ecma-262/#sec-terms-and-definitions-number-value Cheers, Cay Il 23/04/20 11:05, forax at univ-mlv.fr ha scritto: > ----- Mail original ----- >> De: "John Rose" >> ?: "Brian Goetz" >> Cc: "Remi Forax" , "discuss" >> Envoy?: Samedi 18 Avril 2020 09:53:58 >> Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? > >> Representing a JSON tree as a tree of Valhalla values (yes, they >> can come in trees!) is very much worth investigating IMO. The nearest >> approximation before Valhalla would be to design a JSON node model >> which consists only of (or mainly of) value-based classes. Or, perhaps >> Valhalla-style cursors (iterators which say x=x.next() instead of x.next()) >> can express a walk over a mutable tree, whose structure is hidden (and >> thus can be materialized lazily or streamily). The nearest approximation >> to that would be an iterator-style API, where one object updates itself >> in place. >> >> One other thought: Sometimes that parsing and unparsing logic >> for a syntax like JSON or XML can be expressed as operating on a >> linear event-stream, instead of a nested tree model. You can build >> the latter on top of the former, and event streams have a simpler >> object structure than tree-like models. It might be safer to start >> with a parser/unparser API that runs on event streams, and then >> build node models (with cursors, pattern matching, etc.) on top >> later when the tools for those appear. ?Not that I?m longing >> to program an event-based API, but it?s better than nothing, and >> not lost effort either. > > It's what the usual stream APIs do, at least what jackson and the json api from Java EE does. > The stream is just a tokenization of the text format > [1, 3, { "foo": null } ] -> START_ARRAY NUMBER NUMBER START_OBJECT FIELD_NAME NULL END_OBJECT END_ARRAY > > From that you can construct an API like ASM when you call a method visit for each value token (NUMBER, STRING, NULL, TRUE, etc) and a method visit that ask for a sub visitor for START_ARRAY and START_OBJECT. > > The next step, providing a tree/node API is harder and i think stupid mostly because JSON NUMBER is not typed so you have no idea if you should represent a number (as an int, a long, a float, a double, a BigInteger or a DigDecimal*, ??). You can introduce your own JsonNumber has an inline type but you will incompatible with any existing Java values. It's better to have a type representation that provide the type the user want for a number. > > That why most APIs do not provide a generic tree API but a data binding API that takes a tree of types (a bean/record/interface tree) and a stream of JSON tokens and construct the corresponding Java representation (creating instances Bean/Record/Proxy). This tree of type can also be seen as a tokenization of the type informations (START_OBJECT_TYPE, START_ARRAY_TYPE, NUMBER_TYPE, etc). > Annotations on the tree of types are used to derive an adapter (or several with the concept of jackson views) to the visitor that will filter/map the field name and the value before creating the instances. > > So IMO, a tree of instance of Valhalla inline types, a tree of nodes, is not useful but inline types can still be useful to simplify the visitor interface (instead of visitNumberInt/visitNumberLong/visitNumberDouble, etc, have a sole visitNumber(JsonNumber) with the inline type only existing on the stack to transfer the information from the reader to the deserializer (and vice-versa). > > regards, > R?mi > * ECMA 404 doesn't define a size for the numbers ! > From forax at univ-mlv.fr Thu Apr 23 20:18:09 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 23 Apr 2020 22:18:09 +0200 (CEST) Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <8faeedab-14b5-ec6d-133b-9721f385e74e@gmail.com> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> <8faeedab-14b5-ec6d-133b-9721f385e74e@gmail.com> Message-ID: <542509140.465375.1587673089893.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "cay horstmann" > ?: "discuss" > Envoy?: Jeudi 23 Avril 2020 20:53:02 > Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? > A JavaScript number is a double-precision floating-point number. JSON > cannot currently serialize BigInt. > > https://www.ecma-international.org/ecma-262/#sec-json.stringify > https://www.ecma-international.org/ecma-262/#sec-terms-and-definitions-number-value The problem is that at least Jackson can use BigInteger flawlessly, so it only works if you have Java at both ends or don't use JSON.stringify() but this is a use case that exist. Said differently, the ECMA-404 which defines JSON is not a proper subset of ECMA-262 (3rd edition). > > Cheers, > > Cay regards, R?mi > > Il 23/04/20 11:05, forax at univ-mlv.fr ha scritto: >> ----- Mail original ----- >>> De: "John Rose" >>> ?: "Brian Goetz" >>> Cc: "Remi Forax" , "discuss" >>> Envoy?: Samedi 18 Avril 2020 09:53:58 >>> Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? >> >>> Representing a JSON tree as a tree of Valhalla values (yes, they >>> can come in trees!) is very much worth investigating IMO. The nearest >>> approximation before Valhalla would be to design a JSON node model >>> which consists only of (or mainly of) value-based classes. Or, perhaps >>> Valhalla-style cursors (iterators which say x=x.next() instead of x.next()) >>> can express a walk over a mutable tree, whose structure is hidden (and >>> thus can be materialized lazily or streamily). The nearest approximation >>> to that would be an iterator-style API, where one object updates itself >>> in place. >>> >>> One other thought: Sometimes that parsing and unparsing logic >>> for a syntax like JSON or XML can be expressed as operating on a >>> linear event-stream, instead of a nested tree model. You can build >>> the latter on top of the former, and event streams have a simpler >>> object structure than tree-like models. It might be safer to start >>> with a parser/unparser API that runs on event streams, and then >>> build node models (with cursors, pattern matching, etc.) on top >>> later when the tools for those appear. ?Not that I?m longing >>> to program an event-based API, but it?s better than nothing, and >>> not lost effort either. >> >> It's what the usual stream APIs do, at least what jackson and the json api from >> Java EE does. >> The stream is just a tokenization of the text format >> [1, 3, { "foo": null } ] -> START_ARRAY NUMBER NUMBER START_OBJECT FIELD_NAME >> NULL END_OBJECT END_ARRAY >> >> From that you can construct an API like ASM when you call a method visit for >> each value token (NUMBER, STRING, NULL, TRUE, etc) and a method visit that ask >> for a sub visitor for START_ARRAY and START_OBJECT. >> >> The next step, providing a tree/node API is harder and i think stupid mostly >> because JSON NUMBER is not typed so you have no idea if you should represent a >> number (as an int, a long, a float, a double, a BigInteger or a DigDecimal*, >> ??). You can introduce your own JsonNumber has an inline type but you will >> incompatible with any existing Java values. It's better to have a type >> representation that provide the type the user want for a number. >> >> That why most APIs do not provide a generic tree API but a data binding API that >> takes a tree of types (a bean/record/interface tree) and a stream of JSON >> tokens and construct the corresponding Java representation (creating instances >> Bean/Record/Proxy). This tree of type can also be seen as a tokenization of the >> type informations (START_OBJECT_TYPE, START_ARRAY_TYPE, NUMBER_TYPE, etc). >> Annotations on the tree of types are used to derive an adapter (or several with >> the concept of jackson views) to the visitor that will filter/map the field >> name and the value before creating the instances. >> >> So IMO, a tree of instance of Valhalla inline types, a tree of nodes, is not >> useful but inline types can still be useful to simplify the visitor interface >> (instead of visitNumberInt/visitNumberLong/visitNumberDouble, etc, have a sole >> visitNumber(JsonNumber) with the inline type only existing on the stack to >> transfer the information from the reader to the deserializer (and vice-versa). >> >> regards, >> R?mi >> * ECMA 404 doesn't define a size for the numbers ! From john.r.rose at oracle.com Thu Apr 23 22:41:04 2020 From: john.r.rose at oracle.com (John Rose) Date: Thu, 23 Apr 2020 15:41:04 -0700 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> Message-ID: <303B6825-9B90-4935-9C09-5B44DFC03E7D@oracle.com> On Apr 23, 2020, at 11:05 AM, forax at univ-mlv.fr wrote: > > most APIs do not provide a generic tree API but a data binding API that takes a tree of types (a bean/record/interface tree) and a stream of JSON tokens and construct the corresponding Java representation (creating instances Bean/Record/Proxy). Yep. A Lisp refugee like me will immediately think ?S-expressions are enough; why use anything else? and that me to look for some sort of clunky prefab node structure. The more modern side of me will look for a mapping or transcoding to/from the event stream and whatever AST or other data I choose to work with at the moment. My point here is that the event stream ?layer? of the library is a safe-ish first step, even if it fails to use value types in the clever way you suggest. The next layer up will surely be data binding, and maybe a prefab tree for folks like me. IMO, the upper layers will want to work well (IMO) with *both* mutable vs. immutable (easy to edit vs. safe) AST representations, just as ju.List supplies both options, and will also want to perform more ad hoc transcodings (not just standard ASTs) to both mutable and immutable objects. The immutable objects are likely to be inline objects, sometimes. (For that matter, inlines can also express logically mutable objects.) The longer we delay nailing down the transcoding layer, the more future features of the language we can use to make that layer more flexible and powerful. ? John From larry.cable at oracle.com Fri Apr 24 00:05:07 2020 From: larry.cable at oracle.com (Laurence Cable) Date: Thu, 23 Apr 2020 17:05:07 -0700 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <303B6825-9B90-4935-9C09-5B44DFC03E7D@oracle.com> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> <303B6825-9B90-4935-9C09-5B44DFC03E7D@oracle.com> Message-ID: <112754ca-e01a-c2c9-f19c-6137581fa81c@oracle.com> On 4/23/20 3:41 PM, John Rose wrote: > On Apr 23, 2020, at 11:05 AM, forax at univ-mlv.fr wrote: >> most APIs do not provide a generic tree API but a data binding API that takes a tree of types (a bean/record/interface tree) and a stream of JSON tokens and construct the corresponding Java representation (creating instances Bean/Record/Proxy). > Yep. A Lisp refugee like me will immediately think ?S-expressions > are enough; why use anything else? and that me to look for some > sort of clunky prefab node structure. The more modern side of me > will look for a mapping or transcoding to/from the event stream > and whatever AST or other data I choose to work with at the moment. > > My point here is that the event stream ?layer? of the library is > a safe-ish first step, even if it fails to use value types in the clever > way you suggest. The next layer up will surely be data binding, > and maybe a prefab tree for folks like me. IMO, the upper layers > will want to work well (IMO) with *both* mutable vs. immutable > (easy to edit vs. safe) AST representations, just as ju.List supplies > both options, and will also want to perform more ad hoc transcodings > (not just standard ASTs) to both mutable and immutable objects. > The immutable objects are likely to be inline objects, sometimes. > (For that matter, inlines can also express logically mutable objects.) > _/*The longer we delay nailing down the transcoding layer, the more > future features of the language we can use to make that layer more > flexible and powerful*/_. +1! > > ? John From cay.horstmann at gmail.com Fri Apr 24 05:50:28 2020 From: cay.horstmann at gmail.com (Cay Horstmann) Date: Fri, 24 Apr 2020 07:50:28 +0200 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: <542509140.465375.1587673089893.JavaMail.zimbra@u-pem.fr> References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> <8faeedab-14b5-ec6d-133b-9721f385e74e@gmail.com> <542509140.465375.1587673089893.JavaMail.zimbra@u-pem.fr> Message-ID: I see...and people want to use this to serialize monetary values as "number". See https://stackoverflow.com/questions/11319445/java-to-jackson-json-serialization-money-fields and https://github.com/zalando/jackson-datatype-money. That seems like a terrible idea, but whatever. But why is that problematic for a tree API? The tree node needs to hold the string of digits (and sign etc.) since that's what an ECMA-404 "number" is. It's up to the user of the API to interpret the string as a double, BigDecimal, or String. Cheers, Cay Il 23/04/20 13:18, Remi Forax ha scritto: > ----- Mail original ----- >> De: "cay horstmann" >> ?: "discuss" >> Envoy?: Jeudi 23 Avril 2020 20:53:02 >> Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? > >> A JavaScript number is a double-precision floating-point number. JSON >> cannot currently serialize BigInt. >> >> https://www.ecma-international.org/ecma-262/#sec-json.stringify >> https://www.ecma-international.org/ecma-262/#sec-terms-and-definitions-number-value > > The problem is that at least Jackson can use BigInteger flawlessly, so it only works if you have Java at both ends or don't use JSON.stringify() but this is a use case that exist. > > Said differently, the ECMA-404 which defines JSON is not a proper subset of ECMA-262 (3rd edition). > >> >> Cheers, >> >> Cay > > regards, > R?mi > >> >> Il 23/04/20 11:05, forax at univ-mlv.fr ha scritto: >>> ----- Mail original ----- >>>> De: "John Rose" >>>> ?: "Brian Goetz" >>>> Cc: "Remi Forax" , "discuss" >>>> Envoy?: Samedi 18 Avril 2020 09:53:58 >>>> Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? >>> >>>> Representing a JSON tree as a tree of Valhalla values (yes, they >>>> can come in trees!) is very much worth investigating IMO. The nearest >>>> approximation before Valhalla would be to design a JSON node model >>>> which consists only of (or mainly of) value-based classes. Or, perhaps >>>> Valhalla-style cursors (iterators which say x=x.next() instead of x.next()) >>>> can express a walk over a mutable tree, whose structure is hidden (and >>>> thus can be materialized lazily or streamily). The nearest approximation >>>> to that would be an iterator-style API, where one object updates itself >>>> in place. >>>> >>>> One other thought: Sometimes that parsing and unparsing logic >>>> for a syntax like JSON or XML can be expressed as operating on a >>>> linear event-stream, instead of a nested tree model. You can build >>>> the latter on top of the former, and event streams have a simpler >>>> object structure than tree-like models. It might be safer to start >>>> with a parser/unparser API that runs on event streams, and then >>>> build node models (with cursors, pattern matching, etc.) on top >>>> later when the tools for those appear. ?Not that I?m longing >>>> to program an event-based API, but it?s better than nothing, and >>>> not lost effort either. >>> >>> It's what the usual stream APIs do, at least what jackson and the json api from >>> Java EE does. >>> The stream is just a tokenization of the text format >>> [1, 3, { "foo": null } ] -> START_ARRAY NUMBER NUMBER START_OBJECT FIELD_NAME >>> NULL END_OBJECT END_ARRAY >>> >>> From that you can construct an API like ASM when you call a method visit for >>> each value token (NUMBER, STRING, NULL, TRUE, etc) and a method visit that ask >>> for a sub visitor for START_ARRAY and START_OBJECT. >>> >>> The next step, providing a tree/node API is harder and i think stupid mostly >>> because JSON NUMBER is not typed so you have no idea if you should represent a >>> number (as an int, a long, a float, a double, a BigInteger or a DigDecimal*, >>> ??). You can introduce your own JsonNumber has an inline type but you will >>> incompatible with any existing Java values. It's better to have a type >>> representation that provide the type the user want for a number. >>> >>> That why most APIs do not provide a generic tree API but a data binding API that >>> takes a tree of types (a bean/record/interface tree) and a stream of JSON >>> tokens and construct the corresponding Java representation (creating instances >>> Bean/Record/Proxy). This tree of type can also be seen as a tokenization of the >>> type informations (START_OBJECT_TYPE, START_ARRAY_TYPE, NUMBER_TYPE, etc). >>> Annotations on the tree of types are used to derive an adapter (or several with >>> the concept of jackson views) to the visitor that will filter/map the field >>> name and the value before creating the instances. >>> >>> So IMO, a tree of instance of Valhalla inline types, a tree of nodes, is not >>> useful but inline types can still be useful to simplify the visitor interface >>> (instead of visitNumberInt/visitNumberLong/visitNumberDouble, etc, have a sole >>> visitNumber(JsonNumber) with the inline type only existing on the stack to >>> transfer the information from the reader to the deserializer (and vice-versa). >>> >>> regards, >>> R?mi >>> * ECMA 404 doesn't define a size for the numbers ! From larry.cable at oracle.com Fri Apr 24 15:09:43 2020 From: larry.cable at oracle.com (Laurence Cable) Date: Fri, 24 Apr 2020 08:09:43 -0700 Subject: Revival of JEP 198 (Light-Weight JSON API)? In-Reply-To: References: <1b36d358-cded-ef25-7e7d-3e3c40f3112d@oracle.com> <1248493129.412574.1587051715750.JavaMail.zimbra@u-pem.fr> <74BD3167-D5A8-44D0-9F52-0111C71E3053@oracle.com> <3EE20EA0-1DD7-4791-8B49-90739E0E6C0B@oracle.com> <92965CED-DA0D-4709-978B-EB5472EB0FFC@oracle.com> <560354821.440529.1587665109404.JavaMail.zimbra@u-pem.fr> <8faeedab-14b5-ec6d-133b-9721f385e74e@gmail.com> <542509140.465375.1587673089893.JavaMail.zimbra@u-pem.fr> Message-ID: <1e5f7ee7-6e2e-3f32-96d1-7dcc2ad92c26@oracle.com> On 4/23/20 10:50 PM, Cay Horstmann wrote: > I see...and people want to use this to serialize monetary values as > "number". See > https://stackoverflow.com/questions/11319445/java-to-jackson-json-serialization-money-fields > and https://github.com/zalando/jackson-datatype-money. That seems like > a terrible idea, but whatever. +1! > > But why is that problematic for a tree API? The tree node needs to > hold the string of digits (and sign etc.) since that's what an > ECMA-404 "number" is. It's up to the user of the API to interpret the > string as a double, BigDecimal, or String. +1 > > Cheers, > > Cay > > Il 23/04/20 13:18, Remi Forax ha scritto: >> ----- Mail original ----- >>> De: "cay horstmann" >>> ?: "discuss" >>> Envoy?: Jeudi 23 Avril 2020 20:53:02 >>> Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? >> >>> A JavaScript number is a double-precision floating-point number. JSON >>> cannot currently serialize BigInt. >>> >>> https://www.ecma-international.org/ecma-262/#sec-json.stringify >>> https://www.ecma-international.org/ecma-262/#sec-terms-and-definitions-number-value >>> >> >> The problem is that at least Jackson can use BigInteger flawlessly, >> so it only works if you have Java at both ends or don't use >> JSON.stringify() but this is a use case that exist. >> >> Said differently, the ECMA-404 which defines JSON is not a proper >> subset of ECMA-262 (3rd edition). >> >>> >>> Cheers, >>> >>> Cay >> >> regards, >> R?mi >> >>> >>> Il 23/04/20 11:05, forax at univ-mlv.fr ha scritto: >>>> ----- Mail original ----- >>>>> De: "John Rose" >>>>> ?: "Brian Goetz" >>>>> Cc: "Remi Forax" , "discuss" >>>>> >>>>> Envoy?: Samedi 18 Avril 2020 09:53:58 >>>>> Objet: Re: Revival of JEP 198 (Light-Weight JSON API)? >>>> >>>>> Representing a JSON tree as a tree of Valhalla values (yes, they >>>>> can come in trees!) is very much worth investigating IMO. The nearest >>>>> approximation before Valhalla would be to design a JSON node model >>>>> which consists only of (or mainly of) value-based classes. Or, >>>>> perhaps >>>>> Valhalla-style cursors (iterators which say x=x.next() instead of >>>>> x.next()) >>>>> can express a walk over a mutable tree, whose structure is hidden >>>>> (and >>>>> thus can be materialized lazily or streamily). The nearest >>>>> approximation >>>>> to that would be an iterator-style API, where one object updates >>>>> itself >>>>> in place. >>>>> >>>>> One other thought: Sometimes that parsing and unparsing logic >>>>> for a syntax like JSON or XML can be expressed as operating on a >>>>> linear event-stream, instead of a nested tree model. You can build >>>>> the latter on top of the former, and event streams have a simpler >>>>> object structure than tree-like models. It might be safer to start >>>>> with a parser/unparser API that runs on event streams, and then >>>>> build node models (with cursors, pattern matching, etc.) on top >>>>> later when the tools for those appear. ?Not that I?m longing >>>>> to program an event-based API, but it?s better than nothing, and >>>>> not lost effort either. >>>> >>>> It's what the usual stream APIs do, at least what jackson and the >>>> json api from >>>> Java EE does. >>>> The stream is just a tokenization of the text format >>>> [1, 3, { "foo": null } ] -> START_ARRAY NUMBER NUMBER >>>> START_OBJECT FIELD_NAME >>>> NULL END_OBJECT END_ARRAY >>>> >>>> From that you can construct an API like ASM when you call a >>>> method visit for >>>> each value token (NUMBER, STRING, NULL, TRUE, etc) and a method >>>> visit that ask >>>> for a sub visitor for START_ARRAY and START_OBJECT. >>>> >>>> The next step, providing a tree/node API is harder and i think >>>> stupid mostly >>>> because JSON NUMBER is not typed so you have no idea if you should >>>> represent a >>>> number (as an int, a long, a float, a double, a BigInteger or a >>>> DigDecimal*, >>>> ??). You can introduce your own JsonNumber has an inline type but >>>> you will >>>> incompatible with any existing Java values. It's better to have a type >>>> representation that provide the type the user want for a number. >>>> >>>> That why most APIs do not provide a generic tree API but a data >>>> binding API that >>>> takes a tree of types (a bean/record/interface tree) and a stream >>>> of JSON >>>> tokens and construct the corresponding Java representation >>>> (creating instances >>>> Bean/Record/Proxy). This tree of type can also be seen as a >>>> tokenization of the >>>> type informations (START_OBJECT_TYPE, START_ARRAY_TYPE, >>>> NUMBER_TYPE, etc). >>>> Annotations on the tree of types are used to derive an adapter (or >>>> several with >>>> the concept of jackson views) to the visitor that will filter/map >>>> the field >>>> name and the value before creating the instances. >>>> >>>> So IMO, a tree of instance of Valhalla inline types, a tree of >>>> nodes, is not >>>> useful but inline types can still be useful to simplify the visitor >>>> interface >>>> (instead of visitNumberInt/visitNumberLong/visitNumberDouble, etc, >>>> have a sole >>>> visitNumber(JsonNumber) with the inline type only existing on the >>>> stack to >>>> transfer the information from the reader to the deserializer (and >>>> vice-versa). >>>> >>>> regards, >>>> R?mi >>>> * ECMA 404 doesn't define a size for the numbers ! From mark.reinhold at oracle.com Mon Apr 27 16:38:55 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 27 Apr 2020 09:38:55 -0700 (PDT) Subject: Call for Discussion: New Project: Leyden Message-ID: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> I hereby invite discussion of a new Project, Leyden, whose primary goal will be to address the long-term pain points of Java?s slow startup time, slow time to peak performance, and large footprint. Leyden will address these pain points by introducing a concept of _static images_ to the Java Platform, and to the JDK. - A static image is a standalone program, derived from an application, which runs that application -- and no other. - A static image is a closed world: It cannot load classes from outside the image, nor can it spin new bytecodes at run time. These two constraints enable build-time analyses that can remove unused classes and identify class initializers which can be run at build time, thereby reducing both the size of the image and its startup time. These constraints also enable aggressive ahead-of-time compilation, thereby reducing the image?s time to peak performance. Static images are not for everyone, due to the closed-world constraint, nor are they for every type of application. They often require manual configuration in order to achieve the best results. We do, however, expect the results to be worthwhile in important deployment scenarios such as small embedded devices and the cloud. Project Leyden will take inspiration from past efforts to explore this space, including the GNU Compiler for Java [1] and the Native Image feature of GraalVM [2]. Leyden will add static images to the Java Platform Specification, and we expect that GraalVM will evolve to implement that Specification. Developers who use only the standard, specified static-image feature will then be able to switch with ease between Leyden (in the JDK), Native Image (in GraalVM), and whatever other conforming implementations may arise, choosing amongst tradeoffs of compile time, startup time, and image size. We do not intend to implement Leyden by merging the Native Image code from GraalVM into the JDK. Leyden will, rather, be based upon existing components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time compiler [3], application class-data sharing [4], and the `jlink` linking tool [5]. I propose to lead this Project with an initial set of Reviewers that includes, but is not limited to, Alex Buckley, Bob Vandette, Claes Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. This Project will start with a clone of the current JDK main-line release, JDK 15, and track main-line releases going forward. We expect to deliver Leyden over time, in a series of JEPs that will likely span multiple feature releases. Comments? - Mark [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java [2] https://www.graalvm.org/docs/reference-manual/native-image/ [3] https://openjdk.java.net/jeps/295 [4] https://openjdk.java.net/jeps/310 [5] https://openjdk.java.net/jeps/282 From eric.vergnaud at wanadoo.fr Mon Apr 27 17:09:04 2020 From: eric.vergnaud at wanadoo.fr (Eric Vergnaud) Date: Tue, 28 Apr 2020 01:09:04 +0800 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: <076910E3-0353-430E-BED0-B2E86C0841D8@wanadoo.fr> Interesting. I did something similar 20 years ago for a quasi JVM running on PalmOS. The strategy then was to dump the memory once a known entry point was reached, and restore it if present on the next launch. It was not overly difficult because PalmOS was single threaded, and it was done on the device itself, so guaranteed to be fit for that particular device. The below sounds much more ambitious ... > Le 28 avr. 2020 ? 00:38, mark.reinhold at oracle.com a ?crit : > > I hereby invite discussion of a new Project, Leyden, whose primary goal > will be to address the long-term pain points of Java?s slow startup time, > slow time to peak performance, and large footprint. > > Leyden will address these pain points by introducing a concept of _static > images_ to the Java Platform, and to the JDK. > > - A static image is a standalone program, derived from an application, > which runs that application -- and no other. > > - A static image is a closed world: It cannot load classes from outside > the image, nor can it spin new bytecodes at run time. > > These two constraints enable build-time analyses that can remove unused > classes and identify class initializers which can be run at build time, > thereby reducing both the size of the image and its startup time. These > constraints also enable aggressive ahead-of-time compilation, thereby > reducing the image?s time to peak performance. > > Static images are not for everyone, due to the closed-world constraint, > nor are they for every type of application. They often require manual > configuration in order to achieve the best results. We do, however, > expect the results to be worthwhile in important deployment scenarios > such as small embedded devices and the cloud. > > Project Leyden will take inspiration from past efforts to explore this > space, including the GNU Compiler for Java [1] and the Native Image > feature of GraalVM [2]. Leyden will add static images to the Java > Platform Specification, and we expect that GraalVM will evolve to > implement that Specification. Developers who use only the standard, > specified static-image feature will then be able to switch with ease > between Leyden (in the JDK), Native Image (in GraalVM), and whatever > other conforming implementations may arise, choosing amongst tradeoffs > of compile time, startup time, and image size. > > We do not intend to implement Leyden by merging the Native Image code > from GraalVM into the JDK. Leyden will, rather, be based upon existing > components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time > compiler [3], application class-data sharing [4], and the `jlink` linking > tool [5]. > > I propose to lead this Project with an initial set of Reviewers that > includes, but is not limited to, Alex Buckley, Bob Vandette, Claes > Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. > > This Project will start with a clone of the current JDK main-line > release, JDK 15, and track main-line releases going forward. We expect > to deliver Leyden over time, in a series of JEPs that will likely span > multiple feature releases. > > Comments? > > - Mark > > > [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java > [2] https://www.graalvm.org/docs/reference-manual/native-image/ > [3] https://openjdk.java.net/jeps/295 > [4] https://openjdk.java.net/jeps/310 > [5] https://openjdk.java.net/jeps/282 From aph at redhat.com Mon Apr 27 17:28:12 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 27 Apr 2020 18:28:12 +0100 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: On 4/27/20 5:38 PM, mark.reinhold at oracle.com wrote: > This Project will start with a clone of the current JDK main-line > release, JDK 15, and track main-line releases going forward. We expect > to deliver Leyden over time, in a series of JEPs that will likely span > multiple feature releases. > > Comments? It's the end of my working day here and I haven't got time to produce a fully-considered reply, but this is excellent news. It's about time to bring static AOT compilation into the fold. Let's get it done! -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From fw at deneb.enyo.de Mon Apr 27 17:38:09 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 27 Apr 2020 19:38:09 +0200 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> (mark reinhold's message of "Mon, 27 Apr 2020 09:38:55 -0700 (PDT)") References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: <87ees8vnge.fsf@mid.deneb.enyo.de> * mark reinhold: > I hereby invite discussion of a new Project, Leyden, whose primary goal > will be to address the long-term pain points of Java?s slow startup time, > slow time to peak performance, and large footprint. > > Leyden will address these pain points by introducing a concept of _static > images_ to the Java Platform, and to the JDK. > > - A static image is a standalone program, derived from an application, > which runs that application -- and no other. > > - A static image is a closed world: It cannot load classes from outside > the image, nor can it spin new bytecodes at run time. Static images will still rely on dynamic linking to C system libraries, right? Unfortunately, the most apparent alternative terms appear to be overloaded, too. From aph at redhat.com Mon Apr 27 17:43:00 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 27 Apr 2020 18:43:00 +0100 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <87ees8vnge.fsf@mid.deneb.enyo.de> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> <87ees8vnge.fsf@mid.deneb.enyo.de> Message-ID: <53e09235-b129-0676-da6e-122e3e3dff74@redhat.com> On 4/27/20 6:38 PM, Florian Weimer wrote: > Static images will still rely on dynamic linking to C system > libraries, right? I'm sure that will be possible. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From mark.reinhold at oracle.com Mon Apr 27 17:46:56 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 27 Apr 2020 10:46:56 -0700 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <87ees8vnge.fsf@mid.deneb.enyo.de> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> <87ees8vnge.fsf@mid.deneb.enyo.de> Message-ID: <20200427104656.44018386@eggemoggin.niobe.net> 2020/4/27 10:38:09 -0700, Florian Weimer : > * mark reinhold: >> ... >> >> Leyden will address these pain points by introducing a concept of _static >> images_ to the Java Platform, and to the JDK. >> >> - A static image is a standalone program, derived from an application, >> which runs that application -- and no other. >> >> - A static image is a closed world: It cannot load classes from outside >> the image, nor can it spin new bytecodes at run time. > > Static images will still rely on dynamic linking to C system > libraries, right? I expect that will be the default behavior, but an implementation could certainly offer the option to create a static image that requires no dynamic linking to other native libraries. > Unfortunately, the most apparent alternative terms > appear to be overloaded, too. Indeed, they are. ?Static? Java code != ?static? native code but, like you, I couldn?t find a better term. - Mark From hohensee at amazon.com Mon Apr 27 17:51:03 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 27 Apr 2020 17:51:03 +0000 Subject: Call for Discussion: New Project: Leyden Message-ID: <9FF97384-731D-4600-974E-7E5EC762FFF9@amazon.com> Very happy to see this proposal. In my experience, time to optimized performance and memory footprint are the primary user complaints when using Java for microservices. If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. Thanks, Paul ?On 4/27/20, 9:42 AM, "discuss on behalf of mark.reinhold at oracle.com" wrote: I hereby invite discussion of a new Project, Leyden, whose primary goal will be to address the long-term pain points of Java?s slow startup time, slow time to peak performance, and large footprint. Leyden will address these pain points by introducing a concept of _static images_ to the Java Platform, and to the JDK. - A static image is a standalone program, derived from an application, which runs that application -- and no other. - A static image is a closed world: It cannot load classes from outside the image, nor can it spin new bytecodes at run time. These two constraints enable build-time analyses that can remove unused classes and identify class initializers which can be run at build time, thereby reducing both the size of the image and its startup time. These constraints also enable aggressive ahead-of-time compilation, thereby reducing the image?s time to peak performance. Static images are not for everyone, due to the closed-world constraint, nor are they for every type of application. They often require manual configuration in order to achieve the best results. We do, however, expect the results to be worthwhile in important deployment scenarios such as small embedded devices and the cloud. Project Leyden will take inspiration from past efforts to explore this space, including the GNU Compiler for Java [1] and the Native Image feature of GraalVM [2]. Leyden will add static images to the Java Platform Specification, and we expect that GraalVM will evolve to implement that Specification. Developers who use only the standard, specified static-image feature will then be able to switch with ease between Leyden (in the JDK), Native Image (in GraalVM), and whatever other conforming implementations may arise, choosing amongst tradeoffs of compile time, startup time, and image size. We do not intend to implement Leyden by merging the Native Image code from GraalVM into the JDK. Leyden will, rather, be based upon existing components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time compiler [3], application class-data sharing [4], and the `jlink` linking tool [5]. I propose to lead this Project with an initial set of Reviewers that includes, but is not limited to, Alex Buckley, Bob Vandette, Claes Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. This Project will start with a clone of the current JDK main-line release, JDK 15, and track main-line releases going forward. We expect to deliver Leyden over time, in a series of JEPs that will likely span multiple feature releases. Comments? - Mark [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java [2] https://www.graalvm.org/docs/reference-manual/native-image/ [3] https://openjdk.java.net/jeps/295 [4] https://openjdk.java.net/jeps/310 [5] https://openjdk.java.net/jeps/282 From aph at redhat.com Mon Apr 27 17:57:03 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 27 Apr 2020 18:57:03 +0100 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <9FF97384-731D-4600-974E-7E5EC762FFF9@amazon.com> References: <9FF97384-731D-4600-974E-7E5EC762FFF9@amazon.com> Message-ID: On 4/27/20 6:51 PM, Hohensee, Paul wrote: > If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. LOL! Contributions welcome, right? :-) -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bob.vandette at oracle.com Mon Apr 27 18:11:58 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 27 Apr 2020 14:11:58 -0400 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <9FF97384-731D-4600-974E-7E5EC762FFF9@amazon.com> References: <9FF97384-731D-4600-974E-7E5EC762FFF9@amazon.com> Message-ID: <54FB0D3F-CA85-4FA3-AFE7-EA36D3555FF4@oracle.com> > On Apr 27, 2020, at 1:51 PM, Hohensee, Paul wrote: > > Very happy to see this proposal. In my experience, time to optimized performance and memory footprint are the primary user complaints when using Java for microservices. > > If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. I believe Mark is proposing leveraging jaotc. The jaotc behavior you are describing, beyond the closed world image support, is already in the mainline. Bob. > > Thanks, > Paul > > ?On 4/27/20, 9:42 AM, "discuss on behalf of mark.reinhold at oracle.com" wrote: > > I hereby invite discussion of a new Project, Leyden, whose primary goal > will be to address the long-term pain points of Java?s slow startup time, > slow time to peak performance, and large footprint. > > Leyden will address these pain points by introducing a concept of _static > images_ to the Java Platform, and to the JDK. > > - A static image is a standalone program, derived from an application, > which runs that application -- and no other. > > - A static image is a closed world: It cannot load classes from outside > the image, nor can it spin new bytecodes at run time. > > These two constraints enable build-time analyses that can remove unused > classes and identify class initializers which can be run at build time, > thereby reducing both the size of the image and its startup time. These > constraints also enable aggressive ahead-of-time compilation, thereby > reducing the image?s time to peak performance. > > Static images are not for everyone, due to the closed-world constraint, > nor are they for every type of application. They often require manual > configuration in order to achieve the best results. We do, however, > expect the results to be worthwhile in important deployment scenarios > such as small embedded devices and the cloud. > > Project Leyden will take inspiration from past efforts to explore this > space, including the GNU Compiler for Java [1] and the Native Image > feature of GraalVM [2]. Leyden will add static images to the Java > Platform Specification, and we expect that GraalVM will evolve to > implement that Specification. Developers who use only the standard, > specified static-image feature will then be able to switch with ease > between Leyden (in the JDK), Native Image (in GraalVM), and whatever > other conforming implementations may arise, choosing amongst tradeoffs > of compile time, startup time, and image size. > > We do not intend to implement Leyden by merging the Native Image code > from GraalVM into the JDK. Leyden will, rather, be based upon existing > components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time > compiler [3], application class-data sharing [4], and the `jlink` linking > tool [5]. > > I propose to lead this Project with an initial set of Reviewers that > includes, but is not limited to, Alex Buckley, Bob Vandette, Claes > Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. > > This Project will start with a clone of the current JDK main-line > release, JDK 15, and track main-line releases going forward. We expect > to deliver Leyden over time, in a series of JEPs that will likely span > multiple feature releases. > > Comments? > > - Mark > > > [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java > [2] https://www.graalvm.org/docs/reference-manual/native-image/ > [3] https://openjdk.java.net/jeps/295 > [4] https://openjdk.java.net/jeps/310 > [5] https://openjdk.java.net/jeps/282 > From hohensee at amazon.com Mon Apr 27 18:28:03 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 27 Apr 2020 18:28:03 +0000 Subject: Call for Discussion: New Project: Leyden Message-ID: <78530EA3-D4A5-4E91-9820-95FD2331B84C@amazon.com> Right. What I'm proposing is that the two projects effectively be merged, not just have Leyden leverage jaotc. Paul ?On 4/27/20, 11:13 AM, "Bob Vandette" wrote: > On Apr 27, 2020, at 1:51 PM, Hohensee, Paul wrote: > > Very happy to see this proposal. In my experience, time to optimized performance and memory footprint are the primary user complaints when using Java for microservices. > > If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. I believe Mark is proposing leveraging jaotc. The jaotc behavior you are describing, beyond the closed world image support, is already in the mainline. Bob. > > Thanks, > Paul > > On 4/27/20, 9:42 AM, "discuss on behalf of mark.reinhold at oracle.com" wrote: > > I hereby invite discussion of a new Project, Leyden, whose primary goal > will be to address the long-term pain points of Java?s slow startup time, > slow time to peak performance, and large footprint. > > Leyden will address these pain points by introducing a concept of _static > images_ to the Java Platform, and to the JDK. > > - A static image is a standalone program, derived from an application, > which runs that application -- and no other. > > - A static image is a closed world: It cannot load classes from outside > the image, nor can it spin new bytecodes at run time. > > These two constraints enable build-time analyses that can remove unused > classes and identify class initializers which can be run at build time, > thereby reducing both the size of the image and its startup time. These > constraints also enable aggressive ahead-of-time compilation, thereby > reducing the image?s time to peak performance. > > Static images are not for everyone, due to the closed-world constraint, > nor are they for every type of application. They often require manual > configuration in order to achieve the best results. We do, however, > expect the results to be worthwhile in important deployment scenarios > such as small embedded devices and the cloud. > > Project Leyden will take inspiration from past efforts to explore this > space, including the GNU Compiler for Java [1] and the Native Image > feature of GraalVM [2]. Leyden will add static images to the Java > Platform Specification, and we expect that GraalVM will evolve to > implement that Specification. Developers who use only the standard, > specified static-image feature will then be able to switch with ease > between Leyden (in the JDK), Native Image (in GraalVM), and whatever > other conforming implementations may arise, choosing amongst tradeoffs > of compile time, startup time, and image size. > > We do not intend to implement Leyden by merging the Native Image code > from GraalVM into the JDK. Leyden will, rather, be based upon existing > components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time > compiler [3], application class-data sharing [4], and the `jlink` linking > tool [5]. > > I propose to lead this Project with an initial set of Reviewers that > includes, but is not limited to, Alex Buckley, Bob Vandette, Claes > Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. > > This Project will start with a clone of the current JDK main-line > release, JDK 15, and track main-line releases going forward. We expect > to deliver Leyden over time, in a series of JEPs that will likely span > multiple feature releases. > > Comments? > > - Mark > > > [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java > [2] https://www.graalvm.org/docs/reference-manual/native-image/ > [3] https://openjdk.java.net/jeps/295 > [4] https://openjdk.java.net/jeps/310 > [5] https://openjdk.java.net/jeps/282 > From hohensee at amazon.com Mon Apr 27 18:28:47 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 27 Apr 2020 18:28:47 +0000 Subject: Call for Discussion: New Project: Leyden Message-ID: Indeed. :) ?On 4/27/20, 10:57 AM, "aph at redhat.com" wrote: On 4/27/20 6:51 PM, Hohensee, Paul wrote: > If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. LOL! Contributions welcome, right? :-) -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bob.vandette at oracle.com Mon Apr 27 18:37:17 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 27 Apr 2020 14:37:17 -0400 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <78530EA3-D4A5-4E91-9820-95FD2331B84C@amazon.com> References: <78530EA3-D4A5-4E91-9820-95FD2331B84C@amazon.com> Message-ID: <3EE574C3-B905-46F6-9D70-25F3CB4B5479@oracle.com> We?ll see how Mark decides to organize the project but jaotc was not a project. It was implemented with a JEP just like the other technologies mentioned in this proposal (jlink and App CDS). I don?t think it makes sense to treat jaotc any different than other dependent features. If you are suggesting that while enhancing jaotc, we maintain it?s current functionality, I?m good with that. Bob. > On Apr 27, 2020, at 2:28 PM, Hohensee, Paul wrote: > > Right. What I'm proposing is that the two projects effectively be merged, not just have Leyden leverage jaotc. > > Paul > > ?On 4/27/20, 11:13 AM, "Bob Vandette" wrote: > >> On Apr 27, 2020, at 1:51 PM, Hohensee, Paul wrote: >> >> Very happy to see this proposal. In my experience, time to optimized performance and memory footprint are the primary user complaints when using Java for microservices. >> >> If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. > > I believe Mark is proposing leveraging jaotc. > > The jaotc behavior you are describing, beyond the closed world image support, is already in the mainline. > > Bob. > >> >> Thanks, >> Paul >> >> On 4/27/20, 9:42 AM, "discuss on behalf of mark.reinhold at oracle.com" wrote: >> >> I hereby invite discussion of a new Project, Leyden, whose primary goal >> will be to address the long-term pain points of Java?s slow startup time, >> slow time to peak performance, and large footprint. >> >> Leyden will address these pain points by introducing a concept of _static >> images_ to the Java Platform, and to the JDK. >> >> - A static image is a standalone program, derived from an application, >> which runs that application -- and no other. >> >> - A static image is a closed world: It cannot load classes from outside >> the image, nor can it spin new bytecodes at run time. >> >> These two constraints enable build-time analyses that can remove unused >> classes and identify class initializers which can be run at build time, >> thereby reducing both the size of the image and its startup time. These >> constraints also enable aggressive ahead-of-time compilation, thereby >> reducing the image?s time to peak performance. >> >> Static images are not for everyone, due to the closed-world constraint, >> nor are they for every type of application. They often require manual >> configuration in order to achieve the best results. We do, however, >> expect the results to be worthwhile in important deployment scenarios >> such as small embedded devices and the cloud. >> >> Project Leyden will take inspiration from past efforts to explore this >> space, including the GNU Compiler for Java [1] and the Native Image >> feature of GraalVM [2]. Leyden will add static images to the Java >> Platform Specification, and we expect that GraalVM will evolve to >> implement that Specification. Developers who use only the standard, >> specified static-image feature will then be able to switch with ease >> between Leyden (in the JDK), Native Image (in GraalVM), and whatever >> other conforming implementations may arise, choosing amongst tradeoffs >> of compile time, startup time, and image size. >> >> We do not intend to implement Leyden by merging the Native Image code >> from GraalVM into the JDK. Leyden will, rather, be based upon existing >> components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time >> compiler [3], application class-data sharing [4], and the `jlink` linking >> tool [5]. >> >> I propose to lead this Project with an initial set of Reviewers that >> includes, but is not limited to, Alex Buckley, Bob Vandette, Claes >> Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. >> >> This Project will start with a clone of the current JDK main-line >> release, JDK 15, and track main-line releases going forward. We expect >> to deliver Leyden over time, in a series of JEPs that will likely span >> multiple feature releases. >> >> Comments? >> >> - Mark >> >> >> [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java >> [2] https://www.graalvm.org/docs/reference-manual/native-image/ >> [3] https://openjdk.java.net/jeps/295 >> [4] https://openjdk.java.net/jeps/310 >> [5] https://openjdk.java.net/jeps/282 >> > > From hohensee at amazon.com Mon Apr 27 18:55:14 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 27 Apr 2020 18:55:14 +0000 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <3EE574C3-B905-46F6-9D70-25F3CB4B5479@oracle.com> References: <78530EA3-D4A5-4E91-9820-95FD2331B84C@amazon.com> <3EE574C3-B905-46F6-9D70-25F3CB4B5479@oracle.com> Message-ID: <8B6B537F-8E3E-472D-89B7-C36D6D34FC97@amazon.com> Yes. No reduction in current jaotc functionality, just use Leyden functionality in jaotc where useful. Paul ?On 4/27/20, 11:38 AM, "Bob Vandette" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. We?ll see how Mark decides to organize the project but jaotc was not a project. It was implemented with a JEP just like the other technologies mentioned in this proposal (jlink and App CDS). I don?t think it makes sense to treat jaotc any different than other dependent features. If you are suggesting that while enhancing jaotc, we maintain it?s current functionality, I?m good with that. Bob. > On Apr 27, 2020, at 2:28 PM, Hohensee, Paul wrote: > > Right. What I'm proposing is that the two projects effectively be merged, not just have Leyden leverage jaotc. > > Paul > > On 4/27/20, 11:13 AM, "Bob Vandette" wrote: > >> On Apr 27, 2020, at 1:51 PM, Hohensee, Paul wrote: >> >> Very happy to see this proposal. In my experience, time to optimized performance and memory footprint are the primary user complaints when using Java for microservices. >> >> If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. > > I believe Mark is proposing leveraging jaotc. > > The jaotc behavior you are describing, beyond the closed world image support, is already in the mainline. > > Bob. > >> >> Thanks, >> Paul >> >> On 4/27/20, 9:42 AM, "discuss on behalf of mark.reinhold at oracle.com" wrote: >> >> I hereby invite discussion of a new Project, Leyden, whose primary goal >> will be to address the long-term pain points of Java?s slow startup time, >> slow time to peak performance, and large footprint. >> >> Leyden will address these pain points by introducing a concept of _static >> images_ to the Java Platform, and to the JDK. >> >> - A static image is a standalone program, derived from an application, >> which runs that application -- and no other. >> >> - A static image is a closed world: It cannot load classes from outside >> the image, nor can it spin new bytecodes at run time. >> >> These two constraints enable build-time analyses that can remove unused >> classes and identify class initializers which can be run at build time, >> thereby reducing both the size of the image and its startup time. These >> constraints also enable aggressive ahead-of-time compilation, thereby >> reducing the image?s time to peak performance. >> >> Static images are not for everyone, due to the closed-world constraint, >> nor are they for every type of application. They often require manual >> configuration in order to achieve the best results. We do, however, >> expect the results to be worthwhile in important deployment scenarios >> such as small embedded devices and the cloud. >> >> Project Leyden will take inspiration from past efforts to explore this >> space, including the GNU Compiler for Java [1] and the Native Image >> feature of GraalVM [2]. Leyden will add static images to the Java >> Platform Specification, and we expect that GraalVM will evolve to >> implement that Specification. Developers who use only the standard, >> specified static-image feature will then be able to switch with ease >> between Leyden (in the JDK), Native Image (in GraalVM), and whatever >> other conforming implementations may arise, choosing amongst tradeoffs >> of compile time, startup time, and image size. >> >> We do not intend to implement Leyden by merging the Native Image code >> from GraalVM into the JDK. Leyden will, rather, be based upon existing >> components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time >> compiler [3], application class-data sharing [4], and the `jlink` linking >> tool [5]. >> >> I propose to lead this Project with an initial set of Reviewers that >> includes, but is not limited to, Alex Buckley, Bob Vandette, Claes >> Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. >> >> This Project will start with a clone of the current JDK main-line >> release, JDK 15, and track main-line releases going forward. We expect >> to deliver Leyden over time, in a series of JEPs that will likely span >> multiple feature releases. >> >> Comments? >> >> - Mark >> >> >> [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java >> [2] https://www.graalvm.org/docs/reference-manual/native-image/ >> [3] https://openjdk.java.net/jeps/295 >> [4] https://openjdk.java.net/jeps/310 >> [5] https://openjdk.java.net/jeps/282 >> > > From slim at slimdude.com Mon Apr 27 19:16:51 2020 From: slim at slimdude.com (Slim Dude) Date: Mon, 27 Apr 2020 15:16:51 -0400 Subject: UNSUBSCRIBE In-Reply-To: References: Message-ID: <44D29051-0004-46F5-8835-1714409C33AD@slimdude.com> > On Apr 27, 2020, at 2:28 PM, Hohensee, Paul wrote: > > Indeed. :) > > ?On 4/27/20, 10:57 AM, "aph at redhat.com" wrote: > > On 4/27/20 6:51 PM, Hohensee, Paul wrote: >> If I may, I suggest that jaotc, etc. be folded into this project. It should be "straightforward" :) for the proposed AOT compiler to emit, in addition to closed world images, open world images containing nmethods for execution by Hotspot. Code in the nmethods would be predicated on AOT compile assumptions and de-optimized and recompiled as usual if those changed. > > LOL! Contributions welcome, right? :-) > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > > From volker.simonis at gmail.com Mon Apr 27 20:05:06 2020 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 27 Apr 2020 22:05:06 +0200 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: On Mon, Apr 27, 2020 at 6:40 PM wrote: > > I hereby invite discussion of a new Project, Leyden, whose primary goal > will be to address the long-term pain points of Java?s slow startup time, > slow time to peak performance, and large footprint. > > Leyden will address these pain points by introducing a concept of _static > images_ to the Java Platform, and to the JDK. > Definitely interesting and highly welcome! Hope I'll find some time to contribute :) > - A static image is a standalone program, derived from an application, > which runs that application -- and no other. > > - A static image is a closed world: It cannot load classes from outside > the image, nor can it spin new bytecodes at run time. > > These two constraints enable build-time analyses that can remove unused > classes and identify class initializers which can be run at build time, > thereby reducing both the size of the image and its startup time. These > constraints also enable aggressive ahead-of-time compilation, thereby > reducing the image?s time to peak performance. > > Static images are not for everyone, due to the closed-world constraint, > nor are they for every type of application. They often require manual > configuration in order to achieve the best results. We do, however, > expect the results to be worthwhile in important deployment scenarios > such as small embedded devices and the cloud. > > Project Leyden will take inspiration from past efforts to explore this > space, including the GNU Compiler for Java [1] and the Native Image > feature of GraalVM [2]. Leyden will add static images to the Java > Platform Specification, and we expect that GraalVM will evolve to > implement that Specification. I think "adding static images" to the Java Platform Specification will be the real tough part of this JEP. What are you envisioning here? Something like the "compact profiles" [1] on Language/VM level? Which Java language/VM features are you ready to sacrifice for better startup and lower footprint? GraalVM has its "small and well-guarded secret" :) called "LIMITATIONS.md". Is this the set of limitations you'd expect for the Java Platform Specification of static images as well? And finally, how do you plan to prevent the fragmentation of the Java ecosystem into "full Java" and "static Java"? [1] https://docs.oracle.com/javase/8/docs/technotes/guides/compactprofiles/compactprofiles.html [2] https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md > Developers who use only the standard, > specified static-image feature will then be able to switch with ease > between Leyden (in the JDK), Native Image (in GraalVM), and whatever > other conforming implementations may arise, choosing amongst tradeoffs > of compile time, startup time, and image size. > > We do not intend to implement Leyden by merging the Native Image code > from GraalVM into the JDK. Leyden will, rather, be based upon existing > components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time > compiler [3], application class-data sharing [4], and the `jlink` linking > tool [5]. > While I personally like this approach and would very much welcome it if it succeeds, I also think we have to be realistic. What makes you believe that one JVM "can rule them all"? HotSpot's current runtime system and GCs are quite sophisticated and complex. Adding one more use case (i.e. "native image") which they'll have to support, will further increase the code complexity and long term maintenance costs. Have you evaluated these costs compared to a complete new implantation or an implementation which leverages the extensive work done in the GrallVM project? I'm sincerely hoping that project "Leyden" is more than just an instrument of one Oracle product team fighting against another one and wish that the OpenJDK and the GraalVM teams could join forces for the advance and benefit of the Java platform as a whole. > I propose to lead this Project with an initial set of Reviewers that > includes, but is not limited to, Alex Buckley, Bob Vandette, Claes > Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. > > This Project will start with a clone of the current JDK main-line > release, JDK 15, and track main-line releases going forward. We expect > to deliver Leyden over time, in a series of JEPs that will likely span > multiple feature releases. > > Comments? > > - Mark > > > [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java > [2] https://www.graalvm.org/docs/reference-manual/native-image/ > [3] https://openjdk.java.net/jeps/295 > [4] https://openjdk.java.net/jeps/310 > [5] https://openjdk.java.net/jeps/282 From stumon01 at arm.com Mon Apr 27 20:59:07 2020 From: stumon01 at arm.com (Stuart Monteith) Date: Mon, 27 Apr 2020 21:59:07 +0100 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: This seems like the right time for such a project. Having a specification for what is "not-quite-Java" will be especially important for both applications and alternative JVM implementations. Will we be expecting a JSR with an Expert group, or will this be done purely through the OpenJDK JEP process? Have there been thoughts as to the awareness Java code will have to their environment? I'm wondering whether there be a means for classes/jars/modules to behave different under the two regimes, or whether different packages would need to be provided. My last assumption is that this will be developed primarily for x86_64 and will rely on outside assistance for the other platforms. BR, Stuart On 27/04/2020 17:38, mark.reinhold at oracle.com wrote: > I hereby invite discussion of a new Project, Leyden, whose primary goal > will be to address the long-term pain points of Java?s slow startup time, > slow time to peak performance, and large footprint. > > Leyden will address these pain points by introducing a concept of _static > images_ to the Java Platform, and to the JDK. > > - A static image is a standalone program, derived from an application, > which runs that application -- and no other. > > - A static image is a closed world: It cannot load classes from outside > the image, nor can it spin new bytecodes at run time. > > These two constraints enable build-time analyses that can remove unused > classes and identify class initializers which can be run at build time, > thereby reducing both the size of the image and its startup time. These > constraints also enable aggressive ahead-of-time compilation, thereby > reducing the image?s time to peak performance. > > Static images are not for everyone, due to the closed-world constraint, > nor are they for every type of application. They often require manual > configuration in order to achieve the best results. We do, however, > expect the results to be worthwhile in important deployment scenarios > such as small embedded devices and the cloud. > > Project Leyden will take inspiration from past efforts to explore this > space, including the GNU Compiler for Java [1] and the Native Image > feature of GraalVM [2]. Leyden will add static images to the Java > Platform Specification, and we expect that GraalVM will evolve to > implement that Specification. Developers who use only the standard, > specified static-image feature will then be able to switch with ease > between Leyden (in the JDK), Native Image (in GraalVM), and whatever > other conforming implementations may arise, choosing amongst tradeoffs > of compile time, startup time, and image size. > > We do not intend to implement Leyden by merging the Native Image code > from GraalVM into the JDK. Leyden will, rather, be based upon existing > components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time > compiler [3], application class-data sharing [4], and the `jlink` linking > tool [5]. > > I propose to lead this Project with an initial set of Reviewers that > includes, but is not limited to, Alex Buckley, Bob Vandette, Claes > Redestad, Igor Veresov, Ioi Lam, Mandy Chung, and Vladimir Kozlov. > > This Project will start with a clone of the current JDK main-line > release, JDK 15, and track main-line releases going forward. We expect > to deliver Leyden over time, in a series of JEPs that will likely span > multiple feature releases. > > Comments? > > - Mark > > > [1] https://en.wikipedia.org/wiki/GNU_Compiler_for_Java > [2] https://www.graalvm.org/docs/reference-manual/native-image/ > [3] https://openjdk.java.net/jeps/295 > [4] https://openjdk.java.net/jeps/310 > [5] https://openjdk.java.net/jeps/282 > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. From adinn at redhat.com Mon Apr 27 22:30:24 2020 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 27 Apr 2020 23:30:24 +0100 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: On 27/04/2020 17:38, mark.reinhold at oracle.com wrote: > This Project will start with a clone of the current JDK main-line > release, JDK 15, and track main-line releases going forward. We expect > to deliver Leyden over time, in a series of JEPs that will likely span > multiple feature releases. > > Comments? I am immensely pleased by the suggestion that the OpenJDK project should regularize and adopt the notion of a static Java application. I will be very glad to contribute in whatever ways I can, both to specification and implementation. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From mark.reinhold at oracle.com Mon Apr 27 22:31:03 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 27 Apr 2020 15:31:03 -0700 Subject: Call for Discussion: New Project: Leyden In-Reply-To: References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: <20200427153103.335536469@eggemoggin.niobe.net> 2020/4/27 13:59:07 -0700, Stuart Monteith : > ... Will we be expecting a JSR with an Expert group, or > will this be done purely through the OpenJDK JEP process? There will not be a separate JSR for this work. The specification and the implementation will be developed in the OpenJDK Community and then submitted for review and approval by the JCP for inclusion in a future version of the Java Platform. > Have there been thoughts as to the awareness Java code will have to > their environment? I'm wondering whether there be a means for > classes/jars/modules to behave different under the two regimes, or > whether different packages would need to be provided. There will almost certainly be a way to determine whether code is running in a static image vs. a full implementation. I?d be alarmed if there turns out to be a need for separate packages or modules. > My last assumption is that this will be developed primarily for x86_64 > and will rely on outside assistance for the other platforms. I don?t expect there to be much, if any, processor-specific code here. To the extent that there is then, yes, Oracle contributors would likely focus on x64 first. Contributions of support on other architectures would, as usual, be welcome. - Mark From youngty1997 at gmail.com Tue Apr 28 00:24:39 2020 From: youngty1997 at gmail.com (Ty Young) Date: Mon, 27 Apr 2020 19:24:39 -0500 Subject: Call for Discussion: New Project: Leyden Message-ID: <140d6eae-12b9-f883-1447-67fa467a4139@gmail.com> For clarity, does this have anything to do with heap or total JVM memory usage? Will it be possible to include otherwise independent projects into a single image for dynamic loading? Compile-time dependencies will be just fine, right? Is there any way or consideration of a way to take an existing image at runtime, add some plugins on top, and create a new image with those plugins? I haven't used the new jPakcager, so forgive me if this is a dumb question, but will jPackager support this? Back in the day with the old jPackager you'd get .deb, .rpm, and a native image alongside each other and it was kinda nice... until it was deprecated and removed. Personally, I'd like to have that back if possible. From mark.reinhold at oracle.com Tue Apr 28 16:40:38 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 28 Apr 2020 09:40:38 -0700 Subject: Call for Discussion: New Project: Leyden In-Reply-To: References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> Message-ID: <20200428094038.5375939@eggemoggin.niobe.net> 2020/4/27 13:05:06 -0700, volker.simonis at gmail.com: > On Mon, Apr 27, 2020 at 6:40 mark.reinhold at oracle.com wrote: >> ... >> >> Project Leyden will take inspiration from past efforts to explore this >> space, including the GNU Compiler for Java [1] and the Native Image >> feature of GraalVM [2]. Leyden will add static images to the Java >> Platform Specification, and we expect that GraalVM will evolve to >> implement that Specification. > > I think "adding static images" to the Java Platform Specification will > be the real tough part of this JEP. What are you envisioning here? > Something like the "compact profiles" [1] on Language/VM level? (I?m not sure what you mean by this.) > Which > Java language/VM features are you ready to sacrifice for better > startup and lower footprint? Short, preliminary answer: Loading classes not known at build time, plus arbitrary uses of method handles and the `invokedynamic` bytecode. (Uses of those features to, e.g., implement lambda expressions will work.) > GraalVM has its "small and well-guarded > secret" :) called "LIMITATIONS.md". Is this the set of limitations > you'd expect for the Java Platform Specification of static images as > well? I expect there to be some overlap, but it?s too early to say in detail. > And finally, how do you plan to prevent the fragmentation of the > Java ecosystem into "full Java" and "static Java"? Recall the first constraint: A static image runs just one application, and no other. A static image cannot run any code from outside the image, so there?s no risk that any particular image will somehow effectively define an alternate, non-standard version of the platform. It is true that the set of applications that can run in ?static Java? will be a subset of those that require ?full Java,? but then so is the set of applications that can run with just the `java.base` module vs. those that need more of the platform?s modules. >> ... >> >> We do not intend to implement Leyden by merging the Native Image code >> from GraalVM into the JDK. Leyden will, rather, be based upon existing >> components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time >> compiler [3], application class-data sharing [4], and the `jlink` linking >> tool [5]. > > While I personally like this approach and would very much welcome it > if it succeeds, I also think we have to be realistic. What makes you > believe that one JVM "can rule them all"? > > HotSpot's current runtime system and GCs are quite sophisticated and > complex. Adding one more use case (i.e. "native image") which they'll > have to support, will further increase the code complexity and long > term maintenance costs. I don?t expect that we?ll have to make significant changes to HotSpot in support of static images. I?d be surprised if, e.g., the GCs will require any change at all. The HotSpot that you get in a static image will, however, most likely be stripped down to omit features that are of less interest in static images. (I?m sure we?ll have a thorough discussion of exactly what those features are at some point.) > I'm sincerely hoping that project "Leyden" is more than just an > instrument of one Oracle product team fighting against another one and > wish that the OpenJDK and the GraalVM teams could join forces for the > advance and benefit of the Java platform as a whole. This is absolutely not about ?one Oracle product team fighting against another one.? The two teams will work together on this effort, hopefully with additional contributions from non-Oracle developers. I fully expect regular participation from Christian Wimmer, Thomas Wuerthinger, Vojin Jovanovic, and other members of the GraalVM team. - Mark From volker.simonis at gmail.com Wed Apr 29 14:13:37 2020 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 29 Apr 2020 16:13:37 +0200 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200428094038.5375939@eggemoggin.niobe.net> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> <20200428094038.5375939@eggemoggin.niobe.net> Message-ID: On Tue, Apr 28, 2020 at 6:42 PM wrote: > > 2020/4/27 13:05:06 -0700, volker.simonis at gmail.com: > > On Mon, Apr 27, 2020 at 6:40 mark.reinhold at oracle.com wrote: > >> ... > >> > >> Project Leyden will take inspiration from past efforts to explore this > >> space, including the GNU Compiler for Java [1] and the Native Image > >> feature of GraalVM [2]. Leyden will add static images to the Java > >> Platform Specification, and we expect that GraalVM will evolve to > >> implement that Specification. > > > > I think "adding static images" to the Java Platform Specification will > > be the real tough part of this JEP. What are you envisioning here? > > Something like the "compact profiles" [1] on Language/VM level? > > (I?m not sure what you mean by this.) My question was if the "static images" features (and restrictions) will be: a) a part of the Java SE specification (e.g. like compact profiles) b) a separate platform specification (e.g. like Java ME). If the answer is a), will they be an optional or mandatory part of the new Java SE specification? If they will be a part of the Java SE specification, will they be a true subset of Java SE or do you expect that they may contain extensions which will not be part of Java SE? > > > Which > > Java language/VM features are you ready to sacrifice for better > > startup and lower footprint? > > Short, preliminary answer: Loading classes not known at build time, plus > arbitrary uses of method handles and the `invokedynamic` bytecode. (Uses > of those features to, e.g., implement lambda expressions will work.) > > > GraalVM has its "small and well-guarded > > secret" :) called "LIMITATIONS.md". Is this the set of limitations > > you'd expect for the Java Platform Specification of static images as > > well? > > I expect there to be some overlap, but it?s too early to say in detail. > > > And finally, how do you plan to prevent the fragmentation of the > > Java ecosystem into "full Java" and "static Java"? > > Recall the first constraint: A static image runs just one application, > and no other. A static image cannot run any code from outside the image, > so there?s no risk that any particular image will somehow effectively > define an alternate, non-standard version of the platform. > > It is true that the set of applications that can run in ?static Java? > will be a subset of those that require ?full Java,? but then so is the > set of applications that can run with just the `java.base` module vs. > those that need more of the platform?s modules. Yes, that's clear. But you can't currently create a "java.base" JDK and pretend it to be "java.base" compatible. You can either claim Java SE compatibility (in which case you'll have support the full Java language specification, the full Java VM specification and everything included in the "java.se" module) or no compatibility at all. Everybody who is currently writing a Java application against the Java SE specification can be sure that it will execute correctly on every Java SE implementation. As you mentioned, "static Java" won't be able to execute every Java SE application any more. So we need a new "Java StaticEdition" specification which people can develop against. We will get "Java StaticEdition" compatible applications which will run on "Java SE" (if "Java StaticEdition" will be a true subset of "Java SE"). But we might also get "Java StaticEdition" implementations which won't be able to run full "Java SE" applications. > > >> ... > >> > >> We do not intend to implement Leyden by merging the Native Image code > >> from GraalVM into the JDK. Leyden will, rather, be based upon existing > >> components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time > >> compiler [3], application class-data sharing [4], and the `jlink` linking > >> tool [5]. > > > > While I personally like this approach and would very much welcome it > > if it succeeds, I also think we have to be realistic. What makes you > > believe that one JVM "can rule them all"? > > > > HotSpot's current runtime system and GCs are quite sophisticated and > > complex. Adding one more use case (i.e. "native image") which they'll > > have to support, will further increase the code complexity and long > > term maintenance costs. > > I don?t expect that we?ll have to make significant changes to HotSpot > in support of static images. I?d be surprised if, e.g., the GCs will > require any change at all. The HotSpot that you get in a static image > will, however, most likely be stripped down to omit features that are > of less interest in static images. (I?m sure we?ll have a thorough > discussion of exactly what those features are at some point.) > > > I'm sincerely hoping that project "Leyden" is more than just an > > instrument of one Oracle product team fighting against another one and > > wish that the OpenJDK and the GraalVM teams could join forces for the > > advance and benefit of the Java platform as a whole. > > This is absolutely not about ?one Oracle product team fighting against > another one.? The two teams will work together on this effort, hopefully > with additional contributions from non-Oracle developers. I fully expect > regular participation from Christian Wimmer, Thomas Wuerthinger, Vojin > Jovanovic, and other members of the GraalVM team. > Thanks for the clarification. That sounds very promising. > - Mark From volker.simonis at gmail.com Wed Apr 29 14:21:38 2020 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 29 Apr 2020 16:21:38 +0200 Subject: Call for Discussion: New Project: Leyden In-Reply-To: References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> <20200428094038.5375939@eggemoggin.niobe.net> Message-ID: On Tue, Apr 28, 2020 at 10:23 PM Kevin Regan wrote: > > I'm trying to think back over the years and I'm thinking most or all of the Java code that I've written would work in this static context. Could you provide real-world examples that wouldn't work? > If you have never used reflection, dynamic proxies, bytecode generation or instrumentation, JVMTI, JMX etc. you might be right. But remember that even if your code has never used these features, you might have pulled them in through library dependencies even without you being aware of them. As I wrote before, the SubstrateVM project has a good overview [1] of Java SE features which are not or only partially supported in a static image. You can expect the new OpenJDK "static image" implementation to have a similar set of restrictions. [1] https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md > On Tue, Apr 28, 2020 at 9:41 AM wrote: >> >> 2020/4/27 13:05:06 -0700, volker.simonis at gmail.com: >> > On Mon, Apr 27, 2020 at 6:40 mark.reinhold at oracle.com wrote: >> >> ... >> >> >> >> Project Leyden will take inspiration from past efforts to explore this >> >> space, including the GNU Compiler for Java [1] and the Native Image >> >> feature of GraalVM [2]. Leyden will add static images to the Java >> >> Platform Specification, and we expect that GraalVM will evolve to >> >> implement that Specification. >> > >> > I think "adding static images" to the Java Platform Specification will >> > be the real tough part of this JEP. What are you envisioning here? >> > Something like the "compact profiles" [1] on Language/VM level? >> >> (I?m not sure what you mean by this.) >> >> > Which >> > Java language/VM features are you ready to sacrifice for better >> > startup and lower footprint? >> >> Short, preliminary answer: Loading classes not known at build time, plus >> arbitrary uses of method handles and the `invokedynamic` bytecode. (Uses >> of those features to, e.g., implement lambda expressions will work.) >> >> > GraalVM has its "small and well-guarded >> > secret" :) called "LIMITATIONS.md". Is this the set of limitations >> > you'd expect for the Java Platform Specification of static images as >> > well? >> >> I expect there to be some overlap, but it?s too early to say in detail. >> >> > And finally, how do you plan to prevent the fragmentation of the >> > Java ecosystem into "full Java" and "static Java"? >> >> Recall the first constraint: A static image runs just one application, >> and no other. A static image cannot run any code from outside the image, >> so there?s no risk that any particular image will somehow effectively >> define an alternate, non-standard version of the platform. >> >> It is true that the set of applications that can run in ?static Java? >> will be a subset of those that require ?full Java,? but then so is the >> set of applications that can run with just the `java.base` module vs. >> those that need more of the platform?s modules. >> >> >> ... >> >> >> >> We do not intend to implement Leyden by merging the Native Image code >> >> from GraalVM into the JDK. Leyden will, rather, be based upon existing >> >> components in the JDK such as the HotSpot JVM, the `jaotc` ahead-of-time >> >> compiler [3], application class-data sharing [4], and the `jlink` linking >> >> tool [5]. >> > >> > While I personally like this approach and would very much welcome it >> > if it succeeds, I also think we have to be realistic. What makes you >> > believe that one JVM "can rule them all"? >> > >> > HotSpot's current runtime system and GCs are quite sophisticated and >> > complex. Adding one more use case (i.e. "native image") which they'll >> > have to support, will further increase the code complexity and long >> > term maintenance costs. >> >> I don?t expect that we?ll have to make significant changes to HotSpot >> in support of static images. I?d be surprised if, e.g., the GCs will >> require any change at all. The HotSpot that you get in a static image >> will, however, most likely be stripped down to omit features that are >> of less interest in static images. (I?m sure we?ll have a thorough >> discussion of exactly what those features are at some point.) >> >> > I'm sincerely hoping that project "Leyden" is more than just an >> > instrument of one Oracle product team fighting against another one and >> > wish that the OpenJDK and the GraalVM teams could join forces for the >> > advance and benefit of the Java platform as a whole. >> >> This is absolutely not about ?one Oracle product team fighting against >> another one.? The two teams will work together on this effort, hopefully >> with additional contributions from non-Oracle developers. I fully expect >> regular participation from Christian Wimmer, Thomas Wuerthinger, Vojin >> Jovanovic, and other members of the GraalVM team. >> >> - Mark From brian.goetz at oracle.com Thu Apr 30 16:24:29 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 30 Apr 2020 12:24:29 -0400 Subject: Call for Discussion: New Project: Leyden In-Reply-To: References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> <20200428094038.5375939@eggemoggin.niobe.net> Message-ID: On 4/29/2020 10:13 AM, Volker Simonis wrote: >>> I think "adding static images" to the Java Platform Specification will >>> be the real tough part of this JEP. What are you envisioning here? >>> Something like the "compact profiles" [1] on Language/VM level? >> (I?m not sure what you mean by this.) > My question was if the "static images" features (and restrictions) will be: > > a) a part of the Java SE specification (e.g. like compact profiles) > b) a separate platform specification (e.g. like Java ME). > > If the answer is a), will they be an optional or mandatory part of the > new Java SE specification? > > If they will be a part of the Java SE specification, will they be a > true subset of Java SE or do you expect that they may contain > extensions which will not be part of Java SE? I think you're thinking about this in the wrong way.? This is a "call for discussion", which should be about discussing the suitability of embarking on such a project.? Part of the work of the project would then be to explore the possible directions, evaluate the impact, assess tradeoffs, propose design candidates, etc. Questions of the form "will it be" rest on the assumption that there already is an answer, and that it just hasn't been revealed.? While in some cases, the proposer might have thought about some of those and have opinions, in general it is not fair to ask "will it be" questions, because most of the time, most of these questions don't have answers.? It might be OK to ask "do you have any thoughts on", but we should still be mindful that many of these questions are more appropriate for the actual project than for the CFD. From mark.reinhold at oracle.com Thu Apr 30 16:54:45 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 30 Apr 2020 09:54:45 -0700 Subject: Call for Discussion: New Project: Leyden In-Reply-To: References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> <20200428094038.5375939@eggemoggin.niobe.net> Message-ID: <20200430095445.765823432@eggemoggin.niobe.net> 2020/4/29 7:13:37 -0700, volker.simonis at gmail.com: > On Tue, Apr 28, 2020 at 6:42 PM wrote: >> 2020/4/27 13:05:06 -0700, volker.simonis at gmail.com: >>> I think "adding static images" to the Java Platform Specification will >>> be the real tough part of this JEP. What are you envisioning here? >>> Something like the "compact profiles" [1] on Language/VM level? >> >> (I?m not sure what you mean by this.) > > My question was if the "static images" features (and restrictions) will be: > > a) a part of the Java SE specification (e.g. like compact profiles) > b) a separate platform specification (e.g. like Java ME). The former (i.e., (a)). > If the answer is a), will they be an optional or mandatory part of the > new Java SE specification? Optional, most likely. > If they will be a part of the Java SE specification, will they be a > true subset of Java SE or do you expect that they may contain > extensions which will not be part of Java SE? The notion of ?subset? is tricky in this context. I expect that most, if not all, of the Java API will be available to applications that run as static images. When used in a static image, however, some APIs will be specified to behave differently, principally by throwing exceptions in situations that cannot be supported in static mode. Invoking `Class.forName("p.Q")` for a class `p.Q` that doesn?t exist in the image would, e.g., throw a `ClassNotFoundException`, or perhaps a new subclass of that exception, `ClassNotFoundInStaticImageException`. The goal is to have all of the functionality required to make good use of static images be part of the Java Platform Specification. There will no doubt be room for implementation-specific extensions, as is the case with other facilities in the Platform, and over time such extensions may motivate further enhancements to the Platform. >> ... >> >> It is true that the set of applications that can run in ?static Java? >> will be a subset of those that require ?full Java,? but then so is the >> set of applications that can run with just the `java.base` module vs. >> those that need more of the platform?s modules. > > Yes, that's clear. But you can't currently create a "java.base" JDK > and pretend it to be "java.base" compatible. You can either claim Java > SE compatibility (in which case you'll have support the full Java > language specification, the full Java VM specification and everything > included in the "java.se" module) or no compatibility at all. Not true. Since Java 9, the Platform Specification has explicitly defined what it means to have a compatible subset of the entire set of standard modules [1]. - Mark [1] https://cr.openjdk.java.net/~iris/se/9/java-se-9-fr-spec/#Constraints-on-all-modules-in-an-Implementation From volker.simonis at gmail.com Thu Apr 30 18:02:51 2020 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 30 Apr 2020 20:02:51 +0200 Subject: Call for Discussion: New Project: Leyden In-Reply-To: <20200430095445.765823432@eggemoggin.niobe.net> References: <20200427163855.10D3C32C99F@eggemoggin.niobe.net> <20200428094038.5375939@eggemoggin.niobe.net> <20200430095445.765823432@eggemoggin.niobe.net> Message-ID: On Thu, Apr 30, 2020 at 6:55 PM wrote: > > 2020/4/29 7:13:37 -0700, volker.simonis at gmail.com: > > On Tue, Apr 28, 2020 at 6:42 PM wrote: > >> 2020/4/27 13:05:06 -0700, volker.simonis at gmail.com: > >>> I think "adding static images" to the Java Platform Specification will > >>> be the real tough part of this JEP. What are you envisioning here? > >>> Something like the "compact profiles" [1] on Language/VM level? > >> > >> (I?m not sure what you mean by this.) > > > > My question was if the "static images" features (and restrictions) will be: > > > > a) a part of the Java SE specification (e.g. like compact profiles) > > b) a separate platform specification (e.g. like Java ME). > > The former (i.e., (a)). > > > If the answer is a), will they be an optional or mandatory part of the > > new Java SE specification? > > Optional, most likely. > > > If they will be a part of the Java SE specification, will they be a > > true subset of Java SE or do you expect that they may contain > > extensions which will not be part of Java SE? > > The notion of ?subset? is tricky in this context. I expect that most, > if not all, of the Java API will be available to applications that run > as static images. When used in a static image, however, some APIs will > be specified to behave differently, principally by throwing exceptions > in situations that cannot be supported in static mode. Invoking > `Class.forName("p.Q")` for a class `p.Q` that doesn?t exist in the image > would, e.g., throw a `ClassNotFoundException`, or perhaps a new subclass > of that exception, `ClassNotFoundInStaticImageException`. > > The goal is to have all of the functionality required to make good use > of static images be part of the Java Platform Specification. There will > no doubt be room for implementation-specific extensions, as is the case > with other facilities in the Platform, and over time such extensions may > motivate further enhancements to the Platform. > > >> ... > >> > >> It is true that the set of applications that can run in ?static Java? > >> will be a subset of those that require ?full Java,? but then so is the > >> set of applications that can run with just the `java.base` module vs. > >> those that need more of the platform?s modules. > > > > Yes, that's clear. But you can't currently create a "java.base" JDK > > and pretend it to be "java.base" compatible. You can either claim Java > > SE compatibility (in which case you'll have support the full Java > > language specification, the full Java VM specification and everything > > included in the "java.se" module) or no compatibility at all. > > Not true. Since Java 9, the Platform Specification has explicitly > defined what it means to have a compatible subset of the entire set > of standard modules [1]. > You're right. I somehow forgot about that although now, after reading it I'm pretty sure I didn't read it for the first time :) Thanks, Volker > - Mark > > > [1] https://cr.openjdk.java.net/~iris/se/9/java-se-9-fr-spec/#Constraints-on-all-modules-in-an-Implementation From joe.darcy at oracle.com Thu Apr 30 23:15:28 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 30 Apr 2020 16:15:28 -0700 Subject: Invitation to Project Skara update, Thursday May 7, 2020 at 8:30 am PST (UTC-8) / 4:30 pm GMT (UTC+0) / 5:30 pm CET (UTC+1) Message-ID: <883c0164-cdef-af90-6c42-74ab9277058b@oracle.com> Hello, Project Skara has been investigating migrating from Mercurial to Git for the OpenJDK Community's source code management needs. [1] Please join the Skara team for an update on this work on Thursday May 7 at ???? 8:30 am PST (UTC-8) / 4:30 pm GMT (UTC+0) / 5:30 pm CET (UTC+1) The update will discuss OpenJDK projects that have transitioned to using the Skara tooling, features of that tooling, and the future roadmap. The meeting will be hosted on Zoom; connection details below. Cheers, -Joe [1] https://wiki.openjdk.java.net/display/SKARA; also see JEPs, JEP 357: "Migrate from Mercurial to Git" (http://openjdk.java.net/jeps/357) and JEP 369: "Migrate to GitHub." (http://openjdk.java.net/jeps/369). -=-=-=-=-=- Topic: Project Skara Update Time: May 7, 2020 08:30 AM Pacific Time (US and Canada) Single click URL to join Zoom: https://oracle.zoom.us/j/95373614916?pwd=NzlXZTllZExWMkh2bDhzOUlDV1BsZz09 Meeting ID: 953 7361 4916 Password: 012487 One tap mobile +16699006833,,95373614916# US (San Jose) +12532158782,,95373614916# US (Tacoma) Dial by your location ??????? +1 669 900 6833 US (San Jose) ??????? +1 253 215 8782 US (Tacoma) ??????? +1 346 248 7799 US (Houston) ??????? +1 301 715 8592 US (Germantown) ??????? +1 312 626 6799 US (Chicago) ??????? +1 646 558 8656 US (New York) Meeting ID: 953 7361 4916 Password: 012487 Find your local number: https://oracle.zoom.us/u/a2RlbxyZ4 Join by SIP 95373614916 at zoomcrc.com Join by H.323 162.255.37.11 (US West) 162.255.36.11 (US East) 115.114.131.7 (India Mumbai) 115.114.115.7 (India Hyderabad) 213.19.144.110 (EMEA) 103.122.166.55 (Australia) 209.9.211.110 (Hong Kong ?China) 64.211.144.160 (Brazil) 69.174.57.160 (Canada) 207.226.132.110 (Japan) Meeting ID: 953 7361 4916 Password: 012487