Call For Discussion: New Project: Atlantis

Fri Nov 30 12:48:55 UTC 2018

Hi JC,

This looks fine and it is a pragmatic approach.

I still think parts of your original vision have merit, namely having
a new mailing list for discussing serviceability proposals which are
larger or span multiple areas.

Cheers, Thomas

On Thu, Nov 29, 2018 at 11:27 PM JC Beyler <jcbeyler at google.com> wrote:
>
> Hi Magnus,
>
> Thanks for the observation and it is much appreciated. You are probably
> right and an easier means to get us going would be to start with a TSAN
> specific project and then we can figure out what to do with the next
> feature that might require to be hosted via a project (ie should it be
> under Atlantis or under Mu).
>
> This would mean that the CFD shifts itself to this new project description:
>
> -------------------------------------------------------
>
> I would like to start a discussion about the possible creation of the
> Atlantis Project with myself, Jean Christophe Beyler, as the Lead and the
> Serviceability group as the sponsoring group.
>
> In accordance with the OpenJDK guidelines [1], this project would provide a
> venue to explore and incubate a Thread Sanitizing feature that could be
> integrated into the Hotspot JVM and the JVM Tool Interface. This includes
> working, evaluating, and incubating a Thread-Sanitizer implementation [2]
> for Java.
>
> The Google platform team has historically worked on a TSAN project
> internally. The Atlantis project proposes to investigate, in the open, how
> these changes could be made general enough to be pushed into the mainline,
> or should be dropped due to being too specific or could be done using other
> existing mechanisms. The project would not be a Google-only project and
> will have a mailing list to discuss and potentially accept any contribution
> trying to extend and/or enhance the TSAN implementation.
>
> The evaluation criteria for the TSAN implementation includes but would not
> be limited to:
>      * Performance: not adding overhead to non-tsan runs is paramount and
> there are various means to do so
>      * General usefulness for the community at large so that the TSAN
> feature is deemed a good feature to have
>
> If the TSAN implementation becomes stable and is deemed useful enough, the
> Atlantis project would shepherd a JEP to integrate the change into the
> mainline of OpenJDK.
>
> The current suggested initial Reviewers and Committers would be but is most
> likely not going to be limited to:
> * Jean Christophe Beyler
> * Serguei Spitsyn
> * Chuck Rasbold
> * Liam Cushon
>   ... TBD ...
>
> [1] http://openjdk.java.net/projects/#new-project
> [2] https://bugs.openjdk.java.net/browse/JDK-8208520
>
> ---------------------------------
>
> What do you all think? :-)
>
> Thanks,
> Jc
>
> On Wed, Nov 28, 2018 at 1:41 AM Magnus Ihse Bursie <
> magnus.ihse.bursie at oracle.com> wrote:
>
> > Hi JC,
> >
> > Just a simple observation. Creating a project for the purpose of working
> > on TSAN should really be a no-brainer. I don't think anyone would object
> > to that. There are many projects like this, e.g. Portola, where there is
> > a need for the infrastructure OpenJDK grants a project (repo, mailing
> > list).
> >
> >
> > I think what maybe makes this proposal slightly more controversial is
> > the broad and not well delimited scope. Maybe a simpler route would be
> > to retarget your ideas for Atlantis from "any kind of servicability
> > stuff that needs a project" to "trying to get TSAN in shape for
> > integration into OpenJDK". And then, later on, when you feel you are
> > ready to start tackling another serviceability feature as a project, ask
> > the community if that should be a separate project Mu[1], or if Atlantis
> > should have it scope widened to also handle this second project.
> >
> > /Magnus
> >
> > [1] https://en.wikipedia.org/wiki/Mu_(lost_continent) :-)
> >
> > On 2018-11-26 20:22, JC Beyler wrote:
> > > Hi Jonathan,
> > >
> > > Thanks for the interest and sounds good to me! We could start
> > conversations
> > > on this topic once the project opens and see what we could do in that
> > > problem-space :)
> > > Jc
> > >
> > > On Tue, Nov 20, 2018 at 7:46 PM Jonathan Lu <
> > chuansheng.lcs at alibaba-inc.com>
> > > wrote:
> > >
> > >> Hi JC,
> > >>
> > >> Personally, I like this idea.
> > >>
> > >> And I'm especially interested in the heap dumping part.
> > >> It costs a lot to create, transfer, and analyze huge heap dump files for
> > >> my applications.
> > >> Maybe some fundamental changes can be explored in this project, I guess,
> > >> such as making an enhanced format of heap dump file :)
> > >>
> > >> Thank you
> > >> Jonathan
> > >>
> > >> ------------------------------------------------------------------
> > >> From:JC Beyler <jcbeyler at google.com>
> > >> Send Time:2018年11月20日(星期二) 02:48
> > >> To:discuss <discuss at openjdk.java.net>
> > >> Cc:serguei.spitsyn <serguei.spitsyn at oracle.com>
> > >> Subject:Call For Discussion: New Project: Atlantis
> > >>
> > >> Hi all,
> > >>
> > >> I would like to start a discussion about the possible creation of the
> > >> Atlantis Project with myself, Jean Christophe Beyler, as the Lead and
> > the
> > >> Serviceability group as the sponsoring group.
> > >>
> > >> In accordance with the OpenJDK guidelines [1], this project would
> > provide a
> > >> venue to explore and incubate monitoring and performance analysis
> > features
> > >> that could be integrated into the Hotspot JVM and the JVM Tool
> > Interface.
> > >> This includes working, evaluating, and incubating a Thread-Sanitizer
> > >> implementation [2] for Java.
> > >>
> > >> The Google platform team has historically worked on providing new
> > features,
> > >> including monitoring hooks due to certain monitoring features not being
> > >> available in a low-overhead, turned on-by-default manner. Therefore, for
> > >> Google Java users, the platform team augmented or added mechanisms to
> > >> assess various metrics such as but not limited to:
> > >>
> > >> - Thread Sanitizer support for Java (see JDK-8208520 [2]) (though this
> > one
> > >> specifically is not a low-overhead one, sorry ;-))
> > >> - Lock contention profiles to better understand where threads are
> > spending
> > >> time in locks
> > >> - Heap dumping mechanisms to improve heap dump times for large heaps
> > >> - Efficient thread tagging to help filter profiling
> > >> - etc.
> > >>
> > >> All of the above have added changes to the JDK in various forms and the
> > >> Atlantis project proposes to investigate, in the open, how these changes
> > >> could be made general enough to be pushed into the mainline, or should
> > be
> > >> dropped due to being too specific or could be done using other existing
> > >> mechanisms. The project would not be a Google-only project and will
> > have a
> > >> mailing list to discuss and potentially accept any contribution trying
> > to
> > >> extend and/or enhance current monitoring systems.
> > >>
> > >> The evaluation criteria to consider a feature/improvement includes but
> > >> would not be limited to:
> > >>
> > >>       * Performance: metrics and new hooks should not be hurting
> > performance
> > >> when disabled
> > >>       * General usefulness for the community at large
> > >>       * No working alternatives: sometimes it may be that there are new
> > >> technologies available (ie JFR events for example) that might be
> > leveraged
> > >> to deprecate existing solutions
> > >>
> > >> If one or more prototypes indicate a new metric, a new monitoring
> > system,
> > >> or a feature offers substantial improvements, the Atlantis project would
> > >> shepherd a JEP to integrate the change into the mainline of OpenJDK.
> > >>
> > >> The current suggested initial Reviewers and Committers would be but is
> > most
> > >> likely not going to be limited to:
> > >> * Jean Christophe Beyler
> > >> * Serguei Spitsyn
> > >> * Chuck Rasbold
> > >> * Liam Cushon
> > >>
> > >> What do you all think? :-)
> > >>
> > >> Thanks,
> > >> Jc
> > >>
> > >> [1] http://openjdk.java.net/projects/#new-project
> > >> [2] https://bugs.openjdk.java.net/browse/JDK-8208520
> > >>
> > >>
> > >>
> >
> >
>
> --
>
> Thanks,
> Jc