Call For Discussion: New Project: Atlantis

Thu Nov 29 22:25:41 UTC 2018

Hi Magnus,

Thanks for the observation and it is much appreciated. You are probably
right and an easier means to get us going would be to start with a TSAN
specific project and then we can figure out what to do with the next
feature that might require to be hosted via a project (ie should it be
under Atlantis or under Mu).

This would mean that the CFD shifts itself to this new project description:

-------------------------------------------------------

I would like to start a discussion about the possible creation of the
Atlantis Project with myself, Jean Christophe Beyler, as the Lead and the
Serviceability group as the sponsoring group.

In accordance with the OpenJDK guidelines [1], this project would provide a
venue to explore and incubate a Thread Sanitizing feature that could be
integrated into the Hotspot JVM and the JVM Tool Interface. This includes
working, evaluating, and incubating a Thread-Sanitizer implementation [2]
for Java.

The Google platform team has historically worked on a TSAN project
internally. The Atlantis project proposes to investigate, in the open, how
these changes could be made general enough to be pushed into the mainline,
or should be dropped due to being too specific or could be done using other
existing mechanisms. The project would not be a Google-only project and
will have a mailing list to discuss and potentially accept any contribution
trying to extend and/or enhance the TSAN implementation.

The evaluation criteria for the TSAN implementation includes but would not
be limited to:
     * Performance: not adding overhead to non-tsan runs is paramount and
there are various means to do so
     * General usefulness for the community at large so that the TSAN
feature is deemed a good feature to have

If the TSAN implementation becomes stable and is deemed useful enough, the
Atlantis project would shepherd a JEP to integrate the change into the
mainline of OpenJDK.

The current suggested initial Reviewers and Committers would be but is most
likely not going to be limited to:
* Jean Christophe Beyler
* Serguei Spitsyn
* Chuck Rasbold
* Liam Cushon
  ... TBD ...

[1] http://openjdk.java.net/projects/#new-project
[2] https://bugs.openjdk.java.net/browse/JDK-8208520

---------------------------------

What do you all think? :-)

Thanks,
Jc

On Wed, Nov 28, 2018 at 1:41 AM Magnus Ihse Bursie <
magnus.ihse.bursie at oracle.com> wrote:

> Hi JC,
>
> Just a simple observation. Creating a project for the purpose of working
> on TSAN should really be a no-brainer. I don't think anyone would object
> to that. There are many projects like this, e.g. Portola, where there is
> a need for the infrastructure OpenJDK grants a project (repo, mailing
> list).
>
>
> I think what maybe makes this proposal slightly more controversial is
> the broad and not well delimited scope. Maybe a simpler route would be
> to retarget your ideas for Atlantis from "any kind of servicability
> stuff that needs a project" to "trying to get TSAN in shape for
> integration into OpenJDK". And then, later on, when you feel you are
> ready to start tackling another serviceability feature as a project, ask
> the community if that should be a separate project Mu[1], or if Atlantis
> should have it scope widened to also handle this second project.
>
> /Magnus
>
> [1] https://en.wikipedia.org/wiki/Mu_(lost_continent) :-)
>
> On 2018-11-26 20:22, JC Beyler wrote:
> > Hi Jonathan,
> >
> > Thanks for the interest and sounds good to me! We could start
> conversations
> > on this topic once the project opens and see what we could do in that
> > problem-space :)
> > Jc
> >
> > On Tue, Nov 20, 2018 at 7:46 PM Jonathan Lu <
> chuansheng.lcs at alibaba-inc.com>
> > wrote:
> >
> >> Hi JC,
> >>
> >> Personally, I like this idea.
> >>
> >> And I'm especially interested in the heap dumping part.
> >> It costs a lot to create, transfer, and analyze huge heap dump files for
> >> my applications.
> >> Maybe some fundamental changes can be explored in this project, I guess,
> >> such as making an enhanced format of heap dump file :)
> >>
> >> Thank you
> >> Jonathan
> >>
> >> ------------------------------------------------------------------
> >> From:JC Beyler <jcbeyler at google.com>
> >> Send Time:2018年11月20日(星期二) 02:48
> >> To:discuss <discuss at openjdk.java.net>
> >> Cc:serguei.spitsyn <serguei.spitsyn at oracle.com>
> >> Subject:Call For Discussion: New Project: Atlantis
> >>
> >> Hi all,
> >>
> >> I would like to start a discussion about the possible creation of the
> >> Atlantis Project with myself, Jean Christophe Beyler, as the Lead and
> the
> >> Serviceability group as the sponsoring group.
> >>
> >> In accordance with the OpenJDK guidelines [1], this project would
> provide a
> >> venue to explore and incubate monitoring and performance analysis
> features
> >> that could be integrated into the Hotspot JVM and the JVM Tool
> Interface.
> >> This includes working, evaluating, and incubating a Thread-Sanitizer
> >> implementation [2] for Java.
> >>
> >> The Google platform team has historically worked on providing new
> features,
> >> including monitoring hooks due to certain monitoring features not being
> >> available in a low-overhead, turned on-by-default manner. Therefore, for
> >> Google Java users, the platform team augmented or added mechanisms to
> >> assess various metrics such as but not limited to:
> >>
> >> - Thread Sanitizer support for Java (see JDK-8208520 [2]) (though this
> one
> >> specifically is not a low-overhead one, sorry ;-))
> >> - Lock contention profiles to better understand where threads are
> spending
> >> time in locks
> >> - Heap dumping mechanisms to improve heap dump times for large heaps
> >> - Efficient thread tagging to help filter profiling
> >> - etc.
> >>
> >> All of the above have added changes to the JDK in various forms and the
> >> Atlantis project proposes to investigate, in the open, how these changes
> >> could be made general enough to be pushed into the mainline, or should
> be
> >> dropped due to being too specific or could be done using other existing
> >> mechanisms. The project would not be a Google-only project and will
> have a
> >> mailing list to discuss and potentially accept any contribution trying
> to
> >> extend and/or enhance current monitoring systems.
> >>
> >> The evaluation criteria to consider a feature/improvement includes but
> >> would not be limited to:
> >>
> >>       * Performance: metrics and new hooks should not be hurting
> performance
> >> when disabled
> >>       * General usefulness for the community at large
> >>       * No working alternatives: sometimes it may be that there are new
> >> technologies available (ie JFR events for example) that might be
> leveraged
> >> to deprecate existing solutions
> >>
> >> If one or more prototypes indicate a new metric, a new monitoring
> system,
> >> or a feature offers substantial improvements, the Atlantis project would
> >> shepherd a JEP to integrate the change into the mainline of OpenJDK.
> >>
> >> The current suggested initial Reviewers and Committers would be but is
> most
> >> likely not going to be limited to:
> >> * Jean Christophe Beyler
> >> * Serguei Spitsyn
> >> * Chuck Rasbold
> >> * Liam Cushon
> >>
> >> What do you all think? :-)
> >>
> >> Thanks,
> >> Jc
> >>
> >> [1] http://openjdk.java.net/projects/#new-project
> >> [2] https://bugs.openjdk.java.net/browse/JDK-8208520
> >>
> >>
> >>
>
>

-- 

Thanks,
Jc