Call For Discussion: New Project: Atlantis

Fri Nov 30 12:01:10 UTC 2018

Looks good to me.
/Jesper

> On 29 Nov 2018, at 23:25, JC Beyler <jcbeyler at google.com> wrote:
> 
> Hi Magnus,
> 
> Thanks for the observation and it is much appreciated. You are probably
> right and an easier means to get us going would be to start with a TSAN
> specific project and then we can figure out what to do with the next
> feature that might require to be hosted via a project (ie should it be
> under Atlantis or under Mu).
> 
> This would mean that the CFD shifts itself to this new project description:
> 
> -------------------------------------------------------
> 
> I would like to start a discussion about the possible creation of the
> Atlantis Project with myself, Jean Christophe Beyler, as the Lead and the
> Serviceability group as the sponsoring group.
> 
> In accordance with the OpenJDK guidelines [1], this project would provide a
> venue to explore and incubate a Thread Sanitizing feature that could be
> integrated into the Hotspot JVM and the JVM Tool Interface. This includes
> working, evaluating, and incubating a Thread-Sanitizer implementation [2]
> for Java.
> 
> The Google platform team has historically worked on a TSAN project
> internally. The Atlantis project proposes to investigate, in the open, how
> these changes could be made general enough to be pushed into the mainline,
> or should be dropped due to being too specific or could be done using other
> existing mechanisms. The project would not be a Google-only project and
> will have a mailing list to discuss and potentially accept any contribution
> trying to extend and/or enhance the TSAN implementation.
> 
> The evaluation criteria for the TSAN implementation includes but would not
> be limited to:
>     * Performance: not adding overhead to non-tsan runs is paramount and
> there are various means to do so
>     * General usefulness for the community at large so that the TSAN
> feature is deemed a good feature to have
> 
> If the TSAN implementation becomes stable and is deemed useful enough, the
> Atlantis project would shepherd a JEP to integrate the change into the
> mainline of OpenJDK.
> 
> The current suggested initial Reviewers and Committers would be but is most
> likely not going to be limited to:
> * Jean Christophe Beyler
> * Serguei Spitsyn
> * Chuck Rasbold
> * Liam Cushon
>  ... TBD ...
> 
> [1] http://openjdk.java.net/projects/#new-project
> [2] https://bugs.openjdk.java.net/browse/JDK-8208520
> 
> ---------------------------------
> 
> What do you all think? :-)
> 
> Thanks,
> Jc
> 
> On Wed, Nov 28, 2018 at 1:41 AM Magnus Ihse Bursie <
> magnus.ihse.bursie at oracle.com> wrote:
> 
>> Hi JC,
>> 
>> Just a simple observation. Creating a project for the purpose of working
>> on TSAN should really be a no-brainer. I don't think anyone would object
>> to that. There are many projects like this, e.g. Portola, where there is
>> a need for the infrastructure OpenJDK grants a project (repo, mailing
>> list).
>> 
>> 
>> I think what maybe makes this proposal slightly more controversial is
>> the broad and not well delimited scope. Maybe a simpler route would be
>> to retarget your ideas for Atlantis from "any kind of servicability
>> stuff that needs a project" to "trying to get TSAN in shape for
>> integration into OpenJDK". And then, later on, when you feel you are
>> ready to start tackling another serviceability feature as a project, ask
>> the community if that should be a separate project Mu[1], or if Atlantis
>> should have it scope widened to also handle this second project.
>> 
>> /Magnus
>> 
>> [1] https://en.wikipedia.org/wiki/Mu_(lost_continent) :-)
>> 
>> On 2018-11-26 20:22, JC Beyler wrote:
>>> Hi Jonathan,
>>> 
>>> Thanks for the interest and sounds good to me! We could start
>> conversations
>>> on this topic once the project opens and see what we could do in that
>>> problem-space :)
>>> Jc
>>> 
>>> On Tue, Nov 20, 2018 at 7:46 PM Jonathan Lu <
>> chuansheng.lcs at alibaba-inc.com>
>>> wrote:
>>> 
>>>> Hi JC,
>>>> 
>>>> Personally, I like this idea.
>>>> 
>>>> And I'm especially interested in the heap dumping part.
>>>> It costs a lot to create, transfer, and analyze huge heap dump files for
>>>> my applications.
>>>> Maybe some fundamental changes can be explored in this project, I guess,
>>>> such as making an enhanced format of heap dump file :)
>>>> 
>>>> Thank you
>>>> Jonathan
>>>> 
>>>> ------------------------------------------------------------------
>>>> From:JC Beyler <jcbeyler at google.com>
>>>> Send Time:2018年11月20日(星期二) 02:48
>>>> To:discuss <discuss at openjdk.java.net>
>>>> Cc:serguei.spitsyn <serguei.spitsyn at oracle.com>
>>>> Subject:Call For Discussion: New Project: Atlantis
>>>> 
>>>> Hi all,
>>>> 
>>>> I would like to start a discussion about the possible creation of the
>>>> Atlantis Project with myself, Jean Christophe Beyler, as the Lead and
>> the
>>>> Serviceability group as the sponsoring group.
>>>> 
>>>> In accordance with the OpenJDK guidelines [1], this project would
>> provide a
>>>> venue to explore and incubate monitoring and performance analysis
>> features
>>>> that could be integrated into the Hotspot JVM and the JVM Tool
>> Interface.
>>>> This includes working, evaluating, and incubating a Thread-Sanitizer
>>>> implementation [2] for Java.
>>>> 
>>>> The Google platform team has historically worked on providing new
>> features,
>>>> including monitoring hooks due to certain monitoring features not being
>>>> available in a low-overhead, turned on-by-default manner. Therefore, for
>>>> Google Java users, the platform team augmented or added mechanisms to
>>>> assess various metrics such as but not limited to:
>>>> 
>>>> - Thread Sanitizer support for Java (see JDK-8208520 [2]) (though this
>> one
>>>> specifically is not a low-overhead one, sorry ;-))
>>>> - Lock contention profiles to better understand where threads are
>> spending
>>>> time in locks
>>>> - Heap dumping mechanisms to improve heap dump times for large heaps
>>>> - Efficient thread tagging to help filter profiling
>>>> - etc.
>>>> 
>>>> All of the above have added changes to the JDK in various forms and the
>>>> Atlantis project proposes to investigate, in the open, how these changes
>>>> could be made general enough to be pushed into the mainline, or should
>> be
>>>> dropped due to being too specific or could be done using other existing
>>>> mechanisms. The project would not be a Google-only project and will
>> have a
>>>> mailing list to discuss and potentially accept any contribution trying
>> to
>>>> extend and/or enhance current monitoring systems.
>>>> 
>>>> The evaluation criteria to consider a feature/improvement includes but
>>>> would not be limited to:
>>>> 
>>>>      * Performance: metrics and new hooks should not be hurting
>> performance
>>>> when disabled
>>>>      * General usefulness for the community at large
>>>>      * No working alternatives: sometimes it may be that there are new
>>>> technologies available (ie JFR events for example) that might be
>> leveraged
>>>> to deprecate existing solutions
>>>> 
>>>> If one or more prototypes indicate a new metric, a new monitoring
>> system,
>>>> or a feature offers substantial improvements, the Atlantis project would
>>>> shepherd a JEP to integrate the change into the mainline of OpenJDK.
>>>> 
>>>> The current suggested initial Reviewers and Committers would be but is
>> most
>>>> likely not going to be limited to:
>>>> * Jean Christophe Beyler
>>>> * Serguei Spitsyn
>>>> * Chuck Rasbold
>>>> * Liam Cushon
>>>> 
>>>> What do you all think? :-)
>>>> 
>>>> Thanks,
>>>> Jc
>>>> 
>>>> [1] http://openjdk.java.net/projects/#new-project
>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8208520
>>>> 
>>>> 
>>>> 
>> 
>> 
> 
> --
> 
> Thanks,
> Jc