Call For Discussion: New Project: Atlantis

[Magnus Ihse Bursie]

Hi JC,

Just a simple observation. Creating a project for the purpose of working 
on TSAN should really be a no-brainer. I don't think anyone would object 
to that. There are many projects like this, e.g. Portola, where there is 
a need for the infrastructure OpenJDK grants a project (repo, mailing 
list).


I think what maybe makes this proposal slightly more controversial is 
the broad and not well delimited scope. Maybe a simpler route would be 
to retarget your ideas for Atlantis from "any kind of servicability 
stuff that needs a project" to "trying to get TSAN in shape for 
integration into OpenJDK". And then, later on, when you feel you are 
ready to start tackling another serviceability feature as a project, ask 
the community if that should be a separate project Mu[1], or if Atlantis 
should have it scope widened to also handle this second project.

/Magnus

[1] https://en.wikipedia.org/wiki/Mu_(lost_continent) :-)

On 2018-11-26 20:22, JC Beyler wrote:
> Hi Jonathan,
>
> Thanks for the interest and sounds good to me! We could start conversations
> on this topic once the project opens and see what we could do in that
> problem-space :)
> Jc
>
> On Tue, Nov 20, 2018 at 7:46 PM Jonathan Lu <chuansheng.lcs at alibaba-inc.com>
> wrote:
>
>> Hi JC,
>>
>> Personally, I like this idea.
>>
>> And I'm especially interested in the heap dumping part.
>> It costs a lot to create, transfer, and analyze huge heap dump files for
>> my applications.
>> Maybe some fundamental changes can be explored in this project, I guess,
>> such as making an enhanced format of heap dump file :)
>>
>> Thank you
>> Jonathan
>>
>> ------------------------------------------------------------------
>> From:JC Beyler <jcbeyler at google.com>
>> Send Time:2018年11月20日(星期二) 02:48
>> To:discuss <discuss at openjdk.java.net>
>> Cc:serguei.spitsyn <serguei.spitsyn at oracle.com>
>> Subject:Call For Discussion: New Project: Atlantis
>>
>> Hi all,
>>
>> I would like to start a discussion about the possible creation of the
>> Atlantis Project with myself, Jean Christophe Beyler, as the Lead and the
>> Serviceability group as the sponsoring group.
>>
>> In accordance with the OpenJDK guidelines [1], this project would provide a
>> venue to explore and incubate monitoring and performance analysis features
>> that could be integrated into the Hotspot JVM and the JVM Tool Interface.
>> This includes working, evaluating, and incubating a Thread-Sanitizer
>> implementation [2] for Java.
>>
>> The Google platform team has historically worked on providing new features,
>> including monitoring hooks due to certain monitoring features not being
>> available in a low-overhead, turned on-by-default manner. Therefore, for
>> Google Java users, the platform team augmented or added mechanisms to
>> assess various metrics such as but not limited to:
>>
>> - Thread Sanitizer support for Java (see JDK-8208520 [2]) (though this one
>> specifically is not a low-overhead one, sorry ;-))
>> - Lock contention profiles to better understand where threads are spending
>> time in locks
>> - Heap dumping mechanisms to improve heap dump times for large heaps
>> - Efficient thread tagging to help filter profiling
>> - etc.
>>
>> All of the above have added changes to the JDK in various forms and the
>> Atlantis project proposes to investigate, in the open, how these changes
>> could be made general enough to be pushed into the mainline, or should be
>> dropped due to being too specific or could be done using other existing
>> mechanisms. The project would not be a Google-only project and will have a
>> mailing list to discuss and potentially accept any contribution trying to
>> extend and/or enhance current monitoring systems.
>>
>> The evaluation criteria to consider a feature/improvement includes but
>> would not be limited to:
>>
>>       * Performance: metrics and new hooks should not be hurting performance
>> when disabled
>>       * General usefulness for the community at large
>>       * No working alternatives: sometimes it may be that there are new
>> technologies available (ie JFR events for example) that might be leveraged
>> to deprecate existing solutions
>>
>> If one or more prototypes indicate a new metric, a new monitoring system,
>> or a feature offers substantial improvements, the Atlantis project would
>> shepherd a JEP to integrate the change into the mainline of OpenJDK.
>>
>> The current suggested initial Reviewers and Committers would be but is most
>> likely not going to be limited to:
>> * Jean Christophe Beyler
>> * Serguei Spitsyn
>> * Chuck Rasbold
>> * Liam Cushon
>>
>> What do you all think? :-)
>>
>> Thanks,
>> Jc
>>
>> [1] http://openjdk.java.net/projects/#new-project
>> [2] https://bugs.openjdk.java.net/browse/JDK-8208520
>>
>>
>>