Parallelizing the full compaction of CMS
Jon Masamitsu
jon.masamitsu at oracle.com
Mon Jun 8 16:00:19 UTC 2015
On 6/8/2015 4:11 AM, Jungwoo Ha wrote:
> Hi Volker,
>
> I am going to post the code some time this week.
> Stay tuned.
Jungwoo,
Will this be based on jdk8 or jdk9?
Jon
>
> --Jungwoo
>
>
> On Mon, Jun 8, 2015 at 3:04 AM, Volker Simonis
> <volker.simonis at gmail.com <mailto:volker.simonis at gmail.com>> wrote:
>
> Hi John, Jeremy,
>
> @John: sorry, but I don't see what's the big problem behind "supported
> and maintained by the community". We already have quite some code in
> the HotSpot which is developed and maintained that way. Take for
> example the C++-Interpreter. It has not been used and supported by
> Sun/Oracle since 1.4 where it was used for the Itanium port.
> Nevertheless it is quite useful for HotSpot ports to new platforms. We
> at SAP used it for our intitial PowerPC port for example. And of
> course we supported it. This includes both fixes as well as the
> implementation of new features (e.g. supports for JSR 292). Another
> user of the C++-interpreter is the Zero-port which is supported mainly
> by RedHat. Again, Oracle does neither build nor test the Zero-port.
> This is all done by RedHat. But it is still a crucial part of the
> OpenJDK for platforms like Linux on zArch, MIPS, Itanium,....
>
> Then we (i.e. SAP and IBM) have our ports (Linux/PPC64, AIX) and
> RedHat have their AArch64 port. Neither of them is supported by Oracle
> but all of them are pretty well supported by their maintainers. We
> have our own platform categories in the bug system, our OpenJDK
> mailing lists. We do nightly builds and tests and we take all this
> pretty seriously. And that all imposes nearly zero overhead on Oracle
> but rather helps to strengthen the VM on all platforms (e.g. we
> contributed quite some GC-fixes which only showed up on a weak memory
> architecture like Power or helped to clean up the code because we use
> different compiler on other platforms).
>
> I don't think it is appropriate for an Open Source project to reject a
> new feature just because one of the maintainers doesn't want it. And
> the OpenJDK is an OpenSource project with a distinct development model
> as specified in the OpenJDK bylaws (http://openjdk.java.net/bylaws).
> But I don't want to go on collision course. Until now the cooperation
> between the various contributors in the OpenJDK has always worked
> quite well. And if a feature can really be hidden behind a flag
> without impacting the other configurations that's great. Of course
> Oracle would not have to maintain or even test it (there already exist
> a myriad of unsupported configurations today :). And I think it is
> self-evident that contributors feel responsible for bugs in their code
> and fix them (as we already do today). So the only commitment on
> Oracle part would be to help with the initial review process which I'd
> consider acceptable.
>
> @Jeremy: could you please post some code so we can see what we're
> really talking about here. It seems that until now nobody ever saw
> these changes. And maybe it helps if several parties are trying
> jointly to 'convince' the maintainer to consider a patch :)
>
> Regards,
> Volker
>
>
> On Fri, Jun 5, 2015 at 5:56 PM, Jon Masamitsu
> <jon.masamitsu at oracle.com <mailto:jon.masamitsu at oracle.com>> wrote:
> >
> >
> > On 6/3/2015 1:00 PM, Jeremy Manson wrote:
> >
> > I'm not on hotspot-gc-dev, so I'm not seeing the responses. But
> I looked
> > online...
> >
> > Jon says that "My understanding was that the change was based on a
> > parallelization of the serial mark-sweep-compact (used by CMS
> for full
> > collections). If it was based on the parallel-old compaction used by
> > ParallelGC, then my bad. It should have been reviewed."
> >
> > His original understanding was correct - we parallelized the serial
> > mark-sweep-compact used by CMS. I'm sorry if I misled anyone there.
> >
> >
> > No problem. It gave me a chance to say again that
> > I'd rather not have to support two implementations of
> > a parallel full stop-the-world collector. We have that
> > issue with the two implementations of a parallel young
> > collector and I'd really rather not go there again.
> >
> >
> > Otherwise, thanks for your thoughtful reply, Volker. In my
> experience of
> > open source projects, if you can't get a maintainer to agree to
> consider
> > your patch, then there is really very little that you can do.
> And I totally
> > get that maintainers don't want / have the time for the burden of
> > understanding huge amounts of complicated code they didn't write!
> >
> > So, the question comes down to what "community supported" means
> in this
> > case. We can't actually submit anything to Hotspot without
> approval from
> > someone within Oracle. We have a vested interest in keeping
> this patch
> > working, but, for major JDK releases, we really only start
> stress testing to
> > identify problems with it when we upgrade to the latest versions
> of the JDK,
> > which usually happens some months after Oracle releases it.
> That could
> > cause some issues for upstream.
> >
> > The upside is that I think we'd be willing to do what it takes,
> if getting
> > it upstreamed is a possibility. This patch is a huge nuisance,
> and it is
> > much simpler not to have to do a week or two's worth of forward
> porting
> > every year. And whatever problems arise are problems that we
> have to fix
> > anyway.
> >
> > Jon, if you are particularly worried, note that we can keep it
> hidden behind
> > a flag without too much difficulty (and did for a year or two).
> >
> >
> > I don't think that helps much. Unless Google is willing to own any
> > bug reports that come in on CMS when it is turned on. My main
> > concern is long term support. And whether we would have to
> > do our testing with it both turned on and turned off. Both those
> > seem like a big commitment on Oracle's part. Only my opinion,
> > of course.
> >
> > Jon
> >
> >
> > Jeremy
> >
> > On Wed, Jun 3, 2015 at 2:32 AM, Volker Simonis
> <volker.simonis at gmail.com <mailto:volker.simonis at gmail.com>>
> > wrote:
> >>
> >> Hi Jeremy,
> >>
> >> thanks for your answer. Maybe you misunderstood Ivan but it is
> definitely
> >> his and our (i.e. SAP's) intention to make this part of the
> OpenJDK.
> >>
> >> I can also fully understand the pains you have with integrating
> changes
> >> into the HotSpot. We've gone through this painful process with
> our ppc64
> >> port a year ago but we think it will finally pay off. After all
> you (i.e.
> >> Google) used it as the base for your ppc64le port (Alexander
> Smundak
> >> contributed most of the LE-specific changes).
> >>
> >> We're facing the exactly same problems regarding the support of
> >> proprietary changes in a downstream version of the OpenJDK.
> That was
> >> actually the main reason why we started contributing to the
> OpenJDK. What we
> >> want to kindly ask you is to make your changes and extensions more
> >> transparently visible to the community if you intend to
> integrate them into
> >> the OpenJDK. If Hiroshi had posted his changes directly to the
> hotspot
> >> mailing lists instead of sending them privately to Jon I'm
> pretty sure we
> >> would have picked them up. And maybe together we would have
> managed to push
> >> them through (we have some reviewers as well :). After all,
> there are
> >> already quite a lot of features in the OpenJDK which are
> supported and
> >> maintained by the community (i.e. RedHat, IBM, SAP, ...) and
> not by Oracle.
> >>
> >> I don't have much experience with other Open Source projects,
> but with
> >> OpenJDK and especially HotSpot it's pretty pointless to try to
> contribute
> >> something by writing mails like "..I've implemented this cool
> feature XXX.
> >> If somebody is interested I can post more details..". You'll
> almost never
> >> get any feedback to such kind of requests. You'll at least have
> to post a
> >> complete buildabel and runnable patch against the head
> revision. I know
> >> that's a lot of overhead and we're having this very same
> discussion if it is
> >> worth doing it nearly every day for most of the
> features/additions we are
> >> implementing. Nevertheless, I think it is worth doing it
> although it
> >> sometimes may take years until you get your changes back in the
> version you
> >> are using productively.
> >>
> >> So coming back to Ivan's initial request, we would kindly ask
> you to just
> >> publish your changes in their current form against your current
> version
> >> (8u45 or whatsoever). We'll take a look at them an may invest
> some time and
> >> effort (also jointly with you if desired) to forward port them
> to 9 and
> >> propose them for integration into the head revision if that
> makes sense.
> >> Thus that sound reasonable to you?
> >>
> >> Thank you and best regards,
> >> Volker
> >>
> >>
> >> On Tue, Jun 2, 2015 at 5:00 PM, Jeremy Manson
> <jeremymanson at google.com <mailto:jeremymanson at google.com>>
> >> wrote:
> >>>
> >>> We'd much rather it be part of OpenJDK, because it is large
> and fairly
> >>> complicated, so that would save us time and energy forward
> porting it every
> >>> year or so. It's pretty large (for 8u45, it is a 2600 line
> change and
> >>> touches 20 files), so it tends to be a lengthy and complicated
> forward port.
> >>> We would much rather spend that time doing something more
> interesting.
> >>>
> >>> Just posting an updated patch whenever we need to update it
> would be a
> >>> little annoying (although not impossible).
> >>>
> >>> Hiroshi Yamauchi tried to offer it to (I think it was) Jon
> Masamitsu a
> >>> few years ago, and Jon didn't know anyone with the bandwidth
> to review it.
> >>> (It was around the same time we contributed the parallel
> initial mark). I
> >>> believe that was when CMS was unowned for a while. Perhaps
> there's a
> >>> different story now? Oracle volunteers?
> >>>
> >>> (Procedural: One problem is that Hotspot reviews have to go
> through
> >>> someone at Oracle. We had it written by an OpenJDK author and
> reviewed by a
> >>> reviewer. We've been using it in production for years without
> too many
> >>> headaches (although we have yet to inflict^Wlaunch 8u45, which
> has the
> >>> latest updates, widely). I totally understand that Oracle
> folks are the
> >>> ones who have to pay the price for a bad check-in, but it's a
> pretty big
> >>> hurdle for us.)
> >>>
> >>> Jeremy
> >>>
> >>> On Tue, Jun 2, 2015 at 6:12 AM, Galkin, Ivan
> <ivan.galkin at sap.com <mailto:ivan.galkin at sap.com>> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>>
> >>>>
> >>>> in the thread „JEP 248: Make G1 the Default Garbage
> Collector” [1] there
> >>>> is an email from Jeremy Manson, who mentions the enhancements
> made by Google
> >>>> to improve CMS.
> >>>>
> >>>> Especially the parallelizing of full compaction is a great
> improvement
> >>>> and we at SAP see the strong demand of it.
> >>>>
> >>>>
> >>>>
> >>>> @Jeremy: Are these changes published somewhere? May we have
> insight into
> >>>> the diffs you've made? We could collaborate in order to try
> to bring your
> >>>> changes into OpenJDK.
> >>>>
> >>>>
> >>>>
> >>>> Kind regards and thank you in advance,
> >>>>
> >>>> Ivan
> >>>>
> >>>>
> >>>>
> >>>> [1]
> >>>>
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-June/018740.html
> >>>
> >>>
> >>
> >
> >
>
>
>
>
> --
> Jungwoo Ha | Java Platform Team | jwha at google.com <mailto:jwha at google.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20150608/de64a1a4/attachment.htm>
More information about the hotspot-gc-dev
mailing list