[8u] Request for enhancement backport approval for 8210352: [hotspot] --with-extra-cxx-flags should not lower optimization

Mon Sep 10 18:04:23 UTC 2018

On Thu, 6 Sep 2018 at 10:46, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>
> On Wed, 2018-09-05 at 20:06 +0100, Andrew Hughes wrote:
> > On Wed, 5 Sep 2018 at 09:55, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > >
> > > Hi,
> > >
> > > I've been asked to proceed with the enhancement request process[1] for
> > > this proposed JDK 8u fix.
> > >
> > > Bug: https://bugs.openjdk.java.net/browse/JDK-8210352
> > >
> > > Currently JDK 8 and JDK 9+ behave differently in terms of hotspot
> > > libjvm optimization flags. In JDK 8 they can be overridden by
> > > --with-extra-c{,xx}-flags configure options. In JDK 9+ this cannot be
> > > done as libjvm optimization flags come after any extra C flags from
> > > configure. This enhancement request for JDK 8 asks for the build of the
> > > JVM to behave the same as JDK 9+.
> > >
> > > The rationale why we'd wanted such behaviour is as follows. In Fedora
> > > distribution wide config exists which sets C flags. Hooks are in place
> > > so that all natives get compiled with these C flags. One goal is to
> > > ensure all distro libraries and programs are being optimized at at
> > > least a certain threshold, say >= -O2. This includes OpenJDK.
> > >
> > > While there is a way to fix the build system to compile each and every
> > > library with some optimization level the risk is that there might not
> > > be a one-size-fits-all downstream distribution consumers. The easier
> > > and more flexible case is to allow for this via extra C flags which can
> > > be passed to the build. This is currently possible.
> > >
> > > The exception to this rule should be libjvm. Optimization levels for
> > > the JVM have been thoroughly tested and to some extent been lowered via
> > > a by-object overrides. Example case is
> > > sharedRuntimeTrans.o and sharedRuntimeTrig.o which have -O0
> > > settings[2]. The point is that for the JVM optimization flags have been
> > > thoroughly crafted to a point where they are today. Letting the user
> > > override this to some other value, like -O2, via a simple configure
> > > option seems dangerous. Currently in JDK 8 it's possible to override
> > > JVM optimization flags via --with-extra-c{,xx}-flags configure option.
> > >
> > > OK to move ahead with this enhancement for JDK 8u?
> > >
> > > Thanks,
> > > Severin
> > >
> > > [1] http://mail.openjdk.java.net/pipermail/build-dev/2018-September/023064.html
> > > [2] http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/80ee2541504e/make/linux/makefiles/amd64.make#l25
> > >
> >
> > I tend to agree with David's comment:
> >
> > "If I set an extra cflag I expect it to be applied to all compilations
> > - cross-compilation would be
> > broken if that were not the case. I don't see why -On should be
> > treated any differently." [0]
>
> This seems to be a comment on an old webrev. It does not apply to the
> updated one here which adds the OPT flags *after* user CFLAGS for the
> JVM:
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210352/webrev.02/
>
> As to "the flags should get applied to all compilations" issue: I'd
> think that's not an issue for the updated webrev, is it? They get
> applied, but OPT flags come before any JVM set flags.

David's comment is on an earlier version, yes, but it still applies.
Specifically, 'I don't see why -On should be treated any differently.'.
By adding -On flags after user -On flags and overriding them, it is
being treated differently from earlier flags, which user cflags would
be able to override.

> > and we've encountered the same issue in Gentoo in the past [1].
> >
>
> > Your example seems a little disingenuous, especially given you're
> > proposing to remove
> > those -O0 examples in another bug [2].
>
> Fair point. I can come up with others. The hotspot build seems to be
> sprinkled with optimization overrides:
>
> http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/80ee2541504e/make/solaris/makefiles/sparc.make#l34
> http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/80ee2541504e/make/linux/makefiles/gcc.make#l259
> http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/80ee2541504e/make/linux/makefiles/fastdebug.make#l35
>
> They could be summarized as work-arounds for GCC bugs. If user
> supplied -O flags override all these, then they might get broken
> builds. I question that's a good approach...

Then that's the choice of the user. Given that two of these are for
old compilers, and the other for an unsupported architecture, I think
the risk is pretty low. Even lower when you consider these failures
are probably based on compiling with -O3.

>
> > Given that the code is
> > generally compiled with -O3,
> > the more common effect of overriding the optimisation is going to be
> > to lower it.
>
> Since the hotspot build already has slowdebug as a build option I'm not
> sure. Do we really have data that somebody wants release builds with
> -O2 (over -O3) with the risk of getting some mis-compiled objects in
> the JVM?

slowdebug means not just no optimisation, but turning on a raft of assertion
code paths. It's a much bigger step than just toning down the optimisation.

The bug that brought you to this webrev in the first place was because -O2
was being ignored in your builds. Unless I misunderstand it, this entire
fix seems to be aimed at allowing you to pass -O2 and then have it bumped
to -O3 by the build in most cases. Wouldn't it make more sense to fix the
few errant cases that lack optimisation in release builds rather than altering
everyone's build to suit your use case?

<snip as it's basically just the same argument>

>
> It seems we disagree. It looks like the better choice to me. After all,
> user defined global optimization for the JVM might produce wrong code.
> Doesn't that make it too easy for somebody to shoot themselves in the
> foot?

I think the fundamental issue here is whether you assume the user to
be a fool or not. In my experience, it's far more likely that someone
will pass arguments to the build for whatever reason, and then get annoyed
when the build blindly ignores them, than they will override optimisations
and then complain when it breaks things. In the latter case, they've made
the choice and get to keep the pieces. There's very much a control
argument here rather than a technical one.

How likely is it that someone will pass their own optimisation in the first
place? The original problem here feels contrived, because you essentially
want to pass your own optimisation flags, but then not use them in the
majority of cases.

>
> > The right solution here seems to me to be to add optimisations to
> > those areas that
> > lack it, in appropriate build modes. It seems likely to me that jsig
> > and saproc were
> > overlooked because of the use of separate makefiles in the old HotSpot
> > build more
> > than anything else (as seems to have been the case with execstack in 8187045).
>
> Sure. It can be done. In that case we'd have to decide on some
> appropriate level of optimization for those. Perhaps -O2?

I think that's a much better solution. -O2 seems a good compromise for
older builds, where no optimisation has been the norm, but perhaps they
could use the system-wide -O3 on 12.

Have you done much testing with these compiled with optimisation? The main
thing we need to ascertain is whether their exclusion was a mistake or
a deliberate choice because it breaks these tools.

>
> > Are they optimised in 9? Or is there any evidence of breakage caused by
> > using optimisation there?
>
> They don't seem to be on 12:
>
> $ grep -C3 SetupJdkLibrary make/lib/Lib-jdk.hotspot.agent.gmk
>
> ################################################################################
>
> $(eval $(call SetupJdkLibrary, BUILD_LIBSA, \
>     NAME := saproc, \
>     OPTIMIZATION := NONE, \
>     DISABLED_WARNINGS_microsoft := 4267, \
>
> I haven't checked on 9. I don't think there is evidence of breakage
> caused by optimization as nobody likely uses optimization for those.
> That doesn't mean it might not break, though :-)

Indeed :-)

So what we really need to be looking at here is these particular cases
and whether they can be optimised, rather than trying to alter the whole
build so that these cases are inadvertently optimised when someone
passes in -O2.

And yes, I should have said 9+ rather than 9. I doubt there are that
many build changes in this area in the last year.

>
> Thanks,
> Severin
>
> > [0] http://mail.openjdk.java.net/pipermail/build-dev/2018-September/023060.html
> > [1] https://bugs.gentoo.org/356991
> > [2] https://bugs.openjdk.java.net/browse/JDK-8210425
>

-- 
Andrew :)

Senior Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Web Site: http://fuseyism.com
Twitter: https://twitter.com/gnu_andrew_java
PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222