Problems in Dist.gmk with --enable-new-hotspot-build

[Erik Joelsson]

Hello Volker,

Very nice to see you are on top of this!

I agree with your assessment of the failure and it is indeed an error to 
use SetupCopyFiles in that way. Splitting into two calls is the right 
solution.

I also hope you aren't too scared by the current state of the new 
makefiles, there is still a lot of work to be done to make them nice and 
tidy.

About making you committer, I would love to. I suppose we have to 
propose and vote about it just like with any other project?

/Erik

On 2015-04-29 10:07, Volker Simonis wrote:
> Hi,
>
> I'm currently trying to adopt the new HotSpot make system to ppc64.
>
> Linux/ppc64 seems quite simple and I get up to the COPY_INCLUDE step in
> Dist.gmk where it fails with:
>
> SetupCopyFiles COPY_INCLUDE
>   [2] SRC := /usr/work/d046063/OpenJDK/jdk9-build
>   [3] DEST :=
> /priv/d046063/OpenJDK/output-jdk9-build-dbg/hotspot/dist/include
>   [4] FLATTEN := true
>   [5] FILES :=
> /usr/work/d046063/OpenJDK/jdk9-build/hotspot/src/share/vm/prims/jni.h
> /priv/d046063/OpenJDK/output-jdk9-build-dbg/hotspot/gensrc/jvmtifiles/jvmti.h
> /usr/work/d046063/OpenJDK/jdk9-build/hotspot/src/share/vm/code/jvmticmlr.h
> /usr/work/d046063/OpenJDK/jdk9-build/hotspot/src/share/vm/services/jmm.h
>
> gmake[4]: Entering directory
> `/net/usr.work/d046063/OpenJDK/jdk9-build/hotspot/makefiles'
> /bin/echo Copying files to dist directory
> Copying files to dist directory
> gmake[4]: *** No rule to make target
> `/usr/work/d046063/OpenJDK/jdk9-build//priv/d046063/OpenJDK/output-jdk9-build-dbg/hotspot/gensrc/jvmtifiles/jvmti.h',
> needed by
> `/priv/d046063/OpenJDK/output-jdk9-build-dbg/hotspot/dist/include/jvmti.h'.
> Stop.
>
> But this also fails in the same way if I build on Linux/amd64.
>
> The problem seems to be in Dist.gmk:
>
> $(eval $(call SetupCopyFiles, COPY_INCLUDE, \
>      SRC := $(SRC_ROOT), \
>      DEST := $(DIST_OUTPUTDIR)/include, \
>      FLATTEN := true, \
>      FILES := $(HOTSPOT_TOPDIR)/src/share/vm/prims/jni.h \
>          $(HOTSPOT_OUTPUTDIR)/gensrc/jvmtifiles/jvmti.h \
>          $(HOTSPOT_TOPDIR)/src/share/vm/code/jvmticmlr.h \
>          $(HOTSPOT_TOPDIR)/src/share/vm/services/jmm.h))
>
> where we mix files from the source directory (e.g.
> $(HOTSPOT_TOPDIR)/src/share/vm/prims/jni.h) with files from the output
> directory ($(HOTSPOT_OUTPUTDIR)/gensrc/jvmtifiles/jvmti.h) which doesn't
> seem to work.
>
> SetupCopyFiles has the following comment:
>
> #   FILES   : List of files to copy with absolute paths, or path relative
> to SRC.
> #             Must be in SRC.
>
> And looking at the implementation of SetupCopyFiles I can see that first
> SRC is substituted from every FILES argument and later on the remaining
> file part is appended to SRC again. However if a file in FILES isn't
> located under SRC this can not work and results in a non-existing path like
> "/usr/work/d046063/OpenJDK/jdk9-build//priv/d046063/OpenJDK/output-jdk9-build-dbg/hotspot/gensrc/jvmtifiles/jvmti.h"
> in my example which is a concatenation of SRC with the complete absolute
> file path from FILES.
>
> Maybe you haven't seen this because you always build into a subdirectory of
> the source tree?
> Nevertheless I think this should be fixed.
>
> The easiest fix I can come up is to just divide the COPY_INCLUDE step into
> two steps like:
>
> $(eval $(call SetupCopyFiles, COPY_INCLUDE, \
>      SRC := $(SRC_ROOT), \
>      DEST := $(DIST_OUTPUTDIR)/include, \
>      FLATTEN := true, \
>      FILES := $(HOTSPOT_TOPDIR)/src/share/vm/prims/jni.h \
>          $(HOTSPOT_TOPDIR)/src/share/vm/code/jvmticmlr.h \
>          $(HOTSPOT_TOPDIR)/src/share/vm/services/jmm.h))
>
> TARGETS += $(COPY_INCLUDE)
>
> $(eval $(call SetupCopyFiles, COPY_GENERATED_INCLUDE, \
>      DEST := $(DIST_OUTPUTDIR)/include, \
>      FLATTEN := true, \
>      FILES := $(HOTSPOT_OUTPUTDIR)/gensrc/jvmtifiles/jvmti.h))
>
> TARGETS += $(COPY_GENERATED_INCLUDE)
>
> With this change I could successfully build HotSpot with the new build
> system on Linux/amd64 (and with my other ppc-related changes also on
> Linux/ppc64).
>
> My configure line is:
>
> /priv/d046063/OpenJDK/output-jdk9-build-dbg$ bash
> /usr/work/d046063/OpenJDK/jdk9-build/configure
> --with-boot-jdk=/usr/work/openjdk/nb/linuxppc64/last_known_good/output-jdk8/images/j2sdk-image
> --with-jvm-variants=server --with-target-bits=64
> --with-debug-level=slowdebug --disable-zip-debug-info
> --enable-new-hotspot-build
>
> Notice that the build and the source directory have no common root!
>
> My build command is:
>
> make hotspot-only LOG=debug
>
> If you'd make me a committer of the build-infr project I'd be happy to
> submit this fix along with the up-coming ppc fixes right into the
> build-infr repo :)
>
> Regards,
> Volker