RFR (S): 8139921: add mx configuration files to support HotSpot IDE configuration generation

Fri Apr 8 18:31:39 UTC 2016

> On Apr 7, 2016, at 11:59 PM, Volker Simonis <volker.simonis at gmail.com> wrote:
> 
> On Thu, Apr 7, 2016 at 6:59 PM, Christian Thalinger
> <christian.thalinger at oracle.com <mailto:christian.thalinger at oracle.com>> wrote:
>> 
>> On Apr 7, 2016, at 2:22 AM, Volker Simonis <volker.simonis at gmail.com> wrote:
>> 
>> Hi Christian,
>> 
>> I'd like to try this but I couldn't figure out how it works.
>> I don't have any experience with Graal/Truffel/JVMCI but as far as I
>> understood the creation of an Eclipse project should work equally well
>> for a vanilla hospot repository, right?
>> 
>> 
>> Correct.
>> 
>> 
>> The first question is where to get mx from (there's different
>> information in the Wiki and this email thread for example) ?
>> 
>> https://bitbucket.org/allr/mx
>> oder
>> https://github.com/graalvm/mx.git
>> 
>> 
>> Since the review the mx repository got moved to github.  That’s the one you
>> want to use.
>> 
>> 
>> Now let's say I cloned the right mx version. How do I use it?
>> Let's say I have a vanilla jdk9 forest under /share/OpenJDK/jdk9
>> I create an output directory under /share/OpenJDK/output-jdk9-dbg and
>> from there I call configure and build the images:
>> 
>> cd /share/OpenJDK/output-jdk9-dbg
>> bash /share/OpenJDK/jdk9/configure --with-boot-jdk=..
>> --with-debug-level=slowdebug
>> make images
>> 
>> Taking this scenario, from which directory am I supposed to call 'mx
>> ideinit', where will the Eclipse project be created at and how is mx
>> supposed to find my configuration (i.e. platform) and generated files?
>> 
>> 
>> I have never tried to have the output directory not in the source directory
>> and mx might not support this.  But it’s not really necessary because
>> everything goes into build/ by default anyway.  If you really want to have a
>> separate output directory I suppose we could add an option to mx.
>> 
> 
> Hi Christian,
> 
> thanks a lot for your explanation. I've never tried WITH the output
> directory in the source tree :) I have my source repositories on a
> network share and on every platform I configure and build from the
> shared sources into a local directory.
> 
> That said, I've tried to configure and build INTO the source directory
> and called 'mx eclipseinit' afterwards. That worked as described and
> the created project worked quite nicely. But after playing around a
> little bit, I found the following points:
> 
> - the project settings are hard-wired for linux-x86_64
> - so it makes no difference if I configure and build on linux-ppc64,
> the generated project still tries to get the generated files from
> 'PARENT-5-PROJECT_LOC/build/linux-x86_64-normal-server-slowdebug'
> which will apparently not work on any other platform.

That is half-correct.  It’s not hardcoded but it picks up the platform you are running mx on.  So, if you had a separate repository for ppc64 and run mx on the ppc64 host it should work.

I mean, there is no real solution to that problem, right?  You can only have one generated directory to be included.

> - the same is true for the preprocessor defines. They are hard wired
> in ".mx.jvmci/hotspot/templates/eclipse/cproject". I don't actually
> understand why you are creating distinct release, fastdebug and
> slowdebug projects at all

The main reason for the different configurations is because the generated directory is different.

> , if all of them have the same settings (e.g.
> ASSERT and DEBUG is also defined for server-release).

This is odd and sounds like a bug.  Doug?  Ah!  There is only one template:

$ find .mx.jvmci/hotspot/templates/
.mx.jvmci/hotspot/templates/
.mx.jvmci/hotspot/templates/eclipse
.mx.jvmci/hotspot/templates/eclipse/cproject
.mx.jvmci/hotspot/templates/eclipse/settings
.mx.jvmci/hotspot/templates/eclipse/settings/org.eclipse.cdt.core.prefs
.mx.jvmci/hotspot/templates/eclipse/settings/org.eclipse.cdt.ui.prefs
.mx.jvmci/hotspot/templates/eclipse/settings/org.eclipse.core.runtime.prefs

A much bigger problem is this:

$ grep TARGET .mx.jvmci/hotspot/templates/eclipse/cproject
                                                                        <listOptionValue builtIn="false" value="TARGET_ARCH_x86=1"/>
                                                                        <listOptionValue builtIn="false" value="TARGET_COMPILER_gcc=1"/>
                                                                        <listOptionValue builtIn="false" value="TARGET_OS_FAMILY_linux=1"/>
                                                                        <listOptionValue builtIn="false" value="TARGET_OS_ARCH_linux_x86=1"/>
                                                                        <listOptionValue builtIn="false" value="TARGET_ARCH_MODEL_x86_64=1"/>

Doug, can we change that to actually use the right defines?

> This of course
> also applies to OS, CPU, etc...
> - so in the end you're right. I can call 'mx eclipseinit' even without
> a build directory and the generated Eclipse project will still be the
> same (i.e. hard-wired for ../build/linux-x86_64-slowdebug).
> 
> I thought that 'mx eclipseinit' will use the output directory to get
> platform and configuration information from there and to create the
> corresponding Eclipse project from it. That means a project which is
> configured for the corresponding platform with the relevant (and only
> the relevant) sources plus preprocessor defines and the corresponding
> generated sources. But in the current from, that doesn't seems to be
> true. Or am I missing something?

As I said above, mx is picking up the information from the host you are running on:

http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/7d9d8ad32fe6/.mx.jvmci/mx_jvmci.py#l718

Here is the configuration when running mx on a SPARC host:

.mx.jvmci/hotspot/eclipse/server-release/.project
61:                    <name>generated</name>
63:                    <locationURI>PARENT-5-PROJECT_LOC/build/solaris-sparcv9-normal-server-release/hotspot/solaris_sparcv9_compiler2/generated</locationURI>

> 
> Thank you and best regards,
> Volker
> 
>> You run mx in the hotspot source directory.  Since I’m using Eclipse I
>> usually run the eclipseinit command directly:
>> 
>> cthaling at macbook:~/ws/jdk9/hs-comp/hotspot$ mx eclipseinit
>> created
>> /Users/cthaling/ws/jdk9/hs-comp/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/.classpath
>> created
>> /Users/cthaling/ws/jdk9/hs-comp/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/.checkstyle
>> created
>> /Users/cthaling/ws/jdk9/hs-comp/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/.project
>> created
>> /Users/cthaling/ws/jdk9/hs-comp/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/.settings/org.eclipse.jdt.core.prefs
>> created
>> /Users/cthaling/ws/jdk9/hs-comp/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/.settings/org.eclipse.jdt.ui.prefs
>> created
>> /Users/cthaling/ws/jdk9/hs-comp/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/.settings/org.eclipse.core.resources.prefs
>> …
>> 
>> You can see it generates all the necessary files into the various source
>> directories.  The hotspot files are here:
>> 
>> cthaling at macbook:~/ws/jdk9/hs-comp/hotspot$ ls .mx.jvmci/hotspot/eclipse/
>> server-fastdebug/ server-release/   server-slowdebug/
>> 
>> So, you have to enable “Search for nested projects” when importing into
>> Eclipse.  Then everything should magically show up.
>> 
>> 
>> Thanks a lot and best regards,
>> Volker
>> 
>> 
>> On Wed, Nov 11, 2015 at 1:42 AM, Christian Thalinger
>> <christian.thalinger at oracle.com> wrote:
>> 
>> [This is kind of a long email but contains pictures :-)]
>> 
>> https://bugs.openjdk.java.net/browse/JDK-8139921
>> http://cr.openjdk.java.net/~twisti/8139921/webrev/
>> 
>> In order to make the IDE experience more pleasant now that JEP 243 is
>> integrated we would like to use mx (https://bitbucket.org/allr/mx) for IDE
>> configuration generation.  For this we have to integrate a few mx support
>> files into the hotspot repository.
>> 
>> The mx support files will be under a dot-directory:
>> 
>> $ hg st --all .mx.jvmci/
>> C .mx.jvmci/.project
>> C .mx.jvmci/.pydevproject
>> C .mx.jvmci/eclipse-settings/org.eclipse.jdt.core.prefs
>> C .mx.jvmci/hotspot/templates/eclipse/cproject
>> C .mx.jvmci/hotspot/templates/eclipse/settings/org.eclipse.cdt.core.prefs
>> C .mx.jvmci/hotspot/templates/eclipse/settings/org.eclipse.cdt.ui.prefs
>> C
>> .mx.jvmci/hotspot/templates/eclipse/settings/org.eclipse.core.runtime.prefs
>> C .mx.jvmci/mx_jvmci.py
>> C .mx.jvmci/suite.py
>> 
>> mx itself is and will stay an external tool.  Some documentation on how to
>> use it can be found here:
>> 
>> https://wiki.openjdk.java.net/display/Graal/Instructions
>> https://wiki.openjdk.java.net/display/Graal/Eclipse
>> 
>> It basically boils down to:
>> 
>> $ mx ideinit
>> 
>> and importing the configuration into your favorite IDE.
>> 
>> This would give every developer the same view of the source code and we can
>> also enforce code-style guidelines.
>> 
>> Here is how the imported projects look like in Eclipse:
>> 
>> 
>> 
>> This is most helpful for Compiler engineers who work on the JVMCI but there
>> is value for others too.
>> 
>> Notice the “hotspot:*” projects at the top?  These are projects for
>> different HotSpot configurations.  The main advantage here is that these
>> include the generated files directory (if the configuration exists and the
>> files are built).  I only configured and built “release” so these can been
>> seen, fastdebug is empty:
>> 
>> 
>> 
>> This makes it possible for Eclipse to find generated source code.  Very
>> helpful.  For example, JVMTI.  First, jvmtiUtils.hpp from the fastdebug
>> configuration:
>> 
>> 
>> 
>> and here is the release one:
>> 
>> 
>> 
>> mx has lots of other commands but most of them are not really useful for us.
>> The only ones worth mentioning besides ideinit are findbugs and checkstyle.
>> 
>> findbugs runs FindBugs (doh!) on all Java projects that mx knows about:
>> 
>> cthaling at macbook:~/ws/8139921/hotspot$ mx findbugs
>> Scanning archives (15 / 30)
>> 2 analysis passes to perform
>> Pass 1: Analyzing classes (524 / 524) - 100% complete
>> Pass 2: Analyzing classes (305 / 305) - 100% complete
>> Done with analysis
>> Calculating exit code...
>> Exit code set to: 0
>> 
>> checkstyle checks the Java projects against some predefined style.  This is
>> particularly helpful for people who don’t use an IDE or to make sure
>> everything matches the style after applying an external patch:
>> 
>> cthaling at macbook:~/ws/8139921/hotspot$ mx checkstyle
>> Running Checkstyle on
>> /Users/cthaling/ws/8139921/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.runtime/src
>> using
>> /Users/cthaling/ws/8139921/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.service/.checkstyle_checks.xml...
>> /Users/cthaling/ws/8139921/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.runtime/src/jdk/vm/ci/runtime/JVMCI.java:33:
>> 'static' modifier out of order with the JLS suggestions.
>> 
>> or:
>> 
>> cthaling at macbook:~/ws/8139921/hotspot$ mx checkstyle
>> Running Checkstyle on
>> /Users/cthaling/ws/8139921/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.runtime/src
>> using
>> /Users/cthaling/ws/8139921/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.service/.checkstyle_checks.xml...
>> /Users/cthaling/ws/8139921/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.runtime/src/jdk/vm/ci/runtime/JVMCI.java:43:
>> Name 'FOO' must match pattern '^[a-z][a-zA-Z0-9]*$'.
>> 
>> That’s all, folks!