Coming soon: second generation prototype of Hg forest consolidation

Tue Apr 11 10:24:43 UTC 2017

I think it sounds like a good idea to refresh the hotspot source 
structure. However, I think we can be much more radical. :-)

If hotspot were to fully fold into the rest of the source structure, the 
majority of the source code would end up in:

src/java.base/share/native/libjvm
(and src/java.base/solaris/native/libjvm, 
src/java.base/x86/native/libjvm, etc...)

I realize that might be a bit hard to swallow :-).

With that in mind however, let's consider "src/hotspot" a shortcut for 
"src/java.base/*/native/libjvm", and create:
* src/hotspot/share,
* src/hotspot/solaris
* src/hotspot/x86
* src/hotspot/linux_aarch64
* etc...
which then would just contain the libjvm sources. That would create 
succinct paths to all vm source code, and let "hotspot" be synonymous 
with "libjvm" in this new source structure.

As for the rest of the "non-vm" directories in the current structure, 
they are not too many and can easily be relocated to better homes. Let's 
break it down!

* launcher:
It's just a single file, bsd/launcher/Info-privileged.plist. It's not 
even used in the build. It is (or at least should be) identical to 
jdk/src/java.base/macosx/native/launcher/Info-privileged.plist, which 
*is* used in the build. Just remove it.

* os/posix/dtrace:
Three *.d files, used in the gensrc stage of hotspot compilation. Their 
presence is essential for hotspot compilation to succeed if dtrace is 
enabled, so they should not be considered separate from the VM. They 
could move to for instance src/hotspot/posix, or possibly 
src/hotspot/share/utilities, next to dtrace.hpp.

* os/bsd/dtrace:
Is not used. This is just remnants of the old macosx port from solaris. 
Can be safely deleted.

* os/solaris/dtrace:
Here are four C/C++ files. Two of them are used to create a build 
helper, and should move to make/hotspot/src/native. One is used to build 
libjvm_dtrace and one to build libjvm_db. According to the standard used 
elsewhere, they should reside in separate directories named after the 
resulting library. If "src/hotspot" is used as a shortcut for 
"src/java.base/*/native/libjvm", then, these files should move to 
"src/java.base/solaris/native/libjvm_dtrace" and 
"src/java.base/*/native/libjvm_db".

Alternatively, let "src/hotspot" be used just as an alias for 
"src/java.base/*/native", and put them in 
"src/hotspot/solaris/libjvm_db", and the VM sources in 
"src/hotspot/*/libjvm" instead.

* hsdis, IdealGraphVisualizer and LogCompilation
These are stand-alone utilities that are not built as part of the normal 
build process. "Co-located tools", if you want. We have a couple of 
similar tools in jdk/make/non-build-utils (or make/jdk/non-build-utils 
in the proposed new layout). Perhaps all of those deserve a better home, 
like "src/utils" or "src/stand-alone-tools"? If that's not acceptable, I 
suggest crammingthe hotspot/share/tools into 
jdk/hotspot/non-build-utils, analog to the jdk tools.

And then finally we have:
* jsig.c:
This is a file that currently resides in ./src/os/*/vm/jsig.c, but 
shouldn't be. It's explicitly excluded from all hotspot builds, and 
instead used to build libjsig. It should reside in 
"src/java.base/*/native/libjsig" (or "src/hotspot/*/libjsig").

/Magnus

On 2017-04-10 19:47, Stefan Karlsson wrote:
> Hi all,
>
> Great to see that the consolidation work is going forward!
>
> I have comment specific to the HotSpot code layout:
>
> We previously had this layout (using a small set of directories as an 
> example):
>  hotspot/src/share/vm
>  hotspot/src/os/linux/vm
>  hotspot/src/os/solaris/vm
>  hotspot/src/os/solaris/dtrace
>  hotspot/src/os/bsd/launcher
>  hotspot/src/share/tools/hsdis
>
> and this is the new layout:
>  src/hotspot/share/vm
>  src/hotspot/os/linux/vm
>  src/hotspot/os/solaris/vm
>  src/hotspot/os/solaris/dtrace
>  src/hotspot/os/bsd/launcher
>  src/hotspot/share/tools/hsdis
>
> should we consider changing the layout of the HotSpot code so that we 
> don't mix the source code for the vm, tools, launcher? For example:
>  src/hotspot/vm/share
>  src/hotspot/vm/os/linux
>  src/hotspot/vm/os/solaris
>  src/hotspot/launcher/os/bsd
>  src/hotspot/tools/hsdis
>  src/hotspot/dtrace/os/solaris
>
> I'm not sure that we would want to include such a change in the 
> consolidation effort, but we are changing all file paths so maybe it's 
> the right time to think about this.
>
> Cheers,
> StefanK
>
> On 10/04/17 18:42, joe darcy wrote:
>> Hello,
>>
>> Second generation prototype now available from
>>
>>     http://hg.openjdk.java.net/jdk10/consol-proto/
>>
>> Sources equivalent to JDK 9 b161.
>>
>> Thanks,
>>
>> -Joe
>>
>>
>> On 4/5/2017 9:44 AM, joe darcy wrote:
>>> Hello,
>>>
>>> As previously discussed [1], we've been exploring consolidating the 
>>> many repositories in an OpenJDK Hg forest (jdk, langtools, hotspot, 
>>> ...) into a single repository for JDK 10. The initial prototype of a 
>>> consolidated repo [2], equivalent to JDK 9 build 138, was published 
>>> in October 2016.
>>>
>>> Erik Joelsson has been working on a refined version of the 
>>> consolidation repo and we plan to publish this second generation 
>>> prototype in the near future. The new prototype presents a more 
>>> faithful view of the sources between tags for promoted builds and 
>>> provides a cleaner overall file layout.
>>>
>>> The corresponding JEP [3] will be updated to reflect the details of 
>>> the new prototype in due course.
>>>
>>> Cheers,
>>>
>>> -Joe
>>>
>>> [1] "Looking ahead: proposed Hg forest consolidation for JDK 10,"
>>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/004987.html 
>>>
>>>
>>> [2] http://hg.openjdk.java.net/jdk9/consol-proto/
>>>
>>> [3] JEP 296: Consolidate the JDK Forest into a Single Repository
>>> http://openjdk.java.net/jeps/296
>>>
>>
>