Fwd: need advice on module structure for ISA-specific sources and classes

[John Rose]

At Mark's suggestion, I sent the following query to jigsaw-dev.

http://mail.openjdk.java.net/pipermail/jigsaw-dev/2014-September/003564.html

Possible use cases:
1. integrate ISA-dependent and platform-dependent code from JNR into JDK source layout.
2. Organize ISA-dependent functions (VIS, AVX, TSX).
3. Organize native code generators if needed.

— John

Begin forwarded message:

From: John Rose <john.r.rose at oracle.com>
Subject: need advice on module structure for ISA-specific sources and classes
Date: September 4, 2014 at 2:13:17 PM PDT
To: jigsaw-dev at openjdk.java.net

We have standard OpenJDK source locations for platform-specific code, as described in http://openjdk.java.net/jeps/201 .

For example:
http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/tip/src/java.base/linux/classes/sun/nio/ch/LinuxAsynchronousChannelProvider.java
http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/tip/src/java.base/linux/native/libnio/fs/LinuxWatchService.c

We have a place for both C and Java sources.  Amazingly, there is even one assembly source file:
http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/tip/src/java.desktop/unix/native/libawt/sun/awt/medialib/mlib_v_ImageCopy_blk.s

But, there is no place at present for (a) ISA-dependent source files, or (b) ISA-and-platform-dependent source files.

Definitions:  An ISA-dependent source file is coded portably (perhaps in Java, or C with ifdefs) but makes use of specific instructions, such as SPARC VIS or Intel AVX.  A ISA-and-platform-dependent source file is coded for a particular platform and ISA; a typical example would be an assembly file whose syntax is platform specific (like linux/x86).

Context:  The JNR system we are importing to Project Panama will include both types of source files.

Straw man proposal:  Allow the folder names "cpu.$CPU" and "$OS.$CPU" to occur as a sibling to "share" and $OS in source paths.

Here's a BNF style description of JEP 201 source paths:
path := 'src/' $MODULE '/' platform_scope '/' ( classes_path | native_path | conf_path )
platform_scope := 'share' | $OS
classes_path := 'classes/' $PACKAGE '/' *.java
native_path := 'native/' ( 'include/' *.{h,hpp} | $LIBRARY '/' *.{c,cpp} )
conf_path := 'conf/' *

The change would be:
platform_scope := 'share' | $OS | 'cpu.' $CPU | $OS '.' $CPU

Observation 1:  This scheme does not entangle the type of source (native vs. classes) with the platform scope.  It thus allows for platform-specific assembly files, but encourages shared but ISA-specific code.

Observation 2:  Less mangled patterns like "share/cpu/$CPU" might be cleaner, but they also perturb the existing pathname lengths, by adding optional pathname components.  Controlling pathname depth seems like a desirable goal.

In cases where single output artifacts have to be built with support for multiple platforms, we would also add the cpu name somewhere in the $PACKAGE component, and/or the base name of the file (class name), as Graal does.

(There is a possible followup question about deploying multiple ISAs and/or platforms in one artifact, kind of like a multi-JAR.  But I think we can tackle that later, and it may be that package-name hacking will allow multiple ISAs to co-exist when needed, as is the case with Project Sumatra.)

Comments?  Does this look like a good starting point for organizing ISA-specific code for Panama?

Thanks,
— John

P.S.  As a point of comparison, the hotspot repo uses this grammar, which supports only C and assembly code:
path := 'src/' platform_scope '/vm/' $GROUP '/' *.{c,cpp,hpp}
platform_scope := 'share' | 'cpu/' $CPU | 'os/' $OS | 'os_cpu/' $OS '_' $CPU
where GROUP is something like 'code', 'asm', 'c1', 'classfile', etc., an informal grouping of source files in a language without packages.
Also, the base name of the file repeats $OS and $CPU components, if present.
Examples:
http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/tip/src/cpu/x86/vm/macroAssembler_x86.hpp
http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/tip/src/os/linux/vm/osThread_linux.hpp
http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/tip/src/os_cpu/linux_x86/vm/thread_linux_x86.hpp