Module-file format (DRAFT)

[Jonathan Gibbons]

I assume that UTF8 always means Java's version of UTF8, as used in 
classfiles and DataInputStream/DataOutputStream.

-- Jon


Mark Reinhold wrote:
> Here's a first cut at a simple module-file format.
>
> Comments welcome!
>
> - Mark
>
> ----
>
> Jigsaw module-file format DRAFT
> ===============================
>
> Mark Reinhold <mr at sun.com>  
> 15 January 2010
>
>
> Goals
> -----
>
>   - Optimized for streamed reading, from beginning to end.  Random access
>     should not be required when reading a module file, though it can be
>     required when writing.
>
>   - Content-specific compression: Pack200+gzip/lzma for classes, bzip2 for
>     native code, etc.
>
>   - Independent of any specific installed-module format or target-filesystem
>     capability.
>
>
> Layout
> ------
>
> A module file has the following sections, in the order listed.
>
> The first two sections are required:
>
>   - Header -- Magic number, format version, sizes
>   - module-info.class (never compressed, so as to be easily readable by tools)
>
> These sections are optional:
>
>   - Classes (excluding module-info.class)
>   - Resources
>   - Native libraries
>   - Native launchers
>   - Configuration (i.e., properties) files
>
> The final section is required:
>
>   - Secure hash of the entire file (except for this section)
>
>
> Module file header
> ------------------
>
>     ModuleFileHeader {
>       u4 magic;                 // FileConstants.MAGIC
>       u2 type;                  // FileConstants.Type.MODULE_FILE
>       u2 major;                 // FileConstants.ModuleFile.MAJOR_VERSION
>       u2 minor;                 // FileConstants.ModuleFile.MINOR_VERSION
>       u4 csize;                 // Size of entire file, compressed
>       u4 usize;                 // Space required for uncompressed contents
>                                 //   (upper bound; need not be exact)
>     }
>
> All integers are in network byte order.
>
>
> Section headers
> ---------------
>
> Each section has a header:
>
>     SectionHeader {
>       u2 type;                  // One of FileConstants.ModuleFile.SectionType
>       u2 compressor;            // One of FileConstants.ModuleFile.Compressor
>     }
>
> Compression is only relevant to section content; it never applies to the
> headers defined here.
>
> For sections that contain only one entity (i.e., module-info, classes, hash)
> the section header is followed by a size pair, which is then followed by the
> content:
>
>     SectionSize {
>       u4 csize;                 // Size of section content, compressed
>       u4 usize;                 // Size of section content, uncompressed
>     }
>
> If a section contains named files (i.e., resources, native code, config) then
> each file within that section has a header, which is then followed by the
> content:
>
>     SectionFileHeader {
>       u4 csize;                 // Size of file, compressed
>       u4 usize;                 // Size of file, uncompressed
>       u2 nameLength;            // Length of file name
>       b* name;                  // File name, in UTF-8
>     }
>
> File names may include directory separators, though they should not be used for
> native-code files.
>
> There is no mode field on files within a section.  The module-installation code
> must set the local mode appropriately; e.g., it must make native-code files
> executable as appropriate.
>
> A hash section has the content:
>
>     HashSection {
>       u2 type;                  // One of FileConstants.ModuleFile.HashType
>       u2 size;                  // Never compressed
>       b* hash;
>     }
>
>
> Open issues
> -----------
>
>   - Signing: Should module files carry their own digital-signature information,
>     as jar files do today, or should such information be provided elsewhere?
>