Pull parser for jmod packages

[Sean Mullan]

On 5/25/12 9:10 AM, Chris Hegarty wrote:
>> I also took a quick look at the writing side of this. It may be
>> advantageous to keep that in mind for the signing case you mention. At the
>> moment ModuleFileWriter also seems to be bound to a certain file-based
>> layout for input. But, i think it should be possible to round trip by
>> hooking up the pull parser/reader to the writer, then signing becomes an
>> man-in-the-middle of that process, and ideally the Signer should only have
>> to deal with encoding details of the signing section.

Keep in mind that the current signed module file format is optimized for
streaming as you read, but not as you write. The entire file has to be written
out and the hashes calculated/read in order to calculate the signature, thus the
Signer basically does a two-pass write in order to do that, once to a temporary
file, and the second time to the signed jmod.

>> I can imagine a module file writer that either keeps references to content
>> (plus the transformation/encoding step), or content in memory, or as temp
>> files on disk until one is ready to commit to the generation of a module
>> file merging it all together as one file and generating the hashes.
>> 
>> I notice that a signed module file may break the streaming approach if the
>> signed section is at the end of the module file. Should the signature
>> section be mandated to always occur after the header and module info? I
>> don't know enough about the hashing and signing to know if it is a problem
>> to have a flag in the module header section indicating the signature
>> section is present after the module info section.
> 
> Sean added the signing support a good while back. The signature section (if
> supplied) comes right after the module-info. I believe to support streaming,
> and also easy verification of section hashes as you read them.
> 
> http://cr.openjdk.java.net/~mullan/jigsaw/signed-module-file-format
> 
> I really want to capture this and update the original module-file-format [1],
> and eventually get it all onto the project page.

That would be great. There were also a few modifications I recommended we make
to the module-file format in order to better support signing:

http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-June/001392.html

(Some of these are already implemented).

--Sean

> 
>> Do you mind if modify jrepo to support modular jar files?
> 
> Absolutely not, this is a great idea ( sorry I did mean to reply to your 
> original mail about this ). I can add it to my list, or if you like please
> feel free?
> 
> -Chris.
> 
> [1] http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/
> 
>> 
>> Paul.
>> 
>> 
>>>> Such a parser can then be used by "jmod install/extract" [1] and by
>>>> other tools, for example IDEs to more easily extract out the class file
>>>> contents for analysis.
>>>> 
>>>> Paul.
>>>> 
>>>> [1] Arguably "jmod extract" should be "jpkg extract"
>>> 
>>> Yes, I agree. If jpkg evolves into a tool for creating and inspecting
>>> module files/native packages, then I think 'extract' is better suited in
>>> jpkg.
>>> 
>>> -Chris.
>>