Pull parser for jmod packages

[Chris Hegarty]

On 25/05/2012 13:33, Paul Sandoz wrote:
> On May 25, 2012, at 12:37 PM, Chris Hegarty wrote:
>> On 25/05/2012 09:35, Paul Sandoz wrote:
>>> Hi
>>>
>>> At the moment the reader of jmod packages, org.openjdk.jigsaw.ModuleFile.Reader, is tightly bound to reading a module file and extracting out the contents into a directory (and tightly bound to the implementation details of the library contents).
>>>
>>> How about we develop a module file pull parser?
>>
>> I really like this idea. I was always a little uncomfortable with the tight relationship between the reading of the module file and the writing out of its data. For example, jsign and the changes to support listing the module file contents does some funnies to avoid writing out the actual contents. A pull parser type API may work well here. I'll take a stab at what this would look like ( unless you want to? ).
>>
>
> This area is your baby, and i guess you will most likely change it for the BDB work so unless you are overloaded, go for it!
>
> I also took a quick look at the writing side of this. It may be advantageous to keep that in mind for the signing case you mention. At the moment ModuleFileWriter also seems to be bound to a certain file-based layout for input. But, i think it should be possible to round trip by hooking up the pull parser/reader to the writer, then signing becomes an man-in-the-middle of that process, and ideally the Signer should only have to deal with encoding details of the signing section.
>
> I can imagine a module file writer that either keeps references to content (plus the transformation/encoding step), or content in memory, or as temp files on disk until one is ready to commit to the generation of a module file merging it all together as one file and generating the hashes.
>
> I notice that a signed module file may break the streaming approach if the signed section is at the end of the module file. Should the signature section be mandated to always occur after the header and module info? I don't know enough about the hashing and signing to know if it is a problem to have a flag in the module header section indicating the signature section is present after the module info section.

Sean added the signing support a good while back. The signature section 
(if supplied) comes right after the module-info. I believe to support 
streaming, and also easy verification of section hashes as you read them.

http://cr.openjdk.java.net/~mullan/jigsaw/signed-module-file-format

I really want to capture this and update the original module-file-format 
[1], and eventually get it all onto the project page.

> Do you mind if modify jrepo to support modular jar files?

Absolutely not, this is a great idea ( sorry I did mean to reply to your 
original mail about this ). I can add it to my list, or if you like 
please feel free?

-Chris.

[1] http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/

>
> Paul.
>
>
>>> Such a parser can then be used by "jmod install/extract" [1] and by other tools, for example IDEs to more easily extract out the class file contents for analysis.
>>>
>>> Paul.
>>>
>>> [1] Arguably "jmod extract" should be "jpkg extract"
>>
>> Yes, I agree. If jpkg evolves into a tool for creating and inspecting module files/native packages, then I think 'extract' is better suited in jpkg.
>>
>> -Chris.
>