Reusing module name token `*` in -d

Fri Jan 27 16:00:54 UTC 2017

Hi,

On 01/27/2017 03:27 PM, Nicolai Parlog wrote:
>> Being able to compile a bunch of modules together, and put the
>> resulting single output directory on the module path would seem to
>> be more convenient and thus more desirable.
> You make it sound like it's "either or" but that would not be the
> case. It would be "either you use the asterisk" or "I can reuse the
> same path for the module path".

Perhaps the asterisk is not the right solution for distributing 
generated classes into non-standard output paths. It implies that 
compiled module classes will be written to a path that has a component 
in common with source path (i.e. the asterisk) and that this component 
is equal to module name. Asterisk is already used in 
--module-source-path to construct paths that aggregate sources for a 
particular module residing in different directories, for example:

--module-source-path 
/Users/Me/MyProject/src/*/{linux,shared}:/Users/Me/MyProject/build/gensrc/*

The asterisk has a special meaning here. It is a placeholder that gets 
replaced with the same module name in all places it appears to construct 
a list of directories containing the package subdirectories with sources 
for a particular module.

You now want to extend the meaning of asterisk to -d option, which 
implies that the output directory containing package subdirectories with 
generated classes for a particular module will contain a component that 
is equal to module name. This might not always be what is desired.

We can already specify individual modules (either modular jars or 
exploded modules) on the --module-path, which is important for tools 
such as Maven. So maybe a complementary feature could be to specify a 
"distribution" list of output paths per module:

--module-output 
module.one=/path/to/module-one/classes,module.two=/path/to/module-two/classes,...

--module-output would be complementary to -d option. When a module it 
not listed in the --module-output, its classes are written to a common 
tree specified by -d.

This is just a thought.

Regards, Peter

>
>
>
> On 26.01.2017 19:15, Jonathan Gibbons wrote:
>>
>> On 1/26/17 6:30 AM, Nicolai Parlog wrote:
>>> Hi,
>>>
>>> I see that this feature is unlikely to be in Java 9 ;) , but I
>>> want to give it one last push before conceding:
>>>
>>>> Comparing the two ways of compiling a set of modules ...
>>>>
>>>> Compiling modules one at a time (when that is possible) is the
>>>> smallest overall change for both the IDE and developer. Each
>>>> module can be compiled and built separately, and the resulting
>>>> modules can be put individually on the module path.
>>>>
>>>> Compiling modules together may yield a faster overall
>>>> compilation times, and the use of a single output directory
>>>> makes it much easier to set up the module path at runtime.
>>> Compiling all modules at once with a module name token in the
>>> path "may yield faster overall compilation times" and "the
>>> resulting modules can be put individually on the module path".
>>> Looks like a mix of characteristics that is otherwise not
>>> possible to achieve convenientl y.
>>>
>>> so long ... Nicolai
>> That does not seem to be an obviously desirable combination.
>>
>> Being able to compile a bunch of modules together, and put the
>> resulting single output directory on the module path would seem to
>> be more convenient and thus more desirable.
>>
>> -- Jon
>>
>>>
>>> On 25.01.2017 23:50, Jonathan Gibbons wrote:
>>>> Stephan,
>>>>
>>>> There will be many cases where one project contains one module
>>>> and for those cases, the single module mode compilation will
>>>> be sufficient and bears the most similarity to the way people
>>>> organize their project today.
>>>>
>>>> At the other end of the scale, there will be cases where a set
>>>> of closely-coupled modules have to be compiled together, and
>>>> multi-module mode was designed to address those cases. In this
>>>> situation, having the class files end up in a single directory
>>>> hierarchy organized by module should not be surprising.
>>>>
>>>> In between, there may be projects that contain a set of
>>>> loosely related modules, such that they can be compiled one
>>>> module at a time in a topologically sorted order. For these
>>>> projects, it becomes a design choice for the IDE vendor and/or
>>>> a use choice for the developer whether to use a series of
>>>> compilations, one per module, into distinct class output
>>>> directories, or whether to compile them all together, and have
>>>> the classes written to a single module-oriented output
>>>> directory. Either way, the effective compilation will be the
>>>> same, and the same checks for strong encapsulation should be in
>>>> effect between the modules.
>>>>
>>>> (For your example of a set of separate projects, each with its
>>>> own sourcefolder and binfolder, I would expect each project to
>>>> have its own independent build "script", which each come down
>>>> to one of the three preceding cases.)
>>>>
>>>> Comparing the two ways of compiling a set of modules ...
>>>>
>>>> Compiling modules one at a time (when that is possible) is the
>>>> smallest overall change for both the IDE and developer. Each
>>>> module can be compiled and built separately, and the resulting
>>>> modules can be put individually on the module path.
>>>>
>>>> Compiling modules together may yield a faster overall
>>>> compilation times, and the use of a single output directory
>>>> makes it much easier to set up the module path at runtime.
>>>>
>>>> -- Jon
>>>>
>>>>
>>>> On 01/25/2017 12:05 PM, Stephan Herrmann wrote:
>>>>> I found the general direction of this proposal interesting,
>>>>> but from the reluctance to accept this, I start to think,
>>>>> that maybe the role of multi-module compilation was
>>>>> overrated?
>>>>>
>>>>> I assume that the majority of source code out there is
>>>>> organized in some structure like Project1 + sourcefolder +
>>>>> binfolder Project2 + sourcefolder + binfolder ... Note there
>>>>> is nothing outside of projects. I don't believe this is going
>>>>> to change any time soon. Tools know how to find .class files
>>>>> in this structure.
>>>>>
>>>>> When projects become modules, at first look it sounds
>>>>> compelling to perform a multi-module compilation on several
>>>>> projects in one invocation. (Compelling for reasons of
>>>>> "simplicity" as well as performance ...). If scattering
>>>>> .class files into unrelated binfolders is not intended, this
>>>>> seems to suggest to forget about multi-module compilation,
>>>>> and keep compiling one project/module at a time, where we can
>>>>> specify individual output folders.
>>>>>
>>>>> Is that the message?
>>>>>
>>>>> Maybe, at the end of the pipeline there will be one use case
>>>>> where multi-module compilation will indeed collect
>>>>> "everything" into one global bin folder, ready for packaging
>>>>> and shipping, but that would be the least frequently
>>>>> performed use case, and then it's much easier to just copy /
>>>>> merge (or link) the individual binfolders into one.
>>>>>
>>>>> Any positive reasons to consider multi-module builds after
>>>>> all?
>>>>>
>>>>> thanks, Stephan
>>>>>
>>>>> On 01/25/2017 07:38 PM, Jonathan Gibbons wrote:
>>>>>> On 1/25/17 12:20 AM, Nicolai Parlog wrote:
>>>> Hi Jonathan,
>>>>
>>>> thanks for considering this.
>>>>
>>>>>>>>> If nothing else, it would require all the
>>>>>>>>> module-specific output directories to be created
>>>>>>>>> ahead of time, so that javac can determine which ones
>>>>>>>>> to use
>>>> Why would that be the case? It is not necessary to create them
>>>> now so why would using asterisk change that?
>>>>>>> OK, I missed that you were not suggesting to adopt all
>>>>>>> of the functionality of module source path, including
>>>>>>> the ability to specify a path of output locations, so
>>>>>>> that module classes could be written into a directory
>>>>>>> that is more closely associated with the source code.
>>>>>>>
>>>>>>>>> It has always been the case that a single
>>>>>>>>> compilation for different packages from different
>>>>>>>>> libraries would result in the classes being placed in
>>>>>>>>> a single output directory hierarchy, and that the
>>>>>>>>> classes could then be selectively packaged into
>>>>>>>>> different files like .jar files.
>>>> It has also always been the case that the compiler had no
>>>> notion of projects/artifacts/modules but just of plain source
>>>> files. ;) That changed, too, so why not do the same for class
>>>> files?
>>>>
>>>>>>>>> If you're compiling modules together, why could you
>>>>>>>>> not do something similar?
>>>> I have no particular use case (except writing some demos) but
>>>> I would guess that it would make it more comfortable for
>>>> existing tools to move towards multi-module compilation.
>>>>>>> I don't see any reason why the current design would make
>>>>>>> it harder for tools to move towards multi-module
>>>>>>> compilation.
>>>>>>>
>>>> I also like this idea for its symmetry. You can define input
>>>> the compilers input, sorted by modules, with *, so why not do
>>>> the same for its output? Conceptually that should be obvious
>>>> (which does not mean that there are not plenty if reasons
>>>> against it).
>>>>>>> There is a much stronger, more compelling relationship
>>>>>>> in play. The current -d option is designed so that you
>>>>>>> can put the output directory on the class path or module
>>>>>>> path as appropriate. That's always been the case for the
>>>>>>> classpath -- even though source may come from a variety
>>>>>>> of directory hierarchies, the compiled classes are put
>>>>>>> into a simple directory hierarchy that can be put on the
>>>>>>> classpath. Now, with modules, that continues to be the
>>>>>>> case: you can put the -d output directory on the runtime
>>>>>>> module path, either as a single "exploded module" or as a
>>>>>>> directory of "exploded modules". That is a compelling
>>>>>>> argument in favor of the current design of the javac
>>>>>>> output directory.
>>>>>>>
>>>>>>> In contrast, I don't see any compelling advantage to
>>>>>>> allowing -d "./*/target/classes" since that would make
>>>>>>> it significantly harder to place such a directory on the
>>>>>>> runtime module path.
>>>>>>>
>>>>>>> -- Jon
>>>>>>>
>>>> so long ... Nicolai
>>>>
>>>>
>>>>
>>>> On 23.01.2017 20:51, Jonathan Gibbons wrote:
>>>>>>>>> Nicolai,
>>>>>>>>>
>>>>>>>>> I don't think this proposal is a good way to go. If
>>>>>>>>> nothing else, it would require all the
>>>>>>>>> module-specific output directories to be created
>>>>>>>>> ahead of time, so that javac can determine which ones
>>>>>>>>> to use, which would require additional setup commands
>>>>>>>>> to be executed after a "make clean" or its equivalent
>>>>>>>>> in other build systems.
>>>>>>>>>
>>>>>>>>> Also, I note that the output directory is typically
>>>>>>>>> never the final location for the compiled classes; it
>>>>>>>>> is typically just a "staging area". It has always
>>>>>>>>> been the case that a single compilation for different
>>>>>>>>> packages from different libraries would result in the
>>>>>>>>> classes being placed in a single output directory
>>>>>>>>> hierarchy, and that the classes could then be
>>>>>>>>> selectively packaged into different files like .jar
>>>>>>>>> files.   If you're compiling modules together, why
>>>>>>>>> could you not do something similar?
>>>>>>>>>
>>>>>>>>> -- Jon
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 01/21/2017 02:00 AM, Nicolai Parlog wrote:
>>>>>>>>>> Hi!
>>>>>>>>>>
>>>>>>>>>> Another feature request from the trenches
>>>>>>>>>> regarding multi-module compilation. (It is possible
>>>>>>>>>> that there was a similar thread a couple of
>>>>>>>>>> days/weeks (?) back but I didn't find it.)
>>>>>>>>>>
>>>>>>>>>> It would be nice to have the ability to specify
>>>>>>>>>> module specific target folders, so they do not
>>>>>>>>>> automatically end up in
>>>>>>>>>> `<whatever-was-given-to-d>/<module-name>`.
>>>>>>>>>>
>>>>>>>>>> It seems obvious (which could very well make it
>>>>>>>>>> stupid) to reuse the asterisk here and allow
>>>>>>>>>> something like
>>>>>>>>>>
>>>>>>>>>> javac --module-path mods --module-source-path
>>>>>>>>>> "./*/src/main/java" -d "./*/target/classes"
>>>>>>>>>> -module initial.module
>>>>>>>>>>
>>>>>>>>>> I have not thought through how this might or might
>>>>>>>>>> not work with multiple module source paths. It
>>>>>>>>>> looks like the only tractable approach would be to
>>>>>>>>>> not allow more than one -d element.
>>>>>>>>>>
>>>>>>>>>> so long ... Nicolai
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>> -- PGP Key:
>>>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>>>
>>>>
>>>>
> Web: http://codefx.org a blog about software development
>>>> https://www.sitepoint.com/java high-quality Java/JVM content
>>>> http://do-foss.de Free and Open Source Software for the City
>>>> of Dortmund
>>>>
>>>> Twitter: https://twitter.com/nipafx