Reusing module name token `*` in -d

Fri Jan 27 22:19:17 UTC 2017

I think this is an important detail to consider as well.  If -d’s argument will now have an implied subdirectory before the class hierarchy, simple Java applications/deployment tools will break, unless the JVM knows exactly what to do about this.  However, think about the case where I am building with JDK-9.  What happens with "—source 1.6”  would that cause this behavior to change?

Gregg

> On Jan 27, 2017, at 10:00 AM, Peter Levart <peter.levart at gmail.com> wrote:
> 
> Hi,
> 
> On 01/27/2017 03:27 PM, Nicolai Parlog wrote:
>>> Being able to compile a bunch of modules together, and put the
>>> resulting single output directory on the module path would seem to
>>> be more convenient and thus more desirable.
>> You make it sound like it's "either or" but that would not be the
>> case. It would be "either you use the asterisk" or "I can reuse the
>> same path for the module path".
> 
> Perhaps the asterisk is not the right solution for distributing generated classes into non-standard output paths. It implies that compiled module classes will be written to a path that has a component in common with source path (i.e. the asterisk) and that this component is equal to module name. Asterisk is already used in --module-source-path to construct paths that aggregate sources for a particular module residing in different directories, for example:
> 
> --module-source-path /Users/Me/MyProject/src/*/{linux,shared}:/Users/Me/MyProject/build/gensrc/*
> 
> The asterisk has a special meaning here. It is a placeholder that gets replaced with the same module name in all places it appears to construct a list of directories containing the package subdirectories with sources for a particular module.
> 
> You now want to extend the meaning of asterisk to -d option, which implies that the output directory containing package subdirectories with generated classes for a particular module will contain a component that is equal to module name. This might not always be what is desired.
> 
> We can already specify individual modules (either modular jars or exploded modules) on the --module-path, which is important for tools such as Maven. So maybe a complementary feature could be to specify a "distribution" list of output paths per module:
> 
> --module-output module.one=/path/to/module-one/classes,module.two=/path/to/module-two/classes,...
> 
> --module-output would be complementary to -d option. When a module it not listed in the --module-output, its classes are written to a common tree specified by -d.
> 
> This is just a thought.
> 
> Regards, Peter
> 
> 
>> 
>> 
>> 
>> On 26.01.2017 19:15, Jonathan Gibbons wrote:
>>> 
>>> On 1/26/17 6:30 AM, Nicolai Parlog wrote:
>>>> Hi,
>>>> 
>>>> I see that this feature is unlikely to be in Java 9 ;) , but I
>>>> want to give it one last push before conceding:
>>>> 
>>>>> Comparing the two ways of compiling a set of modules ...
>>>>> 
>>>>> Compiling modules one at a time (when that is possible) is the
>>>>> smallest overall change for both the IDE and developer. Each
>>>>> module can be compiled and built separately, and the resulting
>>>>> modules can be put individually on the module path.
>>>>> 
>>>>> Compiling modules together may yield a faster overall
>>>>> compilation times, and the use of a single output directory
>>>>> makes it much easier to set up the module path at runtime.
>>>> Compiling all modules at once with a module name token in the
>>>> path "may yield faster overall compilation times" and "the
>>>> resulting modules can be put individually on the module path".
>>>> Looks like a mix of characteristics that is otherwise not
>>>> possible to achieve convenientl y.
>>>> 
>>>> so long ... Nicolai
>>> That does not seem to be an obviously desirable combination.
>>> 
>>> Being able to compile a bunch of modules together, and put the
>>> resulting single output directory on the module path would seem to
>>> be more convenient and thus more desirable.
>>> 
>>> -- Jon
>>> 
>>>> 
>>>> On 25.01.2017 23:50, Jonathan Gibbons wrote:
>>>>> Stephan,
>>>>> 
>>>>> There will be many cases where one project contains one module
>>>>> and for those cases, the single module mode compilation will
>>>>> be sufficient and bears the most similarity to the way people
>>>>> organize their project today.
>>>>> 
>>>>> At the other end of the scale, there will be cases where a set
>>>>> of closely-coupled modules have to be compiled together, and
>>>>> multi-module mode was designed to address those cases. In this
>>>>> situation, having the class files end up in a single directory
>>>>> hierarchy organized by module should not be surprising.
>>>>> 
>>>>> In between, there may be projects that contain a set of
>>>>> loosely related modules, such that they can be compiled one
>>>>> module at a time in a topologically sorted order. For these
>>>>> projects, it becomes a design choice for the IDE vendor and/or
>>>>> a use choice for the developer whether to use a series of
>>>>> compilations, one per module, into distinct class output
>>>>> directories, or whether to compile them all together, and have
>>>>> the classes written to a single module-oriented output
>>>>> directory. Either way, the effective compilation will be the
>>>>> same, and the same checks for strong encapsulation should be in
>>>>> effect between the modules.
>>>>> 
>>>>> (For your example of a set of separate projects, each with its
>>>>> own sourcefolder and binfolder, I would expect each project to
>>>>> have its own independent build "script", which each come down
>>>>> to one of the three preceding cases.)
>>>>> 
>>>>> Comparing the two ways of compiling a set of modules ...
>>>>> 
>>>>> Compiling modules one at a time (when that is possible) is the
>>>>> smallest overall change for both the IDE and developer. Each
>>>>> module can be compiled and built separately, and the resulting
>>>>> modules can be put individually on the module path.
>>>>> 
>>>>> Compiling modules together may yield a faster overall
>>>>> compilation times, and the use of a single output directory
>>>>> makes it much easier to set up the module path at runtime.
>>>>> 
>>>>> -- Jon
>>>>> 
>>>>> 
>>>>> On 01/25/2017 12:05 PM, Stephan Herrmann wrote:
>>>>>> I found the general direction of this proposal interesting,
>>>>>> but from the reluctance to accept this, I start to think,
>>>>>> that maybe the role of multi-module compilation was
>>>>>> overrated?
>>>>>> 
>>>>>> I assume that the majority of source code out there is
>>>>>> organized in some structure like Project1 + sourcefolder +
>>>>>> binfolder Project2 + sourcefolder + binfolder ... Note there
>>>>>> is nothing outside of projects. I don't believe this is going
>>>>>> to change any time soon. Tools know how to find .class files
>>>>>> in this structure.
>>>>>> 
>>>>>> When projects become modules, at first look it sounds
>>>>>> compelling to perform a multi-module compilation on several
>>>>>> projects in one invocation. (Compelling for reasons of
>>>>>> "simplicity" as well as performance ...). If scattering
>>>>>> .class files into unrelated binfolders is not intended, this
>>>>>> seems to suggest to forget about multi-module compilation,
>>>>>> and keep compiling one project/module at a time, where we can
>>>>>> specify individual output folders.
>>>>>> 
>>>>>> Is that the message?
>>>>>> 
>>>>>> Maybe, at the end of the pipeline there will be one use case
>>>>>> where multi-module compilation will indeed collect
>>>>>> "everything" into one global bin folder, ready for packaging
>>>>>> and shipping, but that would be the least frequently
>>>>>> performed use case, and then it's much easier to just copy /
>>>>>> merge (or link) the individual binfolders into one.
>>>>>> 
>>>>>> Any positive reasons to consider multi-module builds after
>>>>>> all?
>>>>>> 
>>>>>> thanks, Stephan
>>>>>> 
>>>>>> On 01/25/2017 07:38 PM, Jonathan Gibbons wrote:
>>>>>>> On 1/25/17 12:20 AM, Nicolai Parlog wrote:
>>>>> Hi Jonathan,
>>>>> 
>>>>> thanks for considering this.
>>>>> 
>>>>>>>>>> If nothing else, it would require all the
>>>>>>>>>> module-specific output directories to be created
>>>>>>>>>> ahead of time, so that javac can determine which ones
>>>>>>>>>> to use
>>>>> Why would that be the case? It is not necessary to create them
>>>>> now so why would using asterisk change that?
>>>>>>>> OK, I missed that you were not suggesting to adopt all
>>>>>>>> of the functionality of module source path, including
>>>>>>>> the ability to specify a path of output locations, so
>>>>>>>> that module classes could be written into a directory
>>>>>>>> that is more closely associated with the source code.
>>>>>>>> 
>>>>>>>>>> It has always been the case that a single
>>>>>>>>>> compilation for different packages from different
>>>>>>>>>> libraries would result in the classes being placed in
>>>>>>>>>> a single output directory hierarchy, and that the
>>>>>>>>>> classes could then be selectively packaged into
>>>>>>>>>> different files like .jar files.
>>>>> It has also always been the case that the compiler had no
>>>>> notion of projects/artifacts/modules but just of plain source
>>>>> files. ;) That changed, too, so why not do the same for class
>>>>> files?
>>>>> 
>>>>>>>>>> If you're compiling modules together, why could you
>>>>>>>>>> not do something similar?
>>>>> I have no particular use case (except writing some demos) but
>>>>> I would guess that it would make it more comfortable for
>>>>> existing tools to move towards multi-module compilation.
>>>>>>>> I don't see any reason why the current design would make
>>>>>>>> it harder for tools to move towards multi-module
>>>>>>>> compilation.
>>>>>>>> 
>>>>> I also like this idea for its symmetry. You can define input
>>>>> the compilers input, sorted by modules, with *, so why not do
>>>>> the same for its output? Conceptually that should be obvious
>>>>> (which does not mean that there are not plenty if reasons
>>>>> against it).
>>>>>>>> There is a much stronger, more compelling relationship
>>>>>>>> in play. The current -d option is designed so that you
>>>>>>>> can put the output directory on the class path or module
>>>>>>>> path as appropriate. That's always been the case for the
>>>>>>>> classpath -- even though source may come from a variety
>>>>>>>> of directory hierarchies, the compiled classes are put
>>>>>>>> into a simple directory hierarchy that can be put on the
>>>>>>>> classpath. Now, with modules, that continues to be the
>>>>>>>> case: you can put the -d output directory on the runtime
>>>>>>>> module path, either as a single "exploded module" or as a
>>>>>>>> directory of "exploded modules". That is a compelling
>>>>>>>> argument in favor of the current design of the javac
>>>>>>>> output directory.
>>>>>>>> 
>>>>>>>> In contrast, I don't see any compelling advantage to
>>>>>>>> allowing -d "./*/target/classes" since that would make
>>>>>>>> it significantly harder to place such a directory on the
>>>>>>>> runtime module path.
>>>>>>>> 
>>>>>>>> -- Jon
>>>>>>>> 
>>>>> so long ... Nicolai
>>>>> 
>>>>> 
>>>>> 
>>>>> On 23.01.2017 20:51, Jonathan Gibbons wrote:
>>>>>>>>>> Nicolai,
>>>>>>>>>> 
>>>>>>>>>> I don't think this proposal is a good way to go. If
>>>>>>>>>> nothing else, it would require all the
>>>>>>>>>> module-specific output directories to be created
>>>>>>>>>> ahead of time, so that javac can determine which ones
>>>>>>>>>> to use, which would require additional setup commands
>>>>>>>>>> to be executed after a "make clean" or its equivalent
>>>>>>>>>> in other build systems.
>>>>>>>>>> 
>>>>>>>>>> Also, I note that the output directory is typically
>>>>>>>>>> never the final location for the compiled classes; it
>>>>>>>>>> is typically just a "staging area". It has always
>>>>>>>>>> been the case that a single compilation for different
>>>>>>>>>> packages from different libraries would result in the
>>>>>>>>>> classes being placed in a single output directory
>>>>>>>>>> hierarchy, and that the classes could then be
>>>>>>>>>> selectively packaged into different files like .jar
>>>>>>>>>> files.   If you're compiling modules together, why
>>>>>>>>>> could you not do something similar?
>>>>>>>>>> 
>>>>>>>>>> -- Jon
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 01/21/2017 02:00 AM, Nicolai Parlog wrote:
>>>>>>>>>>> Hi!
>>>>>>>>>>> 
>>>>>>>>>>> Another feature request from the trenches
>>>>>>>>>>> regarding multi-module compilation. (It is possible
>>>>>>>>>>> that there was a similar thread a couple of
>>>>>>>>>>> days/weeks (?) back but I didn't find it.)
>>>>>>>>>>> 
>>>>>>>>>>> It would be nice to have the ability to specify
>>>>>>>>>>> module specific target folders, so they do not
>>>>>>>>>>> automatically end up in
>>>>>>>>>>> `<whatever-was-given-to-d>/<module-name>`.
>>>>>>>>>>> 
>>>>>>>>>>> It seems obvious (which could very well make it
>>>>>>>>>>> stupid) to reuse the asterisk here and allow
>>>>>>>>>>> something like
>>>>>>>>>>> 
>>>>>>>>>>> javac --module-path mods --module-source-path
>>>>>>>>>>> "./*/src/main/java" -d "./*/target/classes"
>>>>>>>>>>> -module initial.module
>>>>>>>>>>> 
>>>>>>>>>>> I have not thought through how this might or might
>>>>>>>>>>> not work with multiple module source paths. It
>>>>>>>>>>> looks like the only tractable approach would be to
>>>>>>>>>>> not allow more than one -d element.
>>>>>>>>>>> 
>>>>>>>>>>> so long ... Nicolai
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>> -- PGP Key:
>>>>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>>>> 
>>>>> 
>>>>> 
>> Web: http://codefx.org a blog about software development
>>>>> https://www.sitepoint.com/java high-quality Java/JVM content
>>>>> http://do-foss.de Free and Open Source Software for the City
>>>>> of Dortmund
>>>>> 
>>>>> Twitter: https://twitter.com/nipafx
> 