Reusing module name token `*` in -d

Fri Jan 27 23:22:14 UTC 2017

Gregg,

javac's new behavior for the output directory is only enabled when javac 
is compiling one or more modules in multi-module mode, which is 
signified by using the --module-source-path option. In other words, if 
(and only if) you are compiling many modules together using 
--module-source-path, then javac will automatically use the extra 
directory level to keep separate the classes for the different modules 
being compiled.

If you have a "legacy" javac command line that works on JDK 8, it will 
work the same way on JDK 9, and so you will get the expected 
package-oriented output in the class output directory (i.e. without any 
additional directory.) That applies whether or not you also specify 
-source 1.6.

-- Jon


On 01/27/2017 02:19 PM, Gregg Wonderly wrote:
> I think this is an important detail to consider as well.  If -d’s argument will now have an implied subdirectory before the class hierarchy, simple Java applications/deployment tools will break, unless the JVM knows exactly what to do about this.  However, think about the case where I am building with JDK-9.  What happens with "—source 1.6”  would that cause this behavior to change?
>
> Gregg
>
>> On Jan 27, 2017, at 10:00 AM, Peter Levart <peter.levart at gmail.com> wrote:
>>
>> Hi,
>>
>> On 01/27/2017 03:27 PM, Nicolai Parlog wrote:
>>>> Being able to compile a bunch of modules together, and put the
>>>> resulting single output directory on the module path would seem to
>>>> be more convenient and thus more desirable.
>>> You make it sound like it's "either or" but that would not be the
>>> case. It would be "either you use the asterisk" or "I can reuse the
>>> same path for the module path".
>> Perhaps the asterisk is not the right solution for distributing generated classes into non-standard output paths. It implies that compiled module classes will be written to a path that has a component in common with source path (i.e. the asterisk) and that this component is equal to module name. Asterisk is already used in --module-source-path to construct paths that aggregate sources for a particular module residing in different directories, for example:
>>
>> --module-source-path /Users/Me/MyProject/src/*/{linux,shared}:/Users/Me/MyProject/build/gensrc/*
>>
>> The asterisk has a special meaning here. It is a placeholder that gets replaced with the same module name in all places it appears to construct a list of directories containing the package subdirectories with sources for a particular module.
>>
>> You now want to extend the meaning of asterisk to -d option, which implies that the output directory containing package subdirectories with generated classes for a particular module will contain a component that is equal to module name. This might not always be what is desired.
>>
>> We can already specify individual modules (either modular jars or exploded modules) on the --module-path, which is important for tools such as Maven. So maybe a complementary feature could be to specify a "distribution" list of output paths per module:
>>
>> --module-output module.one=/path/to/module-one/classes,module.two=/path/to/module-two/classes,...
>>
>> --module-output would be complementary to -d option. When a module it not listed in the --module-output, its classes are written to a common tree specified by -d.
>>
>> This is just a thought.
>>
>> Regards, Peter
>>
>>
>>>
>>>
>>> On 26.01.2017 19:15, Jonathan Gibbons wrote:
>>>> On 1/26/17 6:30 AM, Nicolai Parlog wrote:
>>>>> Hi,
>>>>>
>>>>> I see that this feature is unlikely to be in Java 9 ;) , but I
>>>>> want to give it one last push before conceding:
>>>>>
>>>>>> Comparing the two ways of compiling a set of modules ...
>>>>>>
>>>>>> Compiling modules one at a time (when that is possible) is the
>>>>>> smallest overall change for both the IDE and developer. Each
>>>>>> module can be compiled and built separately, and the resulting
>>>>>> modules can be put individually on the module path.
>>>>>>
>>>>>> Compiling modules together may yield a faster overall
>>>>>> compilation times, and the use of a single output directory
>>>>>> makes it much easier to set up the module path at runtime.
>>>>> Compiling all modules at once with a module name token in the
>>>>> path "may yield faster overall compilation times" and "the
>>>>> resulting modules can be put individually on the module path".
>>>>> Looks like a mix of characteristics that is otherwise not
>>>>> possible to achieve convenientl y.
>>>>>
>>>>> so long ... Nicolai
>>>> That does not seem to be an obviously desirable combination.
>>>>
>>>> Being able to compile a bunch of modules together, and put the
>>>> resulting single output directory on the module path would seem to
>>>> be more convenient and thus more desirable.
>>>>
>>>> -- Jon
>>>>
>>>>> On 25.01.2017 23:50, Jonathan Gibbons wrote:
>>>>>> Stephan,
>>>>>>
>>>>>> There will be many cases where one project contains one module
>>>>>> and for those cases, the single module mode compilation will
>>>>>> be sufficient and bears the most similarity to the way people
>>>>>> organize their project today.
>>>>>>
>>>>>> At the other end of the scale, there will be cases where a set
>>>>>> of closely-coupled modules have to be compiled together, and
>>>>>> multi-module mode was designed to address those cases. In this
>>>>>> situation, having the class files end up in a single directory
>>>>>> hierarchy organized by module should not be surprising.
>>>>>>
>>>>>> In between, there may be projects that contain a set of
>>>>>> loosely related modules, such that they can be compiled one
>>>>>> module at a time in a topologically sorted order. For these
>>>>>> projects, it becomes a design choice for the IDE vendor and/or
>>>>>> a use choice for the developer whether to use a series of
>>>>>> compilations, one per module, into distinct class output
>>>>>> directories, or whether to compile them all together, and have
>>>>>> the classes written to a single module-oriented output
>>>>>> directory. Either way, the effective compilation will be the
>>>>>> same, and the same checks for strong encapsulation should be in
>>>>>> effect between the modules.
>>>>>>
>>>>>> (For your example of a set of separate projects, each with its
>>>>>> own sourcefolder and binfolder, I would expect each project to
>>>>>> have its own independent build "script", which each come down
>>>>>> to one of the three preceding cases.)
>>>>>>
>>>>>> Comparing the two ways of compiling a set of modules ...
>>>>>>
>>>>>> Compiling modules one at a time (when that is possible) is the
>>>>>> smallest overall change for both the IDE and developer. Each
>>>>>> module can be compiled and built separately, and the resulting
>>>>>> modules can be put individually on the module path.
>>>>>>
>>>>>> Compiling modules together may yield a faster overall
>>>>>> compilation times, and the use of a single output directory
>>>>>> makes it much easier to set up the module path at runtime.
>>>>>>
>>>>>> -- Jon
>>>>>>
>>>>>>
>>>>>> On 01/25/2017 12:05 PM, Stephan Herrmann wrote:
>>>>>>> I found the general direction of this proposal interesting,
>>>>>>> but from the reluctance to accept this, I start to think,
>>>>>>> that maybe the role of multi-module compilation was
>>>>>>> overrated?
>>>>>>>
>>>>>>> I assume that the majority of source code out there is
>>>>>>> organized in some structure like Project1 + sourcefolder +
>>>>>>> binfolder Project2 + sourcefolder + binfolder ... Note there
>>>>>>> is nothing outside of projects. I don't believe this is going
>>>>>>> to change any time soon. Tools know how to find .class files
>>>>>>> in this structure.
>>>>>>>
>>>>>>> When projects become modules, at first look it sounds
>>>>>>> compelling to perform a multi-module compilation on several
>>>>>>> projects in one invocation. (Compelling for reasons of
>>>>>>> "simplicity" as well as performance ...). If scattering
>>>>>>> .class files into unrelated binfolders is not intended, this
>>>>>>> seems to suggest to forget about multi-module compilation,
>>>>>>> and keep compiling one project/module at a time, where we can
>>>>>>> specify individual output folders.
>>>>>>>
>>>>>>> Is that the message?
>>>>>>>
>>>>>>> Maybe, at the end of the pipeline there will be one use case
>>>>>>> where multi-module compilation will indeed collect
>>>>>>> "everything" into one global bin folder, ready for packaging
>>>>>>> and shipping, but that would be the least frequently
>>>>>>> performed use case, and then it's much easier to just copy /
>>>>>>> merge (or link) the individual binfolders into one.
>>>>>>>
>>>>>>> Any positive reasons to consider multi-module builds after
>>>>>>> all?
>>>>>>>
>>>>>>> thanks, Stephan
>>>>>>>
>>>>>>> On 01/25/2017 07:38 PM, Jonathan Gibbons wrote:
>>>>>>>> On 1/25/17 12:20 AM, Nicolai Parlog wrote:
>>>>>> Hi Jonathan,
>>>>>>
>>>>>> thanks for considering this.
>>>>>>
>>>>>>>>>>> If nothing else, it would require all the
>>>>>>>>>>> module-specific output directories to be created
>>>>>>>>>>> ahead of time, so that javac can determine which ones
>>>>>>>>>>> to use
>>>>>> Why would that be the case? It is not necessary to create them
>>>>>> now so why would using asterisk change that?
>>>>>>>>> OK, I missed that you were not suggesting to adopt all
>>>>>>>>> of the functionality of module source path, including
>>>>>>>>> the ability to specify a path of output locations, so
>>>>>>>>> that module classes could be written into a directory
>>>>>>>>> that is more closely associated with the source code.
>>>>>>>>>
>>>>>>>>>>> It has always been the case that a single
>>>>>>>>>>> compilation for different packages from different
>>>>>>>>>>> libraries would result in the classes being placed in
>>>>>>>>>>> a single output directory hierarchy, and that the
>>>>>>>>>>> classes could then be selectively packaged into
>>>>>>>>>>> different files like .jar files.
>>>>>> It has also always been the case that the compiler had no
>>>>>> notion of projects/artifacts/modules but just of plain source
>>>>>> files. ;) That changed, too, so why not do the same for class
>>>>>> files?
>>>>>>
>>>>>>>>>>> If you're compiling modules together, why could you
>>>>>>>>>>> not do something similar?
>>>>>> I have no particular use case (except writing some demos) but
>>>>>> I would guess that it would make it more comfortable for
>>>>>> existing tools to move towards multi-module compilation.
>>>>>>>>> I don't see any reason why the current design would make
>>>>>>>>> it harder for tools to move towards multi-module
>>>>>>>>> compilation.
>>>>>>>>>
>>>>>> I also like this idea for its symmetry. You can define input
>>>>>> the compilers input, sorted by modules, with *, so why not do
>>>>>> the same for its output? Conceptually that should be obvious
>>>>>> (which does not mean that there are not plenty if reasons
>>>>>> against it).
>>>>>>>>> There is a much stronger, more compelling relationship
>>>>>>>>> in play. The current -d option is designed so that you
>>>>>>>>> can put the output directory on the class path or module
>>>>>>>>> path as appropriate. That's always been the case for the
>>>>>>>>> classpath -- even though source may come from a variety
>>>>>>>>> of directory hierarchies, the compiled classes are put
>>>>>>>>> into a simple directory hierarchy that can be put on the
>>>>>>>>> classpath. Now, with modules, that continues to be the
>>>>>>>>> case: you can put the -d output directory on the runtime
>>>>>>>>> module path, either as a single "exploded module" or as a
>>>>>>>>> directory of "exploded modules". That is a compelling
>>>>>>>>> argument in favor of the current design of the javac
>>>>>>>>> output directory.
>>>>>>>>>
>>>>>>>>> In contrast, I don't see any compelling advantage to
>>>>>>>>> allowing -d "./*/target/classes" since that would make
>>>>>>>>> it significantly harder to place such a directory on the
>>>>>>>>> runtime module path.
>>>>>>>>>
>>>>>>>>> -- Jon
>>>>>>>>>
>>>>>> so long ... Nicolai
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 23.01.2017 20:51, Jonathan Gibbons wrote:
>>>>>>>>>>> Nicolai,
>>>>>>>>>>>
>>>>>>>>>>> I don't think this proposal is a good way to go. If
>>>>>>>>>>> nothing else, it would require all the
>>>>>>>>>>> module-specific output directories to be created
>>>>>>>>>>> ahead of time, so that javac can determine which ones
>>>>>>>>>>> to use, which would require additional setup commands
>>>>>>>>>>> to be executed after a "make clean" or its equivalent
>>>>>>>>>>> in other build systems.
>>>>>>>>>>>
>>>>>>>>>>> Also, I note that the output directory is typically
>>>>>>>>>>> never the final location for the compiled classes; it
>>>>>>>>>>> is typically just a "staging area". It has always
>>>>>>>>>>> been the case that a single compilation for different
>>>>>>>>>>> packages from different libraries would result in the
>>>>>>>>>>> classes being placed in a single output directory
>>>>>>>>>>> hierarchy, and that the classes could then be
>>>>>>>>>>> selectively packaged into different files like .jar
>>>>>>>>>>> files.   If you're compiling modules together, why
>>>>>>>>>>> could you not do something similar?
>>>>>>>>>>>
>>>>>>>>>>> -- Jon
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 01/21/2017 02:00 AM, Nicolai Parlog wrote:
>>>>>>>>>>>> Hi!
>>>>>>>>>>>>
>>>>>>>>>>>> Another feature request from the trenches
>>>>>>>>>>>> regarding multi-module compilation. (It is possible
>>>>>>>>>>>> that there was a similar thread a couple of
>>>>>>>>>>>> days/weeks (?) back but I didn't find it.)
>>>>>>>>>>>>
>>>>>>>>>>>> It would be nice to have the ability to specify
>>>>>>>>>>>> module specific target folders, so they do not
>>>>>>>>>>>> automatically end up in
>>>>>>>>>>>> `<whatever-was-given-to-d>/<module-name>`.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems obvious (which could very well make it
>>>>>>>>>>>> stupid) to reuse the asterisk here and allow
>>>>>>>>>>>> something like
>>>>>>>>>>>>
>>>>>>>>>>>> javac --module-path mods --module-source-path
>>>>>>>>>>>> "./*/src/main/java" -d "./*/target/classes"
>>>>>>>>>>>> -module initial.module
>>>>>>>>>>>>
>>>>>>>>>>>> I have not thought through how this might or might
>>>>>>>>>>>> not work with multiple module source paths. It
>>>>>>>>>>>> looks like the only tractable approach would be to
>>>>>>>>>>>> not allow more than one -d element.
>>>>>>>>>>>>
>>>>>>>>>>>> so long ... Nicolai
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>> -- PGP Key:
>>>>>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>>>>>
>>>>>>
>>>>>>
>>> Web: http://codefx.org a blog about software development
>>>>>> https://www.sitepoint.com/java high-quality Java/JVM content
>>>>>> http://do-foss.de Free and Open Source Software for the City
>>>>>> of Dortmund
>>>>>>
>>>>>> Twitter: https://twitter.com/nipafx