RFC: add starred module(s)

[Alexandre (Shura) Iline]

On 12/20/17 9:14 AM, Jiri Vanek wrote:
> On 12/18/2017 10:13 PM, Alexandre (Shura) Iline wrote:
>> Hi.
>>
>> I just wanted to commend on enhancing JTreg to automatically calculate 
>> module dependencies. Yes, compiler knows the direct static 
>> dependencies of the compiler code, but that is not all that is needed 
>> to be known. @module annotation is used for a few not completely 
>> related things. Besides requesting "--add-exports”, it can also 
>> request "--add-opens" and it can influence test selection during 
>> execution. Of this three usages, only one could be covered by static 
>> dependency analysis.
>>
> 
> 
> Thank you very much for this input!
> 
>  From what you are saying, the need for more support of @module-whatever 
> is needed - and doable.
> Current @modules is adding --add-exports and --add-opens  of the modules 
> you specified to the  command.
> If there will be way to add exports and opens of those, which can be 
> covered by static analyse, then it will be already more then helpfull! 
> Especially if you will be able to add some more manually.
> Of course more granular support like @modules-export and @modules-opens 
> and similar would be nice, but Thats more far away from this proposal.

The point I was trying to make was that it is impossible to statically 
compute what --add-opens should be added or what modules a test requires 
to be present.

I am completely with Jonathan on this, sorry for not making it clear.

Shura

> 
> J.
> 
>>
>>> On Dec 14, 2017, at 7:56 AM, Jiri Vanek <jvanek at redhat.com> wrote:
>>>
>>> Hi Jonathan!
>>>
>>> Thank you very much for great answer. Few remarks inline.
>>>
>>>
>>> On 12/09/2017 02:51 AM, Jonathan Gibbons wrote:
>>>> Jiri, Mila,
>>>> There are three significant questions here.
>>>> 1. @modules all-needed
>>>> This is a non-starter. @modules participates in a relatively 
>>>> light-weight test selection mechanism
>>>
>>> Sure. It is last step, if it will be ever started.
>>>
>>>> ... i.e. tests are not selected if the specified modules are not 
>>>> available in the system image. Having jtreg analyse the source code 
>>>> to determine the set of necessary modules is just not practical.
>>>> 2. Starred modules, as in @modules 
>>>> java.security.jgss/sun.security.krb5.internal.*
>>>> Setting aside whether this is desirable or not, this is not 
>>>> necessarily an easy thing to do. It requires that we can list the 
>>>> (internal) packages of a module. Yes, there may be a attribute that 
>>>> provides the info, but the attribute is not guaranteed to be 
>>>> available, and if it's not, you're
>>> terrible truth. see at bottom.
>>>
>>>> reduced to scanning the module contents.  On the desirability point 
>>>> of view, the use of this sort of syntax was discussed, and rejected, 
>>>> for module declarations, and while that is separate from what is
>>>
>>> I was reading most of this discussion, and I can agree with it for 
>>> final deployment, but not for testing.
>>>> proposed here, it does lend guidance.  Neither is the '*' syntax 
>>>> supported by the options for the Java launcher or javac, and this 
>>>> construct is intended to be a thinly-veiled wrapper for those options.
>>>
>>> This is probably the only sentence I moreover disagree with you. 
>>> Jtreg is far away fro being tinny wrapper, and it always  did huge 
>>> help for tests authors.  So helping testers with setting up module 
>>> path a bit, may be good step.
>>>> One minor comment ... you can reduce the "wordiness" of your 
>>>> examples by almost 50% by just having a single @modules directive 
>>>> with many arg values, possibly spanning many lines, as in ...
>>>>    * ....
>>>>    * @modules java.security.jgss/sun.security.krb5.internal
>>>>    * java.security.jgss/sun.security.krb5.internal.ccache
>>>>    * java.security.jgss/sun.security.krb5.internal.crypto
>>>>    * java.security.jgss/sun.security.krb5.internal.ktab
>>>>    * java.base/sun.security.util
>>>>    * java.security.jgss/sun.security.jgss
>>>
>>> Still you need to enumerate them all. That mean to run javac and jmod 
>>> in loops  one by one, untill you get collect them.
>>>> 3. Shell scripts.
>>>> Using shell scripts for jtreg tests is generally discouraged, these 
>>>> days, to the point that we have made efforts to convert many/most 
>>>> JDK shell tests to Java code. The general experience was that such 
>>>> tests were very fragile, and were often wrong/broken. They typically 
>>>> execute slower than equivalent Java code, especially when using 
>>>> libraries providing commonly-used or domain-specific functionality.
>>>
>>> Well, shell lunchers have theirs dark sides, thats right. But are 
>>> awesome feature without replacement.
>>>> And as you note below, processing variables like TESTMODULES is 
>>>> problematic. (It is a bug if present for JDK 8 tests).
>>>
>>> Its a bug :)
>>>> When using Java code instead of shell scripts, you can look to 
>>>> libraries in the various test suites for suggestions on useful 
>>>> methods to provide, such as invoking generating and compiling source 
>>>> code, providing simple equivalents for regex search (simple grep), 
>>>> comparison (simple diff), etc, as well as support for running 
>>>> subprocesses when necessary. One common pattern is to use Java code 
>>>> to validate whether it is appropriate to run a test in the current 
>>>> configuration and to then launch a JVM with a selected combination 
>>>> of options. This sort of mode has explicit support in jtreg:  "@run 
>>>> driver" is similar to "@run main" for running a class, but it is not 
>>>> run with any JVM options intended for test JVMs. This makes it 
>>>> suitable for running "test driver" classes which spawn JVMs to run 
>>>> the "real" test code.  And if you're just writing API tests, then I 
>>>> recommend using plain old Java code and "agentvm" mode to have the 
>>>> tests as fast as possible.
>>>
>>>
>>> You are mostly right. But consider this: The locating of modules is 
>>> painful when you are dealing with private apis. And that is what you 
>>> do in jtregs for half of the time...
>>> So naturally, the help would be easy.
>>> On contrary, the help do not come from JDK itself in any "supported" 
>>> way (or am I missing something?)
>>> Still the javac can do so. So it would be natural to enahnce javac to 
>>> help with this a bit more via some more solid api. Maybe you know 
>>> something here?
>>>
>>> One would say, that writing private api jtreg is one or two modules 
>>> usage only. Unluckily, oposite is true, and Mila's * idea is another 
>>> proof of it. And If I look to past to my recent work, I was facing 
>>> the same. Just got angry later.
>>>
>>>
>>> So generally - some help should be provided in JDK to deal with it. 
>>> And as JDK looks to have its reasons to not to do so, Jtregs should.
>>>
>>>
>>> Thanx for again for your reply!
>>>   J.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
> 
>