RFC: add starred module(s)

[Jiri Vanek]

On 12/18/2017 10:13 PM, Alexandre (Shura) Iline wrote:
> Hi.
> 
> I just wanted to commend on enhancing JTreg to automatically calculate module dependencies. Yes, compiler knows the direct static dependencies of the compiler code, but that is not all that is needed to be known. @module annotation is used for a few not completely related things. Besides requesting "--add-exports”, it can also request "--add-opens" and it can influence test selection during execution. Of this three usages, only one could be covered by static dependency analysis.
> 


Thank you very much for this input!

 From what you are saying, the need for more support of @module-whatever is needed - and doable.
Current @modules is adding --add-exports and --add-opens  of the modules you specified to the  command.
If there will be way to add exports and opens of those, which can be covered by static analyse, then 
it will be already more then helpfull! Especially if you will be able to add some more manually.
Of course more granular support like @modules-export and @modules-opens and similar would be nice, 
but Thats more far away from this proposal.

J.

> 
>> On Dec 14, 2017, at 7:56 AM, Jiri Vanek <jvanek at redhat.com> wrote:
>>
>> Hi Jonathan!
>>
>> Thank you very much for great answer. Few remarks inline.
>>
>>
>> On 12/09/2017 02:51 AM, Jonathan Gibbons wrote:
>>> Jiri, Mila,
>>> There are three significant questions here.
>>> 1. @modules all-needed
>>> This is a non-starter. @modules participates in a relatively light-weight test selection mechanism
>>
>> Sure. It is last step, if it will be ever started.
>>
>>> ... i.e. tests are not selected if the specified modules are not available in the system image. Having jtreg analyse the source code to determine the set of necessary modules is just not practical.
>>> 2. Starred modules, as in @modules java.security.jgss/sun.security.krb5.internal.*
>>> Setting aside whether this is desirable or not, this is not necessarily an easy thing to do. It requires that we can list the (internal) packages of a module. Yes, there may be a attribute that provides the info, but the attribute is not guaranteed to be available, and if it's not, you're
>> terrible truth. see at bottom.
>>
>>> reduced to scanning the module contents.  On the desirability point of view, the use of this sort of syntax was discussed, and rejected, for module declarations, and while that is separate from what is
>>
>> I was reading most of this discussion, and I can agree with it for final deployment, but not for testing.
>>> proposed here, it does lend guidance.  Neither is the '*' syntax supported by the options for the Java launcher or javac, and this construct is intended to be a thinly-veiled wrapper for those options.
>>
>> This is probably the only sentence I moreover disagree with you. Jtreg is far away fro being tinny wrapper, and it always  did huge help for tests authors.  So helping testers with setting up module path a bit, may be good step.
>>> One minor comment ... you can reduce the "wordiness" of your examples by almost 50% by just having a single @modules directive with many arg values, possibly spanning many lines, as in ...
>>>    * ....
>>>    * @modules java.security.jgss/sun.security.krb5.internal
>>>    * java.security.jgss/sun.security.krb5.internal.ccache
>>>    * java.security.jgss/sun.security.krb5.internal.crypto
>>>    * java.security.jgss/sun.security.krb5.internal.ktab
>>>    * java.base/sun.security.util
>>>    * java.security.jgss/sun.security.jgss
>>
>> Still you need to enumerate them all. That mean to run javac and jmod in loops  one by one, untill you get collect them.
>>> 3. Shell scripts.
>>> Using shell scripts for jtreg tests is generally discouraged, these days, to the point that we have made efforts to convert many/most JDK shell tests to Java code. The general experience was that such tests were very fragile, and were often wrong/broken. They typically execute slower than equivalent Java code, especially when using libraries providing commonly-used or domain-specific functionality.
>>
>> Well, shell lunchers have theirs dark sides, thats right. But are awesome feature without replacement.
>>> And as you note below, processing variables like TESTMODULES is problematic. (It is a bug if present for JDK 8 tests).
>>
>> Its a bug :)
>>> When using Java code instead of shell scripts, you can look to libraries in the various test suites for suggestions on useful methods to provide, such as invoking generating and compiling source code, providing simple equivalents for regex search (simple grep), comparison (simple diff), etc, as well as support for running subprocesses when necessary. One common pattern is to use Java code to validate whether it is appropriate to run a test in the current configuration and to then launch a JVM with a selected combination of options. This sort of mode has explicit support in jtreg:  "@run driver" is similar to "@run main" for running a class, but it is not run with any JVM options intended for test JVMs. This makes it suitable for running "test driver" classes which spawn JVMs to run the "real" test code.  And if you're just writing API tests, then I recommend using plain old Java code and "agentvm" mode to have the tests as fast as possible.
>>
>>
>> You are mostly right. But consider this: The locating of modules is painful when you are dealing with private apis. And that is what you do in jtregs for half of the time...
>> So naturally, the help would be easy.
>> On contrary, the help do not come from JDK itself in any "supported" way (or am I missing something?)
>> Still the javac can do so. So it would be natural to enahnce javac to help with this a bit more via some more solid api. Maybe you know something here?
>>
>> One would say, that writing private api jtreg is one or two modules usage only. Unluckily, oposite is true, and Mila's * idea is another proof of it. And If I look to past to my recent work, I was facing the same. Just got angry later.
>>
>>
>> So generally - some help should be provided in JDK to deal with it. And as JDK looks to have its reasons to not to do so, Jtregs should.
>>
>>
>> Thanx for again for your reply!
>>   J.
>>
>>
>>
>>
>>
>>
>>
> 


-- 
Jiri Vanek
Senior QE engineer, OpenJDK QE lead, Mgr.
Red Hat Czech
jvanek at redhat.com    M: +420775390109