Request for guidance on fixing JDK-8075571: Support tier1 and tier2 make targets

[Jonathan Gibbons]

Joe,

These lines in langtools/test/TEST.groups look like they belong, with 
appropriate tweaking, in langtools/test/Makefile

   37 # Support for invoking tiered testing via the Makefile
   38 #langtools_tier1 = :tier1
   39 #langtools_tier2 = :tier2
   40 #langtools_tier3 = :tier3


Or more specifically, there are no updates in langtools/test/Makefile to 
tie the test targets from test/Makefile into test groups.

You probably want to add tier1 tier2 tier3 to this line in 
langtools/test/Makefile


jtreg apt javac javadoc javah javap jdeps tier1 tier2 tier3: 
$(JPRT_CLEAN) jtreg-tests $(JPRT_ARCHIVE_BUNDLE) jtreg-summary
     @echo "Testing completed successfully"

And add some macros to set the TESTDIRS for each of tier*

all:      JTREG_TESTDIRS = .
jtreg:    JTREG_TESTDIRS = .
tier1:    JTREG_TESTDIRS = :tier1
tier2:    JTREG_TESTDIRS = :tier2
tier3:    JTREG_TESTDIRS = :tier3

And, at some point, we should try and rationalize the various 
*/test/Makefile.

-- Jon


On 06/02/2015 10:52 PM, joe darcy wrote:
> Thanks for the pointers Jon.
>
> A not-quite-finished version of this work is up at
>
>     http://cr.openjdk.java.net/~darcy/8075571.0/
>
> The nashorn test makefile is copied from the jaxp one with 
> s/jaxp/nashorn/g.
>
> While I'm at it, I think it is worthwhile to add support nominal 
> support for tier 3 tests since there are at least some tier 3 tests 
> we'll want to add in the future.
>
> The top-level filtering of the langtools targets isn't quite working 
> as desired; a colon to indicate a test group is not getting passed 
> along. Some different filtering of the langtools arguments is needed?
>
> Thanks,
>
> -Joe
>
>
> On 6/2/2015 2:21 PM, Jonathan Gibbons wrote:
>> Joe,
>>
>> The magic you need to manipulate is lines 54-71 (approx) in 
>> test/Makefile.
>>
>> # Default test target (core)
>> default: jdk_core langtools_jtreg jaxp_all
>>
>> # All testing
>> all: jdk_all langtools_all jaxp_all
>>
>> # Test targets
>> langtools_% :
>>     @$(NO_STOPPING)$(call SUBDIR_TEST, $(LANGTOOLS_DIR), 
>> JT_JAVA=$(PRODUCT_HOME) JTREG_HOME=$(JT_HOME) TEST="$(subst 
>> langtools_,,$@)" $(subst langtools_,,$@))
>>
>> jdk_% core_%s svc_%:
>>     @$(NO_STOPPING)$(call SUBDIR_TEST, $(JDK_DIR), TEST="$@" $@)
>>
>> jaxp_%:
>>     @$(NO_STOPPING)$(call SUBDIR_TEST, $(JAXP_DIR), TEST="$@" $@)
>>
>> hotspot_%:
>>     @$(NO_STOPPING)$(call SUBDIR_TEST, $(HOTSPOT_DIR), TEST="$@" $@)
>>
>>
>> These lines set up how test targets are mapped into individual 
>> targets in the individual repos.   Note the pattern-matching rules 
>> for langtools_%, jdk_%, jaxp_%, hotspot%
>>
>> To add a new top level :tier1 target, I would copy the "all" target 
>> and add something like
>>
>> tier1: jdk_tier1 langtools_tier1 jaxp_tier1
>>
>> (i.e. delegating to all repos that support a :tier1 test target).
>>
>> You will want to add a tier1 target into langtools/test/Makefile for 
>> this to work, because the langtools_% target strips off the 
>> "langtools_" prefix
>> For other repos, make sure there is a jdk-tier1 target, jaxp-tier1 
>> target, etc, since no prefix stripping is done on those targets.
>>
>> You might want to consider whether to (or not to) run the test 
>> targets in parallel. Depending how the individual repo test targets 
>> are set up, you could easily swamp your machine if you run too many 
>> invocations of jtreg in parallel.
>>
>> -- Jon
>>
>>
>>
>>
>>
>> On 06/02/2015 01:04 PM, joe darcy wrote:
>>> Hello makefile gurus,
>>>
>>> To provide the next level of support to the tiered testing policy 
>>> [1], I'd like to get some advice on how best to tackle
>>>
>>>     JDK-8075571: Support tier1 and tier2 make targets
>>>
>>> From the bug, currently one can invoke test groups like so:
>>>
>>>     make test-only TEST=jdk_lang
>>>
>>> That this, this above make command will run the tests in the 
>>> ":jdk_lang" test group. By applying the simple edit
>>>
>>> diff -r df4d75f58f15 test/Makefile
>>> --- a/test/Makefile    Thu May 28 11:31:40 2015 -0700
>>> +++ b/test/Makefile    Tue Jun 02 13:00:06 2015 -0700
>>> @@ -263,7 +263,7 @@
>>>
>>>  # ------------------------------------------------------------------
>>>
>>> -jdk_% core_% svc_%:
>>> +jdk_% core_% svc_% tier%:
>>>      $(ECHO) "Running tests: $@"
>>>      for each in $@; do \
>>>              $(MAKE) -j 1 TEST_SELECTION=":$$each" UNIQUE_DIR=$$each 
>>> jtreg_tests; \
>>>
>>> to the test/Makefile, I was hoping
>>>
>>>     make test-only TEST=tier1
>>>
>>> would in turn be able to run the ":tier1" test group. But alas, that 
>>> does not occur and I don't see what is (not) happening for that 
>>> omission to take place.
>>>
>>> Thanks,
>>>
>>> -Joe
>>>
>>> [1] 
>>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html
>>
>