RFR 8215729: Enhance makefiles to allow collecting code coverage with JCov

[Alexandre (Shura) Iline]

> On Jan 15, 2019, at 5:35 AM, Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:
> 
> Hi Shura,
> 
> I see that this is pushed already, but I still have some questions and/or follow-up requests:
> 
> 1) Do you expect the jcov-test-* targets to be used often? We've recently ran into problems with having too many top-level targets, and the large amount of targets generated for the jcov-test-* targets worries me. Before Christmas, I pushed a change which drastically cut down on the number of these auto-generated target names, and now this fix causes a partial regression on that front. To me, running jcov is such a special case, that the general solution using make jcov-test TEST="..." would suffice.

I mainly added those for consistency originally, but I see a value in them. That is, when you want to collect code coverage, you want to do that for a particular set of tests. If I want to run tier1 tests with coverage, then, I would be more comfortable running make jcov-test-tier1 that make TEST=:tier1 run-test. That said, I do not think the targets will be used “often”. Definitely much less often that the test-* counterparts.

> 
> 2) I try hard to keep the doc/testing.md documentation relevant and up to date. Can you please write something in that document about what jcov-test does, and how to use it?

Oh, sure. I will do that as a follow-on fix.

> 
> 3) What is the reasoning for removing the
> 
>  598   $1: run-test-$1 parse-test-$1
>  599
>  600   TARGETS += $1
> 
> construct?

This is because jcov grabber needs to be started upfront:
$(TARGETS): jcov-start-grabber

Shura


> This created a "meta" rule that included both running and parsing tests, for each test. I don't think it's used right now (otherwise testing would have broke), I think it's a sane abstraction to have, and in line with what we have elsewhere in the build system.
> 
> /Magnus
> 
> On 2019-01-08 19:35, Alexandre (Shura) Iline wrote:
>> Hi,
>> 
>> Could you please take a look on a change which allows to run tests while collecting code coverage with JCov. This is a continuation of work done in
>> JDK-8214309: Enhance makefiles to allow generating JCov instrumented build.
>> 
>> This adds make targets jcov-test and others like jcov-test-tier1 etc. After running the tests, one is left with a coverage report and also a data file containing coverage, for further analysis.
>> 
>> Part of the changes are related to increasing maximum and initial heap size, which is needed to be done because instrumented classes are bigger in size. The way it is done for JTReg tests is by setting both _JAVA_OPTIONS and JAVA_TOOL_OPTIONS environment variables.
>> 
>> As it stands now, not all tests pass while running with code coverage, which pass normally. This particular enhancement is not addressing failing tests. The tests will need to be fixed separately if ever. For example, there are 35 tests which fail in open/test/jdk:jdk_core. A quick look tells that the majority of the failures are in tests which verify error output and discover an unexpected line "Picked up _JAVA_OPTIONS: -Xmx4g”.
>> 
>> Enhancement: https://bugs.openjdk.java.net/browse/JDK-8215729
>> Webrev: http://cr.openjdk.java.net/~shurailine/8215729/webrev.01/
>> 
>> Shura
>