RFR 8215729: Enhance makefiles to allow collecting code coverage with JCov

[Magnus Ihse Bursie]

Hi Shura,

I see that this is pushed already, but I still have some questions 
and/or follow-up requests:

1) Do you expect the jcov-test-* targets to be used often? We've 
recently ran into problems with having too many top-level targets, and 
the large amount of targets generated for the jcov-test-* targets 
worries me. Before Christmas, I pushed a change which drastically cut 
down on the number of these auto-generated target names, and now this 
fix causes a partial regression on that front. To me, running jcov is 
such a special case, that the general solution using make jcov-test 
TEST="..." would suffice.

2) I try hard to keep the doc/testing.md documentation relevant and up 
to date. Can you please write something in that document about what 
jcov-test does, and how to use it?

3) What is the reasoning for removing the

  598   $1: run-test-$1 parse-test-$1
  599
  600   TARGETS += $1

construct? This created a "meta" rule that included both running and 
parsing tests, for each test. I don't think it's used right now 
(otherwise testing would have broke), I think it's a sane abstraction to 
have, and in line with what we have elsewhere in the build system.

/Magnus

On 2019-01-08 19:35, Alexandre (Shura) Iline wrote:
> Hi,
>
> Could you please take a look on a change which allows to run tests while collecting code coverage with JCov. This is a continuation of work done in
> JDK-8214309: Enhance makefiles to allow generating JCov instrumented build.
>
> This adds make targets jcov-test and others like jcov-test-tier1 etc. After running the tests, one is left with a coverage report and also a data file containing coverage, for further analysis.
>
> Part of the changes are related to increasing maximum and initial heap size, which is needed to be done because instrumented classes are bigger in size. The way it is done for JTReg tests is by setting both _JAVA_OPTIONS and JAVA_TOOL_OPTIONS environment variables.
>
> As it stands now, not all tests pass while running with code coverage, which pass normally. This particular enhancement is not addressing failing tests. The tests will need to be fixed separately if ever. For example, there are 35 tests which fail in open/test/jdk:jdk_core. A quick look tells that the majority of the failures are in tests which verify error output and discover an unexpected line "Picked up _JAVA_OPTIONS: -Xmx4g”.
>
> Enhancement: https://bugs.openjdk.java.net/browse/JDK-8215729
> Webrev: http://cr.openjdk.java.net/~shurailine/8215729/webrev.01/
>
> Shura