RFR: 8326831 Clarify test harness control variables in make help

[Erik Joelsson]

On Tue, 27 Feb 2024 13:56:56 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

> I'll note that you still need to run configure to execute make test-only JTREG=help. Should this really be necessary?

I agree that it would be convenient, in some rare usecase, to be able to run this without configure, but the added complexity needed to implement that is hardly worth the effort. In practice, if you are about to run `make test`, and trying to construct your command line, you would have already run configure.

> Also, while `make test-only JTREG=help` lists the valid options, it still does nothing to document their meaning.

The documentation for running tests can be found in `doc/testing.md`. The help output from make is just meant to be a short version reminder for common cases. The list is constructed programmatically from the makefile constructs used to define the possible parameters, so it's guaranteed to be up to date. That is also why you must call the same target as when using the parameter (`test` or `test-only` in this case) as otherwise you wouldn't end up in the makefile where the parameters are defined. You can't really expect makefiles to be fully self documenting as we are limited by what we can reasonably implement in makefiles after all, and these makefiles are really pushing the boundaries of that already.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18028#issuecomment-1967575782