RFR: 8275170: Some jtreg sound tests should be marked headful

[Sergey Bylokhov]

On Fri, 22 Oct 2021 19:01:27 GMT, Phil Race <prr at openjdk.org> wrote:

> This fix adds "headful sound" keywords to a number of javax/sound jtreg tests
> 
> The jtreg javax.sound tests are written in such a way that if a needed audio device
> or other platform resource is not available, they just exit with a "pass" for the test.
> It is as if headful tests just swallowed HeadlessException and issued a pass.
> If we allowed that we'd be effectively testing almost nothing of real importance.
> 
> Currently almost  all sound tests have no keyword like "headful" to indicate they
> may need access to a hardware resource to do anything useful so a similar
> situation arises there except when someone runs those tests manually on
> a local system which has audio devices.
> 
> Of course "headful" and "audio device" aren't exactly interchangeable terms
> but if tests are allowed to be run - or worse usually or always run - in a situation
> where they potentially are on a system without an audio device (a headless VM) or are running in
> a session which doesn't have full desktop access - which may in some
> cases be how audio device access is granted, then they are more likely
> to be running in this scenario where the testing isn't valid.
> 
> Another point is that headful tests must be run one at a time - no concurrency because
> you can't have multiple tests moving the mouse at the same time
> Whereas for headless tests, a test harness may choose to run concurrently - perhaps even in the same VM
> When tests that really need access to an audio device are run it is probably safer to consider
> the headful attribute as implying one test at a time is best .. granted on a desktop it isn't
> usual for a single app to be able to monopolize access to a speaker, but for input devices
> that is what you'd generally expect.
> 
> By no means am I sure that the tests being updated here are the full set that need tagging.
> There are a lot of tests and they are all known to pass on systems with NO audio
> devices, so the search has been for tests which call APIs which will definitely
> need access to some device in order to do anything useful.
> So likely there are more to be added - either by a reviewer pointing them out or by later discovery.
> Alternatively we could mark ALL sound tests headful .. but given where we are starting from
> that might be erring unnecessarily far in the opposite direction.
> 
> By adding both headful and sound keywords it gives options to someone who
> wants to identify the tests and perhaps run just tests which need "sound" on some
> config they know supports audio but isn't headful.

I am not sure this is the right thing to do, as you pointed out the headless system may or may not have the sound configured, similar to the headful system may not have a sound. I see a lot of headful systems in the cloud where the audio device is not configured, and the opposite where the headless system actually has some audio devices. And this change just makes things complicated by assignee the headful keyword to the unrelated sound system.

Actually, it is the step in the opposite direction that was done when these tests were moved to tier3, at that the headful keyword was removed from these tests to make tier3 coverage as much as possible.

I am still not sure what problem do you want to solve? If the problem is to run the tests on the system where the sound is configured then just run them there, so it will not be necessary to validate each test does it really need a sound/headful keyword or not, otherwise what is the benefit?

-------------

PR: https://git.openjdk.java.net/jdk/pull/6086