jtreg FAQ updates

[Chris Hegarty]

Jon,

Thanks for this. A question on testng. 

> On 16 Aug 2018, at 00:46, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
> ...
> 5.6  Can I put more than one test in a file?

I think that you can put several jtreg-invoked testng tests in a single
file. At least it works for me. For example:

   /*
    * @test
    * @run testng/othervm Test
    */
   /*
    * @test
    * @run testng/othervm Test
    */
 
  , but it is not all that useful.

The reason I enquire about this is that we have several long running, and
verbose, testng tests, that have several hundred combinations. These test
take a long time to run and can generate a lot of data. The approach we’ve
taken to date is to effectively create an abstract class that contains all the
test logic and data providers. Then have several “driver” like subtypes that
each override a single supertypes moral "test" method and add the testng
@Test annotation. Example below.

What we’d really like to do is to add several test descriptions to the main
test specify something like:
   /*
    * @test
    * @run testng:groups="synchronous"/othervm TestAsync
    */
   /*
    * @test
    * @run  testng:groups=“asynchronous"/othervm TestAsync
    */

 , or some such syntax that would allow us to pass in a testng test
group. This would allow each group the same timeout, jtr output file
limit ( if any ), etc, without the need for the extra clutter of the “driver”
files. We have several of test drivers in java/net/httpclient, e.g.
ThrowingXXX.java

I’ve looked a little at jtreg’s support for testng and I think that this is
doable. Before filing an enhancement request, or hacking on the code,
is this something that you think would be accepted? I’m not sure if
others have come across a need for it before, but it is certainly useful
to the work being done by the HTTP Client, and the workaround that
exists today is a little crummy.

-Chris.

$ cat test/jdk/java/net/httpclient/AbstractTest.java 

import java.util.concurrent.CompletableFuture;
import org.testng.annotations.*;

public abstract class AbstractTest {

    @DataProvider(name = "values")
    public Object[][] values() {
        return new Object[][] { {1} , {2}, {3} };
    }

    //@Test(dataProvider = "values", groups = "synchronous")
    void testSync(int value) {
        System.out.println(value);
    }

    //@Test(dataProvider = "values", groups = “asynchronous")
    void testAsync(int value) {
        CompletableFuture.runAsync(() -> System.out.println(value)).join();
    }

    @BeforeTest
    public void setup() throws Exception { /* ... */  }

    @AfterTest
    public void teardown() throws Exception { /* ... */  }

    // More common code and infrastructure …
}

$ cat test/jdk/java/net/httpclient/TestSync.java 
/*
 * @test
 * @build AbstractTest
 * @run testng/othervm TestSync
 */

import org.testng.annotations.Test;

public class TestSync extends AbstractTest {

    @Override
    @Test(dataProvider = "values", groups = "synchronous")
    void testSync(int value) {
        super.testSync(value);
    }
}

$ cat test/jdk/java/net/httpclient/TestAsync.java 
/*
 * @test
 * @build AbstractTest
 * @run testng/othervm TestAsync
 */

import org.testng.annotations.Test;

public class TestAsync extends AbstractTest {

    @Override
    @Test(dataProvider = "values", groups = “asynchronous")
    void testAsync(int value) {
        super.testAsync(value);
    }
}