jtreg FAQ updates

[Jonathan Gibbons]

Chris,

This seems generally reasonable, and worth filing as an enhancement request,
although it might also trigger a Q in the FAQ on "How long should a test 
take?"

-- Jon


On 8/16/18 4:30 AM, Chris Hegarty wrote:
> Jon,
>
> Thanks for this. A question on testng.
>
>> On 16 Aug 2018, at 00:46, Jonathan Gibbons <jonathan.gibbons at oracle.com> wrote:
>> ...
>> 5.6  Can I put more than one test in a file?
> I think that you can put several jtreg-invoked testng tests in a single
> file. At least it works for me. For example:
>
>     /*
>      * @test
>      * @run testng/othervm Test
>      */
>     /*
>      * @test
>      * @run testng/othervm Test
>      */
>   
>    , but it is not all that useful.
>
> The reason I enquire about this is that we have several long running, and
> verbose, testng tests, that have several hundred combinations. These test
> take a long time to run and can generate a lot of data. The approach we’ve
> taken to date is to effectively create an abstract class that contains all the
> test logic and data providers. Then have several “driver” like subtypes that
> each override a single supertypes moral "test" method and add the testng
> @Test annotation. Example below.
>
> What we’d really like to do is to add several test descriptions to the main
> test specify something like:
>     /*
>      * @test
>      * @run testng:groups="synchronous"/othervm TestAsync
>      */
>     /*
>      * @test
>      * @run  testng:groups=“asynchronous"/othervm TestAsync
>      */
>
>   , or some such syntax that would allow us to pass in a testng test
> group. This would allow each group the same timeout, jtr output file
> limit ( if any ), etc, without the need for the extra clutter of the “driver”
> files. We have several of test drivers in java/net/httpclient, e.g.
> ThrowingXXX.java
>
> I’ve looked a little at jtreg’s support for testng and I think that this is
> doable. Before filing an enhancement request, or hacking on the code,
> is this something that you think would be accepted? I’m not sure if
> others have come across a need for it before, but it is certainly useful
> to the work being done by the HTTP Client, and the workaround that
> exists today is a little crummy.
>
> -Chris.
>
> $ cat test/jdk/java/net/httpclient/AbstractTest.java
>
> import java.util.concurrent.CompletableFuture;
> import org.testng.annotations.*;
>
> public abstract class AbstractTest {
>
>      @DataProvider(name = "values")
>      public Object[][] values() {
>          return new Object[][] { {1} , {2}, {3} };
>      }
>
>      //@Test(dataProvider = "values", groups = "synchronous")
>      void testSync(int value) {
>          System.out.println(value);
>      }
>
>      //@Test(dataProvider = "values", groups = “asynchronous")
>      void testAsync(int value) {
>          CompletableFuture.runAsync(() -> System.out.println(value)).join();
>      }
>
>      @BeforeTest
>      public void setup() throws Exception { /* ... */  }
>
>      @AfterTest
>      public void teardown() throws Exception { /* ... */  }
>
>      // More common code and infrastructure …
> }
>
> $ cat test/jdk/java/net/httpclient/TestSync.java
> /*
>   * @test
>   * @build AbstractTest
>   * @run testng/othervm TestSync
>   */
>
> import org.testng.annotations.Test;
>
> public class TestSync extends AbstractTest {
>
>      @Override
>      @Test(dataProvider = "values", groups = "synchronous")
>      void testSync(int value) {
>          super.testSync(value);
>      }
> }
>
> $ cat test/jdk/java/net/httpclient/TestAsync.java
> /*
>   * @test
>   * @build AbstractTest
>   * @run testng/othervm TestAsync
>   */
>
> import org.testng.annotations.Test;
>
> public class TestAsync extends AbstractTest {
>
>      @Override
>      @Test(dataProvider = "values", groups = “asynchronous")
>      void testAsync(int value) {
>          super.testAsync(value);
>      }
> }
>
>   
>