RFR: 8337221: CompileFramework: test library to conveniently compile java and jasm sources for fuzzing [v20]

[Christian Hagedorn]

On Mon, 23 Sep 2024 12:32:13 GMT, Emanuel Peter <epeter at openjdk.org> wrote:

>> **Motivation**
>> 
>> I want to write small dedicated fuzzers:
>> - Generate `java` and `jasm` source code: just some `String`.
>> - Quickly compile it (with this framework).
>> - Execute the compiled code.
>> 
>> The primary users of the CompileFramework are Compiler-Engineers. Imagine you are working on some optimization. You already have a list of **hand-written tests**, but you are worried that this does not give you good coverage. You also do not trust that an existing Fuzzer will catch your bugs (at least not fast enough). Hence, you want to **script-generate** a large list of tests. But where do you put this script? It would be nice if it was also checked in on git, so that others can modify and maintain the test easily. But with such a script, you can only generate a **static test**. In some cases that is good enough, but sometimes the list of all possible tests your script would generate is very very large. Too large. So you need to randomly sample some of the tests. At this point, it would be nice to generate different tests with every run: a "mini-fuzzer" or a **fuzzer dedicated to a compiler feature**.
>> 
>> **The CompileFramework**
>> 
>> Java sources are compiled with `javac`, jasm sources with `asmtools` that are delivered with `jtreg`.
>> An important factor: Integration with the IR-Framwrork (`TestFramework`): we want to be able to generate IR-rules for our tests.
>> 
>> I implemented a first, simple version of the framework. I added some tests and examples.
>> 
>> **Example**
>> 
>> 
>> CompileFramework comp = new CompileFramework();
>> comp.add(SourceCode.newJavaSourceCode("XYZ", "<source-code-here>"));
>> comp.compile();
>> comp.invoke("XYZ", "test", new Object[] {5}); // XYZ.test(5);
>> 
>> 
>> https://github.com/openjdk/jdk/blob/e869cce8092ee995cf2f3ad1ab2bca69c5e256ab/test/hotspot/jtreg/testlibrary_tests/compile_framework/examples/SimpleJavaExample.java#L42-L74
>> 
>> **Below some use cases: tests that would have been better with the CompileFramework**
>> 
>> **Use case: test/hotspot/jtreg/compiler/loopopts/superword/TestAlignVectorFuzzer.java**
>> 
>> I needed to test loops with various `init / stride / limit / scale / unrolling-factor / ...`.
>> 
>> For this I used `MethodHandle constant = MethodHandles.constant(int.class, value);`,
>> 
>> to be able to chose different values before the C2 compilation, and then the C2 compilation would see them as constants and optimize assuming those constants. This works, but is difficult to extract reproducers once something i...
>
> Emanuel Peter has updated the pull request incrementally with one additional commit since the last revision:
> 
>   move some code for Christian

Looks great now! Some last nits but I'd consider it done now from my side. Thanks for the patience and applying all the suggestions :-)

test/hotspot/jtreg/testlibrary_tests/compile_framework/examples/IRFrameworkJavaExample.java line 51:

> 49: public class IRFrameworkJavaExample {
> 50: 
> 51:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

test/hotspot/jtreg/testlibrary_tests/compile_framework/examples/MultiFileJasmExample.java line 65:

> 63:     }
> 64: 
> 65:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

test/hotspot/jtreg/testlibrary_tests/compile_framework/examples/MultiFileJavaExample.java line 61:

> 59:     }
> 60: 
> 61:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

test/hotspot/jtreg/testlibrary_tests/compile_framework/examples/SimpleJasmExample.java line 57:

> 55:     }
> 56: 
> 57:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

test/hotspot/jtreg/testlibrary_tests/compile_framework/examples/SimpleJavaExample.java line 53:

> 51:     }
> 52: 
> 53:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

test/hotspot/jtreg/testlibrary_tests/compile_framework/tests/TestBadJasmCompilation.java line 47:

> 45:     }
> 46: 
> 47:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

test/hotspot/jtreg/testlibrary_tests/compile_framework/tests/TestBadJavaCompilation.java line 47:

> 45:     }
> 46: 
> 47:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

test/hotspot/jtreg/testlibrary_tests/compile_framework/tests/TestConcurrentCompilation.java line 90:

> 88:     }
> 89: 
> 90:     public static void main(String args[]) {

Suggestion:

    public static void main(String[] args) {

-------------

Marked as reviewed by chagedorn (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/20184#pullrequestreview-2322198277
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771412267
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771412897
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771413618
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771414312
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771414822
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771415535
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771417239
PR Review Comment: https://git.openjdk.org/jdk/pull/20184#discussion_r1771418447