Adding Microbenchmarks to the JDK forest/trees (JEP-230)

[Staffan Friberg]

Hi,

(Adding the jdk9-dev list to increase the visibility of the discussion)

With the multiple sub-repository commit mechanism improved I believe 
this might be less of an issue. JPRT can push JDK and HS changes at 
together and the same functionality should be possible to use for this 
as well. I wonder if the test issue earlier was that it was a completely 
separate repository outside of the JDK forest, and less of an issue when 
being part of the same forest as the JDK source code. Perhaps someone 
from SQE can chime?

Otherwise the main reason for having a separate sub-repository on the 
top level is making it easier to find what benchmarks are available and 
have a single place to add new once avoid any risk of name duplication. 
JMH is superb in filtering during execution during runtime so running 
just a single test or a group of tests is very straight forward and the 
recommended way, rather than having multiple benchmark JARs. It also 
makes the build process easier as the building can be done using a 
single Makefile and a single benchmark JAR (actually two, one for JDK 8 
compatible tests and one for JDK 9) that can be picked up by automatic 
performance testing.

Cheers,
Staffan

On 12/02/2014 06:48 AM, roger riggs wrote:
> Hi Staffan,
>
> An earlier issue was keeping tests in sync with the code under test, 
> hence
> the use of test directories within each repository.
> I think a structure in which the benchmarks for some function and the 
> function
> itself are in the same repository that is easier to understand and 
> maintain.
>
> $.02, Roger
>
>
> On 12/1/2014 7:08 PM, Staffan Friberg wrote:
>> Hi,
>>
>> Hopefully this is the right list for this discussion.
>>
>> As part of adding Microbenchmarks to the OpenJDK source tree, I'm 
>> trying to understand how we best would add the benchmark sources to 
>> the existing OpenJDK tree structure.
>>
>> Since the microbenchmark suite will cover all parts of the JDK, 
>> covering HotSpot, JDK libraries and Nashorn, it would be preferred to 
>> add the microbenchmark directory as a new top level directory. 
>> Something similar to the following structure. Having "benchmark" as 
>> the top-level directory would allow us to later add different types 
>> of benchmarks without colliding with the microbenchmark suite.
>>
>> <openjdk-root>/
>>    benchmark/microbenchmark/...
>>    hotspot/...
>>    jdk/...
>>    nashorn/...
>>
>> With this as the premise I can see the following 3 options for how 
>> this could be added to the source code layout
>>
>> 1. Part of jdk-root repository
>>      * Only makes sense if we want to move in a direction with fewer
>>        trees (and eventually a single tree)
>> 2. Part of another already existing tree
>>      * Not sure if this is possible without converting and moving the
>>        directory to a subdirectory of that tree
>> 3. New tree in the forest/tree structure
>>      * Most logical option as it follows the current setup and structure
>>
>>
>> Anyone have any comments and/or concerns on the suggested directory 
>> location and the tree structure in option 3.
>>
>> Would the build-dev team be the right group to later help setup a new 
>> tree if decided to be the right way to go?
>>
>> Regards,
>> Staffan
>>
>