Adding Microbenchmarks to the JDK forest/trees (JEP-230)

[joe darcy]

Hello,

On 12/4/2014 4:34 PM, David Holmes wrote:
> Hi Staffan,
>
> On 2/12/2014 10:08 AM, Staffan Friberg wrote:
>> Hi,
>>
>> Hopefully this is the right list for this discussion.
>>
>> As part of adding Microbenchmarks to the OpenJDK source tree, I'm trying
>> to understand how we best would add the benchmark sources to the
>> existing OpenJDK tree structure.
>
> Is there a reason this needs to be inside the OpenJDK instead of a 
> stand-alone project? If it ends up in its own repo and the repo is not 
> needed by anything else, then it is already like a stand-alone project.

I think David is raising a good question here. A related question is if 
we want to update/fix the microbenchmarks, in how many release trains 
will we want to make that fix? If the expected answer is much greater 
than one, to me that would seem to me to argue for a separate forest in 
the overall OpenJDK effort, but not bundled into a particular release.

For example, in the past, the webrev.ksh script was included in the JDK 
forest. That was an improvement over every engineer having his or her 
personal fork, but still made keeping webrev updated unnecessarily 
difficult since any changes would need to be done multiple time and 
there is nothing fundamentally binding a version of webrev to a 
particular JDK release. Likewise, even though (to my knowledge) jtreg is 
only used for testing the JDK, jtreg has its own repository and release 
schedule separate from any JDK release.

So if the microbenchmarks are to a first-approximation version agnostic, 
then they should probably have a forest not associated with a JDK 
release. If they are tightly bound to a release, that argues for putting 
them into the JDK forest itself.

-Joe


>
> David
>
>> Since the microbenchmark suite will cover all parts of the JDK, covering
>> HotSpot, JDK libraries and Nashorn, it would be preferred to add the
>> microbenchmark directory as a new top level directory. Something similar
>> to the following structure. Having "benchmark" as the top-level
>> directory would allow us to later add different types of benchmarks
>> without colliding with the microbenchmark suite.
>>
>> <openjdk-root>/
>>     benchmark/microbenchmark/...
>>     hotspot/...
>>     jdk/...
>>     nashorn/...
>>
>> With this as the premise I can see the following 3 options for how this
>> could be added to the source code layout
>>
>> 1. Part of jdk-root repository
>>       * Only makes sense if we want to move in a direction with fewer
>>         trees (and eventually a single tree)
>> 2. Part of another already existing tree
>>       * Not sure if this is possible without converting and moving the
>>         directory to a subdirectory of that tree
>> 3. New tree in the forest/tree structure
>>       * Most logical option as it follows the current setup and 
>> structure
>>
>>
>> Anyone have any comments and/or concerns on the suggested directory
>> location and the tree structure in option 3.
>>
>> Would the build-dev team be the right group to later help setup a new
>> tree if decided to be the right way to go?
>>
>> Regards,
>> Staffan
>>