Review request: White box testing API for HotSpot

[Mikael Gerdin]

Hi David,
Sorry for the delayed response.

On 2011-12-05 07:18, David Holmes wrote:
> Hi Mikael,
>
> On 2/12/2011 8:46 PM, Mikael Gerdin wrote:
>> On 2011-12-02 06:32, David Holmes wrote:
>>> I'm a little confused as to where wb.jar ends up when I build hotspot. I
>>> see this in a makefile:
>>
>> There are a couple of issues in these make files.
>>>
>>> 26 WB = wb
>>> 27
>>> 28 WBSRCDIR = $(GAMMADIR)/src/share/tools/whitebox/src
>>> 29
>>> 30 WB_JAR = $(GENERATED)/$(WB).jar
>>> 31
>>> 32 DEST_WB_JAR = $(JAVA_HOME)/lib/$(WB_JAR)
>>>
>>> Why JAVA_HOME? That's normally a binary installation of a JDK used for
>>> building, not somewhere I expect my build to try and put something. Plus
>>> the above will expand to:
>>
>> For example, look in jsig.make. It has a target that copies libjsig to
>> JDK_LIBDIR. JDK_LIBDIR is set up in vm.make to point to
>> JAVA_HOME/jre/lib/[arch]. I was only trying to mimic existing behavior
>> with the "install"-targets in the make files.
>
> Our build system is somewhat confusing. The top-level Defs.make will set
> JAVA_HOME based on ABS_BOOTDIR which in turn is set by BOOTDIR which can
> be overridden by ALT_BOOTDIR. BOOTDIR, ALT_BOOTDIR and JAVA_HOME are all
> places where the build expects to find an executable JDK for performing
> build actions like javac compiles, javah etc.
>
> As you point out vm.make then sets JDK_LIBDIR based on JAVA_HOME and
> that is used by the install_* targets, which are dependencies of the
> vm.make install target. The vm.make install target is itself invoked
> from top.make's install target.
>
> So when do these install targets actually get used? Other than by a
> developer on the command-line I don't see these targets actually getting
> used - and they can't be specified as targets for the top-level
> Makefile. I suspect these may be relics from a time when you would do
> something like:
>
> JAVA_HOME=/my/local/jdk/to/test/ make product1 install

You are probably correct.
Do you want me to modify the make file to more closely mimic the 
behavior of the other make files and let the legacy "install" target 
stay or do you want me to just not add an install target?

>
>>> And if I build a full JDK, where does wb.jar end up then?
>>
>> $ find . -name wb.jar
>> ./build/linux-amd64/hotspot/import/jre/lib/endorsed/wb.jar
>> ./build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/generated/wb.jar
>>
>>
>>
>> The JDK makefiles that build the j2{sdk,re}-image directories do not
>> pick up the wb.jar file.
>
> Ok. So what is the expected build process here such that wb is available
> for use? Are you expecting the developer to do some kind of "make install"?

This is primarily intended for use together with our internal test 
infrastructure (nightly testing etc.). When running these tests there is 
already a requirement for a JDK to go together with a VM that we are 
testing. The same goes for jtreg and the HotSpot tests in the test/ 
subdirectory of the HotSpot repository.
So if you want to run tests on your own VM wouldn't you need to use 
something like "make ALT_EXPORT_PATH=/some/jdk all_fastdebug" do 
automatically copy your libjvm to a working JDK?


>
>>> I also see in make/Makefile:
>>>
>>> 370 $(EXPORT_JRE_LIB_DIR)/endorsed/%.jar: $(GEN_DIR)/%.jar
>>> 371 $(install-file)
>>>
>>> Why the endorsed subdirectory? This is nothing to do with an "endorsed
>>> standard":
>>>
>>> http://docs.oracle.com/javase/6/docs/technotes/guides/standards/
>>
>> Because of security requirements and implementation details the Whitebox
>> class must be available on the boot class path.
>> Putting the wb.jar file in the endorsed directory is a quick and easy
>> way to achieve that.
>
> Why not just in lib? Or perhaps lib/ext? endorsed just seems to be the
> least appropriate choice here.

Because jars in lib/endorsed are automatically added to the _boot_ class 
path.
We could put the jar in jre/lib/ext or jre/lib/ or just lib/
but then all tests using the API would need -Xbootclasspath as an 
additional command line flag.

>
> And now your usage model has confused me because the above export is
> done for JPRT builds - right? So you want this to be available in a JPRT
> bundle, but not in a full JDK build?

Nightly testing runs on bundles generated by JPRT, by having the API 
available in those bundles we can have tests that use this API in our 
nighly testing.
If we want to have this API available when doing a full JDK build we can 
revisit that particular point in the future since that would need to be 
synchronized with RE as well.

>
>> Does this clarify your concerns?
>
> Not yet :)

I'll keep trying then :)

/Mikael

>
> Thanks,
> David
>
>> /Mikael Gerdin
>>
>>>
>>> ???
>>>
>>> Thanks,
>>> David
>>> -----
>>>
>>>> If the VM crashes after this API has been accessed a note will be
>>>> written in the hs_err file to signal that the API has been used.
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>>>> (thanks to stefank for hosting my webrev :)
>>>>
>>>> CR:
>>>> I'll file a CR tomorrow.
>>>>
>>>> Change comments:
>>>>
>>>> make/jprt.properties
>>>>
>>>> Add a test target to make sure that the API is available on all
>>>> supported platforms
>>>>
>>>> make/**
>>>>
>>>> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
>>>> JAR file and copy it to the jre/lib/endorsed directory in the export
>>>> targets.
>>>> The BSD makefile changes are not tested since I don't have access to
>>>> any
>>>> BSD/OSX machine to test them on.
>>>>
>>>> src/share/vm/prims/nativeLookup.cpp
>>>>
>>>> Special-case the method sun/hotspot/WhiteBox/registerNatives and
>>>> link it
>>>> to JVM_RegisterWhiteBoxMethods
>>>>
>>>> src/share/vm/prims/whitebox.*
>>>>
>>>> The implementation of the white box API. The actual API functions are
>>>> only examples of what we want to be able to do using the API.
>>>>
>>>> src/share/vm/runtime/globals.hpp
>>>>
>>>> Add the command line flag
>>>>
>>>> src/share/vm/utilities/vmError.cpp
>>>>
>>>> Print a message in hs_err files when white box API has been used.
>>>>
>>>> test/Makefile
>>>>
>>>> Add a makefile test target for the white box API test
>>>>
>>>> test/sanity/wbapi.java
>>>>
>>>> JTreg test to ensure that the API works.
>>>>
>>>>
>>>> Thanks
>>>> /Mikael Gerdin