Review request: White box testing API for HotSpot

[David Holmes]

Hi Mikael,

I'll top-post for convenience :)

Two points:

1. I think it may be simpler to drop the install target - "install" 
seems to be somewhat of a historical relic

2. lib/ext jars have the same permissions as bootclasspath, so it should 
work to place it there.

Thanks,
David

On 9/12/2011 1:11 AM, Mikael Gerdin wrote:
> Hi David,
> Sorry for the delayed response.
>
> On 2011-12-05 07:18, David Holmes wrote:
>> Hi Mikael,
>>
>> On 2/12/2011 8:46 PM, Mikael Gerdin wrote:
>>> On 2011-12-02 06:32, David Holmes wrote:
>>>> I'm a little confused as to where wb.jar ends up when I build
>>>> hotspot. I
>>>> see this in a makefile:
>>>
>>> There are a couple of issues in these make files.
>>>>
>>>> 26 WB = wb
>>>> 27
>>>> 28 WBSRCDIR = $(GAMMADIR)/src/share/tools/whitebox/src
>>>> 29
>>>> 30 WB_JAR = $(GENERATED)/$(WB).jar
>>>> 31
>>>> 32 DEST_WB_JAR = $(JAVA_HOME)/lib/$(WB_JAR)
>>>>
>>>> Why JAVA_HOME? That's normally a binary installation of a JDK used for
>>>> building, not somewhere I expect my build to try and put something.
>>>> Plus
>>>> the above will expand to:
>>>
>>> For example, look in jsig.make. It has a target that copies libjsig to
>>> JDK_LIBDIR. JDK_LIBDIR is set up in vm.make to point to
>>> JAVA_HOME/jre/lib/[arch]. I was only trying to mimic existing behavior
>>> with the "install"-targets in the make files.
>>
>> Our build system is somewhat confusing. The top-level Defs.make will set
>> JAVA_HOME based on ABS_BOOTDIR which in turn is set by BOOTDIR which can
>> be overridden by ALT_BOOTDIR. BOOTDIR, ALT_BOOTDIR and JAVA_HOME are all
>> places where the build expects to find an executable JDK for performing
>> build actions like javac compiles, javah etc.
>>
>> As you point out vm.make then sets JDK_LIBDIR based on JAVA_HOME and
>> that is used by the install_* targets, which are dependencies of the
>> vm.make install target. The vm.make install target is itself invoked
>> from top.make's install target.
>>
>> So when do these install targets actually get used? Other than by a
>> developer on the command-line I don't see these targets actually getting
>> used - and they can't be specified as targets for the top-level
>> Makefile. I suspect these may be relics from a time when you would do
>> something like:
>>
>> JAVA_HOME=/my/local/jdk/to/test/ make product1 install
>
> You are probably correct.
> Do you want me to modify the make file to more closely mimic the
> behavior of the other make files and let the legacy "install" target
> stay or do you want me to just not add an install target?
>
>>
>>>> And if I build a full JDK, where does wb.jar end up then?
>>>
>>> $ find . -name wb.jar
>>> ./build/linux-amd64/hotspot/import/jre/lib/endorsed/wb.jar
>>> ./build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/generated/wb.jar
>>>
>>>
>>>
>>>
>>> The JDK makefiles that build the j2{sdk,re}-image directories do not
>>> pick up the wb.jar file.
>>
>> Ok. So what is the expected build process here such that wb is available
>> for use? Are you expecting the developer to do some kind of "make
>> install"?
>
> This is primarily intended for use together with our internal test
> infrastructure (nightly testing etc.). When running these tests there is
> already a requirement for a JDK to go together with a VM that we are
> testing. The same goes for jtreg and the HotSpot tests in the test/
> subdirectory of the HotSpot repository.
> So if you want to run tests on your own VM wouldn't you need to use
> something like "make ALT_EXPORT_PATH=/some/jdk all_fastdebug" do
> automatically copy your libjvm to a working JDK?
>
>
>>
>>>> I also see in make/Makefile:
>>>>
>>>> 370 $(EXPORT_JRE_LIB_DIR)/endorsed/%.jar: $(GEN_DIR)/%.jar
>>>> 371 $(install-file)
>>>>
>>>> Why the endorsed subdirectory? This is nothing to do with an "endorsed
>>>> standard":
>>>>
>>>> http://docs.oracle.com/javase/6/docs/technotes/guides/standards/
>>>
>>> Because of security requirements and implementation details the Whitebox
>>> class must be available on the boot class path.
>>> Putting the wb.jar file in the endorsed directory is a quick and easy
>>> way to achieve that.
>>
>> Why not just in lib? Or perhaps lib/ext? endorsed just seems to be the
>> least appropriate choice here.
>
> Because jars in lib/endorsed are automatically added to the _boot_ class
> path.
> We could put the jar in jre/lib/ext or jre/lib/ or just lib/
> but then all tests using the API would need -Xbootclasspath as an
> additional command line flag.
>
>>
>> And now your usage model has confused me because the above export is
>> done for JPRT builds - right? So you want this to be available in a JPRT
>> bundle, but not in a full JDK build?
>
> Nightly testing runs on bundles generated by JPRT, by having the API
> available in those bundles we can have tests that use this API in our
> nighly testing.
> If we want to have this API available when doing a full JDK build we can
> revisit that particular point in the future since that would need to be
> synchronized with RE as well.
>
>>
>>> Does this clarify your concerns?
>>
>> Not yet :)
>
> I'll keep trying then :)
>
> /Mikael
>
>>
>> Thanks,
>> David
>>
>>> /Mikael Gerdin
>>>
>>>>
>>>> ???
>>>>
>>>> Thanks,
>>>> David
>>>> -----
>>>>
>>>>> If the VM crashes after this API has been accessed a note will be
>>>>> written in the hs_err file to signal that the API has been used.
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>>>>> (thanks to stefank for hosting my webrev :)
>>>>>
>>>>> CR:
>>>>> I'll file a CR tomorrow.
>>>>>
>>>>> Change comments:
>>>>>
>>>>> make/jprt.properties
>>>>>
>>>>> Add a test target to make sure that the API is available on all
>>>>> supported platforms
>>>>>
>>>>> make/**
>>>>>
>>>>> Makefile changes to build the class sun/hotspot/WhiteBox, put it in a
>>>>> JAR file and copy it to the jre/lib/endorsed directory in the export
>>>>> targets.
>>>>> The BSD makefile changes are not tested since I don't have access to
>>>>> any
>>>>> BSD/OSX machine to test them on.
>>>>>
>>>>> src/share/vm/prims/nativeLookup.cpp
>>>>>
>>>>> Special-case the method sun/hotspot/WhiteBox/registerNatives and
>>>>> link it
>>>>> to JVM_RegisterWhiteBoxMethods
>>>>>
>>>>> src/share/vm/prims/whitebox.*
>>>>>
>>>>> The implementation of the white box API. The actual API functions are
>>>>> only examples of what we want to be able to do using the API.
>>>>>
>>>>> src/share/vm/runtime/globals.hpp
>>>>>
>>>>> Add the command line flag
>>>>>
>>>>> src/share/vm/utilities/vmError.cpp
>>>>>
>>>>> Print a message in hs_err files when white box API has been used.
>>>>>
>>>>> test/Makefile
>>>>>
>>>>> Add a makefile test target for the white box API test
>>>>>
>>>>> test/sanity/wbapi.java
>>>>>
>>>>> JTreg test to ensure that the API works.
>>>>>
>>>>>
>>>>> Thanks
>>>>> /Mikael Gerdin