Review request: White box testing API for HotSpot
Mikael Gerdin
mikael.gerdin at oracle.com
Mon Dec 12 05:13:43 PST 2011
David,
On 2011-12-11 12:10, David Holmes wrote:
> On 9/12/2011 11:42 PM, Mikael Gerdin wrote:
>> On 2011-12-09 05:25, David Holmes wrote:
>>> 2. lib/ext jars have the same permissions as bootclasspath, so it should
>>> work to place it there.
>>
>> I tried that first (before even discovering the existence of the
>> "endorsed" directory) but when I tried it the Whitebox class didn't get
>> have null as its classloader.
>
> Right - lib/ext is loaded by the extensions classloader not the
> bootstraploader. But this is supposed to have as many privileges as the
> bootstrap loader:
Yes, but does the VM know anything about the ext class loader?
Is there any way to check from inside the privilege level of a class? (I
suppose I can do some sort of upcall to the JDK to check that but is
there a shorter way?)
Also, there is one more detail about the boot class loader, see below.
>
> "By default, installed optional packages in this standard directory are
> trusted. That is, they are granted the same privileges as if they were
> core platform classes (those in rt.jar). This default privilege is
> specified in the system policy file (in
> <java-home>/jre/lib/security/java.policy), but can be overridden for a
> particular optional package by adding the appropriate policy file entry
> (see Permissions in the JDK).
>
> http://docs.oracle.com/javase/7/docs/technotes/guides/extensions/spec.html
>
> What is it that requires that wb.jar actually be on the bootclasspath?
In order to avoid adding more changes to nativeLookup.cpp i just added
the JVM_RegisterWhiteboxMethods to the array of methods explicitly
menttioned in this file.
The current code checks for the null class loader explicitly here:
156 Handle loader(THREAD,
157
instanceKlass::cast(method->method_holder())->class_loader());
158 if (loader.is_null()) {
159 entry = lookup_special_native(jni_name);
I could add another JNINativeMethod array and check for
"WhiteBox_registerNatives" on all class loaders.
If we go down this way I would have to verify with my security contact
that he's okay with dropping the boot class loader requirement.
/Mikael
>
> Cheers,
> David
>
>> Looking at the code in src/share/vm/runtime/arguments.cpp, the class
>> SysClassPath claims to be responsible for handling the boot class path.
>> I don't see any reference to lib/ext there but I'm not very familiar
>> with how the class loading code actually works.
>>
>> /Mikael
>>
>>>
>>> Thanks,
>>> David
>>>
>>> On 9/12/2011 1:11 AM, Mikael Gerdin wrote:
>>>> Hi David,
>>>> Sorry for the delayed response.
>>>>
>>>> On 2011-12-05 07:18, David Holmes wrote:
>>>>> Hi Mikael,
>>>>>
>>>>> On 2/12/2011 8:46 PM, Mikael Gerdin wrote:
>>>>>> On 2011-12-02 06:32, David Holmes wrote:
>>>>>>> I'm a little confused as to where wb.jar ends up when I build
>>>>>>> hotspot. I
>>>>>>> see this in a makefile:
>>>>>>
>>>>>> There are a couple of issues in these make files.
>>>>>>>
>>>>>>> 26 WB = wb
>>>>>>> 27
>>>>>>> 28 WBSRCDIR = $(GAMMADIR)/src/share/tools/whitebox/src
>>>>>>> 29
>>>>>>> 30 WB_JAR = $(GENERATED)/$(WB).jar
>>>>>>> 31
>>>>>>> 32 DEST_WB_JAR = $(JAVA_HOME)/lib/$(WB_JAR)
>>>>>>>
>>>>>>> Why JAVA_HOME? That's normally a binary installation of a JDK used
>>>>>>> for
>>>>>>> building, not somewhere I expect my build to try and put something.
>>>>>>> Plus
>>>>>>> the above will expand to:
>>>>>>
>>>>>> For example, look in jsig.make. It has a target that copies
>>>>>> libjsig to
>>>>>> JDK_LIBDIR. JDK_LIBDIR is set up in vm.make to point to
>>>>>> JAVA_HOME/jre/lib/[arch]. I was only trying to mimic existing
>>>>>> behavior
>>>>>> with the "install"-targets in the make files.
>>>>>
>>>>> Our build system is somewhat confusing. The top-level Defs.make will
>>>>> set
>>>>> JAVA_HOME based on ABS_BOOTDIR which in turn is set by BOOTDIR which
>>>>> can
>>>>> be overridden by ALT_BOOTDIR. BOOTDIR, ALT_BOOTDIR and JAVA_HOME are
>>>>> all
>>>>> places where the build expects to find an executable JDK for
>>>>> performing
>>>>> build actions like javac compiles, javah etc.
>>>>>
>>>>> As you point out vm.make then sets JDK_LIBDIR based on JAVA_HOME and
>>>>> that is used by the install_* targets, which are dependencies of the
>>>>> vm.make install target. The vm.make install target is itself invoked
>>>>> from top.make's install target.
>>>>>
>>>>> So when do these install targets actually get used? Other than by a
>>>>> developer on the command-line I don't see these targets actually
>>>>> getting
>>>>> used - and they can't be specified as targets for the top-level
>>>>> Makefile. I suspect these may be relics from a time when you would do
>>>>> something like:
>>>>>
>>>>> JAVA_HOME=/my/local/jdk/to/test/ make product1 install
>>>>
>>>> You are probably correct.
>>>> Do you want me to modify the make file to more closely mimic the
>>>> behavior of the other make files and let the legacy "install" target
>>>> stay or do you want me to just not add an install target?
>>>>
>>>>>
>>>>>>> And if I build a full JDK, where does wb.jar end up then?
>>>>>>
>>>>>> $ find . -name wb.jar
>>>>>> ./build/linux-amd64/hotspot/import/jre/lib/endorsed/wb.jar
>>>>>> ./build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/generated/wb.jar
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> The JDK makefiles that build the j2{sdk,re}-image directories do not
>>>>>> pick up the wb.jar file.
>>>>>
>>>>> Ok. So what is the expected build process here such that wb is
>>>>> available
>>>>> for use? Are you expecting the developer to do some kind of "make
>>>>> install"?
>>>>
>>>> This is primarily intended for use together with our internal test
>>>> infrastructure (nightly testing etc.). When running these tests
>>>> there is
>>>> already a requirement for a JDK to go together with a VM that we are
>>>> testing. The same goes for jtreg and the HotSpot tests in the test/
>>>> subdirectory of the HotSpot repository.
>>>> So if you want to run tests on your own VM wouldn't you need to use
>>>> something like "make ALT_EXPORT_PATH=/some/jdk all_fastdebug" do
>>>> automatically copy your libjvm to a working JDK?
>>>>
>>>>
>>>>>
>>>>>>> I also see in make/Makefile:
>>>>>>>
>>>>>>> 370 $(EXPORT_JRE_LIB_DIR)/endorsed/%.jar: $(GEN_DIR)/%.jar
>>>>>>> 371 $(install-file)
>>>>>>>
>>>>>>> Why the endorsed subdirectory? This is nothing to do with an
>>>>>>> "endorsed
>>>>>>> standard":
>>>>>>>
>>>>>>> http://docs.oracle.com/javase/6/docs/technotes/guides/standards/
>>>>>>
>>>>>> Because of security requirements and implementation details the
>>>>>> Whitebox
>>>>>> class must be available on the boot class path.
>>>>>> Putting the wb.jar file in the endorsed directory is a quick and easy
>>>>>> way to achieve that.
>>>>>
>>>>> Why not just in lib? Or perhaps lib/ext? endorsed just seems to be the
>>>>> least appropriate choice here.
>>>>
>>>> Because jars in lib/endorsed are automatically added to the _boot_
>>>> class
>>>> path.
>>>> We could put the jar in jre/lib/ext or jre/lib/ or just lib/
>>>> but then all tests using the API would need -Xbootclasspath as an
>>>> additional command line flag.
>>>>
>>>>>
>>>>> And now your usage model has confused me because the above export is
>>>>> done for JPRT builds - right? So you want this to be available in a
>>>>> JPRT
>>>>> bundle, but not in a full JDK build?
>>>>
>>>> Nightly testing runs on bundles generated by JPRT, by having the API
>>>> available in those bundles we can have tests that use this API in our
>>>> nighly testing.
>>>> If we want to have this API available when doing a full JDK build we
>>>> can
>>>> revisit that particular point in the future since that would need to be
>>>> synchronized with RE as well.
>>>>
>>>>>
>>>>>> Does this clarify your concerns?
>>>>>
>>>>> Not yet :)
>>>>
>>>> I'll keep trying then :)
>>>>
>>>> /Mikael
>>>>
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>> /Mikael Gerdin
>>>>>>
>>>>>>>
>>>>>>> ???
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>> -----
>>>>>>>
>>>>>>>> If the VM crashes after this API has been accessed a note will be
>>>>>>>> written in the hs_err file to signal that the API has been used.
>>>>>>>>
>>>>>>>> Webrev:
>>>>>>>> http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/
>>>>>>>> (thanks to stefank for hosting my webrev :)
>>>>>>>>
>>>>>>>> CR:
>>>>>>>> I'll file a CR tomorrow.
>>>>>>>>
>>>>>>>> Change comments:
>>>>>>>>
>>>>>>>> make/jprt.properties
>>>>>>>>
>>>>>>>> Add a test target to make sure that the API is available on all
>>>>>>>> supported platforms
>>>>>>>>
>>>>>>>> make/**
>>>>>>>>
>>>>>>>> Makefile changes to build the class sun/hotspot/WhiteBox, put it
>>>>>>>> in a
>>>>>>>> JAR file and copy it to the jre/lib/endorsed directory in the
>>>>>>>> export
>>>>>>>> targets.
>>>>>>>> The BSD makefile changes are not tested since I don't have
>>>>>>>> access to
>>>>>>>> any
>>>>>>>> BSD/OSX machine to test them on.
>>>>>>>>
>>>>>>>> src/share/vm/prims/nativeLookup.cpp
>>>>>>>>
>>>>>>>> Special-case the method sun/hotspot/WhiteBox/registerNatives and
>>>>>>>> link it
>>>>>>>> to JVM_RegisterWhiteBoxMethods
>>>>>>>>
>>>>>>>> src/share/vm/prims/whitebox.*
>>>>>>>>
>>>>>>>> The implementation of the white box API. The actual API functions
>>>>>>>> are
>>>>>>>> only examples of what we want to be able to do using the API.
>>>>>>>>
>>>>>>>> src/share/vm/runtime/globals.hpp
>>>>>>>>
>>>>>>>> Add the command line flag
>>>>>>>>
>>>>>>>> src/share/vm/utilities/vmError.cpp
>>>>>>>>
>>>>>>>> Print a message in hs_err files when white box API has been used.
>>>>>>>>
>>>>>>>> test/Makefile
>>>>>>>>
>>>>>>>> Add a makefile test target for the white box API test
>>>>>>>>
>>>>>>>> test/sanity/wbapi.java
>>>>>>>>
>>>>>>>> JTreg test to ensure that the API works.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> /Mikael Gerdin
More information about the hotspot-dev
mailing list