Review request: White box testing API for HotSpot
David Holmes
david.holmes at oracle.com
Mon Dec 12 23:04:53 PST 2011
On 13/12/2011 4:48 PM, Krystal Mok wrote:
> Hi Mikael and David,
>
> As an alternative, how about inserting wb.jar into the bootclasspath
> in Arguments::parse_vm_init_args() when -XX:+EnableWhiteboxAPI is
> enabled, just like the way alt-rt.jar is inserted when
> -XX:+AggressiveOpts is given, in the product JDK 6 and 7?
And presumably place wb.jar in jre/lib the way alt-rt.jar is.
That seems like a good suggestion - thanks Kris.
David
> - Kris
>
> On Tue, Dec 13, 2011 at 11:16 AM, David Holmes <david.holmes at oracle.com
> <mailto:david.holmes at oracle.com>> wrote:
>
> Hi Mikael,
>
> I see why you are dependent on being loaded by the boot loader, but
> I think using the endorsed mechanism for that is somewhat of a hack
> - this isn't anything to do with being endorsed it is just a way to
> get on the bootclasspath without modifying the bootclasspath.
>
> I'd like to hear what others think of this.
>
> David
>
>
> On 12/12/2011 11:13 PM, Mikael Gerdin wrote:
>
> David,
>
> On 2011-12-11 12:10, David Holmes wrote:
>
> On 9/12/2011 11:42 PM, Mikael Gerdin wrote:
>
> On 2011-12-09 05:25, David Holmes wrote:
>
> 2. lib/ext jars have the same permissions as
> bootclasspath, so it
> should
> work to place it there.
>
>
> I tried that first (before even discovering the
> existence of the
> "endorsed" directory) but when I tried it the Whitebox
> class didn't get
> have null as its classloader.
>
>
> Right - lib/ext is loaded by the extensions classloader not the
> bootstraploader. But this is supposed to have as many
> privileges as the
> bootstrap loader:
>
>
> Yes, but does the VM know anything about the ext class loader?
> Is there any way to check from inside the privilege level of a
> class? (I
> suppose I can do some sort of upcall to the JDK to check that but is
> there a shorter way?)
>
> Also, there is one more detail about the boot class loader, see
> below.
>
>
> "By default, installed optional packages in this standard
> directory are
> trusted. That is, they are granted the same privileges as if
> they were
> core platform classes (those in rt.jar). This default
> privilege is
> specified in the system policy file (in
> <java-home>/jre/lib/security/__java.policy), but can be
> overridden for a
> particular optional package by adding the appropriate policy
> file entry
> (see Permissions in the JDK).
>
> http://docs.oracle.com/javase/__7/docs/technotes/guides/__extensions/spec.html
> <http://docs.oracle.com/javase/7/docs/technotes/guides/extensions/spec.html>
>
>
> What is it that requires that wb.jar actually be on the
> bootclasspath?
>
>
> In order to avoid adding more changes to nativeLookup.cpp i just
> added
> the JVM_RegisterWhiteboxMethods to the array of methods explicitly
> menttioned in this file.
>
> The current code checks for the null class loader explicitly here:
> 156 Handle loader(THREAD,
> 157
> instanceKlass::cast(method->__method_holder())->class___loader());
> 158 if (loader.is_null()) {
> 159 entry = lookup_special_native(jni___name);
>
> I could add another JNINativeMethod array and check for
> "WhiteBox_registerNatives" on all class loaders.
> If we go down this way I would have to verify with my security
> contact
> that he's okay with dropping the boot class loader requirement.
>
> /Mikael
>
>
> Cheers,
> David
>
> Looking at the code in
> src/share/vm/runtime/__arguments.cpp, the class
> SysClassPath claims to be responsible for handling the
> boot class path.
> I don't see any reference to lib/ext there but I'm not
> very familiar
> with how the class loading code actually works.
>
> /Mikael
>
>
> Thanks,
> David
>
> On 9/12/2011 1:11 AM, Mikael Gerdin wrote:
>
> Hi David,
> Sorry for the delayed response.
>
> On 2011-12-05 07:18, David Holmes wrote:
>
> Hi Mikael,
>
> On 2/12/2011 8:46 PM, Mikael Gerdin wrote:
>
> On 2011-12-02 06:32, David Holmes wrote:
>
> I'm a little confused as to where
> wb.jar ends up when I build
> hotspot. I
> see this in a makefile:
>
>
> There are a couple of issues in these
> make files.
>
>
> 26 WB = wb
> 27
> 28 WBSRCDIR =
> $(GAMMADIR)/src/share/tools/__whitebox/src
> 29
> 30 WB_JAR = $(GENERATED)/$(WB).jar
> 31
> 32 DEST_WB_JAR =
> $(JAVA_HOME)/lib/$(WB_JAR)
>
> Why JAVA_HOME? That's normally a
> binary installation of a JDK used
> for
> building, not somewhere I expect my
> build to try and put something.
> Plus
> the above will expand to:
>
>
> For example, look in jsig.make. It has a
> target that copies
> libjsig to
> JDK_LIBDIR. JDK_LIBDIR is set up in
> vm.make to point to
> JAVA_HOME/jre/lib/[arch]. I was only
> trying to mimic existing
> behavior
> with the "install"-targets in the make
> files.
>
>
> Our build system is somewhat confusing. The
> top-level Defs.make will
> set
> JAVA_HOME based on ABS_BOOTDIR which in turn
> is set by BOOTDIR which
> can
> be overridden by ALT_BOOTDIR. BOOTDIR,
> ALT_BOOTDIR and JAVA_HOME are
> all
> places where the build expects to find an
> executable JDK for
> performing
> build actions like javac compiles, javah etc.
>
> As you point out vm.make then sets
> JDK_LIBDIR based on JAVA_HOME and
> that is used by the install_* targets, which
> are dependencies of the
> vm.make install target. The vm.make install
> target is itself invoked
> from top.make's install target.
>
> So when do these install targets actually
> get used? Other than by a
> developer on the command-line I don't see
> these targets actually
> getting
> used - and they can't be specified as
> targets for the top-level
> Makefile. I suspect these may be relics from
> a time when you would do
> something like:
>
> JAVA_HOME=/my/local/jdk/to/__test/ make
> product1 install
>
>
> You are probably correct.
> Do you want me to modify the make file to more
> closely mimic the
> behavior of the other make files and let the
> legacy "install" target
> stay or do you want me to just not add an
> install target?
>
>
> And if I build a full JDK, where
> does wb.jar end up then?
>
>
> $ find . -name wb.jar
> ./build/linux-amd64/hotspot/__import/jre/lib/endorsed/wb.jar
> ./build/linux-amd64/hotspot/__outputdir/linux_amd64___compiler2/generated/wb.jar
>
>
>
>
>
>
>
>
> The JDK makefiles that build the
> j2{sdk,re}-image directories do not
> pick up the wb.jar file.
>
>
> Ok. So what is the expected build process
> here such that wb is
> available
> for use? Are you expecting the developer to
> do some kind of "make
> install"?
>
>
> This is primarily intended for use together with
> our internal test
> infrastructure (nightly testing etc.). When
> running these tests
> there is
> already a requirement for a JDK to go together
> with a VM that we are
> testing. The same goes for jtreg and the HotSpot
> tests in the test/
> subdirectory of the HotSpot repository.
> So if you want to run tests on your own VM
> wouldn't you need to use
> something like "make ALT_EXPORT_PATH=/some/jdk
> all_fastdebug" do
> automatically copy your libjvm to a working JDK?
>
>
>
> I also see in make/Makefile:
>
> 370
> $(EXPORT_JRE_LIB_DIR)/__endorsed/%.jar:
> $(GEN_DIR)/%.jar
> 371 $(install-file)
>
> Why the endorsed subdirectory? This
> is nothing to do with an
> "endorsed
> standard":
>
> http://docs.oracle.com/javase/__6/docs/technotes/guides/__standards/
> <http://docs.oracle.com/javase/6/docs/technotes/guides/standards/>
>
>
> Because of security requirements and
> implementation details the
> Whitebox
> class must be available on the boot
> class path.
> Putting the wb.jar file in the endorsed
> directory is a quick and
> easy
> way to achieve that.
>
>
> Why not just in lib? Or perhaps lib/ext?
> endorsed just seems to be
> the
> least appropriate choice here.
>
>
> Because jars in lib/endorsed are automatically
> added to the _boot_
> class
> path.
> We could put the jar in jre/lib/ext or jre/lib/
> or just lib/
> but then all tests using the API would need
> -Xbootclasspath as an
> additional command line flag.
>
>
> And now your usage model has confused me
> because the above export is
> done for JPRT builds - right? So you want
> this to be available in a
> JPRT
> bundle, but not in a full JDK build?
>
>
> Nightly testing runs on bundles generated by
> JPRT, by having the API
> available in those bundles we can have tests
> that use this API in our
> nighly testing.
> If we want to have this API available when doing
> a full JDK build we
> can
> revisit that particular point in the future
> since that would need
> to be
> synchronized with RE as well.
>
>
> Does this clarify your concerns?
>
>
> Not yet :)
>
>
> I'll keep trying then :)
>
> /Mikael
>
>
> Thanks,
> David
>
> /Mikael Gerdin
>
>
> ???
>
> Thanks,
> David
> -----
>
> If the VM crashes after this API
> has been accessed a note will be
> written in the hs_err file to
> signal that the API has been used.
>
> Webrev:
> http://cr.openjdk.java.net/~__stefank/mgerdin/wbapi.0/__webrev/
> <http://cr.openjdk.java.net/~stefank/mgerdin/wbapi.0/webrev/>
> (thanks to stefank for hosting
> my webrev :)
>
> CR:
> I'll file a CR tomorrow.
>
> Change comments:
>
> make/jprt.properties
>
> Add a test target to make sure
> that the API is available on all
> supported platforms
>
> make/**
>
> Makefile changes to build the
> class sun/hotspot/WhiteBox, put it
> in a
> JAR file and copy it to the
> jre/lib/endorsed directory in the
> export
> targets.
> The BSD makefile changes are not
> tested since I don't have
> access to
> any
> BSD/OSX machine to test them on.
>
> src/share/vm/prims/__nativeLookup.cpp
>
> Special-case the method
> sun/hotspot/WhiteBox/__registerNatives
> and
> link it
> to JVM_RegisterWhiteBoxMethods
>
> src/share/vm/prims/whitebox.*
>
> The implementation of the white
> box API. The actual API functions
> are
> only examples of what we want to
> be able to do using the API.
>
> src/share/vm/runtime/globals.__hpp
>
> Add the command line flag
>
> src/share/vm/utilities/__vmError.cpp
>
> Print a message in hs_err files
> when white box API has been used.
>
> test/Makefile
>
> Add a makefile test target for
> the white box API test
>
> test/sanity/wbapi.java
>
> JTreg test to ensure that the
> API works.
>
>
> Thanks
> /Mikael Gerdin
>
>
More information about the hotspot-dev
mailing list