Using lookups in place of reflection

[Andrew Dinn]

Being a good OpenJDK citizen I have attempted to make my agent use
lookups in jdk9 in place of using reflection. This is mostly working
fine. However, I have run across a rather arbitrary limitation (not just
my conclusion -- the jdk source code acknowledges it as such) and am
wondering whether there will be any chance of remedying it before (or
after) jdk9 release.

The problem (I'll come to the motivation as to /why/ it is a problem in
a second) is this method of class java.lang.invoke.MethodHandles.Lookup

private static void checkUnprivilegedlookupClass(Class<?> lookupClass,
int allowedModes) {
    String name = lookupClass.getName();
    if (name.startsWith("java.lang.invoke."))
        throw newIllegalArgumentException("illegal lookupClass:
"+lookupClass);
    // For caller-sensitive MethodHandles.lookup() disallow lookup from
    // restricted packages.  This a fragile and blunt approach.
    // TODO replace with a more formal and less fragile mechanism
    // that does not bluntly restrict classes under packages within
    // java.base from looking up MethodHandles or VarHandles.
    if (allowedModes == FULL_POWER_MODES && lookupClass.getClassLoader()
== null) {
        if ((name.startsWith("java.") &&
             !name.equals("java.lang.Thread") &&
             !name.startsWith("java.util.concurrent.")) ||
            (name.startsWith("sun.") &&
             !name.startsWith("sun.invoke."))) {
            throw newIllegalArgumentException("illegal lookupClass: " +
lookupClass);
        }
    }
}

This is indeed a 'blunt approach' -- I'm not sure  about the fragility
and it would be good to know what motivated this specific choice of
blunt instrument and what might constitute a more honed instrument.

What makes this so egregious is that this check method is invoked from
method Lookup.privateLookupIn() which, before instituting the check,
ensures, amongst other things, that the target class is open to the
module of the Lookup's 'caller' class. What that adds up to is this:

  the the original Lookup cannot be used to access protected or private
fields or methods of java.* or sun.* classes using a method handle

  reflection /can/ be used to access those same fields or methods

That seems like a rather perverse incentive to stick with reflection.

Now, why am I so concerned about this detail? Well, as an example, here
is a snippet of a Byteman rule used to test HttpURLConnection code:

  RULE java-net: Java HttpUrlConnection connect entry
  CLASS ^java.net.HttpURLConnection
  METHOD connect()
  HELPER io.opentracing.contrib.agent.OpenTracingHelper
  AT ENTRY
  IF !$0.connected && retrieveSpan($0) == null && !ignore($0)
  DO associateSpan($0, getTracer().buildSpan($0.getRequestMethod());
     ...
  ENDRULE

I won't provide a full account of what this rule does but basically it
defines code for the agent to inject into the connect() method of
java.net.HttpURLConnection and its subclasses (the latter indicated by
the ^). The purpose is to track the connection state so operation can be
validated at end of the test run. HELPER class OpenTracingHelper is just
a POJO that defines methods retrieveSpan(), ignore(), buildSpan() etc
which construct and maintain this tracking info n.b. there are several
more rules, defined on other methods of HttpURLConnection, which update
the Span object.

A few more specifics are called for. $0 identifies the target instance
for the connect() call i.e. an instance of HttpURLConnection or one of
its subclasses. In practice the rule code actually gets injected in
method connect() of the underlying implementation class
sun.net.www.protocol.http.HttpURLConnection so at runtime that will be
the type of $0.

The critical expression to attend to is the first term in the IF
condition '$0.connected'. This reads boolean field 'connected' defined
by superclass java.net.URLConnection of class HttpURLConnection. A test
of this field is needed because it is only appropriate to track
connection information for a URLConnection which has actually been
connected.

Unfortunately, field 'connected' is protected. Worse, none of the
classes in the URLConnection hierarchy provide an accessor for it. So,
the only way to implement this test/rule is to strong-arm a read of the
protected field. The Byteman agent has to apply one or other form of
leverage -- either reflection or a getter method handle. In jdk8 it
successfully reads the field using reflection. In jdk9 it tries to
behave well and use a getter but that is ruled out by the rather
arbitrary check above. Other rules run into problems with fields/methods
of the target class java.net.HttpURLConnection.

So, what am I asking:

  Is there a clearer rationale for this check which might lead to a less
restrictive variant?

  Can I haz it in jdk9 pleeze?

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander