#ReflectiveAccessByInstrumentationAgents

[Andrew Dinn]

Hi Alan,

I'm finally replying to your previous post and also reporting news of
the solution I have currently investigated to retain existing Byteman
agent capabilities (I have a working implementation and am interested to
see how high it causes your eyebrows to rise :-).

> If the instrumentation is just injecting code to call into Byteman then
> I assume it just works now. That is, it doesn't appear that Byteman has
> embraced modules yet and so I assume it must be an agent specified via
> -javaagent with perhaps supporting code in other JAR files. In that
> case, the Byteman code is the unnamed module of the app class loader and
> the VM will automatically add a read edge when the transformer touches
> code in any named module. This is the compatibility support that I
> mentioned in another mail.

All the Byteman code which matters is in a single jar loaded using
-javaagent (or the VM_Attach API for dynamic load). The Main class
(implementing premain/agentmain) is completely isolated from the rest of
the code (i.e. no link references -- it loads by name and executes by
reflection).

That isolation is necessary in order to be able to inject into JDK
runtime classes. Main is loaded by the system classloader. However,
Main.premain/agentmain will 'hoist' the jar into the bootstrap path on
request, ensuring that the rest of the agent jar contents are loaded by
the bootstrap loader. In particular this means JDK classes can reference
the 3 Byteman exception types and 1 static method that appear in
injected code.

n.b. I have zero desire to make Byteman (Jigsaw-)modular. That would
definitely require a dual source implementation (since I need to keep it
working on JDK6+).

> If you right this it should be simple as you say. On the other hand,
> some of the mails mention accessing non-public members, setAccessible,
> and even (it would appear) adjusting class loader delegation on the fly
> to facilitate the patches.

Byteman is mostly used for

  i) unit, integration and system tests
  ii) monitoring/debugging apps -- normally (but not always :-) in a sandbox

  In the former case, breaking the app (injection of code which
generates a fault scenario) or validating execution (injecting code
which asserts invariants) is often only only feasible if injected code
can access non-public APIs and/or data. In any significant size app the
stuff you need to check/call at test time is almost always stuff you
also want to hide behind an encapsulation barrier at runtime.

  In the latter case identifying what is broken when trouble-shooting a
problem generally requires the same license. If you are lucky your error
might affect public code/data but you'd be very lucky.

So, if Jigsaw were able to remove the existing opportunity for an agent
to access non-public code/data then that would seriously limit the
usefulness of Byteman -- almost to the point of making it useless. Test
code can often get by in cases where there is public access to data
(modulo getting a handle on the relevant objects). It's precisely the
private stuff that cannot easily be tested.

The reason I use setAccessible in the current implementation is that it
is the only way for injected code to access private data, methods and
constructors. Field, Method and Constructor instances are used by the
interpreter (which executes reflectively) and also as a backdoor to
perform such accesses when Byteman translates injected code to bytecode.

I am not (yet) interested in the issue of injecting into non-public app
classes/methods embedded in Modules nor am I (yet) interested in
allowing injected code to access non-public data/methods of such app
classes. That's not really an important problem for me (yet) because no
such code exists. When I need to address this issue I hope to resolve
any problems by extending the existing module IMPORT syntax which is
used to notify Byteman that injected code needs access to classes (&
their methods/data) deployed in JBoss Modules.

My current concern is for injected code to retain the ability to access
non-public fields of classes residing in JDK modules, in particular
java.base but also the other modules into which the JDK code base has
been partitioned. This currently works on JDK8- and a lot of projects
are relying on it continuing to work in order to be able to test their code.

Luckily, java.base exports to any module and also exports many of the
packages which users tend to inject into (e.g. java.lang.Thread,
java.io.FileInputStream, etc). So ,the checks performed by method
AccessibleObject.checkCanSetAccessible(Class<?>,Class<?>()) tend to vote
yes for most attempts to execute setAccessible i.e. the legacy problem
is not as pressing as it appears at first glance. However, there will
still be cases where existing Byteman usage will fail on JDK9.

I looked at several ways of making this work and decided the best thing
was to have the agent redefine AccessibleObject.checkCanSetAccessible so
that it grants Byteman code (specifically one Byteman class called Rule)
free reign when it calls this method. This still retains security
manager checks made around this call. Choosing this method targets the
code which does module access checks.

Basically I transformed this method so that it looks like this

  public boolean checkCanSetAccessible(Class<?> caller, Class<?>
declaringClass)
  {
    // injected code
    if ("org.jboss.byteman.rule.Rule".equals(caller.getName()) {
       if (caller.getClassLoader == Classloader.getSystemCalssLoader())
          return true
       }
    }
    // original code
    Module callerModule = caller.getModule();
    Module declaringModule = declaringClass.getModule();
    . . .

n.b. when the agent code is loaded in the bootstrap the 2nd if  compares
against null rather than the system loader. Either way the name+loader
check ensures that it is not possible to spoof calls from any old app
class called Rule. When this code is injected the oonly caller class
which will pass this initial test is Byteman's Rule class.

So, with that in place whenever the agent needs to grant access to a
Field/Method I ensure it calls calls method
Rule.grantAccess(AccessibleObject) which is only accessible from code
provided by the agent. The resulting accessible is private to agent Rule
instances and can only be accessed by the interpreter or by generated
bytecode when executing injected code. So, there is no danger of leaking
these access objects.

The nice thing is that I can do all this using only JDK6 compatible code
with no need for it to know about anything Jigsaw-related other than
whether class java.lang.reflect Module exists (thanks, Peter :-).

Now, I know setting an arbitrary Member accessible looks a tad dangerous
from the point of view that it might foil expectations of the JDK
runtime or JVM (from the point of view of app this is the users
responsibilty -- they ask for it they get it). However, I looked at the
rest of the code in method checkCanSetAccessible and noted that any
caller in the java.base module is allowed to set any Member to be
accessible. So, I adopted the working hypothesis that for now, at least,
making this method say 'yes' is not going to break anything that could
not already potentially be broken -- in particular it's not going to
foil the expectations of the JIT (yet? :-).

So, I'd be very interested to know how horrified you are to hear about
what I am doing and whether you have any caveats to throw my way :-)

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander