Accessing class files for named modules

[Rafael Winterhalter]

Hello everybody,

after I gave feedback a while ago I recently revisited how to port code
generation libraries to Java 9 and unfortunately, this still does not seem
possible with the current implementation. I am speaking based on experience
with Javassist and Byte Buddy which are both broadly used within the Java
ecosystem (several million downloads a year). I think I am competent to
speak on this topic as I wrote Byte Buddy and work as a consultant in the
field of code generation what gives me insight into many open- and
closed-source code bases that depend heavily on the API.

Code instrumentation libraries are often used for creating Java agents. One
challange with defining Java agents is the inavailability of "rich type
descriptions" similar to the Java reflection API which could be used to
gather information about the type that is supplied to a class file
transformer. To overcome this, both Javassist and Byte Buddy offer means of
investigating information about unloaded types and their referred types.
For this purpose, type information is extracted manually from class files
which are read from the class loader that is provided to the
ClassFileTransformer as an argument. A class file is located by calling
ClassLoader::getResource with an argument of "binaryTypeName.replace('.',
'/') + ".class". This includes classes that are defined by the VM (which
are now contained within runtime modules).

In Java 9, the latter method is overloaded by another method that accepts a
module name as a second parameter. The javadoc of the preexisting
getResource(String) now says:

"Resources in a named module are private to that module. This method does
not find resources in named modules defined to this class loader."

Of course, this includes resources that represent class file which are no
longer returned for named modules.

This is the API-change that I struggle with. To explain why, consider this
scenario: I want to create a ClassFileTransformer to instrument any class
that implements Interface A. Doing so, the ClassFileTransformer is notified
of some type Z being loaded. In order to find out if Z implements A, I am
looking at all the interface types that are stored in the class file that
the ClassFileTransformer provides. If any of these interfaces represents A,
I am done. Otherwise, as a next stept, I need to look into the interfaces
that Z does implement, Let's say, Z implements B. I now parse the class
files of B to see if this interface implements A. Before Java 9, I could
call "getResource" on the class loader that the ClassFileTransformer
provides. While I cannot generally assume that a class loader provides the
raw class file, this tactic works surprisingly well as most implementors of
custom class loaders are aware of this convention and stick to it.

With Java 9, this is no longer possible as I cannot determine what module -
if any - the above interface B is defined within. What I really require is
the previous behavior: I just want to get the class file independently of a
module similar to somebody wanting to load a class by name without knowing
what module this class will be defined within.

I am aware that the idea of project Jigsaw is encapsulation but for
scenarios like the above, it is often required to break module bondaries.
In general, I think that project Jigsaw solved this problem elegantly but
changing this single method's contract broke all 26 projects that I looked
into before giving this feedback. Note that I only changed runtime as this
contract change also affects "old" code running on a Java 9 VM as the JVM
classes are now contained in runtime modules where

assert
ClassLoader.getSystemClassLoader().getResourceAsAStram("java/lang/Object.class")
!= null

fails on Java 9 (but not on Java 8) whereas the following behavior is as
expected:

assert
Object.class.getModule().getResourceAsStream("java/lang/Object.class") !=
null;
assert
MyCustomClass.class.getModule().getResourceAsStream("java/lang/Object.class")
== null

This leads me to the suggestion I want to make. Did you consider to simply
retain the ClassLoader's behavior to not respect module boundaries? In the
end, a class loader exists on a level above modules. If I wanted to read a
module-scoped resource I would expect a user to read a resource via a
particular module or via a Class instance. It would be possible to
introduce an interface like:

interface ResourceAware {
  URL getResource(String name);
  InputStream getResourceAsStram(String name);
}

which could be implemented by the ClassLoader, Class and Module classes
where a class loader represents a Module-agnostic view whereas Class and
Module respect the new encapsulation rules. Doing so, the implementation of
Class could rather delegate to its Module rather then to its ClassLoader
such that

assert Object.class.getResourceAsStream("Object.class") != null;

holds again which currently fails on Java 9 but works on Java 8. I think
this outcome is intuitive and it would not introduce a breaking change
unless accessing a "foreign resource" via a class instance what is quite
rare. Also, if the latter was a problem, at least there exists a simple
migration path as a class loader is easily available from any Class
instance. To implement this behavior, it would only be necessary to change
the contract of the getResource(String) and getResourceAsStream(String)
methods of ClassLoader back to the current Java-8 definition.

Note that the requirement of reading class foreign files is not only a
matter of convenience but is also required for low-level instrumentation
when using for example ASM computing stack map frames at runtime where it
is important to find two type's common super type. Typically, this is done
by the same strategy as explained above.

Thank you for your time and considering this problem.
Best regards, Rafael