Suggestion: allow accessible reflection on protected methods of exported types.

[Rafael Winterhalter]

To give a short update on this:

I have now updated my code generation library Byte Buddy to fallback to
sun.misc.Unsafe::defineClass for injecting types. I find this really
unfortunate as I managed to avoid using Unsafe in Byte Buddy in five years
and I am rather unhappy with this solution myself. Byte Buddy and cglib,
another library I help maintain, amount to over 10 million yearly downloads
and are consumed by a lot of infrastructure, also including the Spring
framework which is not even accounted for in these download statistics due
to its approach of copying the code without maintaining an explicit
dependency. I am currently planing to also update cglib to using Unsafe
rather than calling protected methods on ClassLoader.

I am really not trying to work against Jigsaw and I would prefer to not
introduce a dependency to sun.misc.Unsafe to millions of Java projects. At
the same time, I am under a bit of preasure myself to retain compatibility
for my APIs what I cannot currently provide with the given constraints
without any backdoors for library developers. This is especially true for
cglib where even minor changes like the replacing of private field types
often causes a lot of pain for adoperts.

I understand that the latter problem is approached by Jigsaw and I would
also prefer if this was something I did not have to deal with but its about
20 years too late to change that. At the end of the day, there is a good
reason that classes like Vector or Stack are still collections that are
included in the Java packages, removing those classes would most likely
cause a lot of trouble. I just really hope that it is considered that
library maintainers face similar problems and just the many million
downloads of tools that depend on such funcationality hopefully proove that
we have done something useful that made Java a success.

I am still convinced that the capability of calling protected methods after
setting setAccessible(true) would solve many of the current problems. Also,
I think that an outcome where everybody enables such functionality in a
controlled manner would be the much preferred outcome.

2016-12-29 19:17 GMT+01:00 Remi Forax <forax at univ-mlv.fr>:

> ----- Mail original -----
> > De: "Rafael Winterhalter" <rafael.wth at gmail.com>
> > À: "jigsaw-dev" <jigsaw-dev at openjdk.java.net>
> > Envoyé: Jeudi 29 Décembre 2016 17:24:55
> > Objet: Suggestion: allow accessible reflection on protected methods of
>       exported types.
>
> > Hello everybody,
>
> Hi Rafael,
>
> >
> > I did another round of testing a recent build of Java 9 with some
> > applications and framework that I work with or maintain. I ran into an
> > issue that I consider rather severe and I want to make a suggestion on
> how
> > to potentially solve this problem.
>
>
> thank for doing that,
>
> >
> > Some libraries entertain utilities that access protected methods of
> > exported types to enhance their functionality without requiring a user to
> > provide a specific subclass of such an instance but to allow them to use
> > “their“ implementation. To access these protected methods, reflection is
> > used in combination with AccessibleObject::setAccessible.
> >
> > For example, this approach is used by code generation libraries to invoke
> > the ClassLoader::defineClass method which allows the injection of proxy
> > types into the class loader of the proxied class. With the new
> constraints
> > enforced by Jigsaw, attempting to make ClassLoader::defineClass
> accessible
> > fails with:
> >
> > java.lang.reflect.InaccessibleObjectException: Unable to make protected
> > final java.lang.Class
> > java.lang.ClassLoader.defineClass(java.lang.String,
> byte[],int,int,java.security.ProtectionDomain)
> > throws java.lang.ClassFormatError accessible: module java.base does not
> > "opens java.lang" to unnamed module @4cbd6df7
>
> yes, it's #AwkwardStrongEncapsulation [1]
>
> >
> > I argue that this module system should not be responsible to assert the
> > usage of such protected methods on exported types. Such methods are not
> > really encapsulated as they are considered official API by subclasses.
> > Accessing such classes could of course be a security thread but
> preventing
> > such breaches should be the job of the security manager rather than the
> > module system.
> >
> > This problem is currently breaking all consumers of code generation
> > libraries which is about every single enterprise application as this
> > strategy is quite popular. During my tests, I ran into this issue with
> any
> > application using Spring, Hibernate or Mockito where at least one of
> those
> > are used by any enterprise application.
> >
> > To overcome this, I have now added a fallback to Unsafe::defineClass to
> my
> > library Byte Buddy to reenable injection in Hibernate and Mockito. I plan
> > to add the same fallback to cglib in the next days which is used by
> Spring
> > and many others as this is currently a real problem that can only be
> > overcome by adding the export on the command line what is a solution I
> try
> > to avoid for its additional complexity.
> >
> > I also argue that there exist many more such access patterns of protected
> > methods on exported types and it would be quite a regression to no longer
> > be able to access these methods using reflection. Therefore, I really
> hope
> > that my suggestion is considered. Also, it would be too bad if this
> change
> > would result in the additional usage of sun.misc.Unsafe by Java end-users
> > as this is currently the only fallback usable by code generation
> libraries
> > which otherwise need to break their API. I managed to avoid introducing a
> > dependency to sun.misc.Unsafe for 8 years now, it would really be a shame
> > if I needed to start using it now.
>
>
> While protected methods are parts of the API (and a huge burden when you
> want to evolve an API, but that's another story), the difference between a
> protected method and a public method is that you have to control the
> creation of the class to be able to use protected methods. Making them
> public by reflection goes against the whole idea to have protected methods.
> In case of a class loader, being able to inject any classes inside a
> package ruins the whole concept of encapsulation because you can use the
> injected class as a trampoline to get access to any package-private and
> protected methods. That's why defineClass is declared protected in
> ClassLoader.
>
> And yes, currently, if you want to be able to inject a class in a package,
> you have to use Unsafe because ... it's unsafe.
>
> Maybe what can be done by adding a public method that will allow to define
> a class if the package of the class is in a module which is open or the
> unnamed module.
>
> In that case, we should also revisit the rules of Proxy creation using
> java.lang.reflect.Proxy because currently being in a open module or not
> doesn't change anything.
>
> >
> > Thank you for considering this and best regards, Rafael
>
> regards,
> Rémi
>
> [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#
> AwkwardStrongEncapsulation
>