Proposal: #ReflectiveAccessToNonExportedTypes: `exports dynamic`

Tue Jul 5 16:21:19 UTC 2016

 Hi Rafael.

> If you need to explicitly white-list packages that are accessible
> via reflection, this makes things more difficult for most users
> while adding a minimum benefit for very few users (i.e. making it
> more difficult to "jail break" an internal API using reflection).

As I explained in the mail I just sent to the other fork of this
thread, I am convinced that users get harmed by broken encapsulation
even if they are not the ones actually doing it.

> In practice, dynamic exports would not require to export one
> package but most likely a whole bunch of packages where finding out
> about what packages to white-list would require quite some effort
> ...

(Which is of course also true for "regular "exports.)

> ... and the correctnes could only be validated at runtime.

Yes, good point. This sucks. Tools might help (see Alan's comment) but
otherwise you are right. Retroactively adding dynamic exports might
require quite some effort.

> I just checked what this would mean for one of our production 
> applications by setting a break point in Method::invoke and we
> would probably choose to not apply modules rather than adding every
> single package as a dynamic export as there are so many libraries
> that use reflection and we have no overview how this is organized.

It seems to be quite obvious that the application is not ready to be
modularized (no offense intended; ours isn't either!), so why force
it? Just put it on the class path (where reflection continues to work
as today), split of smaller modules over the next couple of years, and
migrate slowly.

Maybe this is the underlying assumption that makes this such an
overwhelming and apparently fruitless task: the idea that all code has
to be put in modules soon. Because that's not so. If a project is
fighting so hard to stay on the class path, then let it (for another
year or two). No harm done.

> [V]ery few developers use reflection to access internal APIs and 
> those mostly know what it means to circumvent offical APIs.

Do they really? Because a lot of those developers were complaining
quite loudly that this is going to break their code in a way they were
obviously not prepared for. And not only did they risk their projects'
livelihood but also that of their users! Because I am pretty sure that
many of those had no idea what fickle internal details they were
indirectly depending on.

> In this case, reflection is really only used as a last resort
> where there is no other way like in the cases of JUnit or Unsafe.

I addressed that in my other mail.

> I therefore still hope Java will continue to allow reflection
> across module boundaries without requireing any additional
> configuration.

:)

I still agree that reflection should loose its super powers and be
rendered unable to break encapsulation (without command line flags).

 so long ... Nicolai

On 04.07.2016 11:36, Rafael Winterhalter wrote:
> This statement is where I disagree: So your module will simply
> "export dynamic package.with.dtos"
> 
> If you need to explicitly white-list packages that are accessible
> via reflection, this makes things more difficult for most users
> while adding a minimum benefit for very few users (i.e. making it
> more difficult to "jail break" an internal API using reflection).
> In practice, dynamic exports would not require to export one
> package but most likely a whole bunch of packages where finding out
> about what packages to white-list would require quite some effort
> and the correctnes could only be validated at runtime.
> 
> I just checked what this would mean for one of our production
> applications by setting a break point in Method::invoke and we
> would probably choose to not apply modules rather than adding every
> single package as a dynamic export as there are so many libraries
> that use reflection and we have no overview how this is organized.
> Again, I do not argue that this is perfectly structured software
> but I think this is still representative of the reality of most
> enterprise apps.
> 
> Finally, I do not believe that the benefit of protecting internal
> APIs is as big as it is argued here. Most of the time, reflection
> is used by libraries to access my code where reflection is used for
> (annotation-based) discovery and susequent interaction. The other
> way around, very few developers use reflection to access internal
> APIs and those mostly know what it means to circumvent offical
> APIs. In this case, reflection is really only used as a last resort
> where there is no other way like in the cases of JUnit or Unsafe.
> 
> I therefore still hope Java will continue to allow reflection
> across module boundaries without requireing any additional
> configuration.
> 
> 2016-07-03 0:50 GMT+02:00 Nicolai Parlog <nipa at codefx.org>:
> 
>> Hi Paul,
>> 
>> either I'm misunderstanding you or one of us misunderstood the
>> proposal.
>> 
>>> So now "dynamic" has put me in a pickle. All I really want to
>>> do is have Hibernate access my private types for JPA, but I
>>> have to roll several dice and pray to be right: (1) I got to
>>> know if Hibernate does its reflection or not (2) Does Hibernate
>>> delegate to third party reflection libraries? (3) Are the third
>>> party reflection libraries pluggable? (4) What is going to
>>> happen to my code if Hibernate alters its reflection
>>> mechanism?
>> 
>> You seem to be talking about qualified exports, which let you
>> export packages _to_ a specific module (at compile and at run
>> time). Dynamic exports, on the other hand, will export packages
>> to every module (although I assume there would also be qualified
>> dynamic exports) but only at run time.
>> 
>> So your module will simply "export dynamic package.with.dtos"
>> and every module can access those types at runtime if it reads
>> the your module. If it uses reflection to access some DTOs, the
>> read edge is added automatically.
>> 
>> To me this seems like a clean solution where
>> 
>> (a) nothing changes for Hibernate (b) a module can expose it's
>> DTOs at run time while preventing compile-time dependencies
>> 
>> so long ... Nicolai
>> 
>> 
>> 
>> On 01.07.2016 17:21, Paul Benedict wrote:
>>> Since Hibernate is a popular example for this topic, let's
>>> talk about it.
>>> 
>>> First, when I build JPA applications, I do have an
>>> implementation use in mind (Hibernate, EclipseLink, etc.), but
>>> my code is standardized JPA as much as possible. So if I am
>>> going to publish my JPA model and I want my private types
>>> reflected to be loaded as entities, who am I going to export it
>>> too?
>>> 
>>> Well I certainly am not going to export it to "jpa" because
>>> that's just a specification...
>>> 
>>> The next logical answer would be the "hibernate" module, but
>>> now I'm gambling on knowing the internals of Hibernate... I
>>> have used Hibernate professionally for several years, but, off
>>> the top of my head, I honestly couldn't tell you if HIbernate
>>> has its own reflection routines or if it delegates it to third
>>> party libraries like "cglib" and "javassist". My gut tells me
>>> it's the latter, but don't quote me on it :-)
>>> 
>>> So now "dynamic" has put me in a pickle. All I really want to
>>> do is have Hibernate access my private types for JPA, but I
>>> have to roll several dice and pray to be right: (1) I got to
>>> know if Hibernate does its reflection or not (2) Does Hibernate
>>> delegate to third party reflection libraries? (3) Are the third
>>> party reflection libraries pluggable? (4) What is going to
>>> happen to my code if Hibernate alters its reflection
>>> mechanism?
>>> 
>>> #4 is pretty important, actually. Tools have been known to
>>> shade some of their third party dependencies. So while, for
>>> example, Hibernate may rely on "cglib" or "javassist" today,
>>> there is a possibility those may be shaded and brought in and
>>> another package name in a future version. I've seen those
>>> discussions. I am not implying what was or wasn't decided, but
>>> just it's a possibility.
>>> 
>>> I just don't see a point in creating such a risky situation
>>> for myself. I'd rather just have everything exported all the
>>> time to every module if this is the way reflection is going.
>>> 
>>> 
>>> Cheers, Paul
>>> 
>>> On Fri, Jul 1, 2016 at 5:20 AM, Jochen Theodorou 
>>> <blackdrag at gmx.org> wrote:
>>> 
>>>> 
>>>> 
>>>> On 01.07.2016 11:15, Rafael Winterhalter wrote: [...]
>>>> 
>>>>> Finally, I do not believe the service loader API offers an 
>>>>> adequate alternative to many use-cases of reflection (while
>>>>> it can perfectly substitute some of them).  Many
>>>>> state-of-the art libraries actually intentionally offer
>>>>> POJO approaches without requiring their users to create
>>>>> interfaces for any shared interaction. I do not think that
>>>>> a gentle push into a direction that contradicts most
>>>>> existing Java libraries would be a taken well by the people
>>>>> implementing this software and in the end, the module
>>>>> system will be as successful as its adoption. I rather see
>>>>> people finding new ways around restrictions to keep things
>>>>> simple (for themselves).
>>>>> 
>>>> 
>>>> even in the cases the service API is a replacement, the usage
>>>> of the library will change and a breaking change is forced.
>>>> 
>>>> For all these reasons I do not think that it is a good idea
>>>> to limit
>>>>> reflection to exported packages.
>>>>> 
>>>> 
>>>> or people will start just exporting everything.
>>>> 
>>>> bye Jochen
>>>> 
>>> 
>> 
>> --
>> 
>> PGP Key: 
>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>
>>
>> 
Web:
>> http://codefx.org a blog about software development 
>> http://do-foss.de Free and Open Source Software for the City of
>> Dortmund
>> 
>> Twitter: https://twitter.com/nipafx
>> 
>> 
> 

-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx