Remove public fields without breaking binary compatibility

Tue Sep 13 12:57:41 PDT 2011

Hi i created a small concept-demo how i think we can manage to remove 
some flaws(public accessible fields) out of the jdk without breaking 
binary-compatibility. I uploaded the "code of cencept" to my 
github-incubator[0] and posted some description on my weblog[1].  It 
would be nice to get some discussion/comments on this, even if it 
destroys my dream that this can solve the above noted problem. I think 
discussion on the mailing list should be fine. Comments on my weblog are 
also welcome but i think discussion should went into the mailing-list.

I copy the weblog-text here in a mailing-list compatible format. Sorry 
for the cross-post, i think core-libs-dev can be interessted in general 
in removing such fields. And I hope some of the mlvm people can say 
something about if this can be solved better on vm level.

[0] 
https://github.com/picpromusic/incubator/tree/master/jdk/compatibleFieldAccess
[1] http://codingwizard.wordpress.com/2011/09/13/remove-flaws-in-java-apis/
... more links at the bottom of the mail

---WEBLOG--COPY
I think i made a small step in a concept how solutions that removes 
design flaws (such as public fields) can be solved in an binary 
compatible way. Public fields in public API are really nasty, and there 
isn't a solution how this can be resolved without completely removing 
this class (with a long deprecation phase in the meantime). It is 
because the missing encapsulation for accessing this field gives you no 
change to change behavoir (such as checking/permitting values and 
state). It is this nasty (for public instance fields) GET_FIELD bytecode 
instruction.

It is like Dalibor Topic told it[2] : " Breaking binary compatibility is 
bad, bad, really, really bad"

But than in saw a video[3] Virtual Extension Method with Brain Goetz 
which solves the compatibility problem introduced by extending the jdk 
for lambda expressions. After that i created a small concept-prototype 
which shows how it can work with simple bytecode transformations and 
invoke dynamic. I think there is much room for improvement and maybe it 
can be better implemented at the vm level, but actually i don't have the 
knowledge to do so. And in special : [4]"If you have a golden hammer,...." .

I placed the implementation of the concept on my github incubator[5]

What is this doing?

1. The build.xml looks complicated but this is not the point here. It 
compiles the needed things to show the concept here. There is much room 
optimizing this, but i don't do it because it is really unnecessary.

2. The Problem is the class OLD[6]. It has a public field cause which 
can be access be anyone. If you put this in an public API you can never 
change/remove this field without breaking binary compatibility.

2.1. There is a class NEW[7] which introduces the incompatibility in 
making the public field private. But is also introduces two methods that 
allows us to access the field in a controlled way.

2.2 There is a class NEW2[8] also which is nearer on my original problem 
(i tried to remove a public cause field in an exception-class in 
openjdk). It can throw an exception if changing the cause in the same 
manner java.lang.Throwable does it.

3. The Class GEN[9] generates three testclasses (TestOld,TestNew and 
TestNew2) with the same testsequence with is like this:

     OLD o = new OLD();
     System.out.println(o.cause);
     o.cause = new RuntimeException("NEW");
     System.out.println(o.cause);

TestNew does it with NEW and TestNew2 does this with NEW2.
I have done it with class generating techniques because it would not 
compile and it would need a really complicated build-file to show how it 
works, so it was easier to generate some bytecode.

4. The Class Main[10] does an overall test with the three testclasses 
TestOld, TestNew and TestNew2.

      [java] <<OLD>>
      [java] java.lang.RuntimeException: INIT_OLD
      [java] java.lang.RuntimeException: NEW
      [java] <<NEW>>
      [java] Exception in thread "main" java.lang.IllegalAccessError: 
tried to access field NEW.cause from class TestNew
      [java]     at TestNew.testIt(TestNew.txt:3)
      [java]     at Main.main(Main.java:7)

The access of the cause field

     System.out.println(o.cause);

crashes.

The Test does not even reach the testcase TestNew2

But if you start the Main class with the jvmarg 
-javaagent:transformer.jar the output is

      [java] MOCKINJECT BCI
      [java] <<OLD>>
      [java] java.lang.RuntimeException: INIT_OLD
      [java] java.lang.RuntimeException: NEW
      [java] <<NEW>>
      [java] java.lang.RuntimeException: INIT_NEW
      [java] java.lang.RuntimeException: NEW
      [java] <<NEW2>>
      [java] java.lang.RuntimeException: INIT_NEW
      [java] ***java.lang.IllegalStateException: Not allowed to change

I think this is realy cool. The methods getCause() and initCause are 
used instead of the field in the cases where the field is not accessable.

4. How does this work?
The classes in the transformer.jar transforms the bytecode before 
loading into the vm.
It replaces the GET_FIELD / PUT_FIELD instruction with something like:

     INVOKEDYNAMIC cause (LNEW;)Ljava/lang/Throwable; 
[Bootstrapper.getFunction(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite; 
(6)]
     INVOKEDYNAMIC cause (LNEW;Ljava/lang/Throwable;)V 
[Bootstrapper.setFunction(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite; 
(6)]

You can look the complete bytecode of the testclases if you search for 
the .txt files in the tmp dir after executing the build script.
It invokes the bootstrap methods Bootstrapper.getFunction and 
Bootstrapper.setFunction in the Bootstrapper[11] class.

5. The Bootstapper class connects the invokedynamic call with the field 
if accessable or with the accordant annotated access method getCause or 
initCause.

So what do you think? Please throw comments on me, also if you think 
binary compatible accessors for public fields is a really bad idea. I 
think there is much room for optimizations here, but first lets discuss 
about the idea and the concept.

-- Sebastian

[2] 
http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-August/007539.html
[3] http://medianetwork.oracle.com/media/show/16999
[4] http://en.wikipedia.org/wiki/Law_of_the_instrument
[5] 
https://github.com/picpromusic/incubator/tree/master/jdk/compatibleFieldAccess
[6] 
https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/testsrc/OLD.java#L1
[7] 
https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/testsrc/NEW.java#L1
[8] 
https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/testsrc/NEW2.java#L1
[9] 
https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/generator/GEN.java#L1
[10] 
https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/testsrc/Main.java#L1
[11] 
https://github.com/picpromusic/incubator/blob/master/jdk/compatibleFieldAccess/src/Bootstrapper.java#L1 