JEP: Allow debug tools to detect mismatch btwn sourcecode and bytecode

[Bruno Borges]

Hi Staffan, thanks for your feedback.

The fingerprint algorithm may be either:
- added to javac as an argument, in order to calculate during 
compilation and inject that to the classfile format "new attribute"
- a new tool provided by JDK, say "javafp" that would be allowed for 
standalone calls
- both solutions above, where javac delegates to "javafp" the 
calculation during compilation, which allows IDEs to reuse this tool 
during debugging

The Debugger API exposes the fingerprint so Debugging agents become 
aware of what sourcecode was used to compile.

Now there's one thing I am still not sure if it holds true, or it's 
something will never happen, or not lead to any errors, by tricking 
developers.

        The sourcecode of the class being debugged may have not changed,
    but instead, its super class or default methods of implemented
    interfaces, which could lead to different behavior. This anomaly
    would require the fingerprint to not be a simple checksum on top of
    the source code but on the full inheritance graph and any
    implemented interface, or subinterface that has default methods.

# interfaces
interface C -> fp based on C's sourcecode
subinterface D with default method -> fp based on D's sourcecode
subinterface E -> fp based on D's sourcecode and E's sourcecode 
(considers D's because of default methods)

# classes
class A -> fp based java.lang.Object's computed class fp (taken from 
JDK) and A's sourcecode
class B extends A -> fp based on A's fp and B's source code
class C extends B implements D -> fp based on B's fp, D's fp, and C's 
source code
class F extends C implements E -> fp based on C's fp, E's fp, and F's 
source code

The cases above exposes the possibilities, in my view, of anomalies 
during debugging that could lead developers to believe they are 
debugging the right source code. Often, developers skip calls to super 
methods (or default methods in interfaces, now in Java SE 8). With a 
fingerprint based not only on source code but also on inheritance graph 
containing actual code (I see no point in doing fp for plain 
interfaces), this anomaly could be avoided easily.

I confess I am still not sure if fingerprints based on inheritance graph 
is really necessary.
That's something I'd like to discuss here.

*# When sourcecode of inheritance graph is not available*
How would fingerprint be calculated in case there is no classfile 
fingerprint attribute available on inheritance graph (say, when 
extending a legacy library compiled on previous versions, or 
deliberately chosen by the developer to not calculate and inject 
fingerprint during compilation (if this is an option).

If we see that fingerprint based on inheritance graph is required, then 
what would happen if there is no fingerprint on super classes, or 
interface with default method, or interface inherited from one with 
default methods?

In summary, questions are:
(1) must source code fingerprint consider inheritance graph?
(2) if -1- is 'yes', how to behave and proceed when dealing with 
bytecode without the fingerprint attribute, or if fingerprint is 
optional during compilation?



On 02/12/2014 05:29, Staffan Larsen wrote:
> Hi Bruno,
>
> This is an interesting proposal and I guess it would avoid a lot of unnecessary confusion for developers.
>
> As I understand it, the exact algorithm used to compute the fingerprint would need to be known to debuggers, or at least an implementation of it would need to be provided by the JDK.
>
> The proposed solution would require an addition to the class file format to store the fingerprint. Most likely in the form of a new “attribute”.
>
> It would be interesting to hear opinions from debugger developers (netbeans, eclipse, intellij…) on this proposal.
>
> Thanks,
> /Staffan
>
>
>> On 29 nov 2014, at 23:45, Bruno Borges <bruno.borges at oracle.com> wrote:
>>
>> Hi all,
>>
>> Based on a common incident that happens not rarely among developers, and as recently felt by Martijn Dashort [1], it seems to be a good idea to: "Enable debugger to detect when there is a mismatch between source code and class being debugged" (thanks to Stuart Marks for the excellent problem statement [2]).
>>
>> [1] https://twitter.com/dashorst/status/538329707549560832
>> [2] https://twitter.com/stuartmarks/status/538722163700674561
>>
>> So I came up with the following JEP draft, and I'd like to have your input in this matter as well any comments, suggestions, etc.
>>
>> --- JEP DRAFT BEGIN ---
>>
>> Summary
>> -------
>> Enable debugger to detect when there is a mismatch between source code and class being debugged
>>
>> Goals
>> -----
>> * Detect mismatch between the source code and the class bytecode being debugged
>> * Allow for IDEs to identify through Debugger API such mismatch, enabling development of extensions
>>
>> Motivation
>> ----------
>> To increase development productivity by avoiding developers on wasting time debugging the wrong source code
>>
>> Description
>> -----------
>> Developers often find themselves debugging a Java application attached to their IDE, where the source code used to compile the bytecode being debugged is similar, but not the same. This leads to time wasted in understanding why some things behave differently, and sometimes developers blame the JVM, or the application server, or something in the runtime environment.
>>
>> By providing a fingerprint to the source code used to compile a bytecode, when javac is configured to do so (example: -g:sourcefingerprint), and injecting that fingerprint into the bytecode, the Debugger API can then expose such information to IDEs and Debugger agents. The jdb command and the Java Debug Interface API may need to be extended to add such information.
>>
>> Testing
>> -------
>> A test use case to validate this JEP is in the form of:
>> - compile a Java class
>> - modify the source code
>> - run the bytecode initially compiled
>> - debug the application
>> - identify the mismatch that source code has been modified
>>
>> Debug with jdb.
>>
>> Risks and Assumptions
>> ---------------------
>> There is a risk of the source code match the bytecode, but not inherited classes, which means not only the class being debugged but its structure and inherited graph must also match.
>>
>> The fingerprint must consider multiple data from the source code in order to be stringent:
>> - sourcecode checksum
>> - Java class structure
>> - inheritance graph
>>
>> It may be required to considered other form of data to build the fingerprint.
>>
>> Dependences
>> -----------
>> Beyond the jdb tool and the Java Debug Interface, there are no other identified dependencies.
>>
>> --- JEP DRAFT END ---
>>
>> -- Bruno Borges // +5511995649058 LAD Principal Product Manager Exalogic | Cloud | CAF | Java

-- 
Bruno Borges // +5511995649058
LAD Principal Product Manager
Exalogic | Cloud | CAF | Java

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20141202/b19d846a/attachment-0001.html>