Bytecode generation, Source code mappings, JCov, Future

[Jonathan Gibbons]

Alex Rau wrote:
> On 17.04.2008, at 00:12, Jonathan Gibbons wrote:
>
>> Jcov is still around, though it has never really got the care and 
>> attention it merited.  Every now and then, there is interest in 
>> making it more available. It generally comes down to lack of demand 
>> and internal resources. :-(
>
>
> What exactly is the purpose of jcov in current JDK's ? The coverage 
> feature itself has been superseded by JVMTI while the jcov 
> implementation still exists, is that correct ? Legacy coverage tools 
> (at least I know) rely anyway on line information exclusively and are 
> therefore not interested in column information or offsets. So what use 
> case is left for JCov ? Am I correct, that it just exists more or less 
> as an artificial piece of code from past times without a real purpose ?
My understanding is that jcov is still used within Sun for determining 
code coverage of the various different test suites.

>
> Well. Actually I think the implementation deserves more attention from 
> the compiler's point of view. JVMTI and JVMPI focus on making the 
> state of the VM (during runtime) more visible. JCov in it's current 
> implementation makes the compiler operation (in a different way via 
> byte code attributes) visible and that's perfectly fine in my opinion 
> and useful for tools which focus on static information on class files 
> (independently what happens during runtime).
>
> It's however open whether the compiler operation information is really 
> required to be stored in the byte code. Another option would be to 
> create a single file or one file for each class which exclusively 
> contains the information. These files could be enhanced with other 
> information from the compiler execution as well (what ever that could 
> be) - some kind of standard compiler tracing output.
>
> Due to my personal interest I'd love to work on that feature. If it 
> all goes well and it's getting along in a way which is useful for 
> general purpose - perfect.
We have had internal "hallway"-type discussions on the future of the 
CharacterRangeTable attribute and ways to go forward. Having separate 
files would certainly make it simpler to distribute such files for 
existing class libraries that do not want to incur the hit of the extra 
class file space required.

I think that experiments like this on the compiler, and on related 
tools, are a great idea. To get started, you can just pull down a copy 
of OpenJDK from the Mercurial repositories. However, note that to 
contribute anything back to OpenJDK, you'll need to sign the SCA form, 
available on this page http://openjdk.java.net/legal/

>
>> The features for jcov support are still in the compiler.  The switch 
>> is -Xjcov, it causes a CharacterRangeTable attribute to be added to 
>> the class file.  The format of the attribute is not great -- in 
>> particular, it uses the old "packed line/offset" format for 
>> coordinates, which may not work for very long lines (over 1024 
>> characters) that sometimes occur in mechanically generated code. 
>> Internally, javac now uses a simple character offset from the 
>> beginning of the file to define a source position. It would arguably 
>> be good to evolve the CharacterRangeTable attribute (in a compatible 
>> way) to using character offsets.
>>
>>
>
> I agree.  I think - given that javac itself  uses (as you mentioned) a 
> single offset value it would make sense to stick with it and use this 
> value for generating appropriate byte code including these offset 
> attributes. Tools using these values in general have access to the 
> source code anyway as most often the code is pretty printed in HTML or 
> something else. So mapping a single file offset to a certain line + 
> offset combination would be easily possible during parsing the source 
> code for such tools.

The code for getting line and offset from character number is in javac, 
in util/Log.java.

>
>
> So in case the usefulness of properly tracing above mentioned 
> information is common sense - what would be the best way ? Here some 
> aspects:
>
> - interleaved output with byte code vs. separate output (file) interface
> - replace/reuse/adapt jcov code vs. keep jcov as it is and implement 
> new stuff from scratch in parallel (compatibility?)
> - absolute file offsets as you mentioned vs. something else ?
It would be interesting to get some jcov folk in the discussion to 
discuss the merits of making jcov available and using that as a basis 
for work, compared to starting over.

>
> regards,
>
>  alex
>
>
>>
>> -- Jon
>>
>>
>>
>>
>> Alex Rau wrote:
>>> Hi all,
>>>
>>> I've searched the web and asked at forum.sun.com but my answer 
>>> regarding the javac compiler couldn't be answered yet. so here we go 
>>> with a rational first:
>>>
>>> Rational: I'd like to achieve the following: The byte code created 
>>> with javac should not only contain line number information in debug 
>>> mode, but additionally I want to track exactly which statement 
>>> (including column information as there can be multiple statements in 
>>> one source code line) leads to a certain bytecode instruction or a 
>>> set of instructions. I strongly need this kind of functionality in 
>>> javac as the project I'm working on is settled in the testing area 
>>> (mutation testing in java on bytecode level  which you can imagine 
>>> as some kind of (beta) code coverage software with a different and 
>>> IMHO superior technique - http://retroduction.org for more details 
>>> if you are interested). The software must be able to report mutated 
>>> *statements* which means that line informations are insufficient.
>>>
>>> My Question is: in older JDK's there was something called JCov which 
>>> enhanced byte code with additional information regarding which 
>>> statements finally led to a/multiple bytecode instruction(s). I 
>>> stumbled upon this while debugging javac when I wanted to learn it's 
>>> design/code. I think it's mostly what I need - however the JCov 
>>> switch is a hidden feature. It was "more" public in JDK 1.2 and was 
>>> pretended to have been ported to the JVMPI interface later. However 
>>> I did not find where the port should have been integrated - I'm no 
>>> JVMPI (and JVMTI) guru. Honestly I doubt that something similar is 
>>> in the JVM(P/T)I toolset...
>>>
>>> So perhaps someone has more knowledge about JCov and what happened 
>>> to it. Furthermore I'd like to push a discussion on whether this 
>>> would be a candidate for the kitchen sink (no JLS changes, just 
>>> internal) with the goal to implement the above described 
>>> functionality *including a supported and public interface* (not a 
>>> hidden feature anymore). The first benefit could then obviously be 
>>> that javac could be mutation tested ;) Furthermore other Java 
>>> developers would have solid information and documentation about this 
>>> API instead of relying on mostly unknown and hidden features.
>>>
>>> Best Regards,
>>>
>>>  Alex
>>>
>>
>