History data for "JDK-8058150: Compile for Specific Platform Version"

[Magnus Ihse Bursie]

On 2015-02-27 16:58, Jan Lahoda wrote:
> On 27.2.2015 14:48, Magnus Ihse Bursie wrote:
>> Hi Jan,
>>
>> On 2015-02-27 09:31, Jan Lahoda wrote:
>>> Hi,
>>>
>>> I have a question on JDK-8058150: "Compile for Specific Platform
>>> Version". To support compilation for older versions of the platform,
>>> javac will need some description of the APIs as they existed in the
>>> target platforms.
>>>
>>> For this, the current proposal is to use lib/ct.sym file (similar, but
>>> different, to the JDK 8 lib/ct.sym), containing classfiles of the
>>> older APIs. This file would be constructed at build time from a
>>> textual representation of the APIs stored in an OpenJDK repository
>>> (currently called ct.sym.txt).
>>>
>>> The current ct.sym.txt is a single file that contains APIs for all
>>> supported versions, reusing entries for multiple versions when needed.
>>> An alternative would be to use ct7.sym.txt for JDK 7 APIs, ct8.sym.txt
>>> for JDK 8 APIs, etc. Using a single file leads to a smaller total size
>>> (as it reuses entries where it can), but needs to be considerably
>>> changed when a new version is added or an obsolete version is removed.
>>>
>>> The size of the file is considerable: for the "ct.sym.txt" that
>>> represents APIs from OpenJDK 7 and 8, the size of the checked-out file
>>> in the working copy is (currently[2]) ~23MB, and inside the .hg
>>> directory, the file has ~1.7MB (Mercurial is apparently able to
>>> compress the ct.sym.txt file very well - but as all history is kept
>>> inside .hg directory, the size of the file inside the .hg directory
>>> increases when the ct.sym.txt is updated).
>>
>> In my opinion, the only size that matters here is in the .hg directory.
>> If the workspace takes 23 MB more or less is a non-issue when a full
>> forest clone is in the gigabyte range. But I don't think 1.7 MB extra
>> for the top-level repo is much of a problem either. Since the top-level
>> repo currently is so tiny, it will grow noticably in percentage. But
>> compare this with hotspot/.hg on 67 MB or jdk/.hg on 351 MB.
>
> Thanks. I also think the size inside the .hg folder is more important.
>
>>
>> If you have a proper text format so future edits can be made as trivial
>> diffs, then the mercurial storage will not grow noticable in the future
>> either.
>
> The file currently contains version numbers on most lines, so adding 
> or removing a support for a platform version means a significant 
> update to the file. I am thinking of some ways to limit that, though.

I peaked at your current solution. Is it possible to store the file as a 
baseline version (JDK 7, or however far back you want to go) and a set 
of deltas? Instead of like:
header extends java/lang/Object flags 31 classAnnotations 
@Ljdk/Profile+Annotation;(value=I4) versions 78
method name <clinit> descriptor ()V flags 8 versions 78
method name <init> descriptor ()V flags 1 versions 8
method name getHostId descriptor ()Ljava/lang/String; flags 1 versions 7

Store it like:
baseline-jdk7.sym.txt
header extends java/lang/Object flags 31 classAnnotations 
@Ljdk/Profile+Annotation;(value=I4)
method name <clinit> descriptor ()V flags 8

jdk8.sym.txt
+method name <init> descriptor ()V flags 1
-method name getHostId descriptor ()Ljava/lang/String; flags 1

?

In fact, if it is not too expensive to generate this kind of file from 
the build, you could perhaps do it the other way round, and create a 
"baseline" for the current JDK just built, and just store the diffs to 
previous versions. (Although that might be a maintainence burden to make 
sure that these reverse diffs are up to date).


>
>>
>>> Another alternative would be to partition the file into several
>>> smaller files - would be easier to grasp, but if the files would be
>>> too small, the compression would be worse (leading to bigger
>>> repositories).
>> What is the actual difference? Having too large files can be burdensome
>> on other tools as well, eg. if you open it (mistakenly or not) in a text
>> editor. I would tend to prefer several smaller files than one huge.
>
> That depends on how is the file split up. Originally, I was thinking 
> of having a file per package, but that produces (too) many small files 
> - 797 files, the biggest one having ~650kB (in the working copy), 
> consumes ~5.9MB total inside .hg.
Can you match it to modules? I realize older JDKs has no module concept, 
but that sounds like it could be a reasonable number of reasonable sized 
chunks.

/Magnus

>
> There was a proposal to have a single ct.sym file per each jar file on 
> the bootclasspath (in the target platform), but this produces ~20MB 
> (in the working copy) file for ct.sym, while the next biggest file is 
> for nashorn.jar: ~1MB (in the working copy). This consumes ~1.7MB 
> inside the .hg directory.
>
> I tried to split the big ct.sym.txt artificially at approximately 1MB, 
> this leads to 23 files, and still consumes ~1.7MB inside the .hg 
> directory.
>
>>> Currently, the proposal is to place the ct.sym.txt file into the
>>> top-level repository. A prototype of this feature is currently in the
>>> jdk9/sandbox forest, on branch JDK-8058150-branch. The current
>>> ct.sym.txt file is <top-level-repository>/make/data/symbols/ct.sym.txt.
>> Sounds like a good place to put such a file. I assume it will be
>> processed during building?
>
> Yes, the file is processed during build and the ct.sym file in 
> produced into the "lib" directory. In my current prototype, I've 
> tweaked the make/Images.gmk to do that:
> http://hg.openjdk.java.net/jdk9/sandbox/file/087692cc2663/make/Images.gmk
> (in the "ct.sym" section).
>
> Thanks for the comments,
>     Jan
>
>>
>> /Magnus