History data for "JDK-8058150: Compile for Specific Platform Version"

Fri Feb 27 16:54:38 UTC 2015

On 02/27/2015 08:35 AM, Magnus Ihse Bursie wrote:
> On 2015-02-27 16:58, Jan Lahoda wrote:
>> On 27.2.2015 14:48, Magnus Ihse Bursie wrote:
>>> Hi Jan,
>>>
>>> On 2015-02-27 09:31, Jan Lahoda wrote:
>>>> Hi,
>>>>
>>>> I have a question on JDK-8058150: "Compile for Specific Platform
>>>> Version". To support compilation for older versions of the platform,
>>>> javac will need some description of the APIs as they existed in the
>>>> target platforms.
>>>>
>>>> For this, the current proposal is to use lib/ct.sym file (similar, but
>>>> different, to the JDK 8 lib/ct.sym), containing classfiles of the
>>>> older APIs. This file would be constructed at build time from a
>>>> textual representation of the APIs stored in an OpenJDK repository
>>>> (currently called ct.sym.txt).
>>>>
>>>> The current ct.sym.txt is a single file that contains APIs for all
>>>> supported versions, reusing entries for multiple versions when needed.
>>>> An alternative would be to use ct7.sym.txt for JDK 7 APIs, ct8.sym.txt
>>>> for JDK 8 APIs, etc. Using a single file leads to a smaller total size
>>>> (as it reuses entries where it can), but needs to be considerably
>>>> changed when a new version is added or an obsolete version is removed.
>>>>
>>>> The size of the file is considerable: for the "ct.sym.txt" that
>>>> represents APIs from OpenJDK 7 and 8, the size of the checked-out file
>>>> in the working copy is (currently[2]) ~23MB, and inside the .hg
>>>> directory, the file has ~1.7MB (Mercurial is apparently able to
>>>> compress the ct.sym.txt file very well - but as all history is kept
>>>> inside .hg directory, the size of the file inside the .hg directory
>>>> increases when the ct.sym.txt is updated).
>>>
>>> In my opinion, the only size that matters here is in the .hg directory.
>>> If the workspace takes 23 MB more or less is a non-issue when a full
>>> forest clone is in the gigabyte range. But I don't think 1.7 MB extra
>>> for the top-level repo is much of a problem either. Since the top-level
>>> repo currently is so tiny, it will grow noticably in percentage. But
>>> compare this with hotspot/.hg on 67 MB or jdk/.hg on 351 MB.
>>
>> Thanks. I also think the size inside the .hg folder is more important.
>>
>>>
>>> If you have a proper text format so future edits can be made as trivial
>>> diffs, then the mercurial storage will not grow noticable in the future
>>> either.
>>
>> The file currently contains version numbers on most lines, so adding 
>> or removing a support for a platform version means a significant 
>> update to the file. I am thinking of some ways to limit that, though.
>
> I peaked at your current solution. Is it possible to store the file as 
> a baseline version (JDK 7, or however far back you want to go) and a 
> set of deltas? Instead of like:
> header extends java/lang/Object flags 31 classAnnotations 
> @Ljdk/Profile+Annotation;(value=I4) versions 78
> method name <clinit> descriptor ()V flags 8 versions 78
> method name <init> descriptor ()V flags 1 versions 8
> method name getHostId descriptor ()Ljava/lang/String; flags 1 versions 7
>
> Store it like:
> baseline-jdk7.sym.txt
> header extends java/lang/Object flags 31 classAnnotations 
> @Ljdk/Profile+Annotation;(value=I4)
> method name <clinit> descriptor ()V flags 8
>
> jdk8.sym.txt
> +method name <init> descriptor ()V flags 1
> -method name getHostId descriptor ()Ljava/lang/String; flags 1
>
> ?
>
> In fact, if it is not too expensive to generate this kind of file from 
> the build, you could perhaps do it the other way round, and create a 
> "baseline" for the current JDK just built, and just store the diffs to 
> previous versions. (Although that might be a maintainence burden to 
> make sure that these reverse diffs are up to date).
>
>
>>
>>>
>>>> Another alternative would be to partition the file into several
>>>> smaller files - would be easier to grasp, but if the files would be
>>>> too small, the compression would be worse (leading to bigger
>>>> repositories).
>>> What is the actual difference? Having too large files can be burdensome
>>> on other tools as well, eg. if you open it (mistakenly or not) in a 
>>> text
>>> editor. I would tend to prefer several smaller files than one huge.
>>
>> That depends on how is the file split up. Originally, I was thinking 
>> of having a file per package, but that produces (too) many small 
>> files - 797 files, the biggest one having ~650kB (in the working 
>> copy), consumes ~5.9MB total inside .hg.
> Can you match it to modules? I realize older JDKs has no module 
> concept, but that sounds like it could be a reasonable number of 
> reasonable sized chunks.
>
> /Magnus
>
>>
>> There was a proposal to have a single ct.sym file per each jar file 
>> on the bootclasspath (in the target platform), but this produces 
>> ~20MB (in the working copy) file for ct.sym, while the next biggest 
>> file is for nashorn.jar: ~1MB (in the working copy). This consumes 
>> ~1.7MB inside the .hg directory.
>>
>> I tried to split the big ct.sym.txt artificially at approximately 
>> 1MB, this leads to 23 files, and still consumes ~1.7MB inside the .hg 
>> directory.
>>
>>>> Currently, the proposal is to place the ct.sym.txt file into the
>>>> top-level repository. A prototype of this feature is currently in the
>>>> jdk9/sandbox forest, on branch JDK-8058150-branch. The current
>>>> ct.sym.txt file is 
>>>> <top-level-repository>/make/data/symbols/ct.sym.txt.
>>> Sounds like a good place to put such a file. I assume it will be
>>> processed during building?
>>
>> Yes, the file is processed during build and the ct.sym file in 
>> produced into the "lib" directory. In my current prototype, I've 
>> tweaked the make/Images.gmk to do that:
>> http://hg.openjdk.java.net/jdk9/sandbox/file/087692cc2663/make/Images.gmk 
>>
>> (in the "ct.sym" section).
>>
>> Thanks for the comments,
>>     Jan
>>
>>>
>>> /Magnus
>

Magnus,

The simplified generalization of your suggestion would be to simply use 
patch/diff files.

We could have a baseline of (say)

jdk7.sym.txt

then have a patch file

jdk8.sym.patch

which can be applied (in the build) to jdk7.sym.txt to get jdk8.sym.txt

Since the .txt and .patch files reflect existing releases, they would 
almost never change, so the .hg files would be constant.

Maybe once in a while, we would prune the history, remove older version 
info, and add a new baseline.

-- Jon