Recording source information in a build

[Kelly O'Hair]

Mercurial does not come with the forest extension by default, so I'm
reluctant to become dependent on it in the build process, and this needs
to be a loose build dependence on 'hg' as it is.
But 'hg fsnap' is the basic kind of thing that is needed.

-kto

Roman Kennke wrote:
> Hi Kelly,
> 
> I'm facing similar problems right now inside aicas (we also want to
> transition to HG). I think you should look at hg fsnap, this gives you a
> snapshot file, which uniquely identifies the state of a forest at a
> given time. Maybe this is helpful?
> 
> /Roman
> 
> Am Donnerstag, den 03.04.2008, 11:36 -0700 schrieb Kelly O'Hair:
>> Problem Statement:
>>
>>    Given a build of the OpenJDK, how can you find out what source was used to
>>      build this binary install?
>>
>> Seed of a Solution:
>>
>>    With Mercurial, a single repository changeset number identifies the state of
>>      the complete source repository. If this changeset (or set of changesets)
>>      could be somehow recorded with the built bits, then given any build you
>>      could quickly and easily reconstruct the exact source files that were used
>>      at build time.
>>
>> Problems:
>>
>>    We have a forest not a single repository.
>>    We often create source bundles (sources minus the SCM management data, e.g. ".hg")
>>      so we need this to work in the face of building from source bundles.
>>
>> Possible Solution:
>>
>>    First issue is identifying a repository of the forest relative to the root of the forest.
>>      So each repository would get a managed file ".identification" which would contain
>>        information to help identify the repository.
>>        For example, the topmost OpenJDK one would have a ".identification" file containing:
>>          root=.
>>         directory=.
>>         description=Root of the JDK Source Tree
>>        and the corba one would have:
>>          root=..
>>          directory=corba
>>          description=Corba Sources
>>        etc. (the directory could be a deeper nested directory, like jdk/src/closed)
>>      This .identification file would be a permanent file in the repository, at the root
>>        of the repository. It's saying that to get to the root of the forest, you
>>        'cd ${root}'. And if this repository is not located at ${root}/${directory}
>>        something is wrong, or the repository is not currently part of a forest.
>>
>>    Second issue, the changeset id.
>>      A second file called ".changeset" would not be a managed file and would be created
>>        before the source bundles are created, and be non-existent if they can't be created
>>        because you don't have repositories (building from raw source trees) or don't have
>>        access to 'hg'. These files would just contain a changeset=id, created with:
>>           hg tip --template 'changeset={node}\n'
>>        So somewhere this needs to happen, before source bundles are created and before
>>          the use of this data:
>>          TREES:=$(shell hg ftrees)
>>          if [ "$(TREES)" != "" ] ; then
>>            for i in $(TREES) ; do
>>              (cd $i && hg tip --template 'changeset={node}\n' > .changeset )
>>            done
>>          fi
>>
>>    Third, all this data needs to be merged together into a file that could be
>>      used later to recreate the source tree by running:
>>         hg clone -r ${changeset} http://hg.openjdk.java.net/jdk7/${directory} ${directory}
>>      as many times as needed.
>>      The Makefiles would be sensitive to the existence of the .changeset files and
>>        allow for them to not exist where they are used, they might not be there in
>>        all cases. But when they are there, do something like:
>>          jdk_source_information.txt:
>>                  $(RM) $@
>>                  echo "# JDK Source Information" > $@
>>                  if [ "$(TREES)" != "" ] ; then
>>                    for i in $(TREES) ; do
>>                      if [ -f ${i}/.identification ] ; then
>>                        cat ${i}/.identification >> $@
>>                        if [ -f ${i}/.changeset ] ; then
>>                          cat ${i}/.changeset >> $@
>>                        fi
>>                      fi
>>                    done
>>                  fi
>>         Resulting in a file:
>>           # JDK Source Information
>>           root=.
>>           directory=.
>>           description=Root of the JDK Source Tree
>>           changeset=BIGHEXNUMBER
>>           root=..
>>           directory=corba
>>           description=Corba Sources
>>           changeset=BIGHEXNUMBER
>>           ...
>>         Left in the jdk install tree.
>>
>> ---
>> Just a first guess at a basic idea as to how this could work...
>>
>> Please don't assume the above is also an implementation, it's the basic idea
>> of having members of the forest identify themselves, and the idea of
>> recording the changesets, and finally of leaving source information in
>> the resulting binary build.
>>
>> Comments?
>>
>> -kto
>>
>> P.S. Full RFE can be seen at: http://bugs.sun.com/view_bug.do?bug_id=6631003
>>