Looking ahead: proposed Hg forest consolidation for JDK 10

[Chris Hegarty]

> On 12 Oct 2016, at 15:48, Erik Helin <erik.helin at oracle.com> wrote:
> 
> On 2016-10-11, Weijun Wang wrote:
>> I am very glad to see history preserved. Is it possible to also merge the
>> history for src files before modularization and after it with this magical
>> script? I can call "hg log -f" on the command line but the web interface
>> only shows partial history.
> 
> You mean that for example [0] only shows you the history up until the
> modularization? Sorry, I don't know if this is possible in the Mercurial
> web interface. The history is there in the repository, but as you've
> already noticed, the web interface from `hg serve` does not use --follow
> when showing the history for a file.

Just to note, in case others may not see it, [0] does contain a link to the 
‘base’ version of the file before the modular source code restructure. It’s
a bit of a pain, but you can follow it through the web interface. History is
better searched / viewed through a source code indexer, e.g. OpenGrok.

-Chris.

> If you want to reach out to Mercurial upstream and see if it possible to
> configure this, then you can find them at mercurial at mercurial-scm.org.
> 
> Thanks,
> Erik
> 
> [0]:
> http://hg.openjdk.java.net/jdk9/dev/jdk/log/138876450c3a/src/java.base/share/classes/java/lang/String.java
>> 
>> Thanks
>> Max
>> 
>> On 10/11/2016 10:11, joe darcy wrote:
>>> Hello,
>>> 
>>> Looking ahead to JDK 10, a group of JDK engineers have been exploring
>>> consolidating the large number of Hg repositories in an open JDK forest
>>> to a single one with the goal of using the consolidated arrangement for
>>> JDK 10.
>>> 
>>> This message is being sent to jdk9-dev since a jdk10-dev alias to
>>> discuss JDK 10 doesn't exist yet.
>>> 
>>> A JEP describing the project has been submitted :
>>> 
>>>   JDK-8167368: Consolidate JDK 10 OpenJDK repositories to a single
>>> repository
>>>   https://bugs.openjdk.java.net/browse/JDK-8167368
>>> 
>>> The text of the JEP describes the motivation and current state of the
>>> work in more detail, including proposed changes to the file layout.
>>> Publication of the prototype consolidated repository is planned, but not
>>> done yet. The email below has a list of additional anticipated questions
>>> and answers.
>>> 
>>> We feel this consolidated arrangement offers some significant structural
>>> advantages for managing the JDK's source code and we are now asking for
>>> feedback on this potential change. In particular, if you feel there is a
>>> show-stopper problem with making this change, please let us know!
>>> 
>>> I'd like to acknowledge the work of Stefan Sarne, Stuart Marks, and
>>> Ingemar Aberg participating in discussions leading up to the prototype
>>> and I'd like to especially recognize the contributions of Erik Helin for
>>> savvy Hg manipulations and Erik Joelsson for skillful build wrangling in
>>> this project.
>>> 
>>> Please send initial comments by October 18, 2016.
>>> 
>>> Cheers,
>>> 
>>> -Joe
>>> 
>>> Q: What about the set of forests for JDK 10? Are we going to have
>>> master, dev, client, hotspot, etc. the same set as in 9?
>>> A: That is a separate question from the repository consolidation, but
>>> there will likely be simplifications here too. Discussions on that point
>>> will come later.
>>> 
>>> Q: I usually just build the code in repo X today. Will I have have to
>>> build the *whole JDK* now?
>>> A: Not necessarily. The same top-level build targets should work in the
>>> consolidated forest.
>>> 
>>> Q: Does disk usage change?
>>> A: The total disk usage of the current forest compared to the
>>> consolidated forest is nearly the same.
>>> 
>>> Q: In more detail, how were the changesets imported?
>>> A: The scripts used for the consolidation conversion are attached to the
>>> JEP.
>>> 
>>> Q: What happens to the Hg hashes?
>>> A: The conversion scheme used in the prototype does *not* preserve Hg
>>> hashes of changesets compared the current forests. However, the bug ids
>>> are preserved and can be searched for. In addition, one or more
>>> pre-consolidation forests should be archived in perpetuity so that URLs
>>> in bug comments continue to work, etc.
>>> 
>>> A mapping of the old hashes to the corresponding new hashes might be
>>> generated and placed in the final new repo.
>>> 
>>> Q: I'm allergic to tabs; what about jcheck?
>>> A: If history is preserved, the checking done by jcheck needs to be
>>> modified for the consolidated forest. One way to do this is to augment
>>> the white lists used in jcheck with the conflicting changesets. This
>>> approach may not be elegant, but it is effective and doesn't appear to
>>> appreciably impact jcheck running times.
>>> 
>>> Q: Will the future 9 update forest also have this consolidation
>>> restructuring?
>>> A: The script used to do the consolidation conversion is deterministic
>>> and could be run to create the  9 update forest in the future at the
>>> discretion of the 9 update team.
>>> 
>>> Q: For backports for forwardports, will there be a script to translate
>>> patch files across the consolidation boundary?
>>> A: That work is planned, but not yet done; see JDK-8165623: Create patch
>>> translator to update paths pre/post consolidation.
>>> 
>>> Q: It's the 21st century and I develop using an IDE. That is still going
>>> to work, right?
>>> A: The prototype to date does include updating the various IDE support
>>> files, but bug JDK-8167142 has been filed to track that work.
>>>