Looking ahead: proposed Hg forest consolidation for JDK 10

[Weijun Wang]

I am very glad to see history preserved. Is it possible to also merge 
the history for src files before modularization and after it with this 
magical script? I can call "hg log -f" on the command line but the web 
interface only shows partial history.

Thanks
Max

On 10/11/2016 10:11, joe darcy wrote:
> Hello,
>
> Looking ahead to JDK 10, a group of JDK engineers have been exploring
> consolidating the large number of Hg repositories in an open JDK forest
> to a single one with the goal of using the consolidated arrangement for
> JDK 10.
>
> This message is being sent to jdk9-dev since a jdk10-dev alias to
> discuss JDK 10 doesn't exist yet.
>
> A JEP describing the project has been submitted :
>
>     JDK-8167368: Consolidate JDK 10 OpenJDK repositories to a single
> repository
>     https://bugs.openjdk.java.net/browse/JDK-8167368
>
> The text of the JEP describes the motivation and current state of the
> work in more detail, including proposed changes to the file layout.
> Publication of the prototype consolidated repository is planned, but not
> done yet. The email below has a list of additional anticipated questions
> and answers.
>
> We feel this consolidated arrangement offers some significant structural
> advantages for managing the JDK's source code and we are now asking for
> feedback on this potential change. In particular, if you feel there is a
> show-stopper problem with making this change, please let us know!
>
> I'd like to acknowledge the work of Stefan Sarne, Stuart Marks, and
> Ingemar Aberg participating in discussions leading up to the prototype
> and I'd like to especially recognize the contributions of Erik Helin for
> savvy Hg manipulations and Erik Joelsson for skillful build wrangling in
> this project.
>
> Please send initial comments by October 18, 2016.
>
> Cheers,
>
> -Joe
>
> Q: What about the set of forests for JDK 10? Are we going to have
> master, dev, client, hotspot, etc. the same set as in 9?
> A: That is a separate question from the repository consolidation, but
> there will likely be simplifications here too. Discussions on that point
> will come later.
>
> Q: I usually just build the code in repo X today. Will I have have to
> build the *whole JDK* now?
> A: Not necessarily. The same top-level build targets should work in the
> consolidated forest.
>
> Q: Does disk usage change?
> A: The total disk usage of the current forest compared to the
> consolidated forest is nearly the same.
>
> Q: In more detail, how were the changesets imported?
> A: The scripts used for the consolidation conversion are attached to the
> JEP.
>
> Q: What happens to the Hg hashes?
> A: The conversion scheme used in the prototype does *not* preserve Hg
> hashes of changesets compared the current forests. However, the bug ids
> are preserved and can be searched for. In addition, one or more
> pre-consolidation forests should be archived in perpetuity so that URLs
> in bug comments continue to work, etc.
>
> A mapping of the old hashes to the corresponding new hashes might be
> generated and placed in the final new repo.
>
> Q: I'm allergic to tabs; what about jcheck?
> A: If history is preserved, the checking done by jcheck needs to be
> modified for the consolidated forest. One way to do this is to augment
> the white lists used in jcheck with the conflicting changesets. This
> approach may not be elegant, but it is effective and doesn't appear to
> appreciably impact jcheck running times.
>
> Q: Will the future 9 update forest also have this consolidation
> restructuring?
> A: The script used to do the consolidation conversion is deterministic
> and could be run to create the  9 update forest in the future at the
> discretion of the 9 update team.
>
> Q: For backports for forwardports, will there be a script to translate
> patch files across the consolidation boundary?
> A: That work is planned, but not yet done; see JDK-8165623: Create patch
> translator to update paths pre/post consolidation.
>
> Q: It's the 21st century and I develop using an IDE. That is still going
> to work, right?
> A: The prototype to date does include updating the various IDE support
> files, but bug JDK-8167142 has been filed to track that work.
>