Looking ahead: proposed Hg forest consolidation for JDK 10

[Robbin Ehn]

I tested this last week, ship it!

Thanks to all involved for doing this.

/Robbin

On 10/11/2016 04:11 AM, joe darcy wrote:
> Hello,
>
> Looking ahead to JDK 10, a group of JDK engineers have been exploring consolidating the large number of Hg repositories in an open JDK forest to a single one with the goal
> of using the consolidated arrangement for JDK 10.
>
> This message is being sent to jdk9-dev since a jdk10-dev alias to discuss JDK 10 doesn't exist yet.
>
> A JEP describing the project has been submitted :
>
>     JDK-8167368: Consolidate JDK 10 OpenJDK repositories to a single repository
>     https://bugs.openjdk.java.net/browse/JDK-8167368
>
> The text of the JEP describes the motivation and current state of the work in more detail, including proposed changes to the file layout. Publication of the prototype
> consolidated repository is planned, but not done yet. The email below has a list of additional anticipated questions and answers.
>
> We feel this consolidated arrangement offers some significant structural advantages for managing the JDK's source code and we are now asking for feedback on this potential
> change. In particular, if you feel there is a show-stopper problem with making this change, please let us know!
>
> I'd like to acknowledge the work of Stefan Sarne, Stuart Marks, and Ingemar Aberg participating in discussions leading up to the prototype and I'd like to especially
> recognize the contributions of Erik Helin for savvy Hg manipulations and Erik Joelsson for skillful build wrangling in this project.
>
> Please send initial comments by October 18, 2016.
>
> Cheers,
>
> -Joe
>
> Q: What about the set of forests for JDK 10? Are we going to have master, dev, client, hotspot, etc. the same set as in 9?
> A: That is a separate question from the repository consolidation, but there will likely be simplifications here too. Discussions on that point will come later.
>
> Q: I usually just build the code in repo X today. Will I have have to build the *whole JDK* now?
> A: Not necessarily. The same top-level build targets should work in the consolidated forest.
>
> Q: Does disk usage change?
> A: The total disk usage of the current forest compared to the consolidated forest is nearly the same.
>
> Q: In more detail, how were the changesets imported?
> A: The scripts used for the consolidation conversion are attached to the JEP.
>
> Q: What happens to the Hg hashes?
> A: The conversion scheme used in the prototype does *not* preserve Hg hashes of changesets compared the current forests. However, the bug ids are preserved and can be
> searched for. In addition, one or more pre-consolidation forests should be archived in perpetuity so that URLs in bug comments continue to work, etc.
>
> A mapping of the old hashes to the corresponding new hashes might be generated and placed in the final new repo.
>
> Q: I'm allergic to tabs; what about jcheck?
> A: If history is preserved, the checking done by jcheck needs to be modified for the consolidated forest. One way to do this is to augment the white lists used in jcheck
> with the conflicting changesets. This approach may not be elegant, but it is effective and doesn't appear to appreciably impact jcheck running times.
>
> Q: Will the future 9 update forest also have this consolidation restructuring?
> A: The script used to do the consolidation conversion is deterministic and could be run to create the  9 update forest in the future at the discretion of the 9 update team.
>
> Q: For backports for forwardports, will there be a script to translate patch files across the consolidation boundary?
> A: That work is planned, but not yet done; see JDK-8165623: Create patch translator to update paths pre/post consolidation.
>
> Q: It's the 21st century and I develop using an IDE. That is still going to work, right?
> A: The prototype to date does include updating the various IDE support files, but bug JDK-8167142 has been filed to track that work.
>