Update on JDK 10 repo consolidation; third generation prototype published at http://hg.openjdk.java.net/jdk10/consol-proto

[Thomas Stüfe]

On Wed, Sep 6, 2017 at 5:35 PM, Volker Simonis <volker.simonis at gmail.com>
wrote:

> On Wed, Sep 6, 2017 at 2:30 PM, Thomas Stüfe <thomas.stuefe at gmail.com>
> wrote:
> > Erik, Volker,
> >
> > On Tue, Sep 5, 2017 at 7:31 PM, Erik Joelsson <erik.joelsson at oracle.com>
> > wrote:
> >>
> >>
> >>
> >> On 2017-09-05 11:38, Volker Simonis wrote:
> >>>
> >>> Hi Joe,
> >>>
> >>> generally looks good!
> >>>
> >>> I don't know if this has discussed before, but in my opinion the
> >>> top-level src/ directory looks a little overloaded. Wouldn't it make
> >>> sense to place all the modules into their own 'src/modules/'
> >>> subdirectory?
> >>
> >> I see what you mean, but it's no more overloaded than the jdk/src dir
> used
> >> to be. Also, the bsd, linux, solaris, demo and sample directories are
> all
> >> going away at some point in the (hopefully near) future. Left are
> hotspot
> >> and utils which I don't think warrant another directory level.
> >>>
> >>> Should jdk10/consol-proto build out of t he box or are there any known
> >>> issues? I'd like to give it a tray on AIX and Linux/ppc64 but if there
> >>> are any known, generic problems I'll wait until they get fixed.
> >>
> >> Yes, please try it. I have put a lot of work into maintaining a set of
> >> patches and scripts to keep the generated consolidated repo working and
> >> (near) equivalent with the current forest. You can review those changes
> in
> >> the prototype forest if you like.
> >>
> >
> > I cloned http://hg.openjdk.java.net/jdk10/consol-proto and built AIX.
> Build
> > runs fine.
> >
>
> Hi Thomas,
>
> thanks for checking the AIX build.
>
> > However, I found cloning the repository painfully slow.
> >
> > On my Linux box clone took ~90 minutes. On Windows, I could not
> successfully
> > clone, as cygwin mercurial did hit timeouts. I really hope this is only
> > temporary and will improve?
> >
>
> I think you should ask your company and/or Internet provider :)
>
> The new, consolidated repository is about 1.6GB in size (i.e. the size
> of .hg) and the cloning speed is actually pretty much proportional to
> the time you need to download this amount of data.
>
> When using our http-proxy I measured a download speed of about 350
> KB/s. The calculation is quite simple:
>
> 1600000 KB / 350 KB/s = 4571 s / 60 = 76 min :(
>
> The actual cloning speed was slightly faster:
>
> $ time hg clone http://hg.openjdk.java.net/jdk10/consol-proto
> jdk10-cons-proto
> requesting all changes
> adding changesets
> adding manifests
> adding file changes
> added 46937 changesets with 397578 changes to 162338 files
> updating to branch default
> 57067 files updated, 0 files merged, 0 files removed, 0 files
> unresolved
>
> real    38m37.424s
> user    5m46.620s
> sys    0m33.976s
>
> which is probably because the 1.6 GB from the .hg repo may be
> compressed for the transport.
>
> Without proxy (i.e. 'transparent proxy') the average download speed is
> about 150 KB/s. I think this pretty much explains the ~90 minutes.
>
> I doubt that these poor download speeds are caused by
> hg.openjdk.java.net because then it would be the same for the proxy
> vs. non-proxy case. Nevertheless it would be interesting to see what
> amount of data hg.openjdk.java.net can actually serve to Europe. So if
> somebody with a decent Internet connection can share his experience,
> that would be interesting.
>
>
I'm on cable, with a download speed of 128Mbit. Cloning the new repo still
took 39min. That was on Linux, with SSDs.

Best Regards, Thomas


> Regards,
> Volker
>
> > Kind Regards, Thomas
> >
> >
> >>
> >> /Erik
> >>
> >>>
> >>> Thank you and best regards,
> >>> Volker
> >>>
> >>>
> >>> On Fri, Sep 1, 2017 at 8:36 PM, joe darcy <joe.darcy at oracle.com>
> wrote:
> >>>>
> >>>> On 8/25/2017 1:52 PM, joe darcy wrote:
> >>>>>
> >>>>> Hello,
> >>>>>
> >>>>> A follow-up to the most recent update [1] on the JDK 10 repo
> >>>>> consolidation
> >>>>> efforts, September 2017 is almost upon us and that month remains the
> >>>>> target
> >>>>> to implement the repo consolidation.
> >>>>>
> >>>>> First, a third generation prototype having tags from both JDK 9 and
> JDK
> >>>>> 10
> >>>>> will be published in the near future.
> >>>>>
> >>>> [snip]
> >>>>
> >>>> Third generation prototype with tags from JDK 9 and JDK 10 available
> for
> >>>> browsing at:
> >>>>
> >>>>      http://hg.openjdk.java.net/jdk10/consol-proto/tags
> >>>>
> >>>> Please sent comments by Wednesday, September 6.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> -Joe
> >>
> >>
> >
>