Update on JDK 10 repo consolidation; third generation prototype published at http://hg.openjdk.java.net/jdk10/consol-proto

[Doug Simon]

> On 7 Sep 2017, at 07:06, Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
> 
> On Wed, Sep 6, 2017 at 5:35 PM, Volker Simonis <volker.simonis at gmail.com>
> wrote:
> 
>> On Wed, Sep 6, 2017 at 2:30 PM, Thomas Stüfe <thomas.stuefe at gmail.com>
>> wrote:
>>> Erik, Volker,
>>> 
>>> On Tue, Sep 5, 2017 at 7:31 PM, Erik Joelsson <erik.joelsson at oracle.com>
>>> wrote:
>>>> 
>>>> 
>>>> 
>>>> On 2017-09-05 11:38, Volker Simonis wrote:
>>>>> 
>>>>> Hi Joe,
>>>>> 
>>>>> generally looks good!
>>>>> 
>>>>> I don't know if this has discussed before, but in my opinion the
>>>>> top-level src/ directory looks a little overloaded. Wouldn't it make
>>>>> sense to place all the modules into their own 'src/modules/'
>>>>> subdirectory?
>>>> 
>>>> I see what you mean, but it's no more overloaded than the jdk/src dir
>> used
>>>> to be. Also, the bsd, linux, solaris, demo and sample directories are
>> all
>>>> going away at some point in the (hopefully near) future. Left are
>> hotspot
>>>> and utils which I don't think warrant another directory level.
>>>>> 
>>>>> Should jdk10/consol-proto build out of t he box or are there any known
>>>>> issues? I'd like to give it a tray on AIX and Linux/ppc64 but if there
>>>>> are any known, generic problems I'll wait until they get fixed.
>>>> 
>>>> Yes, please try it. I have put a lot of work into maintaining a set of
>>>> patches and scripts to keep the generated consolidated repo working and
>>>> (near) equivalent with the current forest. You can review those changes
>> in
>>>> the prototype forest if you like.
>>>> 
>>> 
>>> I cloned http://hg.openjdk.java.net/jdk10/consol-proto and built AIX.
>> Build
>>> runs fine.
>>> 
>> 
>> Hi Thomas,
>> 
>> thanks for checking the AIX build.
>> 
>>> However, I found cloning the repository painfully slow.
>>> 
>>> On my Linux box clone took ~90 minutes. On Windows, I could not
>> successfully
>>> clone, as cygwin mercurial did hit timeouts. I really hope this is only
>>> temporary and will improve?
>>> 
>> 
>> I think you should ask your company and/or Internet provider :)
>> 
>> The new, consolidated repository is about 1.6GB in size (i.e. the size
>> of .hg) and the cloning speed is actually pretty much proportional to
>> the time you need to download this amount of data.
>> 
>> When using our http-proxy I measured a download speed of about 350
>> KB/s. The calculation is quite simple:
>> 
>> 1600000 KB / 350 KB/s = 4571 s / 60 = 76 min :(
>> 
>> The actual cloning speed was slightly faster:
>> 
>> $ time hg clone http://hg.openjdk.java.net/jdk10/consol-proto
>> jdk10-cons-proto
>> requesting all changes
>> adding changesets
>> adding manifests
>> adding file changes
>> added 46937 changesets with 397578 changes to 162338 files
>> updating to branch default
>> 57067 files updated, 0 files merged, 0 files removed, 0 files
>> unresolved
>> 
>> real    38m37.424s
>> user    5m46.620s
>> sys    0m33.976s
>> 
>> which is probably because the 1.6 GB from the .hg repo may be
>> compressed for the transport.
>> 
>> Without proxy (i.e. 'transparent proxy') the average download speed is
>> about 150 KB/s. I think this pretty much explains the ~90 minutes.
>> 
>> I doubt that these poor download speeds are caused by
>> hg.openjdk.java.net because then it would be the same for the proxy
>> vs. non-proxy case. Nevertheless it would be interesting to see what
>> amount of data hg.openjdk.java.net can actually serve to Europe. So if
>> somebody with a decent Internet connection can share his experience,
>> that would be interesting.
>> 
>> 
> I'm on cable, with a download speed of 128Mbit. Cloning the new repo still
> took 39min. That was on Linux, with SSDs.

I'm in Switzerland, get 85Mbps according to fast.com and it took 42min to clone the new repo.

-Doug

> 
> Best Regards, Thomas
> 
> 
>> Regards,
>> Volker
>> 
>>> Kind Regards, Thomas
>>> 
>>> 
>>>> 
>>>> /Erik
>>>> 
>>>>> 
>>>>> Thank you and best regards,
>>>>> Volker
>>>>> 
>>>>> 
>>>>> On Fri, Sep 1, 2017 at 8:36 PM, joe darcy <joe.darcy at oracle.com>
>> wrote:
>>>>>> 
>>>>>> On 8/25/2017 1:52 PM, joe darcy wrote:
>>>>>>> 
>>>>>>> Hello,
>>>>>>> 
>>>>>>> A follow-up to the most recent update [1] on the JDK 10 repo
>>>>>>> consolidation
>>>>>>> efforts, September 2017 is almost upon us and that month remains the
>>>>>>> target
>>>>>>> to implement the repo consolidation.
>>>>>>> 
>>>>>>> First, a third generation prototype having tags from both JDK 9 and
>> JDK
>>>>>>> 10
>>>>>>> will be published in the near future.
>>>>>>> 
>>>>>> [snip]
>>>>>> 
>>>>>> Third generation prototype with tags from JDK 9 and JDK 10 available
>> for
>>>>>> browsing at:
>>>>>> 
>>>>>>     http://hg.openjdk.java.net/jdk10/consol-proto/tags
>>>>>> 
>>>>>> Please sent comments by Wednesday, September 6.
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> -Joe