RFR: JDK-8144039: Enable javac server by default

[Erik Joelsson]

Hello,

On 2015-11-27 02:21, David Holmes wrote:
> Hi Erik,
>
> On 26/11/2015 6:17 PM, Erik Joelsson wrote:
>> Hello,
>>
>> This is not sjavac, but just the rather new feature javac-server. It
>> takes the client/server model from sjavac and applies it to plain old
>> javac. What this means is that when we invoke javac, it will spawn off a
>> background server process that stays alive for the duration of the
>> build. All javac compilations are then sent to this server for
>> processing from a very thin client. The performance gain comes from
>> running all javac compilations in a warm JVM.
>
> Thanks for clarifying - though now I'm unsure what the difference is 
> between this and sjavac :)
>
The other main feature of sjavac is tracking dependencies between java 
classes properly to enable fine grained incremental builds of java code.
> I also couldn't say I saw any performance improvement in build times. :(
>
That's sad. I've looked around and it seems the RE nightly build was a 
bit faster when the hardware was the same. JPRT jobs will naturally 
fluctuate a whole lot because of over allocated hypervisors so need a 
lot of data before you can see clear patterns. On local systems I'm 
seeing consistent improved build times.
>> I believe this has been tested enough to try it as default. If things
>> fail for a certain usecase, the feature can easily be turned off with
>> --disable-javac-server. If things go really haywire, we can disable it
>> again. If we didn't change this now, before feature freeze, it would
>> never get into JDK9, and I think that would be very sad as faster build
>> times is generally something everyone wants.
>
> Something that doesn't affect the product bits shouldn't be affected 
> by feature freeze in my opinion - there's no "feature". I also expect 
> we continue to make whatever improvements we can to the build system.
>
I would tend to agree, but it doesn't seem everyone does.
> My main concern with using the javac server is ensuring it gets 
> shutdown properly, particularly if things go wrong. I also worry about 
> its capacity to handle multiple concurrent requests from compilations 
> in different parts of the source tree. And I worry that a big javac 
> process may encounter OOME and other resource issues that multiple 
> javac processes would not.
>
There is a signaling system using file existence. I think the build 
system removes a file and the server shuts down when it notices (I don't 
have the details in memory). As a backup, the server will timeout after 
2 minutes of inactivity. We configure proper -mx flags for the server in 
configure, which are separate from other mx configurations. We also 
reduce the mx flag for the client processes when the server is enabled 
to reduce resource usage.
> I think it may have been more prudent to phase in this change - eg by 
> having JPRT enable it by default.
>
We could have done it that way. I don't think it was warranted this time 
however. It seems your worry was warranted though as the bootcycle build 
broke. I'm posting a fix shortly.

/Erik
> Thanks,
> David
>
>> I will compose an email to jdk9-dev announcing this change.
>>
>> /Erik
>>
>> On 2015-11-26 05:57, David Holmes wrote:
>>> Hi Erik, Magnus,
>>>
>>> Has this been extensively tested with all our build environments?
>>>
>>> It's been a while so could you give an overview of exactly how the
>>> sjavac thing works.
>>>
>>> Changes that affect every single person building the JDK should not be
>>> rushed through!
>>>
>>> Thanks,
>>> David
>>>
>>> On 26/11/2015 12:47 AM, Magnus Ihse Bursie wrote:
>>>> On 2015-11-25 15:45, Erik Joelsson wrote:
>>>>> Doh, here you go:
>>>>>
>>>>> diff -r 358fb90ae6e6 common/autoconf/build-performance.m4
>>>>> --- a/common/autoconf/build-performance.m4
>>>>> +++ b/common/autoconf/build-performance.m4
>>>>> @@ -411,9 +411,9 @@
>>>>>    AC_MSG_RESULT([$ENABLE_SJAVAC])
>>>>>    AC_SUBST(ENABLE_SJAVAC)
>>>>>
>>>>> -  AC_ARG_ENABLE([javac-server],
>>>>> [AS_HELP_STRING([--enable-javac-server],
>>>>> -      [use only the server part of sjavac for faster javac compiles
>>>>> @<:@disabled@:>@])],
>>>>> -      [ENABLE_JAVAC_SERVER="${enableval}"],
>>>>> [ENABLE_JAVAC_SERVER="no"])
>>>>> +  AC_ARG_ENABLE([javac-server],
>>>>> [AS_HELP_STRING([--disable-javac-server],
>>>>> +      [disable javac server @<:@enabled@:>@])],
>>>>> +      [ENABLE_JAVAC_SERVER="${enableval}"],
>>>>> [ENABLE_JAVAC_SERVER="yes"])
>>>>>    if test "x$JVM_ARG_OK" = "xfalse"; then
>>>>>      AC_MSG_WARN([Could not set -Xms${MS_VALUE}M -Xmx${MX_VALUE}M,
>>>>> disabling javac server])
>>>>>      ENABLE_JAVAC_SERVER="no"
>>>>>
>>>> Looks good to me.
>>>>
>>>> /Magnus
>>>>
>>>>> /Erik
>>>>>
>>>>> On 2015-11-25 15:38, Magnus Ihse Bursie wrote:
>>>>>> On 2015-11-25 15:11, Erik Joelsson wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> The --enable-javac-server feature seems to be working well and is
>>>>>>> providing considerable speedup when building the JDK. We should 
>>>>>>> make
>>>>>>> the option enabled by default.
>>>>>>>
>>>>>>> As an example, on my local machine, a run of "make exploded-image"
>>>>>>> without javac server took 00:04:09 and with javac server 00:03:20.
>>>>>>> On machines with less cpus I have seen even bigger improvements.
>>>>>>>
>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8144039
>>>>>>> Patch:
>>>>>>> diff -r 358fb90ae6e6 common/autoconf/build-performance.m4
>>>>>>> --- a/common/autoconf/build-performance.m4
>>>>>>> +++ b/common/autoconf/build-performance.m4
>>>>>>> @@ -413,7 +413,7 @@
>>>>>>>
>>>>>>>    AC_ARG_ENABLE([javac-server],
>>>>>>> [AS_HELP_STRING([--enable-javac-server],
>>>>>>>        [use only the server part of sjavac for faster javac 
>>>>>>> compiles
>>>>>>> @<:@disabled@:>@])],
>>>>>>> -      [ENABLE_JAVAC_SERVER="${enableval}"],
>>>>>>> [ENABLE_JAVAC_SERVER="no"])
>>>>>>> +      [ENABLE_JAVAC_SERVER="${enableval}"],
>>>>>>> [ENABLE_JAVAC_SERVER="yes"])
>>>>>>>    if test "x$JVM_ARG_OK" = "xfalse"; then
>>>>>>>      AC_MSG_WARN([Could not set -Xms${MS_VALUE}M -Xmx${MX_VALUE}M,
>>>>>>> disabling javac server])
>>>>>>>      ENABLE_JAVAC_SERVER="no"
>>>>>>> diff -r 358fb90ae6e6 common/autoconf/generated-configure.sh
>>>>>>
>>>>>> I like that you enable javac-server by default. But the patch was a
>>>>>> bit too hurried. :)
>>>>>>
>>>>>> When you change the default value, you are also in fact reversing 
>>>>>> the
>>>>>> logic of the configure option. As in other such options, the string
>>>>>> listed for AS_HELP_STRING should be "--disable-javac-server", and 
>>>>>> the
>>>>>> default value listed should be "enabled".
>>>>>>
>>>>>> /Magnus
>>>>>
>>>>
>>