Java heap size defaults when running with CGroups in Linux.

[David Holmes]

Hi Kirk,

On 1/12/2016 2:44 PM, kirk at kodewerk.com wrote:
> Hi David,
>
> Thanks, I missed that one. That said, CPU count is but one of the
> machine sized resources. Another question I ask and rarely get the right
> answer to is how large is heap if you don’t specify -mx… and so on. It
> is awesome that the JVM often picks reasonable results for machine sized
> configurations. It means that people *don’t* have to know the answers to
> these questions and they’ll still get a reasonable result until the don’t

The ergonomics decisions for sizing of various memory related values, 
including heap is not trivial, and may also depend on other runtime factors.

When people develop container technologies like cgroups it would be nice 
if they actually considered how their imposed constraints become visible 
through the API's provided by the OS. Unfortunately OS API's were also 
not written to answer questions that now have 3 or more levels of 
context**: ie how much memory is available to my process; how much is 
available to the virtual environment my process is running on; and how 
much is on the physical machine.

Determining where cgroup memory constraints need to be exposed to the 
JVM is not obvious to me. But we're happy to hear from any cgroup 
experts who can shed light on this. But this work is targeted for JDK 10.

Cheers,
David
-----

** At least for CPUs there are API's to cover the process and "machine" 
cases. But we had to examine all the uses of "processor count" to check 
which ones were interested in the actual available processors, and which 
were concerned with the machine processor count.

.
>
> Regards,
> Kirk
>
>> On Dec 1, 2016, at 5:35 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> Hi Kirk,
>>
>> On 1/12/2016 2:31 PM, kirk at kodewerk.com wrote:
>>> Hi all,
>>>
>>> This is a problem that everyone using Docker faces and it is a very common question, how do I configure the JVM when I’m using Docker. The number of parameters that have to be set in order to catch everything is daunting (through probably not necessary). For example, you wouldn’t believe how many people don’t know that there are machine sized resources in the JVM such as the shared F-J thread pool.
>>
>> Note that the CPU issue is fixed in JDK 9 and 8u122.
>>
>> https://bugs.openjdk.java.net/browse/JDK-6515172
>>
>> David
>> -----
>>
>>> Kind regards,
>>> Kirk
>>>
>>>> On Dec 1, 2016, at 1:10 AM, Steven Schlansker <stevenschlansker at gmail.com> wrote:
>>>>
>>>> We run in this configuration and I just want to point out that there's more going
>>>> on than simply correctly fixing heap size.  It's a great first start but we also found
>>>> that we had to spend a fair amount of time playing with all the other tuneables such
>>>> as Metaspace, Code Cache, etc. in order to get our Java apps to run stably
>>>> in cgroups.
>>>>
>>>> Not in anyway to argue against this change, just that it'd be nice to get the "big picture"
>>>> tackled at some point -- heap is only one part :)
>>>>
>>>> Thanks!
>>>> Steven
>>>>
>>>>> On Nov 30, 2016, at 2:53 PM, Mikael Vidstedt <mikael.vidstedt at oracle.com> wrote:
>>>>>
>>>>>
>>>>> Out of curiosity, why wouldn’t this be the default behavior? That is, in which cases is it not a good idea to use the cgroup information when sizing the JVM?
>>>>>
>>>>> Cheers,
>>>>> Mikael
>>>>>
>>>>>
>>>>>> On Nov 30, 2016, at 1:59 PM, Christine Flood <chf at redhat.com> wrote:
>>>>>>
>>>>>>
>>>>>> The problem is that when running the JVM inside of a cgroup, such as docker, the JVM bases it's default heap parameters on the size of the whole machine's memory not on the memory available to the container.  This causes errors as discussed on this blog entry.  http://matthewkwilliams.com/index.php/2016/03/17/docker-cgroups-memory-constraints-and-java-cautionary-tale/
>>>>>>
>>>>>> Basically the JVM dies in a non-obvious manner.
>>>>>>
>>>>>> The solution I propose is to add a parameter -XX:+UseCGroupLimits to the JVM which states that you should look to the CGroup when calculating default heap sizes.
>>>>>>
>>>>>> Webrev is here:  http://cr.openjdk.java.net/~andrew/rh1390708/webrev.01/
>>>>>>
>>>>>>
>>>>>> Christine
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>