RFR [14]: 8232207: Linux os::available_memory re-reads cgroup configuration on every invocation

[Claes Redestad]

Hi Bob,

it seems I need to clarify that I've not proposed that
-XX:UseContainerSupport should be turned off by default.

Instead, what I've been musing about between the lines here is that
there should probably be some flag like 
-XX:ContainerConfigUpdateThreshold=<ms> that controls the
various grace times, with some value (-1?) to turn off updating
altogether.

I'll stop musing now, move ahead with this patch as-is and leave it up
to you to decide whether or not to consider any flags (and change of
defaults) for 8227006 and beyond.

Thanks!

/Claes

On 2019-10-16 17:11, Bob Vandette wrote:
> The problem is that cgroups is enabled by default on most host platforms and there
> is no documented and supported mechanism to know that we are running in a container.
> 
> In my opinion, opt-in container support is not an option.  We've had way to much bad publicity
> in the past due to our inability to adapt to container limits and I’m not going back there!
> 
> I’ve considered checking for the existence of a /.dockerenv file but this doesn’t solve the
> overhead startup problem when we are running in a container.  This is the reason we have been
> looking for other approaches.
> 
> I propose that we adopt your suggestion of timeout caching on key container metrics that
> are exposed to the VM and JDK core libraries during startup.
> 
> 
> Bob.
> 
> 
>> On Oct 16, 2019, at 5:07 AM, Claes Redestad <claes.redestad at oracle.com> wrote:
>>
>>
>>
>> On 2019-10-16 13:52, David Holmes wrote:
>>> On 16/10/2019 6:42 pm, Claes Redestad wrote:
>>>> On 2019-10-16 06:08, David Holmes wrote:
>>>>>>> If the typical scenario is a static configuration then perhaps that
>>>>>>> would have been a better default. A fully adaptive mode (with the
>>>>>>> overheads that comes) sounds like something that should be an opt-in.
>>>>>>
>>>>>> +1 on fully adaptive mode should be opt-in.
>>>>>>
>>>>>> IIRC, the premise was that "we cannot prove that the config stays
>>>>>> static in all cases". Hence, the adaptive mode conclusion. I'm in no
>>>>>> way in that court, though.
>>>>>
>>>>> Yes the underlying cgroup functionality allows for dynamism. The fact some particular container framework chooses not to allow it is not something we know a-priori or can readily detect. Until container management APIs catch up we're stuck at assuming the worst-case - unless we add flags to choose otherwise.
>>>>
>>>> No. What we *must* do is to not regress the platform for existing users
>>>> and applications, and by making this assumption that the JVM should
>>>> default to re-reading cgroup settings all the time, that's just what
>>>> we've done. I now think we should treat this as the regression it is and
>>>> make the behavior to adapt to a change in constraints an opt-in.
>>> There is no "regression" here. If you're running in a container then we have to report what the container settings are and they come from reading the cgroup information which can be dynamically updated. Unless we have something to tell us the information is static and can be cached we have no choice for correctly implementing the functionality.
>>
>> Wrong. I'm not running in a container: I'm seeing the overheads on any
>> java program on all my Linux systems - no docker or anything involved.
>>
>> So I'm paying a price for something I'm not using and need to add a flag
>> to not have to pay that price (-XX:-UseContainerSupport)
>>
>> Regardless: True dynamic adaptiveness needs much more than what's
>> currently in place in the VM (and libraries). Instead now a lot of
>> common use - with or without containers - is paying for that overhead
>> with no upside.
>>
>> Adhering to container / cgroup limits by default: great, tiny one-off
>> overhead.
>>
>> Adapting dynamically at runtime: questionable, cause for measurable
>> regressions (8232207, 8227006)
>>
>> /Claes
>