Proposed Ergonomics Profiles

[Severin Gehwolf]

Hi Stephanie,

In principle it would be useful to have, so I'd be on board with such a
proposal. It would free us from rolling our own tuning in downstream
images. This boils down to having different profiles for physical
deployments (status quo) vs. deployment in containers (on k8s), right?

Of course, such a feature would have a non-free cost item on the
continuous test column. Any thoughts how you'd plan to ensure that both
profiles behave as they're supposed to behave?

On Tue, 2023-05-16 at 20:18 +0000, Stephanie Crater wrote:
> Hi, 
>  
> The Java Engineering Group at Microsoft is currently working on a JEP
> to introduce Ergonomics Profiles as a new JVM feature, with a
> `shared` profile for the existing JVM ergonomics and a `dedicated`
> option for when the JVM is running on systems with dedicated
> resources for the one JVM process.
>  
> The current default JVM ergonomics were designed with the
> understanding that the JVM must share resources with other processes.
> However, a recent study done by an APM vendor (New Relic) identified
> that more than 70% of monitored JVMs [1] in production are running in
> dedicated environments (e.g., containers) as opposed to being shared.
> Many of these JVMs are running without explicit JVM tuning flags,
> once more confirming that JVM tuning is a challenging exercise many
> developers have no experience with. Introducing updated ergonomics
> for when the JVM is running in specific environments would allow the
> JVM to consume available resources more effectively instead of
> running with default ergonomics aimed at shared environments.
>  
> For example, our customer data from Azure Spring Apps shows that 83%
> of monitored JVMs do not use JVM flags to set the heap size. Using
> the current JVM ergonomics, the default maximum heap size of the JVM
> varies from 50% to 25%, depending on how much memory is available in
> the environment: up to 256MB, or 512MB or more, respectively, with a
> fixed amount of ~127MB for systems with anywhere between 256MB and
> 512MB of memory. These amounts do not adequately map the intended
> resource plan of dedicated environments. The user may have already
> considered to allocating, e.g., 4GB of memory to the JVM and expect
> it to use more than only 1GB of the heap (25%). 
>  
> The `dedicated` ergonomics profile will contain different heuristics
> to increase resource consumption in the environment, compared to
> `shared`.

Perhaps it would sense from an high level understanding perspective to
sketch out what you envision such a 'dedicated' profile would actually
amount to? Do you have some concrete ideas?

> The ergonomics we target include heuristics for maximum heap size, GC
> selection, active processor counting, and thread pool sizes internal
> to the JVM. If it would help, we have started writing this proposal
> in a JEP format.

Any thoughts why active processor counting would need adjustment for
such a profile? Why would the current way how the container detection
code abstracts that metric be insufficient?

> We would love to hear what the community thinks about this proposed
> enhancement and any suggestions you may have for the dedicated
> ergonomics profile. For example, this profile will likely increase
> heap size allocation to 60%-70% by default, but GC selection and
> active processor counting are much more complex. This JEP would also
> provide a framework for OpenJDK to include more ergonomics profiles
> for specific machines, environments, or workloads.

Greater than 2 profiles seem concerning. Why do you think more than two
would be necessary?

Thanks,
Severin

> Thank you for the feedback! 
>  
> [1]: https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem