JVM heaps can scale up but not down

[Jesper Wilhelmsson]

Andrew Ash skrev 21/8/14 12:40:
> Hi Jesper,
>
> Many thanks for the extensive answer!
>
> It sounds like with the newer features the scale-down scenario will be possible.
>   Here's how I think it could work from the new features:
>
> 1. Mesos starts a task with X memory of Y memory on a machine
> 2. JVM starts with -Xmx=Y and a MinHeapFree ratio of (X-Y)/Y to keep the max
> actual utilization at X
> 3. Application running in the JVM does work
> 4. Mesos decides to take back memory and sends a message to the app that it
> should shrink to Z memory (Z < X)
> 5. App reduces its memory usage by the appropriate amount
> 6. App signals to the JVM to use a lower MaxHeapFreeRatio and triggers Full GC
> 7. JVM performs Full GC and shrinks heap
> 8. App checks if memory usage is below Z; goto step 5 if not
> 9. App tells Mesos it has shrunk its usage
> 10. Mesos gives the memory to some other task

Seems reasonable.

>
> The Xmx/MinHeapFree calculation in step 2 is to support increasing the memory
> usage up to the maximum of the machine (is Xmx manageable now?).

No, Xmx is not manageable. If the JVM decides that the application needs a 
bigger heap to run with proper performance the heap will grow again. If you want 
to force the heap to stay small you'd probably have to keep a lower 
MaxHeapFreeRatio and be willing to pay with lower performance.

>
> There are some difficulties in step 5 of how the app knows how much memory to
> remove.  In my motivating case (Apache Spark) the app keeps a pool of cached
> data with size estimates, and could potentially drop the appropriate amount of
> data from its cache.  It would require recalculation but that can be streamed
> and never held in a JVM's heap at once.

Well I guess only the app developer knows how much memory the app can release 
and still function properly. If it's a cache then you'd again have to consider 
performance vs memory usage.

>
> The combination of step 6 and step 7 are what the recent JDK feature work will
> make possible.
>
>
> Is this how you envision the scale-down scenario working in practice?

Yes, this is exactly how I envisioned it :)
Let me know if it works out IRL.
/Jesper

>
>
> Many thanks,
> Andrew
>
>
> On Wed, Aug 20, 2014 at 11:47 PM, Jesper Wilhelmsson
> <jesper.wilhelmsson at oracle.com <mailto:jesper.wilhelmsson at oracle.com>> wrote:
>
>     Hi Andrew,
>
>     The flag -XX:MaxHeapFreeRatio was recently made manageable so that you can
>     change its value during runtime. The motivating use case behind that change
>     was exactly the scenario you describe, that you want to force a running JVM
>     to shrink the heap.
>
>     Lowering MaxHeapFreeRatio may decrease performance since it will likely
>     increase garbage collection frequency. The way to use it in a production
>     server would probably be to lower the ratio, force a System.gc to shrink the
>     heap and then increase the ratio again.
>
>     Unfortunately there can never be any guaranties that the heap can shrink
>     unless you force a full GC that compacts the entire heap. The heap has to be
>     allocated as one consecutive memory area and if you're unlucky there can be
>     some long lived object allocated near the top of the heap which makes it
>     impossible to shrink the memory area.
>
>     Recently there has been plenty of work in the G1 collector to deal with this
>     situation. In 8u20 we started sorting the memory regions to increase the
>     chances of allocating long lived objects at the bottom of the heap. In 8u40
>     we will introduce the possibility to decommit memory within the heap. This
>     will make it possible to shrink the heap by cutting out a hole somewhere
>     within the heap rather than being forced to only shrink from the top. This
>     change will most likely be pushed to 8u40 within the next few days, it's
>     already present in the JDK 9 repository.
>
>     These features are all default behavior in G1, so if you haven't already,
>     switch to G1 (-XX:+UseG1GC) and try out the latest bits to see if it solves
>     your problem.
>
>     Best regards,
>     /Jesper
>
>
>     Andrew Ash skrev 21/8/14 00:42:
>
>         // please redirect if this is the wrong list
>
>         Hi JDK devs,
>
>         I'm occasionally in a situation where I have a JVM running and want to tell
>         it to "scale down" to use less heap than its current active set.  This
>         would be in the space between -Xms and -Xmx bounds.  Scaling "up" from -Xms
>         towards -Xmx obviously happens automatically, and I've heard conventional
>         wisdom that you can't decrease heap size on a running JVM, but I've never
>         heard of any work being done to make that possible.
>
>         The use case is when I have a long-running JVM in an Apache Mesos [1]
>         context, and Mesos wants to "take back" some memory resources from a
>         running JVM task.
>
>         Is there any work being done to make scaling down possible?
>
>         Cheers,
>         Andrew
>
>
>         [1] https://mesos.apache.org/
>
>