Question regarding JEP 8204089 Timely Reducing Unused Committed Memory

[Thomas Schatzl]

Hi Aleksey,

On Mon, 2018-06-04 at 15:13 +0200, Aleksey Shipilev wrote:
> On 06/04/2018 02:26 PM, Thomas Schatzl wrote:
> > Implementing something like Shenandoah, i.e. uncommit the regions
> > that stayed empty for more than ShenandoahUncommitDelay
> > milliseconds would mean iterating through all regions and check if
> > they are "old" enough to uncommit (and do so) at regular intervals
> > in some STW pause. When allocating, also give them some sort of
> > timestamp. Again, you need to make sure that the regions are looked
> > through in some regular interval (either piggy-backing by some
> > existing regular pauses or force one).
> 
> I am not sure why STW pause is needed. Nor why timestamps on
> allocation path are needed.
> 
> In Shenandoah, region FSM has a few additional states: Empty-
> Uncommitted, Empty-Committed, Regular, Humongous. Allocation path
> makes state transitions a la {Empty-*, Regular} -> Regular, etc.
> Cleanup moves reclaimed regions to Empty-Committed, and records the
> timestamp when that transition happened. Uncommit does Empty-
> Committed -> Empty-Uncommitted for all the regions that have a
> fitting timestamp.
> All these transitions are sharing the same mechanics as the
> allocation path, so it does not require pause to work.

I was wrong about the suggestion to take the timestamp in the
allocation path, you are right you need them in the deallocation path.
Thanks for pointing out my mistake.

There are different interesting uses for a timestamp in the allocation
path that are not relevant here.

> The uncommit checks are done by the concurrent control thread that
> normally drives the GC cycle, but also does auxiliary work outside of
> GC cycle too.

You are right that you do not necessarily need a STW pause to do the
check-for-regions-to-uncommit work. However you do need at some point
synchronize with the free region list (this is the list of regions new
regions are allocated from in G1) before uncommit.

At this point this free region list access uses a global lock, and we
really want to get rid of this global lock as this restrains
(allocation) throughput already in some applications. So *more* global
locking is something we would prefer to avoid. (I am aware that this is
a very infrequent use anyway, and only grabbed when there is not much
application activity).

In G1 we do not mind about some small extra work in an STW pause - G1
by definition will always do stw pauses for evacuation work (it
wouldn't be G1 at that point), and iterating over all regions is
already done there in various places (and is very fast anyway even for
many regions, and if needed trivially parallelizable).
(I am also talking only about removing these regions from the free
region list, and not necessarily the uncommit call which can be
deferred; I think however that is very cheap.)

The VM already does some (regular) non-GC work in non-GC STW pauses
which as mentioned would be candidates for putting that stuff in there.
In the future this non-gc work will probably be moved into concurrent
phases step-by-step, but at that point, when hopefully an API usable
for all collectors for that has emerged, it may be easiest to move this
work there at that time.

Also somebody needs to implement that, and putting this into an
existing stw pause for now seems to be the easiest way to do. But that
is just my opinion, we can certainly help Rodrigo and Ruslan
implementing that if they want it.

Note that having an additional "background worker" thread adds (some)
overhead in terms of startup and shutdown (time, memory) that some
people might want to avoid too. I am not saying that this would be
prohibitive, just that there is that concern.

This is my current view on that particular problem :)

> > This would not be particularly hard to implement either, but only
> > seems extra work: as you might still want to compact the heap in
> > some way (this is optional) by e.g. doing a marking + mixed cycle
> > (and then wait for the ShenandoahUncommitDelay), so you may as well
> > uncommit within these pauses already.
> 
> Periodic GC does indeed help to compact things, but that is a second-
> order concern. Even uncommitting the regions that became free after
> the cycle provides a very substantial win.

I did mention that this additional heap compaction is fully optional.
However in case of containers you might want to take these extra space
savings with you, assuming that there memory is the main constraint to
put more containers on the same machine, not cpu time.

As you mentioned, one could do both at the same time.

> The beauty of Shenandoah-style uncommit is that it becomes orthogonal
> to the GC cycles themselves: you can first implement uncommits, and
> then figure out in which form and shape to issue periodic GC cycles
> to knock out the rest of the garbage. We might not even bother with
> that part, and just instruct users to say
> -XX:+ExplicitGCInvokesConcurrent, and wait for System.gc() to happen
> for the most compaction.
> 
> The caveat with piggybacking on pauses, is that you don't really want
> to uncommit the memory that would be committed back by allocations 
> in active application. It does make sense to do this on
> explicit GC though! You can do this under the pause too, but let's be
> mindful about the allocation costs, especially if we are about to 
> commit memory back while holding the allocation lock :)

There are already some safeguards in place to prevent overzealous
uncommit in all collectors.

I am not sure that leaving heap compaction to an explicit (potentially)
concurrent GC is what is wanted here.

Thanks,
  Thomas