G1 with Solr - thread from dev at lucene.apache.org

[Shawn Heisey]

On 1/7/2015 5:52 AM, Thomas Schatzl wrote:
> Good to see that it works for you.
>
> One final comment on how to achieve higher throughput (if needed): from
> the logs I can see that the background marking is running all the time,
> which may be wasted CPU cycles.
>
> I.e. heap occupancy seems to hover roughly around 2.75G (=67% of the
> usual 4G that are used - G1 never wants to expand the heap due to pause
> time problems - only once slightly because it cannot get a large enough
> contiguous amount of space for a large object).
>
> By default (and currently) marking starts at 45% of heap occupancy. You
> may want to try to set this to something higher like 70-75% to not have
> it eat up CPU all the time.

Looks like I need to change -XX:InitiatingHeapOccupancyPercent=nn for that.  Thanks for the info.  CPU utilization is not a problem for these machines, but anything that drops resource usage will make performance better, even if limits are not being pushed.


> As mentioned, in the log file G1 keeps the specified minimum heap size
> of 4G (-Xms setting). If you have the memory you may want to give it
> more, or lower the pause time goal.

The machine has 64GB of RAM, so there's plenty of memory ... but I want
to keep as much memory as possible available for OS disk caching.  I
might increase Xms so it's the same as Xmx (6GB).

> Lowering the pause time goal to something below 200ms (with the intent
> of achieving 200ms) may also help achieving better medians and
> percentiles.

I can certainly do this.  I do not want to be greedy, which is why I did
not try for a pause target of 100ms. :)  I chose 200ms because the
setting I see in many places around the Internet is 100, so 200 seemed
more realistic.  I didn't know when I chose that number that the default
was 200ms.

Thanks,
Shawn