[9, P1] RFR (S/M): 8180048: Interned string and symbol table leak memory during parallel unlinking

[Thomas Schatzl]

Hi all,

  can I have reviews for this change that fixes a memory leak when
unlinking (removing) elements from the string and symbol tables in
parallel?

The problem is the use of BasicHashtable::free_entry() that in a non-
thread safe way adds elements to the hash table's bucket free list.

If multiple threads do that, the unlinking process looses
elements/memory forever.

This can be quite serious, the reproducer attached to the CR leaks ~1GB
of memory within like a minute or so.

It only affects G1 as it is the only collector that does string/symbol
table unlinking in parallel.

There is no test for this issue at this time: the minimum duration of
the test (that is similar to the reproducer) takes a significant amount
of time (>15s in product build per GC on my machine) to leak enough
memory so that a leak can be detected with reasonable accuracy. This
makes it kind of unusable to run every time during jprt.
I am going to add a CR to add a proper test, trying to improve what I
have, and if I can't get it down to a reasonable runtime make it so
that is run at the appropriate tier only.

I am also going to try to get this change into 8u.

Ideally I would also like to have a reviewer from the runtime team
(cc'ed).

CR:
https://bugs.openjdk.java.net/browse/JDK-8180048
Webrev:
http://cr.openjdk.java.net/~tschatzl/8180048/webrev/
Testing:
jprt, manual testing

Thanks,
  Thomas