Idea for making ResourceArea allocation signal safe

[Thomas Stüfe]

Hi,

I wanted to run an idea by the group.

In the context of signal handlers, we may allocate RA memory. That is not
ideal but may happen. One example is error reporting - there is no
guarantee, even if we are careful, that some code down the stack may not
use RA. Another example is things like AsyncGetCallTrace, used in the async
profiler. I'm sure there may be more examples.

The problem is that if the signal interrupts the thread while it was
modifying the arena underlaying its RA, the arena is left in an
inconsistent state and subsequent allocations from it may malfunction. RA
allocation is not reentry-safe.

A very simple solution to this would be double buffering. We could hold,
per thread, a secondary resource area, to be used only in signal handling.
At the entrance of the hotspot 0signal handler (which everyone goes
through, even in chain scenarios like with AsyncGetCallTrace) we would
switch over to the secondary resource area, and switch back when leaving
the hotspot signal handler.

The question here is whether to create the secondary RA upfront, at Thread
creation time, or on-demand only. Since not every thread may enter signal
handling, and even if it does it may never allocate RA memory in signal
handlers.

1) If we create it at thread creation time, we pay a small overhead for the
allocation of the RA itself as well as for the first init chunk in its
arena. It is not much, about 1K on 64-bit platforms, but still.

2) If we create it on demand only, we cause allocations in the signal
handler. Arguably this may happen anyway if the Thread were to allocate RA
in the signal handler, and that allocation were to cause arena expansion.
But today its rare. (2) has two variants:
 a) unconditionally create the secondary RA upon first invocation of a
signal handler if it does not exist
 b) hide the on-demand creation under the Thread::resource_area()
accessor() (if _resource_area == NULL create resource area).

(1) is safe and simple, but atm would cost two mallocs and about 1K of
memory per thread

(2a) means we pay for the arena of threads that enter signal handling but
never would allocate RA while in signal handler. This may be almost every
Thread. It also means we are now guaranteed to always malloc at least once
in signal handling.

(2b) seems elegant - we only create the secondary RA if a thread actually
needs it - but this affects every RA allocation in- and outside signal
handling, we now have a NULL check everywhere in the
Thread::resource_area() accessor.

There may be some further changes we could do to mitigate the costs:

- In RA, delay the creation of the first chunk until the resource area is
actually needed
- Or, the other way around, embed the first chunk of a Thread's resource
area into the Thread structure itself. But this would increase the size of
Thread, and it is already a monster.

What do you think?

Cheers, Thomas