RFR(XS): 8204652: [aix] fix thread stack allocation, too big stack sizes not recognized

[Thomas Stüfe]

Hi Goetz,

I see what you intent to do, but I think we need to think about error
handling a bit more. The pattern here (which was broken you try to fix
it with this patch) is that we try, sequentially:

- pthread_attr_setstacksize()

- if ok-so-far pthread_attr_setguardsize

- if ok-so-far pthread_ceate

- logging and error handling


Problem with that is that in the end, we log information about a
failing thread creation, printing errno as if pthread_create() failed,
but that errno may have come actually from one of the preceeding OS
calls. So the error message would be misleading.

I think it would be better to handle errors of each system call explicitly:

pthread_attr_setstacksize()
if failed
   - log out "pthread_attr_setsize failed" and errno
   - [ also: optionally decide what to do: failing thread creation?
retry with a default stack size? ]
   - return false

pthread_attr_setguardsize()
if failed
   - log out "pthread_attr_setguardsize failed" and errno
   - decide what to do - ignore? assert? fail thread creation? Should
we even care?
   - return false

and then call pthread_create with the existing error handling as follow up.

What do you think?`

(Also, if pthread_attr_setstacksize() fails, should the thread not be
created with the pthread default stack size, which should be ~2MB? Why
is that stack tiny?)

Best Regards, Thomas


On Sun, Jun 10, 2018 at 5:33 PM, Lindenmaier, Goetz
<goetz.lindenmaier at sap.com> wrote:
> Hi,
>
> This fixes a crash in thread creation with too big stack sizes
> http://cr.openjdk.java.net/~goetz/wr18/8204652-aixHugeStacksCrash/01/
> Please review.
>
> For Java and Compiler threads the return code of pthread_attr_setstacksize
>  is overwritten. If pthread_attr_setstacksize does not succeed, a tiny
>  stack size with not enough space for the yellow/red pages is allocated.
>  VM crashes with SIGSEGV right away.
>
> Best regards,
>   Goetz