RFR(XS): 8204652: [aix] fix thread stack allocation, too big stack sizes not recognized

[Lindenmaier, Goetz]

Hi,

No, this is not matched to the thread types, but to the flags.
There is 
  CompilerThreadStackSize
  VMThreadStackSize
  ThreadStackSize.

internal, there are more types, but you can not set the stack size for them.
I want to tell here what flag you got wrong. 

Best, Goetz.

> -----Original Message-----
> From: Thomas Stüfe [mailto:thomas.stuefe at gmail.com]
> Sent: Montag, 11. Juni 2018 12:03
> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>
> Cc: David Holmes <david.holmes at oracle.com>; hotspot-runtime-
> dev at openjdk.java.net
> Subject: Re: RFR(XS): 8204652: [aix] fix thread stack allocation, too big stack
> sizes not recognized
> 
> Hi,
> 
> yes, that is better. However, there are many more thread types (see
> ThreadType in os.hpp) so your message now misrepresents a number of
> thread types as VM thread.
> 
> I would either remove the thr_type printout, or just print the
> numerical value of the enum.
> 
> If you just remove the printout, I do not need to see a new webrev.
> 
> Best Regards, Thomas
> 
> 
> On Mon, Jun 11, 2018 at 11:47 AM, Lindenmaier, Goetz
> <goetz.lindenmaier at sap.com> wrote:
> > Hi,
> >
> > Is this better?
> > http://cr.openjdk.java.net/~goetz/wr18/8204652-aixHugeStacksCrash/02/
> >
> > It reports now:
> > bin/java -XX:CompilerThreadStackSize=80000000000000
> > [1.989s][warning][os,thread] The compiler thread stack size specified is
> invalid: 80000000000000k
> > Error occurred during initialization of VM
> > java.lang.OutOfMemoryError: unable to create native thread: possibly out
> of memory or process/resource limits reached
> >
> > Best,
> >   Goetz.
> >
> >> -----Original Message-----
> >> From: David Holmes [mailto:david.holmes at oracle.com]
> >> Sent: Montag, 11. Juni 2018 10:36
> >> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Thomas Stüfe
> >> <thomas.stuefe at gmail.com>
> >> Cc: hotspot-runtime-dev at openjdk.java.net
> >> Subject: Re: RFR(XS): 8204652: [aix] fix thread stack allocation, too big
> stack
> >> sizes not recognized
> >>
> >> On 11/06/2018 6:07 PM, Lindenmaier, Goetz wrote:
> >> > Hi,
> >> >
> >> > Yes, reporting the pthread_create is a bit misleading. But it is the same
> >> > as reported on linux x86_64:
> >> >
> >> > bin/java -XX:CompilerThreadStackSize=8000000000000
> >> > [0.271s][warning][os,thread] Failed to start thread - pthread_create
> failed
> >> (EAGAIN) for attributes: stacksize: 8000000000000k, guardsize: 0k,
> detached.
> >> > Error occurred during initialization of VM
> >> > java.lang.OutOfMemoryError: unable to create native thread: possibly
> out
> >> of memory or process/resource limits reached
> >>
> >> But on linux you _do_ call pthread_create and it did fail!
> >>
> >> > As we know linux is bad at implementing the pthread functionality here:
> >> > It accepts setting the big value as stack size, and only when you call
> create
> >> > you get the failure.
> >>
> >> Does POSIX require otherwise?
> >>
> >> > So it's not true that the VM starts with any other value, it also crashes on
> >> > linux.  And you get the same output as on linux. So I realy don't want to
> >> > start implementing any clever tracing, or even implement other stack
> >> > handling (setting defaults) as on linux.  I would just like to keep it as is.
> >> > Currently we get a SIGSEGV which is hard to debug on aix.
> >>
> >> The point is the error should be reported accurately in the logging
> >> information. It's not coming from pthread_create in your AIX case so
> >> don't report that it is.
> >>
> >> David
> >>
> >> > Best,
> >> >    Goetz
> >> >
> >> >> -----Original Message-----
> >> >> From: Thomas Stüfe [mailto:thomas.stuefe at gmail.com]
> >> >> Sent: Montag, 11. Juni 2018 06:26
> >> >> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>
> >> >> Cc: hotspot-runtime-dev at openjdk.java.net
> >> >> Subject: Re: RFR(XS): 8204652: [aix] fix thread stack allocation, too big
> >> stack
> >> >> sizes not recognized
> >> >>
> >> >> Hi Goetz,
> >> >>
> >> >> On Sun, Jun 10, 2018 at 9:11 PM, Lindenmaier, Goetz
> >> >> <goetz.lindenmaier at sap.com> wrote:
> >> >>> Hi Thomas,
> >> >>>
> >> >>> The stack is started with 192k.  See the output:
> >> >>>
> >> >>
> >> >> Ah you are right. With 64K pages this would not be enough to establish
> >> >> red/yellow zones and still have breathing space left.
> >> >>
> >> >>> bash-4.4$ ~/oJ/vms/aix_j-1/bin/java -
> >> >> XX:CompilerThreadStackSize=8000000000000
> >> >>> [1.482s][warning][os,thread] The compiler thread stack size specified
> is
> >> >> invalid: 8000000000000k
> >> >>> [1.483s][warning][os,thread] Failed to start thread - pthread_create
> >> failed
> >> >> (22=EINVAL) for attributes: stacksize: 192k, guardsize: 4k, detached.
> >> >>> Error occurred during initialization of VM
> >> >>> java.lang.OutOfMemoryError: unable to create native thread:
> possibly
> >> out
> >> >> of memory or process/resource limits reached
> >> >>>
> >> >>> I think the output is not that bad ...
> >> >>
> >> >> See my mail to David: I think the output is misleading, since
> >> >> pthread_create() did not even run. So, "pthread_create failed" is
> >> >> wrong.
> >> >>
> >> >> Best Regards, Thomas
> >> >>
> >> >>>
> >> >>> Best,
> >> >>>    Goetz.
> >> >>>
> >> >>>> -----Original Message-----
> >> >>>> From: Thomas Stüfe <thomas.stuefe at gmail.com>
> >> >>>> Sent: Sunday, June 10, 2018 7:33 PM
> >> >>>> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>
> >> >>>> Cc: hotspot-runtime-dev at openjdk.java.net
> >> >>>> Subject: Re: RFR(XS): 8204652: [aix] fix thread stack allocation, too
> big
> >> >> stack
> >> >>>> sizes not recognized
> >> >>>>
> >> >>>> Hi Goetz,
> >> >>>>
> >> >>>> I see what you intent to do, but I think we need to think about error
> >> >>>> handling a bit more. The pattern here (which was broken you try to
> fix
> >> >>>> it with this patch) is that we try, sequentially:
> >> >>>>
> >> >>>> - pthread_attr_setstacksize()
> >> >>>>
> >> >>>> - if ok-so-far pthread_attr_setguardsize
> >> >>>>
> >> >>>> - if ok-so-far pthread_ceate
> >> >>>>
> >> >>>> - logging and error handling
> >> >>>>
> >> >>>>
> >> >>>> Problem with that is that in the end, we log information about a
> >> >>>> failing thread creation, printing errno as if pthread_create() failed,
> >> >>>> but that errno may have come actually from one of the preceeding
> OS
> >> >>>> calls. So the error message would be misleading.
> >> >>>>
> >> >>>> I think it would be better to handle errors of each system call
> explicitly:
> >> >>>>
> >> >>>> pthread_attr_setstacksize()
> >> >>>> if failed
> >> >>>>     - log out "pthread_attr_setsize failed" and errno
> >> >>>>     - [ also: optionally decide what to do: failing thread creation?
> >> >>>> retry with a default stack size? ]
> >> >>>>     - return false
> >> >>>>
> >> >>>> pthread_attr_setguardsize()
> >> >>>> if failed
> >> >>>>     - log out "pthread_attr_setguardsize failed" and errno
> >> >>>>     - decide what to do - ignore? assert? fail thread creation? Should
> >> >>>> we even care?
> >> >>>>     - return false
> >> >>>>
> >> >>>> and then call pthread_create with the existing error handling as
> follow
> >> up.
> >> >>>>
> >> >>>> What do you think?`
> >> >>>>
> >> >>>> (Also, if pthread_attr_setstacksize() fails, should the thread not be
> >> >>>> created with the pthread default stack size, which should be ~2MB?
> >> Why
> >> >>>> is that stack tiny?)
> >> >>>>
> >> >>>> Best Regards, Thomas
> >> >>>>
> >> >>>>
> >> >>>> On Sun, Jun 10, 2018 at 5:33 PM, Lindenmaier, Goetz
> >> >>>> <goetz.lindenmaier at sap.com> wrote:
> >> >>>>> Hi,
> >> >>>>>
> >> >>>>> This fixes a crash in thread creation with too big stack sizes
> >> >>>>> http://cr.openjdk.java.net/~goetz/wr18/8204652-
> >> >> aixHugeStacksCrash/01/
> >> >>>>> Please review.
> >> >>>>>
> >> >>>>> For Java and Compiler threads the return code of
> >> >>>> pthread_attr_setstacksize
> >> >>>>>   is overwritten. If pthread_attr_setstacksize does not succeed, a
> tiny
> >> >>>>>   stack size with not enough space for the yellow/red pages is
> >> allocated.
> >> >>>>>   VM crashes with SIGSEGV right away.
> >> >>>>>
> >> >>>>> Best regards,
> >> >>>>>    Goetz