[OpenJDK 2D-Dev] <AWT Dev> [8] Review request for 8005607: Recursion in J2DXErrHandler() Causes a Stack Overflow on Linux

[Anthony Petrov]

Yes, I agree. I think the simplest fix would be to install the 
J2DXErrHandler() just once, and either simply relay to the 
ANY_PREVIOUS_HANDLER, or do its current job depending on whether the 2D 
code currently wants it.

--
best regards,
Anthony

On 01/10/13 05:44, Jim Graham wrote:
> I think I'd rather see some way to prevent double-adding the handler in
> the first place as well. Since it is only ever used on errors I also
> think it is OK to set it once and leave it there forever...
>
> ...jim
>
> On 1/9/13 8:08 AM, Anthony Petrov wrote:
>> Hi Anton et al.,
>>
>> If I read the description of the bug correctly, specifically this part:
>>
>>> The problem occurs, if another thread (for example, GTK thread) is
>>> doing the same sort of thing concurrently. This can lead to the
>>> following situation.
>>> JVM thread: Sets J2DXErrHandler(), saves ANY_PREVIOUS_HANDLER as
>>> previous GTK thread: Sets some GTK_HANDLER, saves
>>> J2DXErrHandler() as previous JVM thread: Restores
>>> ANY_PREVIOUS_HANDLER GTK thread: Restores J2DXErrHandler() JVM
>>> thread: Sets J2DXErrHandler(), saves J2DXErrHandler() as previous
>>
>> It is obvious that at this final step 2D is in an inconsistent state. We
>> don't expect to replace our own error handler (and it shouldn't have
>> been there in the first place).
>>
>> I realize that the fix you propose works around this problem. But this
>> doesn't look like an ideal solution to me.
>>
>> BTW, IIRC, in JDK7 (and 6?) we decided to set the actual X11 error
>> handler only once and never replace it. All the rest of the
>> push_handler/pop_handler logic is now located in Java code (see
>> XToolkit.SAVED_ERROR_HANDLER() and the surrounding logic). I believe
>> that we should somehow share this machinery with the 2D code to avoid
>> this sort of problems. Though I'm not sure if this will eliminate this
>> exact issue.
>>
>>
>> 2D/AWT folks: any other thoughts?
>>
>> --
>> best regards,
>> Anthony
>>
>> On 12/29/2012 17:44, Anton Litvinov wrote:
>>> Hello,
>>>
>>> Please review the following fix for a bug.
>>>
>>> Bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8005607
>>> https://jbs.oracle.com/bugs/browse/JDK-8005607
>>> Webrev: http://cr.openjdk.java.net/~alitvinov/8005607/webrev.00
>>>
>>> The bug consists in a crash which is caused by a stack overflow for
>>> the reason of an infinite recursion in AWT native function
>>> J2DXErrHandler() under certain circumstances on 32-bit Linux OS. The
>>> fix is based on introduction of the logic, which detects indirect
>>> recursive calls to J2DXErrHandler() by means of a simple counter, to
>>> J2DXErrHandler() native function. Such a solution requires minimum
>>> code changes, does not alter the handler's code significantly and
>>> eliminates this bug.
>>>
>>> Adding 2d-dev at openjdk.java.net e-mail alias to the list of recipients
>>> of this letter, because the edited function's name is related to Java
>>> 2D area of JDK, despite of the fact that the edited file is located in
>>> AWT directory.
>>>
>>> Thank you,
>>> Anton