[OpenJDK 2D-Dev] [8u Backport] Fix for JDK-8158356 : SIGSEGV when attempting to rotate BufferedImage using AffineTransform by NaN degrees

[Philip Race]

+1

-phil

On 9/14/16, 2:52 AM, Ajit Ghaisas wrote:
> Hi Jim,
>
>      I agree upon aborting the operation in case the non-finite slope is not due to divide by 0 case.
>      As this is the 8u backport of the fix done in 9, I will address the review comment in JDK-8166009.
>
>      The original backport webrev is still the same.
>      http://cr.openjdk.java.net/~aghaisas/8158356/8u_backport/webrev.00/
>
>      Request you to approve this 8u backport.
>
> Regards,
> Ajit
>
> -----Original Message-----
> From: Jim Graham
> Sent: Friday, September 02, 2016 1:16 AM
> To: Ajit Ghaisas; Philip Race; 2d-dev
> Subject: Re: [8u Backport] Fix for JDK-8158356 : SIGSEGV when attempting to rotate BufferedImage using AffineTransform by NaN degrees
>
> Hi Ajit,
>
> In the cases where you "continue" on a non-finite slope, doesn't that mean that the edges will be mismatched?  If you can't determine the bounding polygon, perhaps the entire operation should be aborted instead...?
>
> It's different from the case of dy1==dy2 which also results in a "continue" because in that case there are no edges to insert in the array because that edge of the bounding box doesn't occupy any vertical space.  In the case of an infinite slope, there could be edges expected to be produced (in fact there should be because we've already tested that dy1 != dy2), but the loop doesn't produce any edge values and stale data is left in the array.
>
> 			...jim
>
> On 9/1/16 2:14 AM, Ajit Ghaisas wrote:
>> Hi,
>>
>>
>>
>>     This is a review request for 8u back-port of the fix done in JDK-9.
>>
>>
>>
>> Bug :
>>
>>     https://bugs.openjdk.java.net/browse/JDK-8158356
>>
>>
>>
>> Root cause :
>>
>>     The root cause of the crash is - NaN is converted to an integer and used as array index in mlib_ImageScanPoly.c.
>>
>>
>>
>> Fix :
>>
>>     The native method previously did not check the validity of the
>> input double argument. Now, I have added a check for finite double values.
>>
>>
>>
>> Webrev :
>>
>>
>> http://cr.openjdk.java.net/~aghaisas/8158356/8u_backport/webrev.00/
>>
>>
>>
>> Regards,
>>
>> Ajit
>>