[PATCH] 6788196: Array bounds checks in io_util.c rely on undefined behaviour

[Andrew Haley]

>  >> In C, the result of an overflowing add of two signed integers is
>  >> undefined.
>
> Strewth! That's a surprise to me. I always thought that C defined
> integer arithmetic to always wrap. Checking for a negative to detect
> overflow is a common pattern - heck it's THE pattern for detecting
> overflow according to "Hacker's Delight"! But it isn't valid C ! ???

Yep, it's really true.  gcc assumes that signed arithmetic can never
overflow, so

  if ((off < 0) || (off > datalen) ||
    (len < 0) || ((off + len) > datalen) || ((off + len) < 0))

can be optimized to

  if ((off < 0) || (off > datalen) ||
    (len < 0) || ((off + len) > datalen))

because we know that off and len are positive.

This optimization doesn't seem very useful here, but it means you can
do things like transforming

   for (int i = N; i < N + m; i++)
     foo();

into

   for (int i = m; --i >= 0;)
     foo();

This is not a change.  gcc has done this for years, but it's a bit
more aggressive than it used to be.

Andrew.