[PATCH] 6788196: Array bounds checks in io_util.c rely on undefined behaviour

[Gary Benson]

Alan Bateman wrote:
> I checked the pre-OpenJDK history and this bounds check has been so
> since 1.2 (10+ years old). It's kinda surprising this hasn't been
> noticed with other ports. Did you run into this with an existing
> test (JCK or regression/unit)? Ideally we should have more tests to
> catch these issues (one existing test that we could update is
> java/io/readBytes/ReadBytesBounds.java).

The JCK caught it.  There's a testcase attached to my original mail
too, but it could be extended perhaps to test more values.

> Speaking of, is there a related HotSpot issue that lead to the
> assert datalen >= 0? It's likely many things would break if
> GetArrayLength return a negative value so just curious.

If datalen is negative then the bounds check can overflow: all three
values need to be positive to avoid this.  I've not come across a
situation where datalen was negative, but it didn't feel right to
check the other two without also checking datalen.

Cheers,
Gary

-- 
http://gbenson.net/