Undefined behaviour in hotspot

[David Chase]

On 2014-04-21, at 11:19 AM, Omair Majid <omajid at redhat.com> wrote:
> I recently tried to build OpenJDK on i686 using a prerelease version of GCC
> 4.9. It turns out that new optimizations have been enabled in GCC which now
> cause hotspot to break in a few places where hotspot relies on undefined
> behaviour.
> ...
> Dereferencing a NULL pointer is undefined behaviour in C++ [5] and it seems
> perfectly acceptable for a compiler to compile hotspot in a way so that it ends
> up not working.


> Obviously, I would like to get hotspot to work correctly when compiled against
> GCC 4.9. But I believe the micro-optimizations made to use the address of the
> object instead of using a field in the object is to save space and I would
> rather not blindly undo it. Does anyone here have any advice on what the best
> course of action there is?

Is there a chance that (because you work for redhat) you might have some influence
over the gcc implementers, and get them to implement/document a flag or pragma that
will cause it to cut that crap out?  I've seen discussions of other "it's undefined, ha-ha,
let's party!" optimizations that make overflow detection impossible, e.g.:

if (buf + len < buf) { // check for address space wraparound and overflow
    return EINVAL;
}

is optimized to 

if ( len < 0) {
    return EINVAL;
}

and

if (a > 0 && b > 0) {
   sum = (a + b)
   if (sum < a) { // detect integer overflow
      ...
   } 
}

becomes

if (a > 0 && b > 0) {
   sum = (a + b)
}

We used to joke about doing this kind of thing back when I worked on optimizing C/C++
compilers, but we always assumed that we could never get away with it because saving
a few nanoseconds is not as important as the checks in the two examples above.  Which
is to say, the modeled execution speed of code that I can't run because I don't trust it is
zero, so it's not really an optimization.  Just because we can get hotspot to run on correct
inputs without crashing doesn't mean that we haven't accidentally disabled security-critical
checking by using this hyperactive optimizer.  I would think that this is even more important
now than it was 20 years ago.

So, step one is to locate all the places where undefined behavior occurs, and then we have
to figure out what to do next, but we can't do anything without knowing the full scope of the
problem.

David