RFR: JDK-8266506: [aix] Treat mapping attempt too close to BRK as a mapping error

[Thomas Stuefe]

On AIX, in os::attempt_reserve_memory_at(), if the user specified a wish address too close to the BRK segment, we silently ignore the wish address and just map anywhere.

This is not the right behavior since the normal behavior when failing to map to a wish address is to return NULL (see JDK-8266419). Therefore we should return NULL in these cases instead.

This is a day zero bug.

-------------

Commit messages:
 - start

Changes: https://git.openjdk.java.net/jdk/pull/3859/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3859&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8266506
  Stats: 6 lines in 1 file changed: 2 ins; 0 del; 4 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3859.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3859/head:pull/3859

PR: https://git.openjdk.java.net/jdk/pull/3859