RFR: 8326227: Rounding error that may distort computeNextGaussian results [v14]

[Chris Hennick]

On Sun, 11 Aug 2024 07:30:12 GMT, Chris Hennick <duke at openjdk.org> wrote:

>> This provides a slightly more accurate bounding limit for `computeNextExponentialSoftCapped`. Currently, if the maximum is specified as 12.0, it won't actually return a value larger than `1.5 * exponentialX0` (11.353912041222094); and the error gets worse as we go further into the tail. (This affects slightly less than 12 outputs per million for an ideal RNG.) This could cause the `while (computeNextExponentialSoftCapped(rng, limit) < limit)` check in `computeNextGaussian` on line 1402 to always be true, making `nextGaussian` loop infinitely in the worst case (with probability $$e^{-2^{63} exponentialX0} \approx 10^{-3.022 \times 10^{19}}$$ for an ideal RNG); but more likely, it would give a result that was truncated too close to zero.
>> 
>> This change is being tested prior to submission to OpenJDK by https://github.com/openjdk/jdk/pull/17703/commits/b8be051cbf40a6a05fafc6a2c76942e9e0b11fdf.
>
> Chris Hennick has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Lower limit to fix timeout
>  - Fix?
>  - Revert "Fix?"
>    
>    This reverts commit 5e29b6f924269410b800c4f5a367d7bc259be5b2.

Update: having read the McFarland paper, I'm no longer confident this bug actually exists or that any performance improvements are possible.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17703#issuecomment-2282898111