RFR: 8310537: Fix -Wconversion warnings in gcUtil.hpp [v3]
Coleen Phillimore
coleenp at openjdk.org
Thu Jun 22 15:56:32 UTC 2023
On Thu, 22 Jun 2023 12:28:51 GMT, Axel Boldt-Christmas <aboldtch at openjdk.org> wrote:
>> What I mean is that we are doing a least square fit on a bunch of datapoints. The algorithm requires the mean, sum and sum of squares. If the datapoints truly required double precision or magnitude then storing the sum and sum of square in a double will overflow and/or be inaccurate.
>>
>> AdaptiveWeightedAverage having a float interface strengthens the case that the samples should be floats, as the running mean should never overflow or underflow the samples `min sample ≤ mean sample ≤ max sample`, (may lose precision, but if the samples are float precision, it seems resonable that the mean also have this precision)
>>
>> It was not about safety but about having the interface express the capabilities of the class. It can sample float x,y points and provide a least square fit linear function`y(x)`
>
> At least that is my interpretation when I ever see intermediate sums use a type with more bits of information than the samples/parts. But maybe that was never the intent.
I tried changing update to take floats but there are callers that cast unsigned ints to double to call this function, so they get -Wconversion messages. Since this seems to not lose precision, I added a comment and suggest further cleanup in this area.
-------------
PR Review Comment: https://git.openjdk.org/jdk/pull/14585#discussion_r1238727112
More information about the hotspot-gc-dev
mailing list