Affine transforms - matrix algebra: equals

[Jim Graham]

An essentially equivalent metric is "multiple of Math.ulp()" which gives 
you an FP number representing the quantum increment between adjacent FP 
values for a given input number.  So, Math.ulp(v) is essentially a 
measure of the size of a "mantissa bit in v".

In the case of the "measuring visible differences over a region" command 
that I was proposing, though, you don't necessarily want "to within a 
few mantissa bits", you might actually want "to within this absolute 
dimensional error that is much, much bigger than a mantissa bit".  In 
other words, you'd specify "within 1/Nth of a pixel", not "within N bits 
of mantissa" and if you are measuring over the dimensions of a typical 
screen (0-2K for example) then "1/Nth of a pixel" is around 12 bits of 
mantissa for N=1 (or ~(2^12)/N multiple of ulp) for a "float", and 
around 40 bits of mantissa for N=1 for a "double".  This form of the 
test would probably do better with an absolute error measurement.

But, for API unit testing or "did the object compute the proper result?" 
type testing, then you'd want mantissa/ulp types of error comparisons...

			...jim

On 8/23/2012 9:29 AM, Kirill.Prazdnikov wrote:
> Hi Pavel
>
> On 8/21/2012 10:05 PM, Pavel Safrata wrote:
>> The errbound is a double wich says that particular elements can differ
>> by its value to be considered "equal", right? Or should it rather be
>> some multiplicative coefficient?
>
> I have been using "number of equal mantissa bits ( assuming the floating
> point power component are equal)" when I measure FP equality.
> This allows to "equally" compare numbers with different exponent.
>
> Thanks
> -Kirill