Posted on behalf of a colleague who wishes to remain nameless...

UniBasic code that should work

The results of the following code are proving to be troublesome:

A = 4.57 + (-4.58)
B = (-0.01)
PRINT 'TEST ONE: ARE ':A:' AND ':B:' THE SAME? ':
IF A = B THEN
PRINT 'YES'
END ELSE
PRINT 'NO'
END

A = OCONV(A,'MD20')
B = OCONV(B,'MD20')
PRINT 'TEST TWO: ARE ':A:' AND ':B:' THE SAME? ':
IF A = B THEN
PRINT 'YES'
END ELSE
PRINT 'NO'
END

You would expect both tests to return 'YES', but in fact, only the
second test returns 'YES'.

TEST ONE: ARE -0.01 AND -0.01 THE SAME? NO
TEST TWO: ARE -0.01 AND -0.01 THE SAME? YES

Is there something simple that I am missing?

TEST ONE: ARE -0.01 AND -0.01 THE SAME? YES
TEST TWO: ARE -0.01 AND -0.01 THE SAME? YES

Works in D3 Linux.

Patrick, <;=)


Mike Preece wrote:

Try

IF (A + 0) = (B + 0) THEN

OR try the assignment

B = -0.01

without the brackets - I think you need to coercise the varables to be
of the same (numeric) type

A Classic case of binary system not being able to represent certain
decimals accurately.
I tested this in Excel, a1=0.58, b1=0.57,c1="a1-b1", d1=0.01
the formula in e1 was =if(c1=d1,"yes","no")
The result in e1 was no.
Seleting the cells and formatting the number of decimal places showed
the reason
up to 15 decimals c1 showed 0.010000000000000
increasing the decimals to 17 showed 0.00999999999999979

try multiplying A by 1,000,000,000,000,000 & see if the result isnt a
list of 99999...
(before you do the "a=oconv(a,'md20'))

Jeremy Thomson

Hi
Welcome to the world of wide zero. Always do conversions before
comparisons. Openqm has a flag to force rounding at a reasonable level and
I suspect so does Unidata. D3 uses x significant digits (16 I think) so
skips the problem in normal maths.
Regards
Peter McMurray
"Mike Preece" <michael (AT) preece (DOT) net> wrote

My colleague has accepted that he has to put oconvs around the values.
I'm sure he's not totally happy with that - but it works.

As for me - I am appalled! It's not nice to realise that there is a ton
of code out there that I have written over the years that will break if
migrated to dUniData or dUniVerse. It's ridiculous for 4.57 minus 4.58
not to equal zero minus .01. If this is "wide zero" then wide zero is a
pile of poo.

Mike.

Excalibur wrote:

Hi
I must admit that there are significant problems with having too great a
precision with miles of digits.
However the alternative of having N significant digits I can't remember
whether it is 14 or 16 also has its moments of glory. I did the rental
programs for Australia arcade back in the 70's where rent was 10% of
turnover or a fixed figure whichever was greater. I happily followed normal
maths practise and scaled all at the end. Everything went swimmingly for a
few months then Murphy kicked in. A Melbourne Winter when fur coats where
de riguer, no pc then, minks were $50,000 plus, turnover went through the
roof, and my program said we should pay the fur shop for being there. It
transpired that when one hit the maximum digits it was the most significant
digit that fell off the end. Big OOPS!
I scale faithfully on all varieties at every step of the calculation.
Regards
Peter McMurray
"Mike Preece" <michael (AT) preece (DOT) net> wrote

Yeah but yeah but yeah but... (too much "Little Britain" maybe)

What's to scale when we're comparing 4.57 minus 4.58 with zero minus
0.01???

Mike.

Excalibur wrote:

"Two plus two approaches four for very large values of two."

D3 does purely binary math until the numbers get really big, then it does
"string math".

Most of the newer implementations, epecially those written in C++, use
floating point math, where 1 + 1 = 1.99999999etc


Mark Brown

<panzerboy (AT) gmail (DOT) com> wrote

I have tested the above code using UniVerse 10.1.x on Windows
and Unix, results are as expected.

My wide zero settings are the default value of 0x3dc00000.

HTH

Barry