in reply to Re^4: Integers sometimes turn into Reals after substraction (printed accuracy and float subtraction)
in thread Integers sometimes turn into Reals after substraction

[Ignore. I just noticed that both expressions are completely constant.]

So aside from the output precision issues, a root problem is creating an NV in the first place?

Not quite.

The root problem is trying to store a number that's periodic in binary into a floating point number.

____________________ 0.16 base 10 = 0.00101000111101011100 base 2

That means that it can't be stored exactly as a float. (Obviously, it can't be stored exactly as an integer either.)

The compiler apparently uses an alternate means of calculating 1000*(4/25) such that it produces exactly 160. Whether that's stored as an IV, UV or NV is irrelevant.

  • Comment on Re^5: Integers sometimes turn into Reals after substraction (printed accuracy and float subtraction)
  • Download Code