This behavior is shocking to you because you think in base 10. The computer thinks in base 2 and a number that may appear to you to only take a finite amount of digits in base 10, but it may take an infinite number of digits in base 2.

This issue exists in C, Perl, Java and all languages that do computations in base 2, native binary machine code. For business applications (e.g. accounting), special hardware and software packages exist that do math in BCD, Binary Coded Decimal - that is an encoding where 4 binary bits only represents 0-9 instead of 0-15 like in binary.

If you take an accounting class and we do straight line depreciation of 1/3 per year over 3 years. At "the end of day" these "round down" things come back to haunt us and one way to handle this is that the last year gets just slightly more depreciation so that the total depreciation works out to the initial value of the asset. These are "real world" considerations and they matter - heck anything having to do with money matters!