I would think it's the other way around. Floating-point numbers are not exact, and there is a little fuzzyness to them because there is a finite number of bits. So, a value x in a register is really x±δ. If you compute f(x) and the function exists and is well-behaved for all points in the interval x-δ...x+δ, but happens to be undefined only
at exactly x, then perhaps the caller really meant some other value in that range of uncertainty, and could not represent it exactly in the register. It makes sence to use the limit, when the limit is well-defined and the same for positive and negative directions.
For division by zero, division by a very very small number that got rounded to zero, if you did keep the actual number, would be an overflow anyway. The limit here is infinity. The negative and positive directions are not equal. It can't guess whether your number was actually a tad above zero or a tad under zero, and even if it did would get an overflow anyway, so it really can't help.