Behavior of evaluation at prompt can be confusing (from gforge #21596)
Imported issue: Initially reported by @chevilla in https://gforge.inria.fr/tracker/?group_id=1015&aid=21596
The following behavior is really confusing:
eps = (round(14426/10000,prec,RN) * (x - 1) - round(72135/100000,prec,RN) * (x - 1)^2)/log2(x) - 1;
eps(1);
Warning: rounding has happened. The value displayed is a faithful rounding to 165 bits of the true result.
-6.5877324222896634700941584434482894283506171891075e-5
abs(eps(1));
Warning: the given expression is undefined or numerically unstable.
NaN
If we are able to evalute eps(1) with faithful rounding, why aren't we able to compute something relevant for abs(eps(1))?
The reason is indeed that the behavior of autoprint for eps(1) is an exception, which has been implemented on purpose to make autoprint behaves like evaluate, but this behavior is hardly generalizable: eps(1) really is the expression defining eps, where all occurrences of the free variable have been replaced by 1. This reduces to 0/0-1 in this case. But, as a special case, when such a substitution is made on the prompt, if the resulting expression evaluates to NaN, it is replaced by a call to evaluate. This explains why eps(1) returns something meaningful in the above example.
However, this behavior is hard to generalize to any expression, and currently even simple expressions such as abs(eps(1)) do not try to be smart and are simply evaluated as abs(0/0-1).
Notice in particular that tests like eps(1) < eps(2) do not behave as one could expect (evaluate eps at 1 and at 2, with adapted precision, so as to ensure the result, or when increasing precision several times is not successful, return a warning mentioning that they are probably equal, but that it cannot be proven). And there is indeed no simple way to currently emulate this desired behavior.
A solution that could be considered elegant -- and this is the purpose of this feature request -- would be to change the behavior of substitution with a constant: instead of being replaced by 0/0-1, the expression eps(1) would be like a libraryconstant: a constant that could be evaluated at any precision on demand, by remembering that it is constructed as the expression eps, evaluated at point 1. Notice that, whenever an expression f evaluated at some constant point x0 takes a value which is exactly representable as a floating-point, f(x0) is not the expression where x0 has been substituted to x, but it is the exact result of the evaluation. The proposed feature request would generalize this behavior to any constant expression, seen as a kind of libraryconstant.