numbers: the clamps are recorded, not silent (#865)

CHANGELOG.md

@@ -6,6 +6,48 @@ All notable changes to EigenScript are documented here.
 
 ### Added
 
+- **`math_flags` / `clear_math_flags` — the numeric clamps are no longer
+  undetectable (#865).** "Finite by construction" (NaN → 0, overflow →
+  ±1e308) keeps a program running, which is the point, but it kept it
+  running with a *plausible* number and no way to tell. Two consequences
+  the contract never mentioned: reassociation silently changes a result
+  by 292 orders of magnitude — `(1e300 * 1e300) / 1e300` is `1e8`, which
+  passes any sanity check a caller applies, while `1e300 * (1e300 /
+  1e300)` is `1e300` — and a saturated value compares equal to itself
+  under further growth, so no in-language predicate could distinguish
+  "this is 1e308" from "this overflowed". NaN was worse: it collapses to
+  `0`, indistinguishable from a real zero.
+  The fix is IEEE-754's own answer, sticky exception flags. Arithmetic
+  results are **unchanged** — the finite invariant is load-bearing for
+  the JIT's bail comparison, the observer's entropy, `str of`, and the
+  JSON encoders, so abandoning it is a far larger change than the defect
+  warrants — but every clamp now sets a bit you can read:
+
+  ```eigenscript
+  clear_math_flags of null
+  result is risky of xs
+  if (math_flags of null).overflow:
+      print of "a value saturated; this result is contaminated"
+  ```
+
+  Set inside `num_guard`'s existing clamp branches, so the arithmetic
+  fast path is untouched and all ~54 call sites are covered at once. The
+  JIT needs no mirror: it already bails to the interpreter on any result
+  past `EIGS_NUM_MAX` (including ±Inf/NaN), so the interpreter's
+  `num_guard` runs and the flag cannot go tier-dependent.
+  String conversion is a route too, and it was the quietest case of
+  all: `num of "nan"` is `0` and `num of "inf"` is `1e308`, so a data
+  column containing either parsed to a plausible number with nothing to
+  check. Both now set a bit.
+  The audit also turned up **three more undocumented domain
+  substitutions**, all of which now set `invalid`: `log of 0` returns
+  `log(1e-10)` = `-23.025850929940457` (the one the issue named),
+  `sqrt of -1` returns `0` — indistinguishable from `sqrt of 0` — and
+  `asin`/`acos` silently clamp an out-of-range argument, so `asin of 5`
+  answers `asin of 1`. Their values are unchanged; only the silence is.
+  Documented in the Numbers promise, including the associativity trade,
+  so it is visible rather than discovered.
+
 - **`gfx_read` — pixel readback, the render-decode oracle primitive
   (#823).** `gfx_read of [x, y]` returns the back-buffer pixel as
   `[r, g, b]` (call after drawing, before `gfx_present`). Containment

docs/BUILTINS.md

@@ -49,6 +49,8 @@ audio (`audio_open`, `audio_close`, `audio_pause`, `audio_play`,
 | `str` | `str of value` | Convert to string representation |
 | `num` | `num of value` | Convert to number (parse string or coerce) |
 | `type` | `type of value` | Return type name: "num", "str", "list", "dict", "buffer", "text_builder", "fn", "builtin", "none" (the null value — SPEC.md is normative and its gated example prints `none`; the string `"null"` is never produced) |
+| `math_flags` | `math_flags of null` | Sticky numeric status: `{overflow, invalid}` — 1 when a clamp has fired since the last `clear_math_flags` (#865) |
+| `clear_math_flags` | `clear_math_flags of null` | Reset both status bits |
 | `assert` | `assert of [cond, msg]` | Raise catchable error `"ASSERT FAIL: <msg>"` if condition is false |
 | `exit` | `exit of N` | Terminate the program with exit code `N` (default 0). **Uncatchable** — a `try`/`catch` does not intercept it — and unwinds through normal teardown, so it is leak-clean even with live closures. Code after it does not run. The request is scoped to the evaluating thread and cleared at each host eval entry, so under the embedding API a script that calls `exit` does not disable `try`/`catch` for the host's *next* eval (#739). |
 | `coalesce` | `coalesce of [value, default]` | Return value unless empty/null, else default |

docs/LANGUAGE_CONTRACT.md

@@ -105,6 +105,37 @@ examples (executed by the suite).
 - Finite by construction: no NaN, no Infinity. NaN-producing operations
   return 0; overflow saturates at ±1e308; division by zero warns and
   yields 0.
+- **Every clamp is recorded.** The finite invariant keeps a program
+  running, but it keeps it running with a plausible number, so the
+  clamps are readable as sticky status flags — IEEE-754's own model
+  (`fetestexcept`) — rather than being undetectable (#865):
+
+  ```eigenscript
+  clear_math_flags of null
+  result is risky of xs
+  if (math_flags of null).overflow:
+      print of "a value saturated; this result is contaminated"
+  ```
+
+  `overflow` is set by the ±1e308 clamp. `invalid` is set by the
+  out-of-domain substitutions: `log of x` for `x <= 1e-10` (which
+  returns `log(1e-10)`, i.e. `-23.025850929940457`), `sqrt of x` for
+  negative `x` (returns 0, otherwise indistinguishable from
+  `sqrt of 0`), and `asin`/`acos` outside [-1, 1] (argument clamped).
+  `invalid` is also set when a NaN is collapsed, which arithmetic
+  cannot produce (there is no way to obtain an Inf to combine) but
+  string conversion can: `num of "nan"` is `0` and `num of "inf"` is
+  `1e308`, so a data column containing either used to parse to a
+  plausible number with nothing to check. Both bits are sticky until
+  `clear_math_flags`, so bracket a computation the way you would on an
+  FPU.
+- **Saturation is not associative, and that is not detectable from the
+  value alone.** `(1e300 * 1e300) / 1e300` is `1e8`; `1e300 * (1e300 /
+  1e300)` is `1e300`. The first overflowed and came back down, and
+  `1e8` will pass any plausibility check a caller applies. The results
+  are what the finite invariant requires — the `overflow` flag is how
+  you tell. Stated here because the trade should be visible rather than
+  discovered.
 - `str of` produces the shortest representation that round-trips back to
   the same double; `num of (str of x) == x`.
 - **Every producer of number text obeys that same rule** — `str of`,
@@ -115,7 +146,8 @@ examples (executed by the suite).
 - `%` follows the dividend's sign (C semantics): `-7 % 3 == -1`.
 
 **Status:** Enforced — `tests/test_number_format.eigs`,
-`tests/test_numeric_guard.eigs`, `tests/test_json_roundtrip.eigs`.
+`tests/test_numeric_guard.eigs` (NG20–NG30 cover the flags),
+`tests/test_json_roundtrip.eigs`.
 
 ## Strings
 

src/builtins.c

@@ -772,6 +772,42 @@ Value* builtin_classify(Value *arg) {
     return out;
 }
 
+/* ---- #865: sticky numeric status ----
+ *
+ * The finite-number invariant (NaN -> 0, overflow -> +/-1e308) keeps a
+ * program running, which is the point, but it kept it running with a
+ * plausible number and no way to tell. Two consequences the contract did not
+ * mention: reassociation silently changes a result by 292 orders of magnitude
+ * ((1e300*1e300)/1e300 is 1e8, not 1e300), and an overflowed value compares
+ * equal to itself under further growth, so no in-language predicate could
+ * distinguish "this is 1e308" from "this overflowed".
+ *
+ * These are IEEE-754's sticky exception flags. The results are unchanged —
+ * the finite invariant is load-bearing for the JIT's bail comparison, the
+ * observer's entropy, `str of`, and the JSON encoders — but the clamp is no
+ * longer undetectable. Bracket a computation the way you would an FPU:
+ *
+ *     clear_math_flags of null
+ *     result is risky_computation of xs
+ *     if (math_flags of null).overflow:
+ *         ...
+ */
+Value* builtin_math_flags(Value *arg) {
+    (void)arg;
+    Value *d = make_dict(2);
+    Value *ov = make_num((g_math_flags & EIGS_MATH_OVERFLOW) ? 1 : 0);
+    Value *iv = make_num((g_math_flags & EIGS_MATH_INVALID) ? 1 : 0);
+    dict_set_owned(d, "overflow", ov);
+    dict_set_owned(d, "invalid", iv);
+    return d;
+}
+
+Value* builtin_clear_math_flags(Value *arg) {
+    (void)arg;
+    g_math_flags = 0;
+    return make_null();
+}
+
 Value* builtin_type(Value *arg) {
     if (!arg) return make_str("none");
     switch (arg->type) {
@@ -1874,16 +1910,24 @@ Value* builtin_tan(Value *arg) {
 Value* builtin_asin(Value *arg) {
     if (!arg || arg->type != VAL_NUM) return make_num(0);
     double x = arg->data.num;
-    if (x < -1.0) x = -1.0;
-    if (x > 1.0) x = 1.0;
+    /* #865: an out-of-domain argument is clamped, so `asin of 5` answers
+     * `asin of 1` with no signal. The clamp stays; the invalid bit records it. */
+    if (x < -1.0 || x > 1.0) {
+        g_math_flags |= EIGS_MATH_INVALID;
+        x = (x < -1.0) ? -1.0 : 1.0;
+    }
     return make_num(asin(x));
 }
 
 Value* builtin_acos(Value *arg) {
     if (!arg || arg->type != VAL_NUM) return make_num(0);
     double x = arg->data.num;
-    if (x < -1.0) x = -1.0;
-    if (x > 1.0) x = 1.0;
+    /* #865: an out-of-domain argument is clamped, so `acos of 5` answers
+     * `acos of 1` with no signal. The clamp stays; the invalid bit records it. */
+    if (x < -1.0 || x > 1.0) {
+        g_math_flags |= EIGS_MATH_INVALID;
+        x = (x < -1.0) ? -1.0 : 1.0;
+    }
     return make_num(acos(x));
 }
 
@@ -5816,6 +5860,8 @@ void register_builtins(Env *env) {
     env_set_local_owned(env, "observe", make_builtin(builtin_observe));
     env_set_local_owned(env, "classify", make_builtin(builtin_classify));
     env_set_local_owned(env, "type", make_builtin(builtin_type));
+    env_set_local_owned(env, "math_flags", make_builtin(builtin_math_flags));            /* #865 */
+    env_set_local_owned(env, "clear_math_flags", make_builtin(builtin_clear_math_flags));
     env_set_local_owned(env, "json_encode", make_builtin(builtin_json_encode));
     env_set_local_owned(env, "json_decode", make_builtin(builtin_json_decode));
     env_set_local_owned(env, "coalesce", make_builtin(builtin_coalesce));

src/builtins_tensor.c

@@ -322,9 +322,22 @@ static Value* tensor_unary(Value *v, UnaryOpFn fn) {
     return make_num(0.0);
 }
 
-static double op_sqrt(double x) { return (x < 0) ? 0.0 : sqrt(x); }
+/* #865: `sqrt of -1` returns 0, which is indistinguishable from `sqrt of 0`.
+ * Like the log clamp below, the substituted value stays and the invalid bit
+ * records that the argument was out of domain. */
+static double op_sqrt(double x) {
+    if (x < 0) { g_math_flags |= EIGS_MATH_INVALID; return 0.0; }
+    return sqrt(x);
+}
 static double op_exp(double x) { return num_guard(exp(x)); }
-static double op_log_safe(double x) { return num_guard(log(x > 1e-10 ? x : 1e-10)); }
+/* #865: `log of 0` returns log(1e-10) = -23.025850929940457, an undocumented
+ * substitution that is neither of the two clamps the Numbers promise covers.
+ * The value stays (kernels depend on it), but the invalid bit now says the
+ * argument was out of domain and the answer is a stand-in. */
+static double op_log_safe(double x) {
+    if (!(x > 1e-10)) { g_math_flags |= EIGS_MATH_INVALID; return num_guard(log(1e-10)); }
+    return num_guard(log(x));
+}
 static double op_neg(double x) { return -x; }
 
 /* ==== BUILTIN: sqrt ==== */
@@ -434,8 +447,10 @@ Value* builtin_tensor_log_softmax(Value *arg) {
     double *flat = tensor_to_flat(tensor, &rows, &cols);
     if (!flat) return make_null();
     ne_softmax_buf(flat, rows, cols);
-    for (int i = 0; i < rows * cols; i++)
+    for (int i = 0; i < rows * cols; i++) {
+        if (!(flat[i] > 1e-10)) g_math_flags |= EIGS_MATH_INVALID;   /* #865 */
         flat[i] = log(flat[i] > 1e-10 ? flat[i] : 1e-10);
+    }
     Value *result;
     if (rows == 1)
         result = flat_to_tensor_1d(flat, cols);

src/eigenscript.h

@@ -671,6 +671,12 @@ struct EigsThread {
     struct Env   *last_obs_slot_env;
     int           last_obs_slot_idx;
     int           unobserved_depth;
+    /* #865: sticky numeric status, IEEE-754's own model (fetestexcept).
+     * Saturation and NaN-collapse keep a program running with a plausible
+     * number and no way to tell it happened; these bits make it detectable.
+     * Set only on the clamp branches, so the arithmetic fast path is
+     * unchanged. Sticky until clear_math_flags. */
+    unsigned      math_flags;
     /* Dynamic caller scope for env-aware builtins (env_get/env_set
      * polymorphic dispatch needs to know "who called me"). */
     struct Env   *builtin_call_env;
@@ -806,6 +812,7 @@ extern __thread EigsThread *eigs_current;
 #define g_last_obs_slot_env (eigs_current->last_obs_slot_env)
 #define g_last_obs_slot_idx (eigs_current->last_obs_slot_idx)
 #define g_unobserved_depth  (eigs_current->unobserved_depth)
+#define g_math_flags        (eigs_current->math_flags)
 #define g_builtin_call_env  (eigs_current->builtin_call_env)
 #define g_vm                  (*eigs_current->vm)
 #define g_loop_stall_count    (eigs_current->loop_stall_count)
@@ -970,10 +977,24 @@ void free_value(Value *v);
  * All numeric operations route through this guard.
  * NaN -> 0; values escaping the finite number line saturate at
  * +/-EIGS_NUM_MAX instead of becoming Infinity. */
+/* #865: sticky status bits for the two clamps below. The finite invariant
+ * keeps a program running past an overflow with a plausible-looking number,
+ * and nothing in the language could tell that apart from a real result:
+ * (1e300 * 1e300) / 1e300 is 1e8, which passes any sanity check a caller
+ * applies, and a NaN collapses to 0, which is indistinguishable from a real
+ * zero. IEEE-754 solved exactly this with sticky exception flags, so these
+ * are those: set on the clamp, readable with `math_flags`, reset with
+ * `clear_math_flags`. Bracket a computation with clear/check the way you
+ * would an FPU. */
+#define EIGS_MATH_OVERFLOW 1u   /* a value saturated at +/-EIGS_NUM_MAX */
+#define EIGS_MATH_INVALID  2u   /* a NaN was collapsed, or a domain clamp fired */
+
 static inline double num_guard(double x) {
-    if (x != x) return 0.0;                        /* NaN */
-    if (x > EIGS_NUM_MAX) return EIGS_NUM_MAX;     /* +Inf or overflow */
-    if (x < -EIGS_NUM_MAX) return -EIGS_NUM_MAX;   /* -Inf or underflow */
+    /* Fast path unchanged: the flag writes live only on the clamp branches,
+     * which a program that does not overflow never takes. */
+    if (x != x) { g_math_flags |= EIGS_MATH_INVALID; return 0.0; }        /* NaN */
+    if (x > EIGS_NUM_MAX)  { g_math_flags |= EIGS_MATH_OVERFLOW; return EIGS_NUM_MAX; }
+    if (x < -EIGS_NUM_MAX) { g_math_flags |= EIGS_MATH_OVERFLOW; return -EIGS_NUM_MAX; }
     return x;
 }
 

tests/test_numeric_guard.eigs

@@ -64,4 +64,117 @@ unobserved:
     unobserved_inf is 1e309
 assert of [unobserved_inf == CAP, "NG19 unobserved reassignment literal caps"]
 
+
+# ---- #865: the clamps are recorded, not silent ----
+# The finite invariant keeps a program running, which is the point, but it kept
+# it running with a plausible number and no way to tell. Reassociation changes
+# a result by 292 orders of magnitude, and a saturated value compares equal to
+# itself under further growth — so nothing in the language could distinguish
+# "this is 1e308" from "this overflowed". These are IEEE-754's sticky flags.
+
+clear_math_flags of null
+f0 is math_flags of null
+assert of [f0.overflow == 0, "NG20 flags start clear"]
+assert of [f0.invalid == 0, "NG20 invalid starts clear"]
+
+# Ordinary arithmetic never sets them.
+ordinary is (2.5 * 4.0) + (1.0 / 8.0)
+f1 is math_flags of null
+assert of [f1.overflow == 0, "NG21 ordinary arithmetic sets no flag"]
+assert of [f1.invalid == 0, "NG21 ordinary arithmetic sets no invalid flag"]
+
+# Saturation sets overflow.
+clear_math_flags of null
+sat is 1e308 * 10
+assert of [sat == CAP, "NG22 saturation still saturates (result unchanged)"]
+assert of [(math_flags of null).overflow == 1, "NG22 saturation is now detectable"]
+
+# The associativity case from the issue: the RESULT is unchanged and still
+# implausible-looking-but-plausible, and it is now flagged.
+clear_math_flags of null
+reassoc is (1e300 * 1e300) / 1e300
+assert of [reassoc == 100000000, "NG23 reassociated result is unchanged"]
+assert of [(math_flags of null).overflow == 1, "NG23 but the contamination is visible"]
+
+# The other association order neither overflows nor flags.
+clear_math_flags of null
+other is 1e300 * (1e300 / 1e300)
+assert of [other == 1e300, "NG24 the other order is exact"]
+assert of [(math_flags of null).overflow == 0, "NG24 and sets no flag"]
+
+# Flags are STICKY across statements until cleared — that is what makes
+# clear/compute/check work the way it does on an FPU.
+clear_math_flags of null
+sticky is 1e308 * 10
+a is 1
+b is 2
+c is a + b
+assert of [(math_flags of null).overflow == 1, "NG25 the flag survives later clean arithmetic"]
+clear_math_flags of null
+assert of [(math_flags of null).overflow == 0, "NG25 and clears on request"]
+
+# A NaN cannot be produced by ARITHMETIC here — there is no way to obtain an
+# Inf to combine (an over-range literal is capped before it is ever an
+# operand, and `x / 0` warns and yields 0 without producing a NaN). So an
+# over-range literal in an expression is capped and quiet:
+clear_math_flags of null
+capped_operand is 0.0 * 1e400
+assert of [capped_operand == 0, "NG26 an over-range literal is capped before use"]
+assert of [(math_flags of null).invalid == 0, "NG26 and produces no NaN to collapse"]
+
+# But a NaN DOES arrive from outside arithmetic — string conversion is the
+# in-language route, and it was the quietest case of all: `num of "nan"` is 0
+# and `num of "inf"` is 1e308, so a data file with either in a column parsed
+# to a plausible number with nothing to check. Both now set a bit. (Values
+# arriving through the embed API take the same num_guard path.)
+clear_math_flags of null
+parsed_nan is num of "nan"
+assert of [parsed_nan == 0, "NG30 num of \"nan\" still collapses to 0"]
+assert of [(math_flags of null).invalid == 1, "NG30 and is no longer confusable with a real 0"]
+
+clear_math_flags of null
+parsed_inf is num of "inf"
+assert of [parsed_inf == CAP, "NG30 num of \"inf\" still saturates"]
+assert of [(math_flags of null).overflow == 1, "NG30 and is no longer confusable with a real 1e308"]
+
+# A plain unparseable string is 0 by the documented parse rule, not a clamp.
+clear_math_flags of null
+parsed_junk is num of "abc"
+assert of [parsed_junk == 0, "NG30 num of a non-numeric string is 0"]
+assert of [(math_flags of null).invalid == 0, "NG30 which is a parse result, not a clamp"]
+
+clear_math_flags of null
+parsed_ok is num of "42"
+assert of [parsed_ok == 42, "NG30 an ordinary parse is exact"]
+assert of [(math_flags of null).invalid + (math_flags of null).overflow == 0, "NG30 and clean"]
+
+# The domain clamps are substitutions, not the two clamps the contract named.
+# Each keeps its value and each now says so.
+clear_math_flags of null
+l0 is log of 0
+assert of [(math_flags of null).invalid == 1, "NG27 log of 0 flags invalid"]
+clear_math_flags of null
+lok is log of 1
+assert of [lok == 0, "NG27 log of 1 is 0"]
+assert of [(math_flags of null).invalid == 0, "NG27 an in-domain log is clean"]
+
+clear_math_flags of null
+s_neg is sqrt of (0 - 1)
+assert of [s_neg == 0, "NG28 sqrt of a negative still returns 0"]
+assert of [(math_flags of null).invalid == 1, "NG28 and is no longer confusable with sqrt of 0"]
+clear_math_flags of null
+s_ok is sqrt of 4
+assert of [s_ok == 2, "NG28 an in-domain sqrt is exact"]
+assert of [(math_flags of null).invalid == 0, "NG28 and clean"]
+
+clear_math_flags of null
+a5 is asin of 5
+assert of [(math_flags of null).invalid == 1, "NG29 asin of an out-of-domain argument flags"]
+clear_math_flags of null
+ah is asin of 0.5
+assert of [(math_flags of null).invalid == 0, "NG29 an in-domain asin is clean"]
+clear_math_flags of null
+ac is acos of (0 - 9)
+assert of [(math_flags of null).invalid == 1, "NG29 acos clamps and flags"]
+
 print of "All numeric-guard tests passed"