From 55aaa6bab63d268e0d70f5401030a783c3fdd272 Mon Sep 17 00:00:00 2001 From: Bill Hlavacek Date: Fri, 21 Aug 2026 17:44:15 -0600 Subject: [PATCH 1/3] docs(adr): a linear observable parameter is profilable only where its own family's residual is linear in it, so the coupled scale/offset projection is not adopted (#572, ADR-0123) #572 proposed profiling an observable's additive offset -- and the coupled (scale, offset) pair -- out of the search by variable projection, and asked for an evaluation before anything was built, with a stated kill criterion. The evaluation was run. It kills the proposal, and the reason is one neither the issue's case-for nor its case-against anticipated. Variable projection is a linear-least-squares identity: it needs the family's residual to be `d - y` with `y` affine in the profiled coefficients. A PyBNF noise family declares the space its residual lives in, and on `lognormal` / `lnnormal` that residual is `log d - log(a*s + b)`, which is affine in `log a` only when `b == 0`. Every coupled pair in the corpus -- Borghans, Elowitz, Schwen -- is lognormal. Of the eight offset-role parameters in the 23 slugs, exactly one has a closed form, and it is one the issue does not mention. The two slugs where `Phi = [s, 1]` is the right object, Weber and Brannmark, carry no offset at all. The closed form is not the obstacle, so the landscape was measured with the profile taken numerically -- the exact conditional optimum whatever the family is -- through PyBNF's own evaluate_multiple, with sigma additionally profiled by ADR-0108. That is #572's evaluation item 2 verbatim. On Borghans, over 76 box draws that integrate, the searched objective spans [-160.9, +339.7] with an interquartile range of 129.5; the profiled objective spans [-167.15, -165.98] with an interquartile range of 0.081. Forty-six of 76 draws land exactly on the no-dynamics score -165.982113, 73 of 76 within 1.0 of it, and 39 of 76 profile the scale below 1e-6 -- discarding the dynamics outright, because on a log residual a wrong-shaped trajectory fits worse than no trajectory. Along a section from the reference optimum out to a box draw the profiled objective is bit-identical at six consecutive points. The reference optimum and the nominal point are untouched, exactly as the envelope theorem says. Profiling does not sharpen this landscape; it deletes it. Elowitz reproduces it on the same mechanism. Two further findings the issue did not have. Where the construction does apply, `noise_profiling` has already collected most of it: on Weber with ADR-0108 switched on, profiling all five observable scales changes the spread of the landscape by nothing (IQR 1667 -> 1715, rank correlation 0.87), and the thirteen-orders-of-magnitude improvement visible with a searched sigma belongs to the sigma profile rather than to the scale profile. And Fiedler is not the only double-bound slug: Raia_CancerResearch2011 binds five `scaling_*` parameters as both the observable scale and a factor inside a `prediction_formula` sigma, so a placeholder-name rule would catch Fiedler and miss Raia. Thirteen of the corpus's 47 affine observable parameters are double-bound. Item 3, head-to-head fits at matched budget, was deliberately not run, and the ADR says so and why: for the offset half a fit comparison on a landscape that is constant over half the box measures the sampler's tie-breaking rather than the objective, and for the pure-scale remainder it cannot be run without first building the feature that #572 exists to gate. It is named instead as the acceptance test for that follow-up, on Smith and Brannmark -- not on Weber, where there is nothing left for it to collect. No behaviour change, no new config key. The ADR exists so the measurement is not re-litigated and so the small, genuinely sound remainder -- a pure multiplicative scale on a linear-scale family, tied across the series that share it, 19 parameters in 4 slugs -- is scoped rather than lost. The two tools that produced these numbers, and their gotchas, live with the corpus in BNGL-Models: `Grein-2026-benchmark-subset-I/tools/linear_scope.py` and `.../linear_profile.py`. --- ...-scale-offset-projection-is-not-adopted.md | 281 ++++++++++++++++++ 1 file changed, 281 insertions(+) create mode 100644 docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md diff --git a/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md b/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md new file mode 100644 index 00000000..480862cd --- /dev/null +++ b/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md @@ -0,0 +1,281 @@ +# An observable's linear parameter is profilable only where its own family's residual is linear in it, so the coupled `(scale, offset)` projection is not adopted (issue #572) + +**Status: Evaluated 2026-08-21; the proposed method is NOT adopted, and #572's own kill +criterion is met.** ADR-0066/0099 profile a declared column's optimal multiplicative **scale** +out analytically, and ADR-0108 (#562) profiles out an estimated **noise scale**. #572 proposed +the third member of that family — an additive **offset**, and the coupled `(scale, offset)` +pair — by variable projection, and asked for an evaluation *before* anything was built, +with a stated kill criterion. The evaluation was run. It kills the proposal, for a reason +neither the issue's case-for nor its case-against anticipated: **variable projection is a +linear-least-squares identity, and every coupled pair and every additive offset in the +corpus is scored on a log residual scale, where that identity does not hold.** Where an +exact profile is taken anyway — numerically, so the missing closed form is not the +obstacle — it collapses essentially the entire search box onto the no-dynamics score. + +No code changes. This ADR exists so the measurement is not re-litigated, and so the small, +genuinely sound remainder is scoped rather than lost. + +## What was asked + +> Fewer dimensions is not on its own a reason to ship a change to what the objective means. +> — #572 + +#572 asked four things: (1) compare the reduced and searched landscapes offline; (2) on +`Borghans`, does the `-165.98` no-dynamics attractor survive when scale, offset and σ are all +profiled; (3) head-to-head fits at matched budget; (4) confirm the `Fiedler` double-binding +refusal and that no other slug hides the same pattern. + +## How it was measured + +Two tools, both in the corpus (`Grein-2026-benchmark-subset-I/tools/`): + +* **`linear_scope.py`** classifies every declared free parameter that reaches an observable: + is the formula affine in it, is it a pure multiplicative factor, what residual space does + its observable's noise family score in (`family.additive_on.ln_base`), how many series is + it tied across, and is it *also* bound as a noise parameter. Static; no simulation. +* **`linear_profile.py`** simulates once per point and then minimizes **PyBNF's own + `evaluate_multiple`** over the linear parameters with θ held fixed. The profile is taken + *numerically*, on purpose: it is the exact conditional optimum whatever the family is, so + the landscape question is answered independently of whether a closed form exists. (ADR-0108 + pinned its own σ closed form against a numeric minimization of the reported objective for + the same reason: a plausible-but-wrong formula survives re-derived algebra and does not + survive this.) + +Neither tool runs a fit. A `Borghans` point costs one simulation plus a few hundred +re-scorings of it. + +Three self-checks carry the result, and all three passed: + +* The independently computed **flat-line reference** — every non-intercept coefficient pinned + to exactly `0`, the intercept profiled — reproduces `-165.982113` at **every** point, spread + `0`, which is the no-dynamics attractor already on record for this slug. +* The **reference optimum** re-scores at `-248.0524` against the recorded `-248.0692`, and the + PEtab nominal point at `-198.207` against the recorded `-198.1017`; both differences are the + σ-profiling and the bngsim build, not the harness. +* The profiled score is **never worse than the searched score** at any of the 250-odd points + measured, which it cannot be, since the searched value is in the set being minimized over. + +## Finding 1 — the scope in #572 is wrong in both directions + +Across all 23 subset-I slugs there are **47** affine observable-layer parameters in **9** +slugs, not 22 in 6. Split by whether a closed-form profile exists at all: + +| slug | k | affine obs params | linear-lsq | log-geomean | none | why not | +|---|---:|---:|---:|---:|---:|---| +| `Schwen_PONE2015` | 30 | 10 | 0 | 1 | 9 | log family, not multiplicative | +| `Smith_BMCSystBiol2013` | 25 | 9 | 9 | 0 | 0 | — | +| `Fiedler_BMCSystBiol2016` | 22 | 8 | 0 | 0 | 8 | double-bound as a noise parameter | +| `Raia_CancerResearch2011` | 39 | 5 | 0 | 0 | 5 | double-bound as a noise parameter | +| `Weber_BMC2015` | 36 | 5 | 5 | 0 | 0 | — | +| `Brannmark_JBC2010` | 22 | 4 | 4 | 0 | 0 | — | +| `Borghans_BiophysChem1997` | 23 | 2 | 0 | 0 | 2 | log family, not multiplicative | +| `Elowitz_Nature2000` | 21 | 2 | 0 | 0 | 2 | log family, not multiplicative | +| `Laske_PLOSComputBiol2019` | 13 | 2 | 1 | 0 | 1 | log family, not multiplicative | + +Two slugs the issue does not list at all: + +* **`Smith_BMCSystBiol2013`** — nine tied per-observable scales, 36 % of `k = 25`, every one of + them closed-form profilable. It is the **largest** opportunity in the corpus and it is + missing from the issue's table. +* **`Raia_CancerResearch2011`** — see finding 5. + +And the issue undercounts `Brannmark` (4, not 2: `k_IRP_1Step` and `k_IRSiP_DosR` are scales +too) and miscounts `Schwen`'s share as 42 % of the search when 9 of its 10 are not profilable +by any closed form. + +## Finding 2 — variable projection does not apply where #572 wants it + +`Phi = [s, 1]`, `c* = (Phi^T W Phi)^-1 Phi^T W d` is an identity about a residual `d - Phi c`. +A PyBNF noise family declares the space its residual lives in. For `lognormal` +(`ln_base = ln 10`) and `lnnormal` (`ln_base = 1`) the residual is `log d - log(a*s + b)`, +which is affine in `log a` when `b == 0` — that is exactly why ADR-0066's geometric-mean ratio +is a closed form for a *pure* scale — and is affine in `b` for no `b` at all. + +Sorted by role, the corpus splits cleanly and unhelpfully: + +* **All three coupled `(scale, offset)` pairs** — `Borghans` (`Z_state*scale + offset`), + `Elowitz` (`GFP*scale + background`), `Schwen` (`scale*(IR1 + IR1in + offset)`) — are + `lognormal`. Zero have a closed form. +* **Of the 8 offset-role parameters in the corpus, exactly one has a closed form**: + `Laske`'s `Int_nuc_off`, a Gaussian offset shared by nine observables — which #572 does not + mention. `Laske`'s `vRNA_offset`, the one offset the issue *does* name, sits on an + `lnnormal` observable and has none. +* The two slugs where `Phi = [s, 1]` is the right object, `Weber` and `Brannmark`, carry **no + offset at all**. They are pure scales — "one more scalar", the case the issue explicitly + sets aside as not the interesting one. + +So the interesting half of the proposal has no instances, and the instances have no +interesting half. `Schwen`'s `scale` is the single log-family parameter with a closed form, +and it is the geometric-mean ratio ADR-0066 already implements — though note it is a +*different object*: ADR-0066 profiles an **undeclared, per-series** scale, while `Schwen`'s is +one **declared coefficient tied across 7 series**, so it is a stacked solve, not ADR-0066's. + +## Finding 3 — the exact profile collapses the box onto the no-dynamics score + +This is the kill. The closed form is not the obstacle: the profile below is the exact +conditional optimum, taken numerically, with σ additionally profiled by ADR-0108 — #572's +item 2 verbatim, "scale, offset and σ all profiled". + +`Borghans_BiophysChem1997`, 76 box draws that integrate, `noise_profiling = 1`: + +| | searched | profiled | +|---|---:|---:| +| range over the box | `[-160.88, +339.69]` | **`[-167.15, -165.98]`** | +| interquartile range | **129.5** | **0.081** | +| median − flat line | +374.5 | −2.8e−14 | +| draws scoring better than the flat line | 0 / 76 | 40 / 76 | +| draws scoring **exactly** the flat line | 0 / 76 | **46 / 76** | +| draws within 1.0 of the flat line | 0 / 76 | **73 / 76** | +| draws whose profiled `scale < 1e-6` | — | **39 / 76** | +| Spearman rank correlation, searched ~ profiled | — | **0.035** | + +The reference optimum is untouched (`-248.0524 -> -248.0525`, gain `1.1e-4`) and so is the +nominal point (`-198.207 -> -198.262`, gain `0.055`) — the envelope theorem doing exactly what +#572 says it does. What moves is everything else: **the entire box collapses into a 1.16-unit +band sitting on the no-dynamics score**, while the reference optimum stays 82 units below it. + +The mechanism is worse than "the flat line becomes available". At 39 of 76 draws the profile +drives the scale below `1e-6` — it *discards the dynamics entirely*, because on a log residual +a wrong-shaped trajectory makes the fit worse than no trajectory. The no-dynamics answer is +not a floor the reduced surface can reach; over half the box it **is** the reduced surface. + +`Elowitz_Nature2000` reproduces it on the same mechanism, less extremely: 57 draws, IQR +`70.21 -> 7.77`, 27/57 exactly at the flat line `-53.656041`, 19/57 discarding the dynamics, +rank correlation **−0.43**. + +### The section is the picture + +Thirteen points interpolated in sampling space from the reference optimum (`t00`) out to a box +draw (`t12`), nine of which integrate: + +| | t00 | t01 | t02 | t03 | t04 | t05 | t06 | t07 | t08 | +|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| +| searched | −248.05 | −152.29 | −66.49 | +112.97 | +150.42 | +166.09 | +178.64 | +189.77 | +199.87 | +| profiled | −248.05 | −166.58 | −167.01 | **−165.9821** | **−165.9821** | **−165.9821** | **−165.9821** | **−165.9821** | **−165.9821** | + +The searched objective rises monotonically away from the optimum — a continuous descent +direction all the way in from the box. The profiled objective is **bit-identical at six +consecutive section points**. Profiling does not sharpen this landscape; it deletes it. + +### Answering (1) and (2) directly + +1. **Worse, decisively.** The reduced surface does not separate the reference optimum from the + flat-line floor better — it removes the separation between the flat-line floor and + *everything else*, which is the only structure a global sampler had. +2. **The `-165.98` attractor does not merely remain a converged endpoint. It becomes the value + of the search space.** Half the box scores it to the last bit, and 96 % of the box scores + within 1.0 of it. + +The conclusion does not depend on the parametrization. Repeating `Borghans` with the profile +taken on the real line rather than in each parameter's sampling space (`--linear-space`, +i.e. what an unconstrained variable projection would do) gives the same collapse, and +additionally trips ADR-0108's degenerate-scale refusal, because an unconstrained offset drives +predictions out of the log family's domain. An unconstrained projection is not merely +unavailable on these slugs; it is infeasible. + +## Finding 4 — where it does apply, `noise_profiling` has already collected most of it + +`Weber_BMC2015`, five pure Gaussian scales, 20 box draws, measured twice: + +| | searched IQR | profiled IQR | rank corr | +|---|---:|---:|---:| +| σ **searched** (as the conf ships) | 1.96e18 | 3.38e5 | 0.43 | +| σ **profiled** (`noise_profiling = 1`) | 1.67e3 | **1.72e3** | **0.87** | + +With ADR-0108 switched on, profiling all five observable scales changes the spread of the +landscape by **nothing** — it is marginally *larger* — and leaves the ranking almost intact. +The thirteen-orders-of-magnitude improvement in the first row is real, and it is the σ +profile's, not the scale profile's: a wrong scale inflates residuals, and a profiled σ absorbs +that inflation. #572's case-for is "better conditioning, every draw linear-optimal". On the +one corpus slug where both switches are available, the shipped one already delivers it. + +`Brannmark_JBC2010` is the counter-case and it is instructive: `noise_profiling = 1` is +**refused** there (`IRS1_P` estimates σ from a `PerMeasurementFormulaSigma`), so the scale +profile has the field to itself and does compress the landscape (IQR `1.01e17 -> 3.94e12`, +rank corr 0.62). Which also disposes of #572's proposed ordering rule: "linear-first, then σ +in closed form" describes a situation that does not arise on the slug it would matter for. + +`Laske`'s `Int_nuc_off` — the corpus's one genuine closed-form additive offset — behaves +exactly as a good nuisance should: it profiles to ≈ `57.13` at essentially every draw +regardless of θ, compresses the IQR from `1.01e9` to `5.01e7`, and moves the nominal point by +`0.02`. It is a real, small, well-behaved win. It is one parameter of 13 in one slug. + +## Finding 5 — the double-binding refusal is needed, and `Fiedler` is not the only slug + +#572 asks for `Fiedler_BMCSystBiol2016` to be refused by name and asks whether anything else +hides the pattern. Both halves check out, and the answer to the second is **yes**: + +* **`Fiedler`** — confirmed exactly as described. All eight `s_pErk_*` / `s_pMek_*` tokens are + bound to `observableParameter1_*` *and* `noiseParameter1_*` in the same per-measurement + tables. +* **`Raia_CancerResearch2011`** — the same defect by a different route, and **not mentioned in + the issue**. Five `scaling_*` parameters are the observable scale *and* appear inside a + `prediction_formula` σ (`CD274mRNA*scaling_CD274mRNA*sd_CD274mRNA_rel + sd_CD274mRNA_abs`), + so moving the scale moves that observable's σ too. Thirteen of the corpus's 47 affine + observable parameters are double-bound this way — more than a quarter. + +A "detect the `noiseParameter` placeholder" rule catches `Fiedler` and misses `Raia`. Any +implementation must test the resolved parameter *names* a noise source reads, formula sources +included. + +## The decision + +**The coupled `(scale, offset)` variable projection of #572 is not implemented**, and #572 is +closed against its own kill criterion. Restating that criterion: + +> If (1) and (2) show the reduced landscape is no better — or worse — for global search, and +> (3) shows no reliable improvement, then close this as a tidiness item and do **not** +> implement. + +(1) and (2) are worse by the largest margin the measurement could have produced: the reduced +objective is constant to 0.08 units over the interquartile range of the box, and bit-identical +across six consecutive points of a section. Fewer dimensions was never the argument, and it is +all that is left. + +### What was deliberately not measured, and why + +**Item 3, head-to-head fits at matched budget, was not run.** For the offset half that is a +consequence of the result rather than a gap in it: a fit comparison on a landscape that is +constant over half the box measures the sampler's tie-breaking, not the objective. Reporting +"profiled did not converge better" from such a run would attribute to the search what the +measurement already attributes to the surface. + +For the pure-scale remainder, item 3 *is* the right test — and it cannot be run without first +building the feature, which is the thing #572 exists to gate. It is therefore named below as +the acceptance test for the follow-up rather than performed here. + +### What survives, and what it would have to be + +A **pure multiplicative scale, on a linear-scale family, tied across the series that share +it** is sound: the closed form is real, the corpus has 19 such parameters in 4 slugs, and +`Laske`'s `Int_nuc_off` shows the additive case behaving well where the family permits it. +That is a materially different, smaller feature than #572 proposes, and it inherits three +constraints this evaluation established: + +* It must **refuse a parameter that any noise source also reads**, by resolved name, formula + sources included — 13 of 47 corpus parameters, across `Fiedler` **and** `Raia`. +* It must refuse any parameter whose observable scores on a **log** family unless the formula + is homogeneous in it, and it must apply the geometric-mean form rather than least squares + when it is. +* Its acceptance test is **item 3 on `Smith` and `Brannmark`** — the two slugs where it is not + redundant with `noise_profiling` — not on `Weber`, where finding 4 shows there is nothing + left for it to collect. + +Whether that is worth building is a separate question from this one, and this ADR does not +answer it. It is not opened here. + +## Consequences + +* No behaviour change. No new config key. `noise_profiling` (ADR-0108) and the ADR-0066 + `normalization = ... , scale` chain are unaffected. +* `docs/adr/0108` is unchanged; its "pyPESTO/AMICI profile scale, offset and σ analytically by + default" prior-art line stands, with the qualification recorded here that those defaults are + a linear-Gaussian construction and PyBNF's log families are outside it. +* Two reusable tools land in the corpus, with their gotchas written down: + `Grein-2026-benchmark-subset-I/tools/linear_scope.py` and `.../linear_profile.py`. + +## Prior art + +Variable projection (Golub & Pereyra 1973); hierarchical optimization for ODE models (Loos et +al. 2018); pyPESTO/AMICI. All three are stated over an additive-Gaussian residual. That +assumption is load-bearing and is the one this corpus violates. From 8231754b8b5b22662902aa64054df1ed3a9e6f80 Mon Sep 17 00:00:00 2001 From: Bill Hlavacek Date: Fri, 21 Aug 2026 17:46:11 -0600 Subject: [PATCH 2/3] docs(adr): quote the box run's own optimum figures and the real point count (#572, ADR-0123) The self-check paragraph cites the section run's re-score of the reference optimum (-248.0524) and finding 3 cites the box run's gain (1.1e-4); the two came from different simulations, and finding 3 had the section's value against the box's gain. Use the box run's own pair, -248.054331 -> -248.054446, and give the exact number of scored points behind the never-worse check (279) rather than an approximation. --- ...t-so-the-coupled-scale-offset-projection-is-not-adopted.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md b/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md index 480862cd..962c5c00 100644 --- a/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md +++ b/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md @@ -52,7 +52,7 @@ Three self-checks carry the result, and all three passed: * The **reference optimum** re-scores at `-248.0524` against the recorded `-248.0692`, and the PEtab nominal point at `-198.207` against the recorded `-198.1017`; both differences are the σ-profiling and the bngsim build, not the harness. -* The profiled score is **never worse than the searched score** at any of the 250-odd points +* The profiled score is **never worse than the searched score** at any of the **279** points measured, which it cannot be, since the searched value is in the set being minimized over. ## Finding 1 — the scope in #572 is wrong in both directions @@ -129,7 +129,7 @@ item 2 verbatim, "scale, offset and σ all profiled". | draws whose profiled `scale < 1e-6` | — | **39 / 76** | | Spearman rank correlation, searched ~ profiled | — | **0.035** | -The reference optimum is untouched (`-248.0524 -> -248.0525`, gain `1.1e-4`) and so is the +The reference optimum is untouched (`-248.054331 -> -248.054446`, gain `1.1e-4`) and so is the nominal point (`-198.207 -> -198.262`, gain `0.055`) — the envelope theorem doing exactly what #572 says it does. What moves is everything else: **the entire box collapses into a 1.16-unit band sitting on the no-dynamics score**, while the reference optimum stays 82 units below it. From 99276c8a4e994e98ad80d8b7cc805264b56778e5 Mon Sep 17 00:00:00 2001 From: Bill Hlavacek Date: Fri, 21 Aug 2026 18:07:29 -0600 Subject: [PATCH 3/3] docs(adr): #572 is narrowed, not killed -- the log-family half is refused on the math, the linear-Gaussian half is untested (#572, ADR-0123) Rewrites ADR-0123. The previous version concluded "not adopted" on two arguments that do not hold, and the numbers underneath them were fine; it was the inference that was wrong. Both corrections are recorded in the ADR rather than quietly applied. The first bad argument was "the corpus has no instances". PyBNF is general-purpose, and a 23-slug corpus of PEtab imports is evidence about Benchmark-Models-PEtab, not about the merit of a facility. The sample skews lognormal because PEtab problems do; a conf written by hand uses sos or chi_sq, which is a LINEAR scale -- the case where the construction works. Smith, the one `objective = sos` slug, carries nine profilable scales. If anything the sample under-counts the feature's domain relative to PyBNF's native idiom. The second was that the landscape measurement kills it. It does not, because of where it was taken. The Borghans/Elowitz collapse happens BECAUSE the residual is on a log scale -- a wrong-shaped trajectory has more log-spread than a flat line, so the inner solve picks a = 0 -- and log families are exactly what the mathematics already excludes. The evaluation landscape-tested the method only in the regime where it cannot be used. The coupled (scale, offset) pair on a linear-scale family, which is the case #572 is actually about, was never tested, because the corpus has no instance of it. That collapse is now demoted to a footnote about what it does show: do not offer a numeric profile as a fallback where the closed form is absent. What survives, and now leads: * Finding 1, general and mathematical: the residual's space decides what is profilable. A log family has a closed form for a homogeneous parameter (the ADR-0066 geometric-mean ratio) and for nothing else, so the coupled form must be refused there rather than approximated. * Finding 2, a design constraint on any implementation: a parameter that any noise source also reads must be refused by RESOLVED NAME. The obvious rule -- detect the noiseParameter placeholder -- catches Fiedler and misses Raia, where the collision is inside a prediction_formula sigma. Thirteen of the corpus's 47 affine observable parameters are double-bound, so this is a quarter of the population, not a corner. * Finding 3, promoted from a buried aside: the only measurement taken inside the construction's real domain points FOR the feature. Laske's Int_nuc_off, a Gaussian offset in nine observables, profiles to about 57.13 at essentially every box draw regardless of theta, compresses the spread ~20x, and moves the nominal point by 0.02 -- a textbook redundant nuisance parameter. * Finding 4, the question that is actually open: how much of this is a re-delivery of ADR-0108. On Weber with noise_profiling on, profiling all five scales changes the landscape spread by nothing (IQR 1667 -> 1715, rank corr 0.87); Brannmark disagrees, but only because noise_profiling is refused there. Two slugs at n = 20 and n = 25 disagreeing for an explicable reason is not an answer. The status becomes "narrowed, not decided", and the ADR names the two measurements that would decide it: a synthetic linear-Gaussian fixture carrying a real coupled pair -- which the corpus cannot supply and which is cheap to build -- and #572's item 3 on Smith and Brannmark, the two slugs where the scale profile is not redundant with a profiled sigma. It also records the scope the feature would have if those come back positive. Also corrects the tied-parameter count to 9 of 19, across 2 to 11 experiments. No behaviour change. #572 stays open. --- ...on-a-measurement-the-corpus-cannot-make.md | 254 ++++++++++++++++ ...-scale-offset-projection-is-not-adopted.md | 281 ------------------ 2 files changed, 254 insertions(+), 281 deletions(-) create mode 100644 docs/adr/0123-a-linear-observable-parameter-is-profilable-only-in-the-space-its-own-noise-family-scores-so-the-log-family-half-is-refused-on-the-math-and-the-linear-gaussian-half-stays-open-on-a-measurement-the-corpus-cannot-make.md delete mode 100644 docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md diff --git a/docs/adr/0123-a-linear-observable-parameter-is-profilable-only-in-the-space-its-own-noise-family-scores-so-the-log-family-half-is-refused-on-the-math-and-the-linear-gaussian-half-stays-open-on-a-measurement-the-corpus-cannot-make.md b/docs/adr/0123-a-linear-observable-parameter-is-profilable-only-in-the-space-its-own-noise-family-scores-so-the-log-family-half-is-refused-on-the-math-and-the-linear-gaussian-half-stays-open-on-a-measurement-the-corpus-cannot-make.md new file mode 100644 index 00000000..8a795b6b --- /dev/null +++ b/docs/adr/0123-a-linear-observable-parameter-is-profilable-only-in-the-space-its-own-noise-family-scores-so-the-log-family-half-is-refused-on-the-math-and-the-linear-gaussian-half-stays-open-on-a-measurement-the-corpus-cannot-make.md @@ -0,0 +1,254 @@ +# A linear observable parameter is profilable only in the space its own noise family scores, so the log-family half is refused on the math and the linear-Gaussian half stays open on a measurement the corpus cannot make (issue #572) + +**Status: Narrowed, not decided (2026-08-21).** ADR-0066/0099 profile a declared column's +optimal multiplicative **scale** out analytically, and ADR-0108 (#562) profiles out an +estimated **noise scale**. #572 proposed the third member — an additive **offset**, and the +coupled `(scale, offset)` pair — by variable projection, and asked for an evaluation before +anything was built. + +The evaluation settles two things and leaves the main question open: + +* **Settled, on the mathematics.** Variable projection needs the family's residual to be + `d - y` with `y` affine in the profiled coefficients. PyBNF's **log** families + (`lognormal`, `lnnormal`) score `log d - log(a*s + b)`, which is affine in `log a` only when + `b == 0` and in `b` never. The coupled form has no closed form there and must be **refused**, + not approximated. The pure-scale form survives as ADR-0066's geometric-mean ratio. +* **Settled, on the interface.** A parameter that any noise source also reads is not a free + linear coefficient and must be refused **by resolved name**, formula sources included — + because the obvious rule misses one of the two real instances. +* **Open.** Whether profiling is worth having on a **linear-scale** family — the case #572 is + actually about — is *not* answered here, and this ADR is explicit that the evaluation did not + answer it. The two measurements that would are named at the end. + +No code changes. + +## This ADR replaces a wrong conclusion + +The first version of this ADR concluded "not adopted", on two arguments that do not hold: + +1. *"The corpus has no instances."* PyBNF is general-purpose. Our 23-slug corpus is PEtab + imports, which skew `lognormal`; a conf written by hand uses `sos` or `chi_sq`, which is a + **linear** scale — the case where the construction works. `Smith_BMCSystBiol2013`, the one + `objective = sos` slug, carries nine profilable scales. If anything the sample under-counts + the feature's domain relative to PyBNF's native idiom. Instance counts in this corpus are + evidence about Benchmark-Models-PEtab, not about the merit of a general facility. +2. *"The landscape measurement kills it."* It does not, because of where it was taken. The + collapse documented below happens **because** the residual is on a log scale — and log + families are exactly what argument (1) above already excludes. The evaluation landscape-tested + the method only in the regime where it cannot be used. See "What was not measured". + +Both corrections are recorded rather than quietly applied, because the numbers underneath them +are sound and reusable; it was the inference from them that was wrong. + +## How it was measured + +Two tools, both in the corpus (`Grein-2026-benchmark-subset-I/tools/`): `linear_scope.py` +classifies every declared free parameter reaching an observable (affine? pure multiplicative +factor? what residual space? tied across how many series? also read by a noise source?), and +`linear_profile.py` simulates once per point and then minimizes **PyBNF's own +`evaluate_multiple`** over the linear parameters with θ held fixed. + +The profile is taken *numerically* on purpose: it is the exact conditional optimum whatever the +family is, so the landscape question separates from the closed-form question. ADR-0108 pinned +its own σ closed form against a numeric minimization of the reported objective for the same +reason. **Neither tool runs a fit** — a `Borghans` point is one simulation plus a few hundred +re-scorings of it. + +Three self-checks, all passed: the independently computed flat-line reference reproduces +`-165.982113` at every point with spread `0` (the attractor already on record for that slug); +the reference optimum and PEtab nominal point re-score to their recorded values within the +σ-profiling and build difference; and the profiled score never exceeded the searched score at +any of the **279** points measured, which it cannot, since the searched value is in the set +being minimized over. + +## Finding 1 — the residual's space decides what is profilable, and that is general + +`Phi = [s, 1]`, `c* = (Phi^T W Phi)^-1 Phi^T W d` is an identity about a residual `d - Phi c`. +A PyBNF noise family declares the space its residual lives in +(`family.additive_on.ln_base`: `0` linear, `ln 10` for `lognormal`, `1` for `lnnormal`). So: + +| family | pure scale `a*s` | offset `s + b` | coupled `a*s + b` | +|---|---|---|---| +| linear (`gaussian`, `laplace`, `sos`) | least squares | least squares | least squares | +| log (`lognormal`, `lnnormal`) | **geometric-mean ratio** (ADR-0066) | none | none | + +This is the load-bearing result and it has nothing to do with which models we happen to own. +A correct implementation refuses a log-family observable unless the formula is **homogeneous** +in the parameter, and applies the geometric-mean form rather than least squares when it is. +Note "homogeneous" is a property of the whole formula: `Borghans`'s `scale` in +`Z_state*scale + offset` is *not* a pure scale, because setting it to zero leaves `offset`. + +## Finding 2 — the double-binding refusal is needed, and the obvious rule is wrong + +#572 asks for `Fiedler_BMCSystBiol2016` to be refused by name and asks whether anything else +hides the pattern. Both check out, and the answer to the second is **yes**: + +* **`Fiedler`** — confirmed exactly as described: all eight `s_pErk_*` / `s_pMek_*` tokens are + bound to `observableParameter1_*` **and** `noiseParameter1_*` in the same per-measurement + tables. +* **`Raia_CancerResearch2011`** — the same defect by a different route, and absent from the + issue. Five `scaling_*` parameters are the observable scale *and* appear inside a + `prediction_formula` σ + (`CD274mRNA*scaling_CD274mRNA*sd_CD274mRNA_rel + sd_CD274mRNA_abs`), so moving the scale moves + that observable's σ. + +A rule that detects the `noiseParameter` placeholder catches `Fiedler` and **misses `Raia`**. +The test has to be on the resolved parameter *names* every noise source reads, formula sources +included. Thirteen of the corpus's 47 affine observable parameters are double-bound this way — +so this is not a corner, it is a quarter of the population, and it is a design constraint on +any implementation regardless of what is decided below. + +## Finding 3 — the one in-domain measurement is positive + +`Laske`'s `Int_nuc_off` is the corpus's only additive offset on a **linear-scale** family: a +Gaussian offset tied across nine observables. It is exactly #572's case, minus the coupling +with a scale, and it behaves the way a redundant nuisance parameter should: + +* it profiles to ≈ `57.13` at essentially **every** box draw, regardless of θ — i.e. it carries + no information about the dynamics, which is the whole argument for removing it from a search; +* it compresses the spread over the box by ~20× (IQR `1.01e9 -> 5.01e7`, 20 draws); +* it moves the PEtab nominal point by `0.02`. + +One parameter in one slug is thin evidence, and σ is searched rather than profiled there +(`Laske` refuses `noise_profiling`, so the absolute magnitudes are inflated). But it is the only +measurement taken inside the construction's actual domain, and it points **for** the feature, +not against it. + +## Finding 4 — the open question is redundancy with ADR-0108, not soundness + +`Weber_BMC2015`, five pure Gaussian scales, 20 box draws, measured twice: + +| | searched IQR | profiled IQR | rank corr | +|---|---:|---:|---:| +| σ **searched** (as the conf ships) | 1.96e18 | 3.38e5 | 0.43 | +| σ **profiled** (`noise_profiling = 1`) | 1.67e3 | **1.72e3** | **0.87** | + +With ADR-0108 switched on, profiling all five observable scales changes the spread of the +landscape by **nothing** — marginally worse — and leaves the ranking almost intact. The +thirteen-orders-of-magnitude improvement in the first row is real and it is the **σ profile's**: +a wrong scale inflates residuals, and a profiled σ absorbs the inflation. + +`Brannmark_JBC2010` points the other way, and the reason matters: `noise_profiling = 1` is +**refused** there (`IRS1_P` takes σ from a `PerMeasurementFormulaSigma`), so the scale profile +has the field to itself and does compress the landscape (IQR `1.01e17 -> 3.94e12`, rank corr +0.62). + +So the real question this evaluation raises is not "is the construction sound" — on a linear +family it plainly is — but **"how much of it is a re-delivery of `noise_profiling`, and does +that depend on whether `noise_profiling` is available for the slug?"** Two slugs, n = 20 and +n = 25, disagreeing for an explicable reason, is not an answer. + +This also disposes of #572's proposed ordering rule. "Linear-first, then σ in closed form" +describes a situation that does not arise on `Brannmark`, the slug where the linear profile +matters most, because σ is not profilable there at all. A linear-profiling switch has to work +stand-alone. + +## Finding 5 — the scope, read for what it is + +Across all 23 subset-I slugs: **47** affine observable-layer parameters in **9** slugs, against +the 22 in 6 the issue tabulates. + +| slug | k | affine obs params | linear-lsq | log-geomean | none | why not | +|---|---:|---:|---:|---:|---:|---| +| `Schwen_PONE2015` | 30 | 10 | 0 | 1 | 9 | log family, not homogeneous | +| `Smith_BMCSystBiol2013` | 25 | 9 | 9 | 0 | 0 | — | +| `Fiedler_BMCSystBiol2016` | 22 | 8 | 0 | 0 | 8 | double-bound as a noise parameter | +| `Raia_CancerResearch2011` | 39 | 5 | 0 | 0 | 5 | double-bound as a noise parameter | +| `Weber_BMC2015` | 36 | 5 | 5 | 0 | 0 | — | +| `Brannmark_JBC2010` | 22 | 4 | 4 | 0 | 0 | — | +| `Borghans_BiophysChem1997` | 23 | 2 | 0 | 0 | 2 | log family, not homogeneous | +| `Elowitz_Nature2000` | 21 | 2 | 0 | 0 | 2 | log family, not homogeneous | +| `Laske_PLOSComputBiol2019` | 13 | 2 | 1 | 0 | 1 | log family, not homogeneous | + +Two slugs the issue does not list: **`Smith`** (nine tied scales, 36 % of `k = 25`, all +closed-form — the largest opportunity in the corpus) and **`Raia`** (finding 2). It also +undercounts `Brannmark` (4, not 2 — `k_IRP_1Step` and `k_IRSiP_DosR` are scales too) and cites +`Schwen`'s 42 % without noting that 9 of its 10 have no closed form. + +**Read this table as a fact about Benchmark-Models-PEtab, not about the feature.** What it does +support is a *priority* claim about PEtab-imported work, and a warning that #572's motivating +examples — `Borghans`, `Elowitz`, `Schwen` — are all in the refused half. + +## What was not measured, and why that matters + +**The coupled `(scale, offset)` pair on a linear-scale family was never tested, because the +corpus has no instance of it.** That is the case #572 calls "not just one more scalar", and +this evaluation has nothing to say about it. Absence from a 23-problem sample is not evidence +against it. + +**Item 3 (head-to-head fits at matched budget) was not run.** It cannot be, without first +building the feature that #572 exists to gate. + +**`Smith`'s box landscape was not measured** — 71 s per simulation, and a 9-parameter inner +solve on top. Only its nominal point was scored (`888141 -> 114222`, a large gain, but its +PEtab nominal `sc_*` are far from optimal so this says little). + +### The log-family collapse, demoted to what it actually shows + +For completeness, since the numbers are sound and were the previous version's headline: taking +the profile **numerically** on a log family — the obvious "no closed form? do it numerically +then" fallback — destroys the landscape. + +`Borghans_BiophysChem1997`, 76 box draws that integrate, σ profiled by ADR-0108, profiling +`scale` and `offset`: the searched objective spans `[-160.88, +339.69]` with IQR `129.5`; the +profiled objective spans `[-167.15, -165.98]` with IQR **`0.081`**. Forty-six of 76 draws land +**exactly** on the no-dynamics score `-165.982113`, 73 of 76 within `1.0` of it, and 39 of 76 +profile the scale below `1e-6` — the inner solve discards the dynamics, because on a log +residual a wrong-shaped trajectory fits worse than no trajectory at all. Along a section from +the reference optimum out to a box draw, the profiled objective is bit-identical at six +consecutive points where the searched objective rises monotonically. The reference optimum +(`-248.054331 -> -248.054446`) and the nominal point (`-198.207 -> -198.262`) do not move, which +is the envelope theorem. `Elowitz_Nature2000` reproduces it: IQR `70.21 -> 7.77`, 27/57 exactly +at the flat line, 19/57 discarding the dynamics. + +**What this is evidence for:** do not offer a numeric profile as a fallback where the closed +form is absent. **What it is not evidence for:** the linear-Gaussian construction, which has no +such mechanism — there the constant column is genuinely in the span, the projection can only +reduce the residual, and `a = 0` is optimal only in the degenerate case that `s` is orthogonal +to the centred data. `Borghans` is also a landscape already on record as pathological (a +wrong-period oscillator scores ~25 NLL units *worse* than a flat line), which is a further +reason not to generalize from it. + +## What would decide it + +Two measurements, in this order. Neither needs the full feature. + +1. **A synthetic linear-Gaussian fixture carrying a real coupled `(scale, offset)` pair.** The + corpus cannot supply one; building one is a few lines in the tutorial-fixture style, and it + puts the question in its own domain on a landscape nobody thinks is pathological. Run + `linear_profile.py` against it exactly as above. This is the measurement the evaluation is + missing, and it is cheap. +2. **The ADR-0108 redundancy question, on `Smith` and `Brannmark`** — the two slugs where the + scale profile is *not* redundant with a profiled σ (`Smith` has no σ at all, `Brannmark` + refuses `noise_profiling`). Not on `Weber`, where finding 4 shows there is nothing left for + it to collect. This is where #572's item 3 belongs. + +If (1) shows the coupled pair behaves like `Int_nuc_off` did and (2) shows a benefit that +`noise_profiling` is not already delivering, the feature is worth building, scoped to: + +* linear-scale families, plus the ADR-0066 geometric-mean form for a homogeneous parameter on a + log family; every other log-family parameter refused with the reason; +* refusal by resolved parameter name for anything a noise source also reads (finding 2); +* a stacked solve over the series a parameter is tied across — **9** of the corpus's 19 + closed-form-profilable parameters are tied across 2 to 11 experiments (`Smith`'s `sc_GLUT_2B` + and `sc_PI3K` across 11 each), so the per-series solve #572 writes down is not the right + object for them; +* `k` still counting a profiled linear parameter, and a Bayesian-sampler refusal, both by the + same arguments ADR-0108 already makes. + +## Consequences + +* No behaviour change. No new config key. `noise_profiling` (ADR-0108) and the ADR-0066 + `normalization = ..., scale` chain are unaffected. +* #572 stays open, narrowed to the linear-scale case, with the two measurements above as its + gate. +* Two reusable tools land in the corpus with their gotchas written down: + `Grein-2026-benchmark-subset-I/tools/linear_scope.py` and `.../linear_profile.py`. + +## Prior art + +Variable projection (Golub & Pereyra 1973); hierarchical optimization for ODE models (Loos et +al. 2018); pyPESTO/AMICI, which profile scale, offset and σ analytically by default. All three +are stated over an **additive-Gaussian** residual. That assumption is load-bearing, it is the +one finding 1 turns into a refusal, and ADR-0108's citation of the same prior art should be +read with it. diff --git a/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md b/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md deleted file mode 100644 index 962c5c00..00000000 --- a/docs/adr/0123-an-observables-linear-parameter-is-profilable-only-where-its-own-familys-residual-is-linear-in-it-so-the-coupled-scale-offset-projection-is-not-adopted.md +++ /dev/null @@ -1,281 +0,0 @@ -# An observable's linear parameter is profilable only where its own family's residual is linear in it, so the coupled `(scale, offset)` projection is not adopted (issue #572) - -**Status: Evaluated 2026-08-21; the proposed method is NOT adopted, and #572's own kill -criterion is met.** ADR-0066/0099 profile a declared column's optimal multiplicative **scale** -out analytically, and ADR-0108 (#562) profiles out an estimated **noise scale**. #572 proposed -the third member of that family — an additive **offset**, and the coupled `(scale, offset)` -pair — by variable projection, and asked for an evaluation *before* anything was built, -with a stated kill criterion. The evaluation was run. It kills the proposal, for a reason -neither the issue's case-for nor its case-against anticipated: **variable projection is a -linear-least-squares identity, and every coupled pair and every additive offset in the -corpus is scored on a log residual scale, where that identity does not hold.** Where an -exact profile is taken anyway — numerically, so the missing closed form is not the -obstacle — it collapses essentially the entire search box onto the no-dynamics score. - -No code changes. This ADR exists so the measurement is not re-litigated, and so the small, -genuinely sound remainder is scoped rather than lost. - -## What was asked - -> Fewer dimensions is not on its own a reason to ship a change to what the objective means. -> — #572 - -#572 asked four things: (1) compare the reduced and searched landscapes offline; (2) on -`Borghans`, does the `-165.98` no-dynamics attractor survive when scale, offset and σ are all -profiled; (3) head-to-head fits at matched budget; (4) confirm the `Fiedler` double-binding -refusal and that no other slug hides the same pattern. - -## How it was measured - -Two tools, both in the corpus (`Grein-2026-benchmark-subset-I/tools/`): - -* **`linear_scope.py`** classifies every declared free parameter that reaches an observable: - is the formula affine in it, is it a pure multiplicative factor, what residual space does - its observable's noise family score in (`family.additive_on.ln_base`), how many series is - it tied across, and is it *also* bound as a noise parameter. Static; no simulation. -* **`linear_profile.py`** simulates once per point and then minimizes **PyBNF's own - `evaluate_multiple`** over the linear parameters with θ held fixed. The profile is taken - *numerically*, on purpose: it is the exact conditional optimum whatever the family is, so - the landscape question is answered independently of whether a closed form exists. (ADR-0108 - pinned its own σ closed form against a numeric minimization of the reported objective for - the same reason: a plausible-but-wrong formula survives re-derived algebra and does not - survive this.) - -Neither tool runs a fit. A `Borghans` point costs one simulation plus a few hundred -re-scorings of it. - -Three self-checks carry the result, and all three passed: - -* The independently computed **flat-line reference** — every non-intercept coefficient pinned - to exactly `0`, the intercept profiled — reproduces `-165.982113` at **every** point, spread - `0`, which is the no-dynamics attractor already on record for this slug. -* The **reference optimum** re-scores at `-248.0524` against the recorded `-248.0692`, and the - PEtab nominal point at `-198.207` against the recorded `-198.1017`; both differences are the - σ-profiling and the bngsim build, not the harness. -* The profiled score is **never worse than the searched score** at any of the **279** points - measured, which it cannot be, since the searched value is in the set being minimized over. - -## Finding 1 — the scope in #572 is wrong in both directions - -Across all 23 subset-I slugs there are **47** affine observable-layer parameters in **9** -slugs, not 22 in 6. Split by whether a closed-form profile exists at all: - -| slug | k | affine obs params | linear-lsq | log-geomean | none | why not | -|---|---:|---:|---:|---:|---:|---| -| `Schwen_PONE2015` | 30 | 10 | 0 | 1 | 9 | log family, not multiplicative | -| `Smith_BMCSystBiol2013` | 25 | 9 | 9 | 0 | 0 | — | -| `Fiedler_BMCSystBiol2016` | 22 | 8 | 0 | 0 | 8 | double-bound as a noise parameter | -| `Raia_CancerResearch2011` | 39 | 5 | 0 | 0 | 5 | double-bound as a noise parameter | -| `Weber_BMC2015` | 36 | 5 | 5 | 0 | 0 | — | -| `Brannmark_JBC2010` | 22 | 4 | 4 | 0 | 0 | — | -| `Borghans_BiophysChem1997` | 23 | 2 | 0 | 0 | 2 | log family, not multiplicative | -| `Elowitz_Nature2000` | 21 | 2 | 0 | 0 | 2 | log family, not multiplicative | -| `Laske_PLOSComputBiol2019` | 13 | 2 | 1 | 0 | 1 | log family, not multiplicative | - -Two slugs the issue does not list at all: - -* **`Smith_BMCSystBiol2013`** — nine tied per-observable scales, 36 % of `k = 25`, every one of - them closed-form profilable. It is the **largest** opportunity in the corpus and it is - missing from the issue's table. -* **`Raia_CancerResearch2011`** — see finding 5. - -And the issue undercounts `Brannmark` (4, not 2: `k_IRP_1Step` and `k_IRSiP_DosR` are scales -too) and miscounts `Schwen`'s share as 42 % of the search when 9 of its 10 are not profilable -by any closed form. - -## Finding 2 — variable projection does not apply where #572 wants it - -`Phi = [s, 1]`, `c* = (Phi^T W Phi)^-1 Phi^T W d` is an identity about a residual `d - Phi c`. -A PyBNF noise family declares the space its residual lives in. For `lognormal` -(`ln_base = ln 10`) and `lnnormal` (`ln_base = 1`) the residual is `log d - log(a*s + b)`, -which is affine in `log a` when `b == 0` — that is exactly why ADR-0066's geometric-mean ratio -is a closed form for a *pure* scale — and is affine in `b` for no `b` at all. - -Sorted by role, the corpus splits cleanly and unhelpfully: - -* **All three coupled `(scale, offset)` pairs** — `Borghans` (`Z_state*scale + offset`), - `Elowitz` (`GFP*scale + background`), `Schwen` (`scale*(IR1 + IR1in + offset)`) — are - `lognormal`. Zero have a closed form. -* **Of the 8 offset-role parameters in the corpus, exactly one has a closed form**: - `Laske`'s `Int_nuc_off`, a Gaussian offset shared by nine observables — which #572 does not - mention. `Laske`'s `vRNA_offset`, the one offset the issue *does* name, sits on an - `lnnormal` observable and has none. -* The two slugs where `Phi = [s, 1]` is the right object, `Weber` and `Brannmark`, carry **no - offset at all**. They are pure scales — "one more scalar", the case the issue explicitly - sets aside as not the interesting one. - -So the interesting half of the proposal has no instances, and the instances have no -interesting half. `Schwen`'s `scale` is the single log-family parameter with a closed form, -and it is the geometric-mean ratio ADR-0066 already implements — though note it is a -*different object*: ADR-0066 profiles an **undeclared, per-series** scale, while `Schwen`'s is -one **declared coefficient tied across 7 series**, so it is a stacked solve, not ADR-0066's. - -## Finding 3 — the exact profile collapses the box onto the no-dynamics score - -This is the kill. The closed form is not the obstacle: the profile below is the exact -conditional optimum, taken numerically, with σ additionally profiled by ADR-0108 — #572's -item 2 verbatim, "scale, offset and σ all profiled". - -`Borghans_BiophysChem1997`, 76 box draws that integrate, `noise_profiling = 1`: - -| | searched | profiled | -|---|---:|---:| -| range over the box | `[-160.88, +339.69]` | **`[-167.15, -165.98]`** | -| interquartile range | **129.5** | **0.081** | -| median − flat line | +374.5 | −2.8e−14 | -| draws scoring better than the flat line | 0 / 76 | 40 / 76 | -| draws scoring **exactly** the flat line | 0 / 76 | **46 / 76** | -| draws within 1.0 of the flat line | 0 / 76 | **73 / 76** | -| draws whose profiled `scale < 1e-6` | — | **39 / 76** | -| Spearman rank correlation, searched ~ profiled | — | **0.035** | - -The reference optimum is untouched (`-248.054331 -> -248.054446`, gain `1.1e-4`) and so is the -nominal point (`-198.207 -> -198.262`, gain `0.055`) — the envelope theorem doing exactly what -#572 says it does. What moves is everything else: **the entire box collapses into a 1.16-unit -band sitting on the no-dynamics score**, while the reference optimum stays 82 units below it. - -The mechanism is worse than "the flat line becomes available". At 39 of 76 draws the profile -drives the scale below `1e-6` — it *discards the dynamics entirely*, because on a log residual -a wrong-shaped trajectory makes the fit worse than no trajectory. The no-dynamics answer is -not a floor the reduced surface can reach; over half the box it **is** the reduced surface. - -`Elowitz_Nature2000` reproduces it on the same mechanism, less extremely: 57 draws, IQR -`70.21 -> 7.77`, 27/57 exactly at the flat line `-53.656041`, 19/57 discarding the dynamics, -rank correlation **−0.43**. - -### The section is the picture - -Thirteen points interpolated in sampling space from the reference optimum (`t00`) out to a box -draw (`t12`), nine of which integrate: - -| | t00 | t01 | t02 | t03 | t04 | t05 | t06 | t07 | t08 | -|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| -| searched | −248.05 | −152.29 | −66.49 | +112.97 | +150.42 | +166.09 | +178.64 | +189.77 | +199.87 | -| profiled | −248.05 | −166.58 | −167.01 | **−165.9821** | **−165.9821** | **−165.9821** | **−165.9821** | **−165.9821** | **−165.9821** | - -The searched objective rises monotonically away from the optimum — a continuous descent -direction all the way in from the box. The profiled objective is **bit-identical at six -consecutive section points**. Profiling does not sharpen this landscape; it deletes it. - -### Answering (1) and (2) directly - -1. **Worse, decisively.** The reduced surface does not separate the reference optimum from the - flat-line floor better — it removes the separation between the flat-line floor and - *everything else*, which is the only structure a global sampler had. -2. **The `-165.98` attractor does not merely remain a converged endpoint. It becomes the value - of the search space.** Half the box scores it to the last bit, and 96 % of the box scores - within 1.0 of it. - -The conclusion does not depend on the parametrization. Repeating `Borghans` with the profile -taken on the real line rather than in each parameter's sampling space (`--linear-space`, -i.e. what an unconstrained variable projection would do) gives the same collapse, and -additionally trips ADR-0108's degenerate-scale refusal, because an unconstrained offset drives -predictions out of the log family's domain. An unconstrained projection is not merely -unavailable on these slugs; it is infeasible. - -## Finding 4 — where it does apply, `noise_profiling` has already collected most of it - -`Weber_BMC2015`, five pure Gaussian scales, 20 box draws, measured twice: - -| | searched IQR | profiled IQR | rank corr | -|---|---:|---:|---:| -| σ **searched** (as the conf ships) | 1.96e18 | 3.38e5 | 0.43 | -| σ **profiled** (`noise_profiling = 1`) | 1.67e3 | **1.72e3** | **0.87** | - -With ADR-0108 switched on, profiling all five observable scales changes the spread of the -landscape by **nothing** — it is marginally *larger* — and leaves the ranking almost intact. -The thirteen-orders-of-magnitude improvement in the first row is real, and it is the σ -profile's, not the scale profile's: a wrong scale inflates residuals, and a profiled σ absorbs -that inflation. #572's case-for is "better conditioning, every draw linear-optimal". On the -one corpus slug where both switches are available, the shipped one already delivers it. - -`Brannmark_JBC2010` is the counter-case and it is instructive: `noise_profiling = 1` is -**refused** there (`IRS1_P` estimates σ from a `PerMeasurementFormulaSigma`), so the scale -profile has the field to itself and does compress the landscape (IQR `1.01e17 -> 3.94e12`, -rank corr 0.62). Which also disposes of #572's proposed ordering rule: "linear-first, then σ -in closed form" describes a situation that does not arise on the slug it would matter for. - -`Laske`'s `Int_nuc_off` — the corpus's one genuine closed-form additive offset — behaves -exactly as a good nuisance should: it profiles to ≈ `57.13` at essentially every draw -regardless of θ, compresses the IQR from `1.01e9` to `5.01e7`, and moves the nominal point by -`0.02`. It is a real, small, well-behaved win. It is one parameter of 13 in one slug. - -## Finding 5 — the double-binding refusal is needed, and `Fiedler` is not the only slug - -#572 asks for `Fiedler_BMCSystBiol2016` to be refused by name and asks whether anything else -hides the pattern. Both halves check out, and the answer to the second is **yes**: - -* **`Fiedler`** — confirmed exactly as described. All eight `s_pErk_*` / `s_pMek_*` tokens are - bound to `observableParameter1_*` *and* `noiseParameter1_*` in the same per-measurement - tables. -* **`Raia_CancerResearch2011`** — the same defect by a different route, and **not mentioned in - the issue**. Five `scaling_*` parameters are the observable scale *and* appear inside a - `prediction_formula` σ (`CD274mRNA*scaling_CD274mRNA*sd_CD274mRNA_rel + sd_CD274mRNA_abs`), - so moving the scale moves that observable's σ too. Thirteen of the corpus's 47 affine - observable parameters are double-bound this way — more than a quarter. - -A "detect the `noiseParameter` placeholder" rule catches `Fiedler` and misses `Raia`. Any -implementation must test the resolved parameter *names* a noise source reads, formula sources -included. - -## The decision - -**The coupled `(scale, offset)` variable projection of #572 is not implemented**, and #572 is -closed against its own kill criterion. Restating that criterion: - -> If (1) and (2) show the reduced landscape is no better — or worse — for global search, and -> (3) shows no reliable improvement, then close this as a tidiness item and do **not** -> implement. - -(1) and (2) are worse by the largest margin the measurement could have produced: the reduced -objective is constant to 0.08 units over the interquartile range of the box, and bit-identical -across six consecutive points of a section. Fewer dimensions was never the argument, and it is -all that is left. - -### What was deliberately not measured, and why - -**Item 3, head-to-head fits at matched budget, was not run.** For the offset half that is a -consequence of the result rather than a gap in it: a fit comparison on a landscape that is -constant over half the box measures the sampler's tie-breaking, not the objective. Reporting -"profiled did not converge better" from such a run would attribute to the search what the -measurement already attributes to the surface. - -For the pure-scale remainder, item 3 *is* the right test — and it cannot be run without first -building the feature, which is the thing #572 exists to gate. It is therefore named below as -the acceptance test for the follow-up rather than performed here. - -### What survives, and what it would have to be - -A **pure multiplicative scale, on a linear-scale family, tied across the series that share -it** is sound: the closed form is real, the corpus has 19 such parameters in 4 slugs, and -`Laske`'s `Int_nuc_off` shows the additive case behaving well where the family permits it. -That is a materially different, smaller feature than #572 proposes, and it inherits three -constraints this evaluation established: - -* It must **refuse a parameter that any noise source also reads**, by resolved name, formula - sources included — 13 of 47 corpus parameters, across `Fiedler` **and** `Raia`. -* It must refuse any parameter whose observable scores on a **log** family unless the formula - is homogeneous in it, and it must apply the geometric-mean form rather than least squares - when it is. -* Its acceptance test is **item 3 on `Smith` and `Brannmark`** — the two slugs where it is not - redundant with `noise_profiling` — not on `Weber`, where finding 4 shows there is nothing - left for it to collect. - -Whether that is worth building is a separate question from this one, and this ADR does not -answer it. It is not opened here. - -## Consequences - -* No behaviour change. No new config key. `noise_profiling` (ADR-0108) and the ADR-0066 - `normalization = ... , scale` chain are unaffected. -* `docs/adr/0108` is unchanged; its "pyPESTO/AMICI profile scale, offset and σ analytically by - default" prior-art line stands, with the qualification recorded here that those defaults are - a linear-Gaussian construction and PyBNF's log families are outside it. -* Two reusable tools land in the corpus, with their gotchas written down: - `Grein-2026-benchmark-subset-I/tools/linear_scope.py` and `.../linear_profile.py`. - -## Prior art - -Variable projection (Golub & Pereyra 1973); hierarchical optimization for ODE models (Loos et -al. 2018); pyPESTO/AMICI. All three are stated over an additive-Gaussian residual. That -assumption is load-bearing and is the one this corpus violates.