Skip to content

[Decision] Bernoulli's survival convention: P(X >= x) costs the ff/qf duality #344

Description

@derrynknife

Needs a decision. Not a bug — the code does what was specified and is tested. This records a convention trade-off made in 0.19.1 so it is a deliberate choice on the record rather than something rediscovered later.

The situation

Bernoulli uses R(x) = P(X >= x), so R(0) = 1 and R(1) = p. That reads as a one-shot device: p is the probability the unit works when demanded.

The package requires sf + ff == 1. That forces F(x) = P(X < x), which never reaches 1 on the support, because P(X < x) excludes the atom at x and at the top of the support that atom is p itself:

p = 0.3
  x    sf = P(X>=x)    ff = P(X<x)     P(X<=x)
  0          1.00           0.00        0.70
  1          0.30           0.70        1.00

A quantile function answers "smallest x with F(x) >= u". With F only taking the values 0 and 0.7:

u = 0.5:  candidates = [1]
u = 0.7:  candidates = [1]
u = 0.9:  candidates = NONE      <- nothing in {0,1} reaches 0.9

So above u = 1 - p the question has no answer in the support. Bernoulli.qf therefore inverts P(X <= x) — the standard quantile — and is not the inverse of this class's own ff.

Everything else holds: the mass sums to one, h = f/R, H = -ln R, E[X] from the mass equals mean and moment(1), and qf(U) reproduces the distribution under inverse-transform sampling. qf also matches Binomial.qf(u, 1, p) and scipy.stats.binom.ppf exactly on the open interval.

Why no other distribution has this

Every other discrete distribution uses R(k) = P(X > k), so its ff comes out as P(X <= k) — the standard CDF, which climbs to 1 — and ff/qf invert each other cleanly:

Poisson(3)   k      0      1      2      3      4      5
             ff  0.050  0.199  0.423  0.647  0.815  0.916  -> 1

For continuous distributions P(X < x) == P(X <= x), so the distinction does not exist. It only bites on a discrete support, where every point carries an atom.

The options

A — keep R(x) = P(X >= x) (status quo). Keeps the one-shot-device reading and R(0) = 1, R(1) = p. Costs the ff/qf duality, so the package's usual discrete check ff(qf(u)) >= u does not hold for this one distribution. Already implemented, documented in the class and qf docstrings, and pinned by test_bernoulli_qf_does_not_invert_this_ff_and_says_so, which asserts the failure for Bernoulli and that Poisson still satisfies it.

B — switch to R(x) = P(X > x). R(0) = p, R(1) = 0. ff becomes the standard CDF, qf inverts it, and Bernoulli lines up with every other discrete distribution and with Binomial(n=1) on sf as well as df. Costs the R(0) = 1 reading and flips p back to meaning something closer to the failure probability. Small change — sf, ff and qf only; df, hf, Hf, mean, moment, entropy, random and fit are unaffected.

Recommendation

Keep A. In a reliability library, "survives the single demand with probability p" is the reading people want, and the cost is confined to one documented and tested property of one distribution. But it is a genuine convention choice and worth being explicit about before 1.0.

Context

  • Introduced in 0.19.1 when Bernoulli stopped being a flat F(x) = p model and became a real coin flip.
  • The flat model still exists unchanged as FixedEventProbability.
  • Relevant code: surpyval/univariate/parametric/distributions/bernoulli.py
  • Relevant tests: test_bernoulli_qf_is_the_standard_quantile, test_bernoulli_qf_does_not_invert_this_ff_and_says_so, test_bernoulli_qf_drives_inverse_transform_sampling in surpyval/tests/univariate/parametric/test_discrete.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions