json: extract non-finite numeric values without JSON quotes

[Alignyx]

Summary

Fix extra JSON quotation marks in text extracted from numerically constructed
NaN, Infinity, and -Infinity. In #144837, the same typed SQL query returns
different text depending on whether a non-finite JSON constant is evaluated
locally or serialized into a remote expression.

The production change is confined to jsonNumber.AsText: return the decimal's
text instead of its JSON serialization. JSON formatting, constructors, internal
numeric type tags, comparison, and binary/index encoding are unchanged. The PR
includes native unit and SQL regressions, including fixed legacy encoding bytes.

This is a text-extraction consistency fix, not a normalization of non-finite
JSON values into JSON strings throughout the system.

Problem and expected behavior

A reduced query retains the report's FLOAT8 cast and makes the extraction index
a runtime column, so the complete extraction cannot simply be constant-folded:

CREATE TABLE inputs (i INT PRIMARY KEY, f FLOAT8, d DECIMAL);
INSERT INTO inputs VALUES (0, 'NaN', 'NaN'), (1, 'NaN', 'NaN');

SELECT i,
       to_json('NaN'::FLOAT8) AS j,
       to_json('NaN'::FLOAT8) ->> i AS t
FROM inputs
ORDER BY i;

For i = 0, JSON display j is "NaN", but extracted SQL text t should be
NaN, without the two JSON wrapper quotes. For the out-of-range scalar index
i = 1, extraction remains SQL NULL. These are different observations: valid
JSON serialization needs the quotes, whereas scalar text extraction does not.

The baseline local query returns the five-character text "NaN"; the remotely
serialized constant returns the three-character text NaN. Numerically
constructed infinities have the same extraction defect. An explicit JSON string
already extracts without its JSON wrapper quotes.

Metamorphic analysis and correct-result oracle

Primary relation: execution-location invariance

For this deterministic query over the same data and SQL types, moving evaluation
from the gateway to a remote node must preserve the extracted SQL text. Switching
between vectorized and row execution must preserve it as well.

The investigation used the identical query above in a three-node cluster. The
table lease was placed on node 2. With distsql = off, SQL execution was local
on node 1; with distsql = always, the distributed arm executed on node 2.
Actual EXPLAIN ANALYZE node/distribution observations were checked, rather than
treating a setting change alone as proof of remote execution. The data, explicit
FLOAT8 cast, projection, and runtime extraction index were unchanged.

This relation detects an inconsistency without assuming that either plan is
correct simply because it ran locally or remotely. The intended unquoted text
is independently supported by the scalar text-extraction contract, the existing
JSON-string extraction behavior, and the following auxiliary controls.

Auxiliary text oracles, with an important type boundary

The following local controls both extract NaN at index 0:

-- Serialize the JSON value to text, then parse that JSON text again.
SELECT i,
       to_json('NaN'::FLOAT8)::STRING::JSONB ->> i AS t
FROM inputs ORDER BY i;

-- Extract from an explicit JSON string with the same JSON representation.
SELECT i, '"NaN"'::JSONB ->> i AS t
FROM inputs ORDER BY i;

These controls change the representation boundary, not just the physical plan.
They must not be described as preserving every internal JSON property:

SELECT jsonb_typeof(to_json('NaN'::FLOAT8)),
       jsonb_typeof(to_json('NaN'::FLOAT8)::STRING::JSONB),
       jsonb_typeof('"NaN"'::JSONB);
-- number, string, string

That distinction persists with this patch. The controls are oracles for the
extracted text and clues to the representation boundary; the unchanged typed
query under execution-location transformation is the primary metamorphic relation.
Runtime to_json(f) and DECIMAL inputs were also checked to distinguish
constant serialization from numeric construction during execution.

Repeated observations

Three complete baseline runs and three complete candidate runs agreed. The
following table describes text at i = 0; index 1 remains SQL NULL in every arm.

Execution/input path	Baseline extracted text	Candidate extracted text
Local numeric literal, vectorized or row engine	"NaN"	NaN
Local runtime FLOAT column / DECIMAL literal	"NaN"	NaN
Remote serialized numeric constant, vectorized or row engine	NaN	NaN
Remote runtime FLOAT column	"NaN"	NaN
Local JSON-text roundtrip / explicit JSON string	NaN	NaN

In particular, remote execution is not universally correct on the baseline:
the runtime-column arm still constructs a numeric JSON value remotely and exhibits
the defect. This separates the serialization boundary from the node or engine
itself. All nine W/C arms agree after the repair.

Root cause and execution-path localization

tree.AsJSON sends FLOAT and DECIMAL datums to FromFloat64 and FromDecimal.
These constructors create a jsonNumber, including for non-finite decimal forms.
The following existing paths then disagree:

1. JSON formatting: jsonNumber.String() calls Format(). For non-finite
   values, Format() writes quotes around apd.Decimal.String() to produce
   valid JSON text, such as "NaN".
2. Local numeric text extraction: jsonNumber.AsText() previously called
   j.String(), inadvertently returning that JSON-formatted text, quotes included.
3. Remote constant serialization: an already folded JSON constant can cross
   ExprFactory.Make's expression-text boundary. DJSON.Format emits a SQL literal
   containing the JSON text; DeserializeExpr parses/type-checks it on the remote
   side. The quoted non-finite representation is then a genuine jsonString.
   Its existing AsText() returns the string contents without JSON wrapper quotes.
4. Remote runtime construction: to_json(f) constructs the numeric JSON value
   during execution, so it does not benefit from that folded-constant conversion.

This identifies the faulty interface as numeric text extraction reusing JSON
serialization, rather than the decimal value, scan contents, or a particular
execution engine.

The plan comparison supports that source-level explanation:

• The local failing query and local JSON-roundtrip control have byte-identical
  rendered EXPLAIN (OPT, VERBOSE) output. JSON rendering hides the internal
  number/string distinction; identical plan text is not proof of identical datums.
• Local execution records SQL work on node 1 and KV access on node 2. The remote
  constant arm records full distribution and SQL/KV work on node 2, with network
  usage. The scan and render each process two rows.
• Therefore the result discrepancy occurs despite the same input rows and is
  consistent with the expression-representation boundary, not missing data.

These are source-path checks combined with optimizer and actual-plan observations,
not instrumented, ordered branch-coverage traces.

How the repair works and why this boundary was chosen

The change is:

func (j jsonNumber) AsText() (*string, error) {
    // JSON formatting quotes non-finite numbers, but text extraction must not.
    dec := apd.Decimal(j)
    s := dec.String()
    return &s, nil
}

For finite numbers, Format() already writes this exact decimal text without
quotes. Using it directly therefore preserves finite formatting and decimal scale,
including the tested 1.2300 and -12.50 cases. For non-finite numbers, it avoids
only the JSON wrapper quotes.

Scalar/index/key ->> and path #>> extraction ultimately use AsText() on the
selected value, so the correction applies to numeric leaves in arrays and objects
as well as standalone scalars. JSONEncoded.AsText() decodes and delegates to the
same method. Consequently, legacy binary numeric JSON
also receives the corrected extraction behavior without changing its stored type.

The alternative of making constructors return JSON strings would change type tags,
comparison and encoding behavior, and would not by itself fix already encoded
numeric values. Keyside numeric JSON decoding also reconstructs values through
FromDecimal. This PR deliberately leaves those contracts alone.

The historical dea0313b change
added non-finite JSON quoting and a parseability regression. This repair preserves
that behavior: JSON serialization still emits "NaN", "Infinity", and
"-Infinity". Extracting a whole container still returns its JSON representation,
including quoted non-finite leaves; extracting the leaf returns its text contents.

Completed local validation

The following results apply to repair commit
474855b6b74253da0ce4a55ef57bd731c233fa65, based on
8812064a015d2faf99d3fc7e15880f94042954b0. They are completed local validation,
not a claim that upstream PR CI has passed.

• Frozen native unit regression: TestJSONNonFiniteAsText covers FLOAT and
  DECIMAL non-finite values, finite controls, decoded/encoded values with repeated
  extraction, fully
  decoded values, nested leaves and container formatting. Baseline has 21 failing
  non-finite leaf subtests and 15 passing finite leaf subtests; the unchanged
  regression passes completely with the candidate.
• Legacy binary contract: fixed scalar encoding bytes assert the existing
  Number and decimal tags. Tests decode those bytes, check type and decimal
  components, repeat extraction and re-encode to the same bytes. JSON String()
  and Format() remain unchanged and produce parseable JSON.
• Native SQL regression: regression_144837 in json_builtins covers
  FLOAT/DECIMAL NaN and both infinities, valid scalar indices 0/-1, out-of-range
  indices, runtime numeric and stored JSON columns, object/array/path extraction,
  finite scale, JSON formatting/type and string-versus-SQL NULL controls. It fails
  on baseline and passes on the candidate. The baseline run stops at the first
  failing FLOAT query; later statements are not claimed to have separate native-red
  observations.
• Repeated metamorphic replay: all six complete baseline/candidate matrices
  pass the archived verifier's exact expected-outcome checks. These include actual
  local/remote nodes, two-row scan/render counts, strict CSV quote interpretation,
  and type/NULL/finite controls. Twelve corrupted or mislabeled evidence variants
  and four malformed CSV variants are rejected.
• Independent held-outs: 12 scenarios and 123 checks produce 54 baseline
  mismatches and zero candidate mismatches; eight defect scenarios are corrected.
  Coverage includes scalar/key/nested extraction, array/each text extraction,
  stored scalars and container leaves, with finite, string/NULL, full-container
  and internal Number-type controls. Ten verifier rejection self-tests pass.
• Broader regressions: complete json, encoding, keyside, tree, builtins and
  eval unit targets, plus the full SQL target across 16 shards, pass locally.
  Complete json, json_builtins, postgres_jsonb and json_index logic files
  pass under local, local-vec-off, fakedist, fakedist-vec-off and
  fakedist-disk (20 file/configuration executions).

One preliminary candidate replay stopped during lease relocation before any W/C
query because its target replica was not yet ready. That setup-only attempt was
excluded and retained separately; the three complete candidate runs used the
unchanged SQL and strict actual-node checks.

The PR contains three files (+148/-1): the small json.go production change,
json_test.go coverage and the json_builtins regression. Supplemental replay and
held-out logs/verifiers are retained in the local investigation record, not added
as files to this PR.

Current limitations

• Internal JSON type differences remain. Numerically constructed non-finite
  JSON can still have type number, while its parsed textual representation has
  type string. This PR does not unify those types or establish equivalence for
  every type-sensitive comparison, JSONPath expression or other JSON operation.
• The scope is the AsText() extraction path. It does not redesign every
  JSON-to-SQL conversion or change JSON parsing, numeric constructors, Format(),
  Compare(), numeric type tags, or binary/index encoding.
• Previously materialized derived values are not migrated. Existing stored
  extracted TEXT, stored computed results or expression-index entries derived
  from the old quoted extraction are not automatically recomputed. Preserving
  JSON storage encoding does not establish that all such derived data needs no
  operational follow-up; affected cases require separate assessment.
• Upgrade and performance coverage is bounded. SQL storage cases write/read
  each version independently; they are not an upgrade of the same storage directory
  or a mixed-version cluster qualification. Fixed legacy byte tests separately
  cover the numeric binary representation. No performance benchmark or blanket
  backport certification is supplied.

If a broader normalization of non-finite JSON types is desired, it should be
reviewed separately with explicit decisions about legacy numeric values,
comparison/index semantics and derived-data migration. This PR makes no claim
that such a migration or normalization design is already implemented.

Resolves: #144837
Epic: none

Release note (bug fix): Fixed JSON text extraction operators such as ->>
and #>> returning extra quotes around NaN and infinities constructed from
numeric values. Results now agree with text extraction after remote
constant serialization. JSON formatting and stored numeric JSON types
remain unchanged.

[blathers-crl]

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}