Derive the EAMxx source vertical grid file at run time

[chengzhuzhang]

Fixes #863.

Problem

zppy hardcoded an L128v1 vertical coordinate file (e3sm_to_cmip_data/grids/vert_L128.nc) as ncremap --vrt_in whenever prc_typ == 'eamxx'. E3SM#8692 makes L128v4 the default EAMxx grid, and because the level count is still 128, ncremap does not error on v4 input — it interpolates with v1 coefficients and emits silently wrong pressure-level data into ts_vrt_remap/ and cmip_ts/.

Setting vrt_in_file explicitly already worked as a user override, so this is a bad default rather than a missing capability. The problem is that a user who does not know to set it gets no error.

Change

Derive the source vertical grid from the run's own output rather than shipping a grid-version-specific data file:

run_nco ncks -O -v hyai,hybi,hyam,hybm,P0 ${file} vrt_in.nc 2> /dev/null || \
  { run_nco ncks -O -v hyai,hybi,hyam,hybm ${file} vrt_in.nc && \
    run_nco ncap2 -A -s 'P0=100000.0' vrt_in.nc ; }

(the fallback is the operation recorded in vert_L128.nc's own history attribute). This is grid-agnostic — L72, L128v1, L128v4 and any future EAMXX_VGRID work with no staged file and no zppy release.

• ts.bash derives from the raw history file (head -n 1 input.txt), which always carries hy*, so the [ts] path does not depend on extra_vars. Only ps remains load-bearing there, for --ps_nm. Failure gets its own status code 7.
• e3sm_to_cmip.bash derives lazily, inside the loop, from the file it is about to remap, guarded so it runs once. Eager derivation would have been a regression: interp_vars defaults to EAM names, so that loop usually remaps nothing for EAMxx, and a job that never calls ncremap must not fail on a missing derivation. Status code 5, with a message naming [ts] extra_vars as the fix.
• Both templates set a shell vrt_in_file, so an explicit vrt_in_file still takes precedence and short-circuits the derivation entirely.
• vrt_remap_file / cmip_plevdata (the --vrt_out plev19 target) are untouched — the target grid is unaffected by this.

Docs, default.ini comments, and the examples/post.v3.eamxx.cfg header no longer claim a vert_L128.nc default. No references to that file remain anywhere in zppy/, docs/, or examples/.

Tests

New tests/test_vertical_remap.py renders both templates and asserts three cases: derived (no vert_L128, derivation emitted), user-supplied vrt_in_file (path wins, no derivation), and prc_typ = eam (no --vrt_in / --ps_nm at all). 96 unit tests pass; pre-commit clean.

Verification

Checked against the L128v1 EAMxx case from examples/post.v3.eamxx.cfg (ne256pg2 ... F20TR-SCREAMv1, ne30pg2 monthly output, 1995-1999), whose ts/ and ts_vrt_remap/ output from the old staged-file code is still on scratch.

• The derived grid file reproduces vert_L128.nc bit-for-bit. hyai, hyam, hybi, hybm and P0 (double, 100000.0) are bit-identical, whether derived from the raw history file (the [ts] path) or from the split per-variable ts file (the [e3sm_to_cmip] path). lev top/bottom match at 2.5802608 / 998.49646 hPa. The derived file additionally carries ilev, which ncks pulls in as a coordinate; ncremap does not use it.
• ncremap output is bit-identical. Same ncremap line on a two-month subset of T_mid_199501_199912.nc, derived vs staged --vrt_in: maxabsdiff = 0.0 over all (2, 19, 180, 360) values, with no change in missing-value count. The same holds against the ts_vrt_remap/T_mid_199501_199912.nc that the old code wrote in July, so this is a real before/after comparison and not just self-consistency within one session.
• P0 is genuinely required, not assumed by NCO. With P0 stripped from an otherwise identical vrt_in file, ncremap fails: ERROR Failed to vertically interpolate. cmd_rgr[0] failed. The ncap2 step is load-bearing, hence the status check around it.
• P0 handling is future-proof. EAMxx writes no P0 today, but later versions will, so the derivation asks for it first and synthesizes 100000.0 only when that extraction fails. Checked against real files: a source without P0 gives the synthesized 100000; a source carrying P0 = 99999 keeps 99999 rather than being overwritten; a source missing the hybrid coefficients still fails loudly with the NCO error and trips the status code.
• Once an L128v4 run exists, confirm the derived lev top differs from vert_L128.nc's 2.58 hPa. Nothing to test against yet.

Draft pending that last item.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NGCrzkffuiQ1Y5PR9FGv7C