diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 4cbe2fc..6ae817b 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -16,7 +16,24 @@ jobs: python-version: "3.11" - name: Install dependencies - run: pip install -r requirements.txt + run: | + python -m pip install --upgrade pip + pip install -r requirements.txt + pip install --group test - - name: Run tests - run: pytest + # Validates tests/test_manifest.yaml, regenerates tests/README.md, then + # runs pytest with pytest-html. Exits with pytest's own exit code, so a + # test failure still fails this job -- after the report is uploaded. + - name: Run tests and generate report + run: python scripts/generate_test_dashboard.py + + - name: Upload test report + if: always() + uses: actions/upload-artifact@v4 + with: + name: pyrpod-test-report + path: | + reports/pyrpod-pytest-report.html + reports/logs/ + retention-days: 14 + if-no-files-found: warn diff --git a/.gitignore b/.gitignore index 1201f39..b7a2928 100755 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,10 @@ data/stl/tcd/ results/ +# Run-specific test reports (HTML report + failed-test logs). The generated +# tests/README.md and tests/test_manifest.yaml ARE tracked. +reports/ + results.txt *.pyc *.png diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 141bd7d..9d595ec 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -48,8 +48,15 @@ PyRPOD uses [pytest](https://docs.pytest.org/) to ensure stability and prevent r 1. Write tests for your contributions in the `tests` directory, following the existing `__test_NN.py` naming convention (e.g. `rpod_unit_test_04.py`). 2. Run the full suite locally with `pytest` from the repository root before submitting a pull request. -3. Tests are automatically tagged with `unit`, `integration`, `verification`, and subsystem (`mdao`, `mission`, `plume`, `rpod`) markers based on their filename, so you can run a subset with e.g. `pytest -m unit` or `pytest -m rpod`. -4. CI runs the same `pytest` suite automatically on every push and pull request via [GitHub Actions](.github/workflows/tests.yml). +3. Tests are automatically tagged with `unit`, `integration`, `verification`, and subsystem (`logging`, `mdao`, `mission`, `plume`, `rpod`, `tooling`) markers based on their filename, so you can run a subset with e.g. `pytest -m unit` or `pytest -m rpod`. +4. **Add a matching entry to [`tests/test_manifest.yaml`](tests/test_manifest.yaml)** describing the new file, then regenerate the inventory: + ```bash + python -m pip install "pytest-html>=4.2,<5" "PyYAML>=6" # once + python scripts/generate_test_dashboard.py + ``` + Commit the regenerated `tests/README.md` alongside your test. The generator fails with an explicit list of missing or stale entries if the manifest and pytest's collection disagree. +5. Record the *development* status (`implemented`, `placeholder`, `needs_review`, `blocked`, `archived`, `deprecated`) in the manifest — never a pass/fail result. Execution outcomes live only in the run-specific `reports/pyrpod-pytest-report.html`, which is git-ignored. A test with no assertions is a `placeholder`; skip it explicitly so it cannot be mistaken for coverage. +6. CI runs the same command automatically on every push and pull request via [GitHub Actions](.github/workflows/tests.yml), and uploads the HTML report as the `pyrpod-test-report` artifact even when tests fail. ## Code Formatting diff --git a/README.md b/README.md index 39e5813..3b32ed2 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,46 @@ PyRPOD utilizies scientific libraries such as NumPy, SciPy, Matplotlib, and SymP python -m pytest tests/rpod # or just point pytest at a directory/file directly ``` +## Test Reporting + +Test information lives in two places, and the split is deliberate: + +| | Where | Owned by | Committed? | +| --- | --- | --- | --- | +| **Execution outcome** (passed / failed / skipped, for one run) | `reports/pyrpod-pytest-report.html` | pytest | No — `reports/` is git-ignored | +| **Development status** (implemented / placeholder / blocked / ...) | [`tests/test_manifest.yaml`](tests/test_manifest.yaml) → [`tests/README.md`](tests/README.md) | maintainers | Yes | + +A passing test is *not* evidence that a test is finished: several placeholder +tests contain no assertions and are explicitly skipped so they can never be +mistaken for coverage. + +1. Install the reporting dependencies once (declared as the `test` dependency + group in `pyproject.toml`): + ```bash + python -m pip install "pytest-html>=4.2,<5" "PyYAML>=6" + ``` + With pip 25.1 or newer you can instead run `python -m pip install --group test`. + +2. Generate the dashboard and the HTML report: + ```bash + python scripts/generate_test_dashboard.py + ``` + This validates the manifest, cross-checks it against pytest's current + collection, regenerates `tests/README.md`, then runs the suite and writes a + self-contained `reports/pyrpod-pytest-report.html`. It exits with pytest's + own exit code, and the report is written even when tests fail. Use + `--inventory-only` to refresh `tests/README.md` without running the suite, + or `--check` to verify it is up to date. + +3. When you add a test, add a matching entry to `tests/test_manifest.yaml`. + The generator fails with an explicit list of missing or stale entries + otherwise. + +CI runs the same command on every pull request and every push to `master`, and +uploads the report as the **`pyrpod-test-report`** artifact (retained 14 days), +including when tests fail. Download it from the *Artifacts* section of the +workflow run summary on the Actions tab. + ## Logging PyRPOD ships a formal, opt-in operational logging system built entirely on the diff --git a/pyproject.toml b/pyproject.toml index 6fa69a7..2ed903c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,3 +1,15 @@ +# Optional dependencies for generating the test dashboard (PEP 735). PyRPOD is +# not installed as a distribution, so there is no [project] table to hang +# optional-dependencies off; a dependency group adds these without introducing +# packaging metadata or changing how the repository is installed. +# python -m pip install --group test (pip >= 25.1) +[dependency-groups] +test = [ + "pytest", + "pytest-html>=4.2,<5", + "PyYAML>=6", +] + [tool.black] line-length = 88 target-version = ["py311"] @@ -39,4 +51,5 @@ markers = [ "mission: mission subsystem tests", "plume: plume subsystem tests", "rpod: rpod subsystem tests", + "tooling: test-infrastructure tests", ] diff --git a/scripts/generate_test_dashboard.py b/scripts/generate_test_dashboard.py new file mode 100644 index 0000000..c8f894c --- /dev/null +++ b/scripts/generate_test_dashboard.py @@ -0,0 +1,696 @@ +#!/usr/bin/env python +"""Regenerate the PyRPOD test inventory and the pytest HTML execution report. + + python scripts/generate_test_dashboard.py + +The script keeps two deliberately separate concepts apart: + +* **Development status** - inventory-level metadata maintained by hand in + ``tests/test_manifest.yaml`` (implemented, placeholder, needs_review, ...). + It is rendered into the committed ``tests/README.md``. +* **Execution outcome** - pass / fail / skip for a single run, owned by pytest + and rendered into the run-specific ``reports/pyrpod-pytest-report.html``. + +Steps performed: + +1. Validate the manifest schema. +2. Collect the current pytest tests (``pytest --collect-only``). +3. Cross-check the manifest against that collection. +4. Regenerate ``tests/README.md``. +5. Create ``reports/``. +6. Run pytest with ``pytest-html``, writing + ``reports/pyrpod-pytest-report.html``. +7. Exit with pytest's exit code (the report is written either way). + +Requires the ``test`` dependency group (``pytest-html``, ``PyYAML``); see the +Testing section of the root README.md for the install command. +""" + +from __future__ import annotations + +import argparse +import subprocess +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +MANIFEST_PATH = REPO_ROOT / "tests" / "test_manifest.yaml" +README_PATH = REPO_ROOT / "tests" / "README.md" +REPORTS_DIR = REPO_ROOT / "reports" +REPORT_PATH = REPORTS_DIR / "pyrpod-pytest-report.html" + +SUBSYSTEMS = ("logging", "mdao", "mission", "plume", "rpod", "tooling") +SUBSYSTEM_TITLES = { + "logging": "Logging", + "mdao": "MDAO", + "mission": "Mission", + "plume": "Plume", + "rpod": "RPOD", + "tooling": "Tooling", +} +CATEGORIES = ("unit", "integration", "verification") +EXECUTION_MODES = ("automated", "manual") +DEVELOPMENT_STATUSES = ( + "implemented", + "placeholder", + "needs_review", + "blocked", + "archived", + "deprecated", +) +COLLECTION_STATUSES = ("collected", "ignored", "manual", "archived") + +REQUIRED_FIELDS = ( + "path", + "description", + "subsystem", + "category", + "execution_mode", + "development_status", + "collection_status", +) +OPTIONAL_FIELDS = ("reference", "manual_command", "collection_ignore_reason") +KNOWN_FIELDS = REQUIRED_FIELDS + OPTIONAL_FIELDS + +# Files under tests/ that are support code rather than test assets and so are +# deliberately absent from the manifest. +NON_TEST_FILES = frozenset( + { + "tests/conftest.py", + "tests/plume/plume_figure_utils.py", + "tests/plume/plume_impingement_utils.py", + } +) + +REVIEW_REQUIRED = "REVIEW REQUIRED" + +AUTOGEN_HEADER = ( + "" +) + + +class ManifestError(Exception): + """Raised when the manifest cannot be parsed or fails validation.""" + + +# -------------------------------------------------------------------------- +# Manifest loading and validation +# -------------------------------------------------------------------------- + + +def load_manifest(path=MANIFEST_PATH): + """Parse the manifest and return its list of entries. + + Raises ManifestError for anything that makes the file unusable (missing, + unparseable, wrong top-level shape). + """ + try: + import yaml + except ImportError as exc: # pragma: no cover - environment dependent + raise ManifestError( + "PyYAML is required to read the test manifest. Install the test " + 'dependency group: python -m pip install "PyYAML>=6"' + ) from exc + + path = Path(path) + if not path.is_file(): + raise ManifestError(f"manifest not found: {path}") + + try: + data = yaml.safe_load(path.read_text(encoding="utf-8")) + except yaml.YAMLError as exc: + raise ManifestError(f"{path} is not valid YAML: {exc}") from exc + + if not isinstance(data, dict) or "tests" not in data: + raise ManifestError(f"{path} must be a mapping with a top-level 'tests' key") + + entries = data["tests"] + if not isinstance(entries, list) or not entries: + raise ManifestError(f"{path}: 'tests' must be a non-empty list") + + for index, entry in enumerate(entries): + if not isinstance(entry, dict): + raise ManifestError(f"{path}: entry #{index + 1} is not a mapping") + + return entries + + +def validate_manifest(entries, repo_root=REPO_ROOT): + """Return a list of human-readable schema problems (empty when valid).""" + problems = [] + seen = set() + + for index, entry in enumerate(entries): + label = entry.get("path") or f"entry #{index + 1}" + + for field in REQUIRED_FIELDS: + value = entry.get(field) + if value is None or (isinstance(value, str) and not value.strip()): + problems.append(f"{label}: missing required field '{field}'") + + for field in sorted(set(entry) - set(KNOWN_FIELDS)): + problems.append(f"{label}: unknown field '{field}'") + + path = entry.get("path") + if isinstance(path, str) and path: + if path in seen: + problems.append(f"{label}: duplicate path") + seen.add(path) + if "\\" in path: + problems.append(f"{label}: path must use '/' separators") + if not (repo_root / path).is_file(): + problems.append(f"{label}: file does not exist") + + for field, allowed in ( + ("subsystem", SUBSYSTEMS), + ("category", CATEGORIES), + ("execution_mode", EXECUTION_MODES), + ("development_status", DEVELOPMENT_STATUSES), + ("collection_status", COLLECTION_STATUSES), + ): + value = entry.get(field) + if value is not None and value not in allowed: + problems.append( + f"{label}: {field} '{value}' is not one of {', '.join(allowed)}" + ) + + mode = entry.get("execution_mode") + collection = entry.get("collection_status") + if mode == "manual" and not entry.get("manual_command"): + # Blocked/archived assets have no meaningful command to document. + if entry.get("development_status") not in ("blocked", "archived"): + problems.append( + f"{label}: execution_mode 'manual' requires a manual_command" + ) + if mode == "automated" and collection != "collected": + problems.append( + f"{label}: execution_mode 'automated' requires " + f"collection_status 'collected', got '{collection}'" + ) + if collection != "collected" and not entry.get("collection_ignore_reason"): + problems.append( + f"{label}: collection_status '{collection}' requires a " + "collection_ignore_reason" + ) + if collection == "collected" and entry.get("collection_ignore_reason"): + problems.append( + f"{label}: collection_status 'collected' must not set " + "collection_ignore_reason" + ) + + description = entry.get("description") + if isinstance(description, str) and description.startswith(REVIEW_REQUIRED): + if entry.get("development_status") != "needs_review": + problems.append( + f"{label}: '{REVIEW_REQUIRED}' description requires " + "development_status 'needs_review'" + ) + + return problems + + +# -------------------------------------------------------------------------- +# Pytest collection +# -------------------------------------------------------------------------- + + +def collect_pytest_nodes(repo_root=REPO_ROOT): + """Return the sorted node IDs pytest currently collects.""" + completed = subprocess.run( + [ + sys.executable, + "-m", + "pytest", + "--collect-only", + "-q", + "--no-header", + "-p", + "no:cacheprovider", + ], + cwd=repo_root, + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + if completed.returncode != 0: + raise ManifestError( + "pytest collection failed with exit code " + f"{completed.returncode}:\n{completed.stdout}\n{completed.stderr}" + ) + nodes = [ + line.strip() + for line in completed.stdout.splitlines() + if "::" in line and not line.startswith(("=", "_", " ")) + ] + return sorted(set(nodes)) + + +def files_from_nodes(nodes): + """Map collected node IDs to their sorted set of source files.""" + return sorted({node.split("::", 1)[0] for node in nodes}) + + +def cross_check(entries, collected_files, repo_root=REPO_ROOT): + """Compare manifest entries against actual pytest collection. + + Returns a list of human-readable problems: collected tests with no + manifest entry, manifest entries claiming collection that pytest does not + collect, and test files on disk that the manifest does not document. + """ + problems = [] + collected = set(collected_files) + by_path = {entry["path"]: entry for entry in entries if entry.get("path")} + + for path in sorted(collected - set(by_path)): + problems.append(f"{path}: collected by pytest but missing from the manifest") + + for path in sorted(by_path): + entry = by_path[path] + if entry.get("collection_status") == "collected" and path not in collected: + problems.append( + f"{path}: manifest says collection_status 'collected' but pytest " + "collects no tests from it (stale entry)" + ) + if entry.get("collection_status") != "collected" and path in collected: + problems.append( + f"{path}: pytest collects tests from it but the manifest says " + f"collection_status '{entry.get('collection_status')}'" + ) + + tests_dir = Path(repo_root) / "tests" + on_disk = { + p.relative_to(repo_root).as_posix() + for p in tests_dir.rglob("*.py") + if "__pycache__" not in p.parts + } + for path in sorted(on_disk - set(by_path) - NON_TEST_FILES - collected): + problems.append(f"{path}: test file on disk is missing from the manifest") + + return problems + + +# -------------------------------------------------------------------------- +# tests/README.md rendering +# -------------------------------------------------------------------------- + + +def _cell(value, default="—"): + """Collapse a manifest value into a single markdown table cell.""" + if value is None: + return default + text = " ".join(str(value).split()) + return text.replace("|", "\\|") if text else default + + +def _code(value, default="—"): + text = _cell(value, default) + return f"`{text}`" if text != default else default + + +def _sorted(entries): + return sorted( + entries, + key=lambda e: ( + SUBSYSTEMS.index(e["subsystem"]), + CATEGORIES.index(e["category"]), + e["path"], + ), + ) + + +def _table(lines, header, rows): + lines.append("| " + " | ".join(header) + " |") + lines.append("|" + "|".join(" --- " for _ in header) + "|") + lines.extend("| " + " | ".join(row) + " |" for row in rows) + lines.append("") + + +def render_readme(entries, collected_nodes=()): + """Render the full text of tests/README.md. + + Deterministic: the output depends only on the manifest contents and the + set of collected node IDs, never on timestamps or run outcomes. + """ + entries = _sorted(entries) + node_counts = {} + for node in collected_nodes: + path = node.split("::", 1)[0] + node_counts[path] = node_counts.get(path, 0) + 1 + + automated = [e for e in entries if e["collection_status"] == "collected"] + manual = [e for e in entries if e["collection_status"] == "manual"] + ignored = [e for e in entries if e["collection_status"] == "ignored"] + archived = [e for e in entries if e["collection_status"] == "archived"] + placeholders = [e for e in entries if e["development_status"] == "placeholder"] + blocked = [e for e in entries if e["development_status"] == "blocked"] + needs_review = [e for e in entries if e["development_status"] == "needs_review"] + + lines = [AUTOGEN_HEADER, "", "# PyRPOD Test Inventory", ""] + lines += [ + "This inventory documents every test asset in `tests/`: what it is for,", + "which subsystem and category it belongs to, how it is executed, and how", + "far its development has progressed.", + "", + "**It deliberately records no pass/fail results.** Execution outcomes belong", + "to a specific run and are published in the pytest HTML report", + "(`reports/pyrpod-pytest-report.html`); development status is long-lived", + "metadata that a green test run does not change.", + "", + "Test files follow the `__test_NN.py` naming convention,", + "and `tests/conftest.py` uses it to tag every collected test with a subsystem", + "marker (`logging`, `mdao`, `mission`, `plume`, `rpod`, `tooling`) and a", + "category marker (`unit`, `integration`, `verification`).", + "", + "Regenerate this file and the HTML report with:", + "", + "```bash", + "python scripts/generate_test_dashboard.py", + "```", + "", + "Source of truth for the metadata below: " + "[`test_manifest.yaml`](test_manifest.yaml).", + "", + "---", + "", + "## Summary", + "", + ] + _table( + lines, + ["Metric", "Count"], + [ + ["Manifest entries", str(len(entries))], + ["Collected by pytest (files)", str(len(automated))], + ["Collected by pytest (test cases)", str(sum(node_counts.values()))], + ["Manual verification scripts", str(len(manual))], + ["Placeholder tests", str(len(placeholders))], + ["Blocked tests", str(len(blocked))], + ["Archived / legacy tests", str(len(archived))], + ["Needs review", str(len(needs_review))], + ], + ) + + lines += [ + "---", + "", + "## Automated pytest tests", + "", + "Files pytest collects and runs. The **Cases** column is the number of test", + "cases pytest currently collects from the file (parametrized variants count", + "separately).", + "", + ] + for subsystem in SUBSYSTEMS: + subset = [e for e in automated if e["subsystem"] == subsystem] + if not subset: + continue + lines += [f"### {SUBSYSTEM_TITLES[subsystem]}", ""] + for category in CATEGORIES: + rows = [e for e in subset if e["category"] == category] + if not rows: + continue + lines += [f"#### {category.capitalize()}", ""] + _table( + lines, + [ + "Test file", + "Cases", + "Description", + "Development status", + "Reference", + ], + [ + [ + f"[`{Path(e['path']).name}`]({_relative(e['path'])})", + str(node_counts.get(e["path"], 0)), + _cell(e["description"]), + f"`{e['development_status']}`", + _cell(e.get("reference")), + ] + for e in rows + ], + ) + + lines += [ + "---", + "", + "## Manual verification scripts", + "", + "Run directly from the repository root; they are **not** pytest tests and", + "never appear in the HTML execution report. Most reproduce a published", + "figure and write a PNG under `tests/plume/output/`.", + "", + ] + for subsystem in SUBSYSTEMS: + rows = [e for e in manual if e["subsystem"] == subsystem] + if not rows: + continue + lines += [f"### {SUBSYSTEM_TITLES[subsystem]}", ""] + _table( + lines, + ["Script", "Description", "Development status", "Command", "Reference"], + [ + [ + f"[`{Path(e['path']).name}`]({_relative(e['path'])})", + _cell(e["description"]), + f"`{e['development_status']}`", + _code(e.get("manual_command")), + _cell(e.get("reference")), + ] + for e in rows + ], + ) + + lines += [ + "---", + "", + "## Placeholder tests", + "", + "Collected by pytest but containing no assertion or verification behavior -", + "typically an empty body or a fully commented-out one. Each is explicitly", + "skipped in its source file so that the HTML report shows it as **skipped**", + "rather than passed; a placeholder must never be mistaken for coverage.", + "", + ] + _table( + lines, + ["Test file", "Subsystem", "Category", "Description"], + [ + [ + f"[`{Path(e['path']).name}`]({_relative(e['path'])})", + e["subsystem"], + e["category"], + _cell(e["description"]), + ] + for e in placeholders + ], + ) + + lines += [ + "---", + "", + "## Ignored or blocked tests", + "", + "Excluded from pytest collection (see `collect_ignore` in", + "[`conftest.py`](conftest.py)) or otherwise unable to run.", + "", + ] + _table( + lines, + [ + "Test file", + "Subsystem", + "Development status", + "Collection status", + "Reason", + "Command", + ], + [ + [ + f"[`{Path(e['path']).name}`]({_relative(e['path'])})", + e["subsystem"], + f"`{e['development_status']}`", + f"`{e['collection_status']}`", + _cell(e.get("collection_ignore_reason")), + _code(e.get("manual_command")), + ] + for e in ignored + + [b for b in blocked if b["collection_status"] != "ignored"] + ], + ) + + lines += [ + "---", + "", + "## Archived or legacy tests", + "", + "Kept for historical reference under `tests/old/`. They are not repaired,", + "renamed, or deleted as part of routine work.", + "", + ] + _table( + lines, + ["Test file", "Subsystem", "Category", "Description", "Reason"], + [ + [ + f"[`{e['path'].split('tests/', 1)[-1]}`]({_relative(e['path'])})", + e["subsystem"], + e["category"], + _cell(e["description"]), + _cell(e.get("collection_ignore_reason")), + ] + for e in archived + ], + ) + + lines += [ + "---", + "", + "## Legend", + "", + "Three independent axes. A test can be `implemented` and still fail today;", + "a `placeholder` can be green in CI only because it is skipped.", + "", + "**Execution outcome** — owned by pytest, one value per run, published only", + "in `reports/pyrpod-pytest-report.html`:", + "", + "| Outcome | Meaning |", + "| --- | --- |", + "| passed | The test ran and every assertion held. |", + "| failed | The test ran and an assertion or the code under test failed. |", + "| error | The test could not run to completion (setup or teardown raised). |", + "| skipped | The test was not executed (placeholder, or a skip condition). |", + "", + "**Development status** — maintained in `test_manifest.yaml`, long-lived:", + "", + "| Status | Meaning |", + "| --- | --- |", + "| `implemented` | Complete: exercises the code and asserts a result. |", + "| `placeholder` | No assertion or verification behavior yet; skipped. |", + "| `needs_review` | Runs real code but asserts nothing, or its purpose " + "is insufficiently documented. |", + "| `blocked` | Cannot run against the current architecture. |", + "| `archived` | Superseded; kept for historical reference only. |", + "| `deprecated` | Slated for removal. |", + "", + "**Collection status** — whether pytest picks the file up:", + "", + "| Status | Meaning |", + "| --- | --- |", + "| `collected` | Collected and run by pytest; appears in the HTML report. |", + "| `manual` | Run by hand; defines no pytest tests. |", + "| `ignored` | Listed in `collect_ignore` in `conftest.py`. |", + "| `archived` | Under `tests/old/`, excluded from collection wholesale. |", + "", + ] + + return "\n".join(lines).rstrip("\n") + "\n" + + +def _relative(path): + """Manifest paths are repo-relative; README links are tests/-relative.""" + return path.split("tests/", 1)[-1] if path.startswith("tests/") else path + + +# -------------------------------------------------------------------------- +# Entry point +# -------------------------------------------------------------------------- + + +def _report_problems(title, problems): + print(f"\n{title}:", file=sys.stderr) + for problem in problems: + print(f" - {problem}", file=sys.stderr) + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument( + "--inventory-only", + action="store_true", + help="validate the manifest and regenerate tests/README.md without " + "running the test suite", + ) + parser.add_argument( + "--check", + action="store_true", + help="fail instead of writing if tests/README.md is out of date " + "(implies --inventory-only)", + ) + args = parser.parse_args(argv) + + try: + entries = load_manifest() + except ManifestError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + problems = validate_manifest(entries) + if problems: + _report_problems(f"error: {MANIFEST_PATH.name} failed validation", problems) + return 2 + print(f"manifest OK ({len(entries)} entries)") + + print("collecting pytest tests ...") + try: + nodes = collect_pytest_nodes() + except ManifestError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + collected_files = files_from_nodes(nodes) + print(f"collected {len(nodes)} test cases in {len(collected_files)} files") + + problems = cross_check(entries, collected_files) + if problems: + _report_problems("error: manifest is out of sync with pytest", problems) + return 2 + + readme = render_readme(entries, nodes) + if args.check: + current = ( + README_PATH.read_text(encoding="utf-8") if README_PATH.is_file() else "" + ) + if current != readme: + print( + f"error: {README_PATH.relative_to(REPO_ROOT)} is out of date; " + "rerun python scripts/generate_test_dashboard.py", + file=sys.stderr, + ) + return 2 + print(f"{README_PATH.relative_to(REPO_ROOT)} is up to date") + return 0 + + README_PATH.write_text(readme, encoding="utf-8", newline="\n") + print(f"wrote {README_PATH.relative_to(REPO_ROOT)}") + + if args.inventory_only: + return 0 + + REPORTS_DIR.mkdir(parents=True, exist_ok=True) + print(f"running pytest -> {REPORT_PATH.relative_to(REPO_ROOT)}") + completed = subprocess.run( + [ + sys.executable, + "-m", + "pytest", + f"--html={REPORT_PATH}", + "--self-contained-html", + ], + cwd=REPO_ROOT, + ) + if REPORT_PATH.is_file(): + print(f"report written: {REPORT_PATH.relative_to(REPO_ROOT)}") + else: + print( + "warning: pytest did not produce the HTML report; is pytest-html " + "installed?", + file=sys.stderr, + ) + return completed.returncode + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/README.md b/tests/README.md index ffcecf1..0ff3075 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,95 +1,332 @@ -# PyRPOD Test Dashboard + -This dashboard provides an overview of all tests in the PyRPOD project, categorized by their respective modules. Each test is listed with its type, description, and current status. +# PyRPOD Test Inventory + +This inventory documents every test asset in `tests/`: what it is for, +which subsystem and category it belongs to, how it is executed, and how +far its development has progressed. + +**It deliberately records no pass/fail results.** Execution outcomes belong +to a specific run and are published in the pytest HTML report +(`reports/pyrpod-pytest-report.html`); development status is long-lived +metadata that a green test run does not change. + +Test files follow the `__test_NN.py` naming convention, +and `tests/conftest.py` uses it to tag every collected test with a subsystem +marker (`logging`, `mdao`, `mission`, `plume`, `rpod`, `tooling`) and a +category marker (`unit`, `integration`, `verification`). + +Regenerate this file and the HTML report with: + +```bash +python scripts/generate_test_dashboard.py +``` + +Source of truth for the metadata below: [`test_manifest.yaml`](test_manifest.yaml). + +--- + +## Summary + +| Metric | Count | +| --- | --- | +| Manifest entries | 93 | +| Collected by pytest (files) | 44 | +| Collected by pytest (test cases) | 204 | +| Manual verification scripts | 42 | +| Placeholder tests | 12 | +| Blocked tests | 5 | +| Archived / legacy tests | 5 | +| Needs review | 10 | --- -## **MDAO Tests** -| Test Name | Type | Description | Status | -|---------------------------------|----------------|-----------------------------------------------|--------| -| `mdao_unit_test_01.py` | Unit | empty. | ❌ | -| `mdao_unit_test_02.py` | Unit | Generate RCS configs by sweeping cant angle. | ⏳ | -| `mdao_integration_test_01.py` | Integration | empty. | ❌ | -| `mdao_verification_test_01.py` | Verification | Minimizes heat flux by varying axial position of RCS pack. | ⏳ | -| `mdao_verification_test_02.py` | Verification | Minimizes heat flux by varying cant angle of RCS pack. | ⏳ | -| `mdao_verification_test_03.py` | Verification | Evaluates RCS performance by varying cant angle and axial position. | ⏳ | -| `mdao_verification_test_04.py` | Verification | Evaluates RCS performance in axial-overshoot study. | ⏳ | -| `mdao_verification_test_05.py` | Verification | ????? no idea. | ❌ | -| `mdao_verification_test_06.py` | Verification | axial over shoot? | ❌ | +## Automated pytest tests + +Files pytest collects and runs. The **Cases** column is the number of test +cases pytest currently collects from the file (parametrized variants count +separately). + +### Logging + +#### Unit + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`logging_unit_test_01.py`](logging/logging_unit_test_01.py) | 14 | Unit tests for the centralized operational logging system (pyrpod.logging_utils): import side effects, handler ownership, configuration precedence, console/file toggles, runtime-log naming and location, configuration snapshots, input-asset logging and array summaries. | `implemented` | — | + +#### Integration + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`logging_integration_test_01.py`](logging/logging_integration_test_01.py) | 13 | Integration tests for operational logging of the plume-strike workflow: fail-fast validation of required inputs, progress cadence, serial/parallel event equivalence, DEBUG-vs-INFO artifact levels, parallel-to-serial fallback with successful_with_warning status and optional-visualization warn-and-continue behavior. | `implemented` | — | + +### MDAO + +#### Unit + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`mdao_unit_test_01.py`](mdao/mdao_unit_test_01.py) | 1 | Placeholder MDAO unit test; the test body returns immediately and asserts nothing. | `placeholder` | — | +| [`mdao_unit_test_02.py`](mdao/mdao_unit_test_02.py) | 1 | Intended to build an array of cant-angle-swept thruster configurations (symmetric pitch/yaw canting) and visualize each sweep step. The entire body is currently commented out, so the test asserts nothing. | `placeholder` | — | + +#### Integration + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`mdao_integration_test_01.py`](mdao/mdao_integration_test_01.py) | 1 | Placeholder MDAO integration test; the test body returns immediately and asserts nothing. | `placeholder` | — | + +#### Verification + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`mdao_verification_test_04.py`](mdao/mdao_verification_test_04.py) | 1 | Intended variable-sweep study of the maximum overshoot velocity a logistics module can absorb with a fixed thruster configuration and deceleration start distance. The entire body is commented out, so the test asserts nothing. | `placeholder` | — | +| [`mdao_verification_test_05.py`](mdao/mdao_verification_test_05.py) | 1 | Intended trade study sweeping the surface cant angle of the RCS pack. The entire body is commented out, so the test asserts nothing. | `placeholder` | — | +| [`mdao_verification_test_06.py`](mdao/mdao_verification_test_06.py) | 1 | Intended multi-variable trade study sweeping axial overshoot together with surface cant angle. The entire body is commented out, so the test asserts nothing. | `placeholder` | — | + +### Mission + +#### Unit + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`mission_unit_test_01.py`](mission/mission_unit_test_01.py) | 1 | Placeholder mission unit test; the test body returns immediately and asserts nothing. | `placeholder` | — | + +#### Integration + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`mission_integration_test_01.py`](mission/mission_integration_test_01.py) | 1 | Calculates the 6DOF performance (normal vector, force, translational acceleration, torque) of each individual thruster in the logistics module. Expected-value helpers are present but no assertion is made against them, so the test only exercises the code path. | `needs_review` | — | +| [`mission_integration_test_02.py`](mission/mission_integration_test_02.py) | 1 | Intended to calculate the 6DOF performance of thruster working groups. The body only assigns mass-distribution constants; the header records that the analysis was disabled because it created unwanted data files, so the test asserts nothing. | `placeholder` | — | +| [`mission_integration_test_03.py`](mission/mission_integration_test_03.py) | 1 | Calculates RCS performance for a given flight plan, approximating the 1D delta-v requirements. Exercises the flight-evaluation pipeline end to end but makes no assertion. | `needs_review` | — | +| [`mission_integration_test_04.py`](mission/mission_integration_test_04.py) | 1 | Analyzes a notional 1D translation-plus-rotation approach (module header marks it "NEEDS TLC"). Exercises the flight-evaluation pipeline end to end but makes no assertion. | `needs_review` | — | +| [`mission_integration_test_05.py`](mission/mission_integration_test_05.py) | 1 | Graphs thrust versus time or distance for the given design requirements to establish an RCS thrust-requirement flight envelope. Exercises the plotting pipeline but makes no assertion. | `needs_review` | — | +| [`mission_integration_test_06.py`](mission/mission_integration_test_06.py) | 1 | Graphs propellant-mass requirements across the flight envelope for the given design requirements. Exercises the plotting pipeline but makes no assertion. | `needs_review` | — | +| [`mission_integration_test_07.py`](mission/mission_integration_test_07.py) | 1 | Contours the burn-time plot across a range of thrust and Isp values (module header marks it "NEEDS TLC"). Exercises the plotting pipeline but makes no assertion. | `needs_review` | — | +| [`mission_integration_test_08.py`](mission/mission_integration_test_08.py) | 1 | Contours propellant usage across the various delta-v legs of a given flight plan. Exercises the plotting pipeline but makes no assertion. | `needs_review` | — | + +#### Verification + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`mission_verification_test_01.py`](mission/mission_verification_test_01.py) | 1 | Placeholder mission verification test; the test body returns immediately and asserts nothing. | `placeholder` | — | +| [`mission_verification_test_02.py`](mission/mission_verification_test_02.py) | 1 | Builds and summarizes a chain of Hohmann transfers (LEO to MEO to GEO) through MissionPlanner.orbital_transfer. Exercises the orbital transfer path but makes no assertion. | `needs_review` | — | + +### Plume + +#### Unit + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`plume_unit_test_01.py`](plume/plume_unit_test_01.py) | 1 | Placeholder plume unit test; the test body returns immediately and asserts nothing. | `placeholder` | — | +| [`plume_unit_test_02.py`](plume/plume_unit_test_02.py) | 9 | Verifies the closed form of the special factor Q against a direct 50-term truncation of the printed Legendre series, its centerline reduction, and the 0 < Q <= 1 bound that keeps the exp(-S0^2 (1-Q)) combination overflow-safe. | `implemented` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Eq. 9 | +| [`plume_unit_test_03.py`](plume/plume_unit_test_03.py) | 13 | Tests the Simons cosine-law plume model after the gamma generalization: isentropic throat ratios from gamma, zero density beyond the limiting angle, parameterizable beaming exponent kappa and exit-referenced density scaling. | `implemented` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Sec. II.C, Eqs. 25-26 | +| [`plume_unit_test_04.py`](plume/plume_unit_test_04.py) | 3 | Asserts that the NumPy-vectorized plume-strike detection in compute_plume_strikes() reproduces the scalar reference implementation exactly for every firing of case/rpod/1d_approach (geometry only) and case/rpod/multi_thrusters_square (Simplified kinetics, multiple thrusters). | `implemented` | — | + +#### Integration + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`plume_integration_test_01.py`](plume/plume_integration_test_01.py) | 1 | Placeholder plume integration test; the test body returns immediately and asserts nothing. | `placeholder` | — | +| [`plume_integration_test_02.py`](plume/plume_integration_test_02.py) | 3 | Asserts that jfh_plume_strikes() keeps its default serial, return-compatible behavior, produces identical per-firing and cumulative output when the optional process-parallel path is enabled, and rejects invalid worker counts with a clear error. | `implemented` | — | + +#### Verification + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`plume_verification_test_01.py`](plume/plume_verification_test_01.py) | 1 | Intended to plot simple isentropic radial-expansion profiles. Both plotting calls are commented out, leaving only construction of an IsentropicExpansion object, so the test asserts nothing. | `placeholder` | — | +| [`plume_verification_test_02.py`](plume/plume_verification_test_02.py) | 23 | Pinning tests for SimplifiedGasKinetics against reference values computed independently from the paper's equations with mpmath at 40 significant digits, plus far-field asymptote convergence and the near-field divergence of the corrected Eq. 21 centerline temperature quadrature. | `implemented` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Eqs. 13-19, 22-24 | +| [`plume_verification_test_03.py`](plume/plume_verification_test_03.py) | 27 | Verification of the full collisionless model CollisionlessGasKinetics against the paper's internal anchors: centerline reduction to the Eq. 18/19 closed forms, vanishing W, the Eq. 30 far-field error bound, monotonic centerline density decay and asymptote convergence. | `implemented` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Eqs. 5-12, 20, 30 | + +### RPOD + +#### Unit + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`rpod_unit_test_01.py`](rpod/rpod_unit_test_01.py) | 1 | Converts cylinder STL data to VTK and asserts the resulting file has the proper VTK data format, cell counts and point counts. | `implemented` | — | +| [`rpod_unit_test_02.py`](rpod/rpod_unit_test_02.py) | 1 | Reads a jet firing history and asserts that all required keys exist and that every value has the expected data type. | `implemented` | — | +| [`rpod_unit_test_03.py`](rpod/rpod_unit_test_03.py) | 3 | Captures the exact file output of the three JFH printing helpers in pyrpod.util.io.file_print to a temporary directory and compares it against the committed tests/rpod/jfh_outputs snapshot. | `implemented` | — | +| [`rpod_unit_test_04.py`](rpod/rpod_unit_test_04.py) | 31 | Unit tests for the PR #116 plume-mesh consolidation: VisitingVehicle.transform_plume_mesh, the cached thruster-id map, multi-digit cluster ids, stl.compose_meshes and the reusable stl.transform_mesh mesh-object API, pinned against hand-computed golden values. | `implemented` | — | + +#### Integration + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`rpod_integration_test_01.py`](rpod/rpod_integration_test_01.py) | 1 | Asserts the expected number of cell strikes on a flat-plate STL for a notional "sweep" trajectory above it across 20 distinct firings. This is the established base case for RPOD plume-impingement analysis. | `implemented` | — | +| [`rpod_integration_test_02.py`](rpod/rpod_integration_test_02.py) | 1 | Asserts the expected number of cell strikes on a flat-plate STL as a notional visiting vehicle approaches it along a 1D-physics trajectory, across 15 distinct firings. | `implemented` | — | +| [`rpod_integration_test_03.py`](rpod/rpod_integration_test_03.py) | 1 | Analyzes keep-out-zone impingement and asserts the expected strike counts against the committed expected-strikes fixture (module header marks the case "WIP"). | `implemented` | — | +| [`rpod_integration_test_04.py`](rpod/rpod_integration_test_04.py) | 1 | Produces hollow-cube target data and asserts the expected strike counts against the committed expected-strikes fixture. | `implemented` | — | +| [`rpod_integration_test_05.py`](rpod/rpod_integration_test_05.py) | 1 | Runs the plume gas-kinetic models through a JFH with multiple thrusters per firing (case/rpod/multi_thrusters_square). Exercises graph_jfh and jfh_plume_strikes end to end but makes no assertion. | `needs_review` | — | +| [`rpod_integration_test_06.py`](rpod/rpod_integration_test_06.py) | 3 | Performance benchmark for plume-strike computation (scalar vs NumPy-vectorized geometry, serial vs process-parallel). Elapsed times are printed but no speedup threshold is asserted; each benchmark asserts only that the compared paths produce identical strike arrays. | `implemented` | — | +| [`rpod_integration_test_07.py`](rpod/rpod_integration_test_07.py) | 1 | Runs the Cai 2016 inclined-plate case through PlumeStrikeEstimationStudy, converts per-face dimensional loads to the paper's coefficient normalization and compares them face by face against the exact reference functions evaluated at face centroids. | `implemented` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Eqs. 9-13 | +| [`rpod_integration_test_08.py`](rpod/rpod_integration_test_08.py) | 4 | End-to-end geometry-equivalence regressions for the PR #116 plume-mesh consolidation: pins the visualization pipeline's produced geometry, artifact counts and file names against an inline reproduction of the legacy transform/compose sequence, and proves the thruster transform is applied exactly once. | `implemented` | — | + +#### Verification + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`rpod_verification_test_04.py`](rpod/rpod_verification_test_04.py) | 1 | Visualizes STLs after decoupling the thruster configuration data and is intended to verify that plume strikes line up with the thrusters. Exercises graph_jfh and jfh_plume_strikes but makes no assertion, so the alignment must still be checked by eye in ParaView. | `needs_review` | — | +| [`rpod_verification_test_06.py`](rpod/rpod_verification_test_06.py) | 1 | Cai 2016 flat-plate SWEEP verification: runs the 95-firing sweep JFH (19 approach angles x 5 stand-off distances) through the strike pipeline, reduces each pose to the Eq.-15 plate-averaged coefficients, checks mirror symmetry in +/- alpha and compares against the exact reference envelope. | `implemented` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Eq. 15 | +| [`rpod_verification_test_07.py`](rpod/rpod_verification_test_07.py) | 1 | Cylinder-target sweep smoke test: exercises the full PlumeStrikeEstimationStudy pipeline on a curved, closed target (14036-face cylinder) over 19 approach angles x 5 orbit radii. | `implemented` | — | + +### Tooling + +#### Unit + +| Test file | Cases | Description | Development status | Reference | +| --- | --- | --- | --- | --- | +| [`tooling_unit_test_01.py`](tooling/tooling_unit_test_01.py) | 27 | Unit tests for the test-inventory tooling itself: manifest schema validation, detection of missing and stale entries, and deterministic rendering of the generated tests/README.md. | `implemented` | — | --- -## **Mission Tests** -| Test Name | Type | Description | Status | -|---------------------------------|----------------|-----------------------------------------------|--------| -| `mission_integration_test_01.py`| Integration | Assesrts 6DOF performance of individual thrusters. | ⏳ | -| `mission_integration_test_02.py`| Integration | Assesrts 6DOF performance of thruster groups. | ⏳ | -| `mission_integration_test_03.py`| Integration | Assersts Δv requirements for an RCS system (1D). | ⏳ | -| `mission_integration_test_04.py`| Integration | Assersts Δv requirements for an RCS system (2D). | ⏳ | -| `mission_integration_test_05.py`| Integration | Assersts thrust requirements for an RCS system. | ⏳ | -| `mission_integration_test_06.py`| Integration | Assersts mass requirements for an RCS system. | ⏳ | -| `mission_integration_test_07.py`| Integration | Contours RCS performance (thrust vs ISP). | ⏳ | -| `mission_integration_test_08.py`| Integration | Contours propellant usage for all Δv in a flight plan. | ⏳ | -| `mission_unit_test_01.py` | Unit | empty. | ❌ | -| `mission_verification_test_01.py`| Verification | empty. | ❌ | +## Manual verification scripts + +Run directly from the repository root; they are **not** pytest tests and +never appear in the HTML execution report. Most reproduce a published +figure and write a PNG under `tests/plume/output/`. + +### MDAO + +| Script | Description | Development status | Command | Reference | +| --- | --- | --- | --- | --- | +| [`mdao_verification_test_01.py`](mdao/mdao_verification_test_01.py) | OpenMDAO axial-positioning optimizer: minimizes the maximum heat-flux load for a 1D approach by varying the axial position of the thruster packs. | `blocked` | `python tests/mdao/mdao_verification_test_01.py` | — | +| [`mdao_verification_test_02.py`](mdao/mdao_verification_test_02.py) | OpenMDAO cant-angle optimizer: minimizes the maximum heat-flux load for a 1D approach by varying the cant angle of the deceleration thrusters. | `blocked` | `python tests/mdao/mdao_verification_test_02.py` | — | +| [`mdao_verification_test_03.py`](mdao/mdao_verification_test_03.py) | OpenMDAO axial-plus-cant evaluator: reports maximum cumulative heat-flux load and JFH propellant expenditure for a 1D approach given the axial position and cant angle of the deceleration thrusters. | `blocked` | `python tests/mdao/mdao_verification_test_03.py` | — | + +### Plume + +| Script | Description | Development status | Command | Reference | +| --- | --- | --- | --- | --- | +| [`plume_impingement_error_summary.py`](plume/plume_impingement_error_summary.py) | Maximum and mean relative differences between the exact Cai 2016 reference surface coefficients (Eqs. 9-14) and the current PyRPOD approximation chain for the inclined-plate impingement study. | `implemented` | `python tests/plume/plume_impingement_error_summary.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Figs. 17-21 conditions | +| [`plume_verification_error_summary.py`](plume/plume_verification_error_summary.py) | Model-vs-model analog of Table 1: maximum relative differences in density and mass flux between the analytical, simplified and Simons models along the centerline and along r/D = 10 at S0 = 2.0. The DSMC reference columns are emitted as "pending digitized data". | `implemented` | `python tests/plume/plume_verification_error_summary.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Table 1 | +| [`plume_verification_test_04.py`](plume/plume_verification_test_04.py) | Reproduces Fig. 2: plume boundaries (the n/n0 = 0.001 contour of the full analytical model) for exit speed ratios S0 = 1, 2, 3. | `implemented` | `python tests/plume/plume_verification_test_04.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 2 | +| [`plume_verification_test_05.py`](plume/plume_verification_test_05.py) | Reproduces Fig. 3: normalized analytical centerline number density (Eq. 18) versus X/D for S0 = 1, 2, 3. | `implemented` | `python tests/plume/plume_verification_test_05.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 3 | +| [`plume_verification_test_06.py`](plume/plume_verification_test_06.py) | Reproduces Fig. 4: normalized analytical centerline U-velocity (Eq. 19) versus X/D for S0 = 1, 2, 3. | `implemented` | `python tests/plume/plume_verification_test_06.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 4 | +| [`plume_verification_test_07.py`](plume/plume_verification_test_07.py) | Reproduces Fig. 5: normalized analytical centerline temperature (Eq. 21, exact quadrature) versus X/D for S0 = 1, 2, 3. | `implemented` | `python tests/plume/plume_verification_test_07.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 5 | +| [`plume_verification_test_08.py`](plume/plume_verification_test_08.py) | Reproduces Fig. 6: normalized number-density contours at Kn = 100, S0 = 2.0; analytical plus simplified in the upper half-plane, Simons plus the DSMC overlay slot in the lower half-plane. | `implemented` | `python tests/plume/plume_verification_test_08.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 6 | +| [`plume_verification_test_09.py`](plume/plume_verification_test_09.py) | Reproduces Fig. 7: normalized number-density contours at Kn = 0.1, S0 = 2.0. Same analytic content as Fig. 6; the Kn distinction lives in the DSMC overlay slot. | `implemented` | `python tests/plume/plume_verification_test_09.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 7 | +| [`plume_verification_test_10.py`](plume/plume_verification_test_10.py) | Reproduces Fig. 8: normalized number-density contours at Kn = 0.01, S0 = 2.0. Same analytic content as Fig. 6; the Kn distinction lives in the DSMC overlay slot. | `implemented` | `python tests/plume/plume_verification_test_10.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 8 | +| [`plume_verification_test_11.py`](plume/plume_verification_test_11.py) | Reproduces Fig. 9: relative density error between the analytical and Simons cosine-law solutions, \|n_A/n_Simons - 1\| (Eq. 28), S0 = 2.0. | `implemented` | `python tests/plume/plume_verification_test_11.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 9 | +| [`plume_verification_test_12.py`](plume/plume_verification_test_12.py) | Reproduces Fig. 10: relative density error between the analytical and simplified analytical solutions, \|n_A/n_As - 1\| (Eq. 29), S0 = 2.0. | `implemented` | `python tests/plume/plume_verification_test_12.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 10 | +| [`plume_verification_test_13.py`](plume/plume_verification_test_13.py) | Reproduces Fig. 11: normalized pressure contours p1/p0 = (n1/n0)(T1/T0) at Kn = 100, S0 = 2.0. | `implemented` | `python tests/plume/plume_verification_test_13.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 11 | +| [`plume_verification_test_14.py`](plume/plume_verification_test_14.py) | Reproduces Fig. 12: normalized pressure contours p1/p0 at Kn = 0.1, S0 = 2.0. Same analytic content as Fig. 11. | `implemented` | `python tests/plume/plume_verification_test_14.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 12 | +| [`plume_verification_test_15.py`](plume/plume_verification_test_15.py) | Reproduces Fig. 13: normalized temperature contours T1/T0 at Kn = 100, S0 = 2.0 - the figure the paper uses to argue that p = n k T0 is invalid because T1 < T0 everywhere downstream. | `implemented` | `python tests/plume/plume_verification_test_15.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 13 | +| [`plume_verification_test_16.py`](plume/plume_verification_test_16.py) | Reproduces Fig. 14: normalized U-velocity contours U1*sqrt(beta0) at Kn = 100, S0 = 2.0. | `implemented` | `python tests/plume/plume_verification_test_16.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 14 | +| [`plume_verification_test_17.py`](plume/plume_verification_test_17.py) | Reproduces Fig. 15: normalized transverse-velocity contours at Kn = 100, S0 = 2.0. In the plotted XOZ plane the y-component is zero by axisymmetry, so the model's W (Eq. 7) is plotted. | `implemented` | `python tests/plume/plume_verification_test_17.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 15 | +| [`plume_verification_test_18.py`](plume/plume_verification_test_18.py) | Reproduces Fig. 16: normalized radial-velocity Vr contours at Kn = 100, S0 = 2.0. | `implemented` | `python tests/plume/plume_verification_test_18.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 16 | +| [`plume_verification_test_19.py`](plume/plume_verification_test_19.py) | Reproduces Fig. 17: normalized radial-velocity Vr contours at Kn = 0.1, S0 = 2.0. Same analytic content as Fig. 16. | `implemented` | `python tests/plume/plume_verification_test_19.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 17 | +| [`plume_verification_test_20.py`](plume/plume_verification_test_20.py) | Reproduces Fig. 18: normalized radial-velocity Vr contours at Kn = 0.01, S0 = 2.0. Same analytic content as Fig. 16. | `implemented` | `python tests/plume/plume_verification_test_20.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 18 | +| [`plume_verification_test_21.py`](plume/plume_verification_test_21.py) | Reproduces Fig. 19: centerline density profiles at Kn = 100, S0 = 2.0 - analytical (Eq. 18), simplified (Eq. 14), Simons (exit-referenced cosine law) and the DSMC overlay slot. | `implemented` | `python tests/plume/plume_verification_test_21.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 19 | +| [`plume_verification_test_22.py`](plume/plume_verification_test_22.py) | Reproduces Fig. 20: centerline density profiles at Kn = 0.1, S0 = 2.0. Same analytic content as Fig. 19. | `implemented` | `python tests/plume/plume_verification_test_22.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 20 | +| [`plume_verification_test_23.py`](plume/plume_verification_test_23.py) | Reproduces Fig. 21: centerline density profiles at Kn = 0.01, S0 = 2.0. Same analytic content as Fig. 19. | `implemented` | `python tests/plume/plume_verification_test_23.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 21 | +| [`plume_verification_test_24.py`](plume/plume_verification_test_24.py) | Reproduces Fig. 22: density profiles along r/D = 10 at Kn = 100, S0 = 2.0 - analytical, simplified, Simons with kappa = 1.5/2/3 under a single Boyton normalization, and the DSMC overlay slot. | `implemented` | `python tests/plume/plume_verification_test_24.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 22 | +| [`plume_verification_test_25.py`](plume/plume_verification_test_25.py) | Reproduces Fig. 23: density profiles along r/D = 10 at Kn = 0.1, S0 = 2.0. Same analytic content as Fig. 22. | `implemented` | `python tests/plume/plume_verification_test_25.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 23 | +| [`plume_verification_test_26.py`](plume/plume_verification_test_26.py) | Reproduces Fig. 24: density profiles along r/D = 10 at Kn = 0.01, S0 = 2.0. Same analytic content as Fig. 22. | `implemented` | `python tests/plume/plume_verification_test_26.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 24 | +| [`plume_verification_test_27.py`](plume/plume_verification_test_27.py) | Reproduces Fig. 25: normalized mass flux along r/D = 10 versus theta at S0 = 2.0, together with three DSMC overlay slots. The module header records the flux-normalization convention adopted to match the printed magnitudes. | `implemented` | `python tests/plume/plume_verification_test_27.py` | Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 25 | +| [`plume_verification_test_28.py`](plume/plume_verification_test_28.py) | Reproduces Fig. 17: diffuse-plate surface pressure contours Cp,d(s, tau) for the round argon jet on the inclined plate (Eq. 9), overlaid with the current PyRPOD approximation. | `implemented` | `python tests/plume/plume_verification_test_28.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 17 | +| [`plume_verification_test_29.py`](plume/plume_verification_test_29.py) | Reproduces Fig. 18: specular-plate surface pressure contours Cp,s(s, tau) (Eq. 14), overlaid with the current PyRPOD approximation at sigma = 0. | `implemented` | `python tests/plume/plume_verification_test_29.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 18 | +| [`plume_verification_test_30.py`](plume/plume_verification_test_30.py) | Reproduces Fig. 19: diffuse-plate friction coefficient Cf1,d(s, tau) along the inclined direction (Eq. 11), whose zero contour marks the stagnation line, overlaid with the current PyRPOD approximation. | `implemented` | `python tests/plume/plume_verification_test_30.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 19 | +| [`plume_verification_test_31.py`](plume/plume_verification_test_31.py) | Reproduces Fig. 20: diffuse-plate friction coefficient Cf2,d(s, tau) along the horizontal direction (Eq. 12), antisymmetric in s, overlaid with the current PyRPOD approximation. | `implemented` | `python tests/plume/plume_verification_test_31.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 20 | +| [`plume_verification_test_32.py`](plume/plume_verification_test_32.py) | Reproduces Fig. 21: diffuse-plate heat-flux coefficient Cq,d(s, tau) (Eq. 13), peaking just beneath the plate center, overlaid with the current PyRPOD approximation. | `implemented` | `python tests/plume/plume_verification_test_32.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 21 | +| [`plume_verification_test_33.py`](plume/plume_verification_test_33.py) | Reproduces Fig. 5: temperature contours T/T0 for the 2D slot jet impinging on an inclined DIFFUSE planar plate, the combined free-jet plus wall-emission field of Section 3. | `implemented` | `python tests/plume/plume_verification_test_33.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 5 | +| [`plume_verification_test_34.py`](plume/plume_verification_test_34.py) | Reproduces Fig. 6: temperature contours T/T0 for the 2D slot jet impinging on an inclined SPECULAR planar plate, using the paper's virtual-nozzle construction (Eq. 7). | `implemented` | `python tests/plume/plume_verification_test_34.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 6 | +| [`plume_verification_test_35.py`](plume/plume_verification_test_35.py) | Reproduces Fig. 7: 2D diffuse-plate surface pressure profiles Cp,d(s) (Eq. 2) for four (S0, alpha0) combinations at Tw/T0 = 1.5. | `implemented` | `python tests/plume/plume_verification_test_35.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 7 | +| [`plume_verification_test_36.py`](plume/plume_verification_test_36.py) | Reproduces Fig. 8: 2D specular-plate surface pressure profiles Cp,s(s) (Eq. 8) for four (S0, alpha0) combinations. | `implemented` | `python tests/plume/plume_verification_test_36.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 8 | +| [`plume_verification_test_37.py`](plume/plume_verification_test_37.py) | Reproduces Fig. 9: 2D diffuse-plate surface friction profiles Cf,d(s) (Eq. 3), including the near-s/(2H) = -2 zero crossing the paper identifies as a possible flow-separation spot. | `implemented` | `python tests/plume/plume_verification_test_37.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 9 | +| [`plume_verification_test_38.py`](plume/plume_verification_test_38.py) | Reproduces Fig. 10: 2D diffuse-plate surface heat-flux profiles Cq,d(s) (Eq. 4) for four (S0, alpha0) combinations at Tw/T0 = 1.5. | `implemented` | `python tests/plume/plume_verification_test_38.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 10 | +| [`plume_verification_test_39.py`](plume/plume_verification_test_39.py) | Reproduces Fig. 15: static-pressure contours p/p0 in the Y = 0 plane for the round jet impinging on an inclined DIFFUSE rectangular plate. | `implemented` | `python tests/plume/plume_verification_test_39.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 15 | +| [`plume_verification_test_40.py`](plume/plume_verification_test_40.py) | Reproduces Fig. 16: static-pressure contours p/p0 in the Y = 0 plane for the round jet impinging on an inclined SPECULAR rectangular plate, using the paper's 3D virtual-nozzle construction. | `implemented` | `python tests/plume/plume_verification_test_40.py` | Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 16 | --- -## **Plume Tests** -| Test Name | Type | Description | Status | -|---------------------------------|----------------|-----------------------------------------------|--------| -| `plume_integration_test_01.py` | Integration | Tests plume modeling in integrated systems. | ❌ | -| `plume_unit_test_01.py` | Unit | Verifies individual plume calculation methods.| ❌ | -| `plume_verification_test_01.py` | Verification | Validates plume outputs against benchmarks. | ❌ | -| `plume_verification_test_28.py` | Verification | Cai 2016 Fig. 17: diffuse-plate Cp contours (manual-run figure). | ✅ | -| `plume_verification_test_29.py` | Verification | Cai 2016 Fig. 18: specular-plate Cp contours (manual-run figure). | ✅ | -| `plume_verification_test_30.py` | Verification | Cai 2016 Fig. 19: diffuse-plate Cf1 contours (manual-run figure). | ✅ | -| `plume_verification_test_31.py` | Verification | Cai 2016 Fig. 20: diffuse-plate Cf2 contours (manual-run figure). | ✅ | -| `plume_verification_test_32.py` | Verification | Cai 2016 Fig. 21: diffuse-plate Cq contours (manual-run figure). | ✅ | -| `plume_verification_test_33.py` | Verification | Cai 2016 Fig. 5: 2D diffuse-plate flowfield T contours (manual-run figure). | ✅ | -| `plume_verification_test_34.py` | Verification | Cai 2016 Fig. 6: 2D specular-plate flowfield T contours (manual-run figure). | ✅ | -| `plume_verification_test_35.py` | Verification | Cai 2016 Fig. 7: 2D diffuse-plate Cp profiles (manual-run figure). | ✅ | -| `plume_verification_test_36.py` | Verification | Cai 2016 Fig. 8: 2D specular-plate Cp profiles (manual-run figure). | ✅ | -| `plume_verification_test_37.py` | Verification | Cai 2016 Fig. 9: 2D diffuse-plate Cf profiles (manual-run figure). | ✅ | -| `plume_verification_test_38.py` | Verification | Cai 2016 Fig. 10: 2D diffuse-plate Cq profiles (manual-run figure). | ✅ | -| `plume_verification_test_39.py` | Verification | Cai 2016 Fig. 15: 3D diffuse-plate flowfield p contours (manual-run figure). | ✅ | -| `plume_verification_test_40.py` | Verification | Cai 2016 Fig. 16: 3D specular-plate flowfield p contours (manual-run figure). | ✅ | -| `plume_impingement_error_summary.py` | Verification | Cai 2016 reference vs PyRPOD-chain max/mean error table (manual-run generator). | ✅ | +## Placeholder tests + +Collected by pytest but containing no assertion or verification behavior - +typically an empty body or a fully commented-out one. Each is explicitly +skipped in its source file so that the HTML report shows it as **skipped** +rather than passed; a placeholder must never be mistaken for coverage. + +| Test file | Subsystem | Category | Description | +| --- | --- | --- | --- | +| [`mdao_unit_test_01.py`](mdao/mdao_unit_test_01.py) | mdao | unit | Placeholder MDAO unit test; the test body returns immediately and asserts nothing. | +| [`mdao_unit_test_02.py`](mdao/mdao_unit_test_02.py) | mdao | unit | Intended to build an array of cant-angle-swept thruster configurations (symmetric pitch/yaw canting) and visualize each sweep step. The entire body is currently commented out, so the test asserts nothing. | +| [`mdao_integration_test_01.py`](mdao/mdao_integration_test_01.py) | mdao | integration | Placeholder MDAO integration test; the test body returns immediately and asserts nothing. | +| [`mdao_verification_test_04.py`](mdao/mdao_verification_test_04.py) | mdao | verification | Intended variable-sweep study of the maximum overshoot velocity a logistics module can absorb with a fixed thruster configuration and deceleration start distance. The entire body is commented out, so the test asserts nothing. | +| [`mdao_verification_test_05.py`](mdao/mdao_verification_test_05.py) | mdao | verification | Intended trade study sweeping the surface cant angle of the RCS pack. The entire body is commented out, so the test asserts nothing. | +| [`mdao_verification_test_06.py`](mdao/mdao_verification_test_06.py) | mdao | verification | Intended multi-variable trade study sweeping axial overshoot together with surface cant angle. The entire body is commented out, so the test asserts nothing. | +| [`mission_unit_test_01.py`](mission/mission_unit_test_01.py) | mission | unit | Placeholder mission unit test; the test body returns immediately and asserts nothing. | +| [`mission_integration_test_02.py`](mission/mission_integration_test_02.py) | mission | integration | Intended to calculate the 6DOF performance of thruster working groups. The body only assigns mass-distribution constants; the header records that the analysis was disabled because it created unwanted data files, so the test asserts nothing. | +| [`mission_verification_test_01.py`](mission/mission_verification_test_01.py) | mission | verification | Placeholder mission verification test; the test body returns immediately and asserts nothing. | +| [`plume_unit_test_01.py`](plume/plume_unit_test_01.py) | plume | unit | Placeholder plume unit test; the test body returns immediately and asserts nothing. | +| [`plume_integration_test_01.py`](plume/plume_integration_test_01.py) | plume | integration | Placeholder plume integration test; the test body returns immediately and asserts nothing. | +| [`plume_verification_test_01.py`](plume/plume_verification_test_01.py) | plume | verification | Intended to plot simple isentropic radial-expansion profiles. Both plotting calls are commented out, leaving only construction of an IsentropicExpansion object, so the test asserts nothing. | --- -## **RPOD Tests** -| Test Name | Type | Description | Status | -|---------------------------------|----------------|-----------------------------------------------|--------| -| `rpod_integration_test_01.py` | Integration | Asserts plume strikes for RPOD "base case". | ✅ | -| `rpod_integration_test_02.py` | Integration | Asserts plume strikes for notional 1D approach. | ✅ | -| `rpod_integration_test_03.py` | Integration | Asserts plume strikes using KOZ geometry. | ✅ | -| `rpod_integration_test_04.py` | Integration | Asserts plume strikes using hollow cube geometry. | ✅ | -| `rpod_integration_test_07.py` | Integration | Cai 2016 inclined-plate case: pipeline loads vs exact reference. | ✅ | -| `rpod_unit_test_01.py` | Unit | Verifies STL to VTK data conversion. | ✅ | -| `rpod_unit_test_02.py` | Unit | Verifiy behavior of JFH reader. | ✅ | -| `rpod_unit_test_03.py` | Unit | Produces JFH data according to produced equation. | ⏳ | -| `rpod_verification_test_03.py` | Verification | Tests strike counting from multiple thrusters. | ⏳ | -| `rpod_verification_test_04.py` | Verification | Asserts plume strikes after TCD decoupling. | ⏳ | +## Ignored or blocked tests + +Excluded from pytest collection (see `collect_ignore` in +[`conftest.py`](conftest.py)) or otherwise unable to run. + +| Test file | Subsystem | Development status | Collection status | Reason | Command | +| --- | --- | --- | --- | --- | --- | +| [`test_case_25.py`](test_case_25.py) | rpod | `blocked` | `ignored` | Listed in tests/conftest.py collect_ignore: imports the flat `pyrpod.` layout that no longer exists after the rpod -> plume/vehicle refactor. Excluded from collection until it is fixed or removed outright. | — | +| [`rpod_verification_test_05.py`](rpod_verification_test_05.py) | rpod | `blocked` | `ignored` | Listed in tests/conftest.py collect_ignore: imports the flat `pyrpod.` layout that no longer exists after the rpod -> plume/vehicle refactor. Excluded from collection until it is fixed or removed outright. | — | +| [`mdao_verification_test_01.py`](mdao/mdao_verification_test_01.py) | mdao | `blocked` | `manual` | Defines an openmdao ExplicitComponent rather than a pytest test, so pytest collects nothing from it. The module header records "This has been having issues, disregard for now (4/2/24)". | `python tests/mdao/mdao_verification_test_01.py` | +| [`mdao_verification_test_02.py`](mdao/mdao_verification_test_02.py) | mdao | `blocked` | `manual` | Defines an openmdao ExplicitComponent rather than a pytest test, so pytest collects nothing from it. The module header records "This has been having issues, disregard for now (4/3/24)", and the __main__ block contains an incomplete call (prob.set_val with a missing value). | `python tests/mdao/mdao_verification_test_02.py` | +| [`mdao_verification_test_03.py`](mdao/mdao_verification_test_03.py) | mdao | `blocked` | `manual` | Defines an openmdao ExplicitComponent rather than a pytest test, so pytest collects nothing from it. The module header records "This has been having issues, disregard for now (4/3/24)", and the __main__ block contains an incomplete call (prob.set_val with a missing value). | `python tests/mdao/mdao_verification_test_03.py` | --- -## **Legacy/Old Tests** -| Test Name | Type | Description | Status | -|---------------------------------|----------------|-----------------------------------------------|--------| -| `test_case_15.py` | Miscellaneous | Old test for deprecated functionality. | ⏳ | -| `test_case_17.py` | Miscellaneous | Tests legacy feature interactions. | ⏳ | -| `test_case_19.py` | Miscellaneous | Validates compatibility of old methods. | ⏳ | -| `test_case_sweep_cants.py` | Sweep Test | Evaluates various canting angles. | ⏳ | -| `test_case_sweep_coords.py` | Sweep Test | Tests coordinate system transformations. | ⏳ | +## Archived or legacy tests + +Kept for historical reference under `tests/old/`. They are not repaired, +renamed, or deleted as part of routine work. + +| Test file | Subsystem | Category | Description | Reason | +| --- | --- | --- | --- | --- | +| [`old/test_case_sweep_cants.py`](old/test_case_sweep_cants.py) | mdao | unit | Legacy sweep building an array of cant-angle-swept thruster configurations with symmetric pitch/yaw canting. Superseded by tests/mdao/mdao_unit_test_02.py. | tests/old is listed in tests/conftest.py collect_ignore. Imports the flat `pyrpod.` layout removed by the rpod -> plume/vehicle refactor, plus a `test_header` module that no longer exists. | +| [`old/test_case_sweep_coords.py`](old/test_case_sweep_coords.py) | mdao | unit | Legacy sweep building an array of axially swept thruster configurations on a common ring x-coordinate. | tests/old is listed in tests/conftest.py collect_ignore. Imports the flat `pyrpod.` layout removed by the rpod -> plume/vehicle refactor, plus a `test_header` module that no longer exists. | +| [`old/test_case_17.py`](old/test_case_17.py) | rpod | unit | Legacy test of STL-to-VTK conversion, checking the VTK data format. Superseded by tests/rpod/rpod_unit_test_01.py. | tests/old is listed in tests/conftest.py collect_ignore. Imports the flat `pyrpod.` layout removed by the rpod -> plume/vehicle refactor, plus a `test_header` module that no longer exists. | +| [`old/test_case_15.py`](old/test_case_15.py) | rpod | integration | Legacy test of the plume gas-kinetic models in JFH firings. | tests/old is listed in tests/conftest.py collect_ignore. Imports the flat `pyrpod.` layout removed by the rpod -> plume/vehicle refactor, plus a `test_header` module that no longer exists. | +| [`old/test_case_19.py`](old/test_case_19.py) | rpod | integration | Legacy test producing hollow-cube data. Superseded by tests/rpod/rpod_integration_test_04.py. | tests/old is listed in tests/conftest.py collect_ignore. Imports the flat `pyrpod.` layout removed by the rpod -> plume/vehicle refactor, plus a `test_header` module that no longer exists. | --- -## **Status Legend** -- ✅ = Passed -- ❌ = Not Started -- ⏳ = In Progress -- ⚙️ = Under Review -- 🛠️ = Requires Fixes +## Legend + +Three independent axes. A test can be `implemented` and still fail today; +a `placeholder` can be green in CI only because it is skipped. + +**Execution outcome** — owned by pytest, one value per run, published only +in `reports/pyrpod-pytest-report.html`: + +| Outcome | Meaning | +| --- | --- | +| passed | The test ran and every assertion held. | +| failed | The test ran and an assertion or the code under test failed. | +| error | The test could not run to completion (setup or teardown raised). | +| skipped | The test was not executed (placeholder, or a skip condition). | + +**Development status** — maintained in `test_manifest.yaml`, long-lived: + +| Status | Meaning | +| --- | --- | +| `implemented` | Complete: exercises the code and asserts a result. | +| `placeholder` | No assertion or verification behavior yet; skipped. | +| `needs_review` | Runs real code but asserts nothing, or its purpose is insufficiently documented. | +| `blocked` | Cannot run against the current architecture. | +| `archived` | Superseded; kept for historical reference only. | +| `deprecated` | Slated for removal. | + +**Collection status** — whether pytest picks the file up: -This dashboard serves as a quick reference for test organization and tracking within PyRPOD. Update the statuses regularly to ensure it reflects the latest testing outcomes. +| Status | Meaning | +| --- | --- | +| `collected` | Collected and run by pytest; appears in the HTML report. | +| `manual` | Run by hand; defines no pytest tests. | +| `ignored` | Listed in `collect_ignore` in `conftest.py`. | +| `archived` | Under `tests/old/`, excluded from collection wholesale. | diff --git a/tests/conftest.py b/tests/conftest.py index 8b8f979..613e584 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,12 +1,15 @@ +import html import os +import shutil from pathlib import Path import pytest TESTS_DIR = Path(__file__).resolve().parent +REPO_ROOT = TESTS_DIR.parent CATEGORIES = ("unit", "integration", "verification") -GROUPS = ("logging", "mdao", "mission", "plume", "rpod") +GROUPS = ("logging", "mdao", "mission", "plume", "rpod", "tooling") # These files import a flat `pyrpod.` layout that no longer exists after # the rpod -> plume/vehicle refactor. Excluded from collection until they are @@ -49,3 +52,142 @@ def pytest_collection_modifyitems(items): for group in GROUPS: if group in relative_parts or stem.startswith(f"{group}_"): item.add_marker(getattr(pytest.mark, group)) + + +# --------------------------------------------------------------------------- +# pytest-html reporting +# --------------------------------------------------------------------------- +# Inventory metadata (description, subsystem, development status, ...) lives in +# tests/test_manifest.yaml; the hooks below surface it as extra columns in +# reports/pyrpod-pytest-report.html so a reader can tell "this run passed" from +# "this test is finished". Everything here degrades to a plain pytest-html +# report if PyYAML or the manifest is unavailable -- a normal `pytest` run must +# never depend on the reporting extras. + +MANIFEST_PATH = TESTS_DIR / "test_manifest.yaml" +REPORT_LOG_DIR = REPO_ROOT / "reports" / "logs" + +# Extra result-table columns, in insertion order after pytest-html's "Test". +_EXTRA_COLUMNS = ( + ("Description", "description"), + ("Subsystem", "subsystem"), + ("Category", "category"), + ("Mode", "execution_mode"), + ("Dev status", "development_status"), + ("Reference", "reference"), + ("Source", "path"), +) + +_manifest_cache = None + + +def _manifest_by_path(): + """{repo-relative path: manifest entry}, or {} if it cannot be read.""" + global _manifest_cache + if _manifest_cache is None: + entries = {} + try: + import yaml + + data = yaml.safe_load(MANIFEST_PATH.read_text(encoding="utf-8")) + for entry in data["tests"]: + entries[entry["path"]] = entry + except Exception: + # Missing PyYAML / manifest only costs the extra report columns. + entries = {} + _manifest_cache = entries + return _manifest_cache + + +def _metadata_for(item): + """Manifest metadata for the file a test item came from.""" + try: + path = Path(str(item.fspath)).resolve().relative_to(REPO_ROOT).as_posix() + except ValueError: + return {} + entry = _manifest_by_path().get(path) + if entry is None: + return {"path": path} + return {**entry, "path": path} + + +def _associated_log_files(item, start, stop): + """PyRPOD run logs written by this test, matched by modification time. + + Only files under a `results/logs/` directory whose mtime falls inside the + test's own call phase are returned, so a log can never be attributed to a + test that did not produce it. + """ + matched = [] + for log in (REPO_ROOT / "case").rglob("results/logs/*.log"): + try: + mtime = log.stat().st_mtime + except OSError: + continue + if start <= mtime <= stop: + matched.append(log) + return sorted(matched) + + +def _slug(nodeid): + return "".join(c if c.isalnum() or c in "-_." else "_" for c in nodeid)[:120] + + +@pytest.hookimpl(wrapper=True) +def pytest_runtest_makereport(item, call): + report = yield + report.pyrpod_meta = _metadata_for(item) + + if report.when == "call" and report.failed: + _attach_logs(report, item, call) + return report + + +def _attach_logs(report, item, call): + """Copy this test's PyRPOD logs into reports/logs/ and embed them.""" + logs = _associated_log_files(item, call.start, call.stop) + if not logs: + return + try: + from pytest_html import extras + except ImportError: + extras = None + + collected = [] + for index, log in enumerate(logs): + try: + text = log.read_text(encoding="utf-8", errors="replace") + except OSError: + continue + collected.append((log, text)) + try: + REPORT_LOG_DIR.mkdir(parents=True, exist_ok=True) + shutil.copyfile( + log, REPORT_LOG_DIR / f"{_slug(report.nodeid)}.{index}.log" + ) + except OSError: + pass + + if extras is None or not collected: + return + report.extras = list(getattr(report, "extras", [])) + for log, text in collected: + name = log.relative_to(REPO_ROOT).as_posix() + report.extras.append(extras.text(text, name=f"PyRPOD log: {name}")) + + +def pytest_html_report_title(report): + report.title = "PyRPOD test execution report" + + +def pytest_html_results_table_header(cells): + for offset, (title, _) in enumerate(_EXTRA_COLUMNS): + cells.insert(2 + offset, f"{html.escape(title)}") + + +def pytest_html_results_table_row(report, cells): + meta = getattr(report, "pyrpod_meta", {}) or {} + for offset, (_, key) in enumerate(_EXTRA_COLUMNS): + value = meta.get(key) + text = " ".join(str(value).split()) if value else "—" + cells.insert(2 + offset, f"{html.escape(text)}") diff --git a/tests/mdao/mdao_integration_test_01.py b/tests/mdao/mdao_integration_test_01.py index ac173b7..36653cd 100644 --- a/tests/mdao/mdao_integration_test_01.py +++ b/tests/mdao/mdao_integration_test_01.py @@ -11,7 +11,13 @@ import unittest +import pytest + class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the test body returns immediately and asserts " + "nothing (see tests/test_manifest.yaml)." + ) def test_mdao(self): # print("mdao integration test") diff --git a/tests/mdao/mdao_unit_test_01.py b/tests/mdao/mdao_unit_test_01.py index a4c8fa8..9a26a4f 100644 --- a/tests/mdao/mdao_unit_test_01.py +++ b/tests/mdao/mdao_unit_test_01.py @@ -11,7 +11,13 @@ import unittest +import pytest + class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the test body returns immediately and asserts " + "nothing (see tests/test_manifest.yaml)." + ) def test_mdao(self): # print("mdao unit test") diff --git a/tests/mdao/mdao_unit_test_02.py b/tests/mdao/mdao_unit_test_02.py index 550d847..be509e9 100644 --- a/tests/mdao/mdao_unit_test_02.py +++ b/tests/mdao/mdao_unit_test_02.py @@ -18,6 +18,7 @@ import unittest, os, sys import numpy as np +import pytest from pyrpod.rpod import JetFiringHistory, PlumeStrikeEstimationStudy from pyrpod.vehicle import TargetVehicle, VisitingVehicle @@ -25,6 +26,10 @@ class CoordinateSweepCheck(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the entire test body is commented out, so it " + "asserts nothing (see tests/test_manifest.yaml)." + ) def test_cant_sweep(self): # # number of deceleration thrusters to evenly distribute diff --git a/tests/mdao/mdao_verification_test_04.py b/tests/mdao/mdao_verification_test_04.py index 2db0dc7..edb2ca0 100644 --- a/tests/mdao/mdao_verification_test_04.py +++ b/tests/mdao/mdao_verification_test_04.py @@ -12,10 +12,16 @@ import unittest +import pytest + from pyrpod.vehicle import TargetVehicle, LogisticsModule from pyrpod.mdao import TradeStudy class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the entire test body is commented out, so it " + "asserts nothing (see tests/test_manifest.yaml)." + ) def test_mdao(self): # # 1. Set Up diff --git a/tests/mdao/mdao_verification_test_05.py b/tests/mdao/mdao_verification_test_05.py index 0bdc62a..0ace029 100644 --- a/tests/mdao/mdao_verification_test_05.py +++ b/tests/mdao/mdao_verification_test_05.py @@ -11,10 +11,16 @@ import unittest +import pytest + from pyrpod.vehicle import TargetVehicle, LogisticsModule from pyrpod.mdao import TradeStudy class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the entire test body is commented out, so it " + "asserts nothing (see tests/test_manifest.yaml)." + ) def test_mdao(self): # # 1. Set Up diff --git a/tests/mdao/mdao_verification_test_06.py b/tests/mdao/mdao_verification_test_06.py index 8952d01..81a911b 100644 --- a/tests/mdao/mdao_verification_test_06.py +++ b/tests/mdao/mdao_verification_test_06.py @@ -11,10 +11,16 @@ import unittest +import pytest + from pyrpod.vehicle import TargetVehicle, LogisticsModule from pyrpod.mdao import TradeStudy class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the entire test body is commented out, so it " + "asserts nothing (see tests/test_manifest.yaml)." + ) def test_mdao(self): # # 1. Set Up diff --git a/tests/mission/mission_integration_test_02.py b/tests/mission/mission_integration_test_02.py index c05a9d9..1053296 100644 --- a/tests/mission/mission_integration_test_02.py +++ b/tests/mission/mission_integration_test_02.py @@ -9,9 +9,15 @@ # A brief test case to calculate the 6DOF performance of thruster working groups import unittest, os, sys +import pytest from pyrpod.vehicle import LogisticsModule class ThrusterGroupingChecks(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: only local constants are assigned; the analysis " + "itself is commented out, so the test asserts nothing " + "(see tests/test_manifest.yaml)." + ) def test_performance_per_thruster_group(self): # Define LM mass distrubtion properties. diff --git a/tests/mission/mission_unit_test_01.py b/tests/mission/mission_unit_test_01.py index e146e95..9d24f56 100644 --- a/tests/mission/mission_unit_test_01.py +++ b/tests/mission/mission_unit_test_01.py @@ -11,7 +11,13 @@ import unittest +import pytest + class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the test body returns immediately and asserts " + "nothing (see tests/test_manifest.yaml)." + ) def test_mission(self): # print("mission unit test") return diff --git a/tests/mission/mission_verification_test_01.py b/tests/mission/mission_verification_test_01.py index 67bb77d..fff2998 100644 --- a/tests/mission/mission_verification_test_01.py +++ b/tests/mission/mission_verification_test_01.py @@ -12,7 +12,13 @@ import unittest +import pytest + class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the test body returns immediately and asserts " + "nothing (see tests/test_manifest.yaml)." + ) def test_mission(self): # print("mission verification test") return diff --git a/tests/plume/plume_integration_test_01.py b/tests/plume/plume_integration_test_01.py index 043c281..d7dd1f3 100644 --- a/tests/plume/plume_integration_test_01.py +++ b/tests/plume/plume_integration_test_01.py @@ -11,7 +11,13 @@ import unittest +import pytest + class MDAOTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the test body returns immediately and asserts " + "nothing (see tests/test_manifest.yaml)." + ) def test_mdao(self): # print("plume integration test") return diff --git a/tests/plume/plume_unit_test_01.py b/tests/plume/plume_unit_test_01.py index b42cb92..ffbebe2 100644 --- a/tests/plume/plume_unit_test_01.py +++ b/tests/plume/plume_unit_test_01.py @@ -11,7 +11,13 @@ import unittest +import pytest + class PlumeTest(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: the test body returns immediately and asserts " + "nothing (see tests/test_manifest.yaml)." + ) def test_plume(self): # print("plume unit test") return diff --git a/tests/plume/plume_verification_test_01.py b/tests/plume/plume_verification_test_01.py index 2b75412..5e73fea 100644 --- a/tests/plume/plume_verification_test_01.py +++ b/tests/plume/plume_verification_test_01.py @@ -10,9 +10,15 @@ # TODO: Re-factor code to save data in a relevant object. Also add files to save to. import unittest, os, sys +import pytest from pyrpod.plume import IsentropicExpansion class IsentropicExpansionCheck(unittest.TestCase): + @pytest.mark.skip( + reason="Placeholder: both plotting calls are commented out, leaving " + "only object construction, so the test asserts nothing " + "(see tests/test_manifest.yaml)." + ) def test_temp_vs_radial_expansion(self): #define flow and sonic properties. diff --git a/tests/test_manifest.yaml b/tests/test_manifest.yaml new file mode 100644 index 0000000..f6b220a --- /dev/null +++ b/tests/test_manifest.yaml @@ -0,0 +1,1446 @@ +# PyRPOD centralized test manifest +# ================================ +# Source of truth for INVENTORY-level metadata about every test asset in +# tests/ (description, subsystem, category, how it is executed, how far its +# development has progressed). Pytest remains the source of truth for +# EXECUTION outcomes (pass / fail / skip); those never appear here. +# +# Regenerate tests/README.md and the HTML execution report with: +# python scripts/generate_test_dashboard.py +# +# Schema (one entry per file under tests/): +# path repo-relative path, POSIX separators (required) +# description one-line human-readable purpose (required) +# subsystem logging | mdao | mission | plume | rpod | tooling +# category unit | integration | verification +# execution_mode automated | manual +# development_status implemented | placeholder | needs_review +# | blocked | archived | deprecated +# collection_status collected | ignored | manual | archived +# reference publication / benchmark, or null +# manual_command exact command for manual assets, or null +# collection_ignore_reason why the asset is absent from the pytest run, +# or null for collection_status: collected +# +# Helper modules that define no tests are intentionally absent: +# tests/conftest.py, tests/plume/plume_figure_utils.py, +# tests/plume/plume_impingement_utils.py. + +tests: + + # ---------------------------------------------------------------- logging + - path: tests/logging/logging_unit_test_01.py + description: >- + Unit tests for the centralized operational logging system + (pyrpod.logging_utils): import side effects, handler ownership, + configuration precedence, console/file toggles, runtime-log + naming and location, configuration snapshots, input-asset logging + and array summaries. + subsystem: logging + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/logging/logging_integration_test_01.py + description: >- + Integration tests for operational logging of the plume-strike + workflow: fail-fast validation of required inputs, progress cadence, + serial/parallel event equivalence, DEBUG-vs-INFO artifact levels, + parallel-to-serial fallback with successful_with_warning status and + optional-visualization warn-and-continue behavior. + subsystem: logging + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + # ------------------------------------------------------------------- mdao + - path: tests/mdao/mdao_unit_test_01.py + description: >- + Placeholder MDAO unit test; the test body returns immediately and + asserts nothing. + subsystem: mdao + category: unit + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mdao/mdao_unit_test_02.py + description: >- + Intended to build an array of cant-angle-swept thruster + configurations (symmetric pitch/yaw canting) and visualize each + sweep step. The entire body is currently commented out, so the test + asserts nothing. + subsystem: mdao + category: unit + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mdao/mdao_integration_test_01.py + description: >- + Placeholder MDAO integration test; the test body returns immediately + and asserts nothing. + subsystem: mdao + category: integration + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mdao/mdao_verification_test_01.py + description: >- + OpenMDAO axial-positioning optimizer: minimizes the maximum heat-flux + load for a 1D approach by varying the axial position of the thruster + packs. + subsystem: mdao + category: verification + execution_mode: manual + development_status: blocked + collection_status: manual + reference: null + manual_command: python tests/mdao/mdao_verification_test_01.py + collection_ignore_reason: >- + Defines an openmdao ExplicitComponent rather than a pytest test, so + pytest collects nothing from it. The module header records "This has + been having issues, disregard for now (4/2/24)". + + - path: tests/mdao/mdao_verification_test_02.py + description: >- + OpenMDAO cant-angle optimizer: minimizes the maximum heat-flux load + for a 1D approach by varying the cant angle of the deceleration + thrusters. + subsystem: mdao + category: verification + execution_mode: manual + development_status: blocked + collection_status: manual + reference: null + manual_command: python tests/mdao/mdao_verification_test_02.py + collection_ignore_reason: >- + Defines an openmdao ExplicitComponent rather than a pytest test, so + pytest collects nothing from it. The module header records "This has + been having issues, disregard for now (4/3/24)", and the __main__ + block contains an incomplete call (prob.set_val with a missing value). + + - path: tests/mdao/mdao_verification_test_03.py + description: >- + OpenMDAO axial-plus-cant evaluator: reports maximum cumulative + heat-flux load and JFH propellant expenditure for a 1D approach given + the axial position and cant angle of the deceleration thrusters. + subsystem: mdao + category: verification + execution_mode: manual + development_status: blocked + collection_status: manual + reference: null + manual_command: python tests/mdao/mdao_verification_test_03.py + collection_ignore_reason: >- + Defines an openmdao ExplicitComponent rather than a pytest test, so + pytest collects nothing from it. The module header records "This has + been having issues, disregard for now (4/3/24)", and the __main__ + block contains an incomplete call (prob.set_val with a missing value). + + - path: tests/mdao/mdao_verification_test_04.py + description: >- + Intended variable-sweep study of the maximum overshoot velocity a + logistics module can absorb with a fixed thruster configuration and + deceleration start distance. The entire body is commented out, so the + test asserts nothing. + subsystem: mdao + category: verification + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mdao/mdao_verification_test_05.py + description: >- + Intended trade study sweeping the surface cant angle of the RCS pack. + The entire body is commented out, so the test asserts nothing. + subsystem: mdao + category: verification + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mdao/mdao_verification_test_06.py + description: >- + Intended multi-variable trade study sweeping axial overshoot together + with surface cant angle. The entire body is commented out, so the test + asserts nothing. + subsystem: mdao + category: verification + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + # ---------------------------------------------------------------- mission + - path: tests/mission/mission_unit_test_01.py + description: >- + Placeholder mission unit test; the test body returns immediately and + asserts nothing. + subsystem: mission + category: unit + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_01.py + description: >- + Calculates the 6DOF performance (normal vector, force, translational + acceleration, torque) of each individual thruster in the logistics + module. Expected-value helpers are present but no assertion is made + against them, so the test only exercises the code path. + subsystem: mission + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_02.py + description: >- + Intended to calculate the 6DOF performance of thruster working + groups. The body only assigns mass-distribution constants; the header + records that the analysis was disabled because it created unwanted + data files, so the test asserts nothing. + subsystem: mission + category: integration + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_03.py + description: >- + Calculates RCS performance for a given flight plan, approximating the + 1D delta-v requirements. Exercises the flight-evaluation pipeline + end to end but makes no assertion. + subsystem: mission + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_04.py + description: >- + Analyzes a notional 1D translation-plus-rotation approach (module + header marks it "NEEDS TLC"). Exercises the flight-evaluation + pipeline end to end but makes no assertion. + subsystem: mission + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_05.py + description: >- + Graphs thrust versus time or distance for the given design + requirements to establish an RCS thrust-requirement flight envelope. + Exercises the plotting pipeline but makes no assertion. + subsystem: mission + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_06.py + description: >- + Graphs propellant-mass requirements across the flight envelope for + the given design requirements. Exercises the plotting pipeline but + makes no assertion. + subsystem: mission + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_07.py + description: >- + Contours the burn-time plot across a range of thrust and Isp values + (module header marks it "NEEDS TLC"). Exercises the plotting pipeline + but makes no assertion. + subsystem: mission + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_integration_test_08.py + description: >- + Contours propellant usage across the various delta-v legs of a given + flight plan. Exercises the plotting pipeline but makes no assertion. + subsystem: mission + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_verification_test_01.py + description: >- + Placeholder mission verification test; the test body returns + immediately and asserts nothing. + subsystem: mission + category: verification + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/mission/mission_verification_test_02.py + description: >- + Builds and summarizes a chain of Hohmann transfers (LEO to MEO to + GEO) through MissionPlanner.orbital_transfer. Exercises the orbital + transfer path but makes no assertion. + subsystem: mission + category: verification + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + # ------------------------------------------------------------------ plume + - path: tests/plume/plume_unit_test_01.py + description: >- + Placeholder plume unit test; the test body returns immediately and + asserts nothing. + subsystem: plume + category: unit + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_unit_test_02.py + description: >- + Verifies the closed form of the special factor Q against a direct + 50-term truncation of the printed Legendre series, its centerline + reduction, and the 0 < Q <= 1 bound that keeps the + exp(-S0^2 (1-Q)) combination overflow-safe. + subsystem: plume + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Eq. 9 + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_unit_test_03.py + description: >- + Tests the Simons cosine-law plume model after the gamma + generalization: isentropic throat ratios from gamma, zero density + beyond the limiting angle, parameterizable beaming exponent kappa + and exit-referenced density scaling. + subsystem: plume + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: >- + Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Sec. II.C, + Eqs. 25-26 + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_unit_test_04.py + description: >- + Asserts that the NumPy-vectorized plume-strike detection in + compute_plume_strikes() reproduces the scalar reference + implementation exactly for every firing of case/rpod/1d_approach + (geometry only) and case/rpod/multi_thrusters_square (Simplified + kinetics, multiple thrusters). + subsystem: plume + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_integration_test_01.py + description: >- + Placeholder plume integration test; the test body returns immediately + and asserts nothing. + subsystem: plume + category: integration + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_integration_test_02.py + description: >- + Asserts that jfh_plume_strikes() keeps its default serial, + return-compatible behavior, produces identical per-firing and + cumulative output when the optional process-parallel path is enabled, + and rejects invalid worker counts with a clear error. + subsystem: plume + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_verification_test_01.py + description: >- + Intended to plot simple isentropic radial-expansion profiles. Both + plotting calls are commented out, leaving only construction of an + IsentropicExpansion object, so the test asserts nothing. + subsystem: plume + category: verification + execution_mode: automated + development_status: placeholder + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_verification_test_02.py + description: >- + Pinning tests for SimplifiedGasKinetics against reference values + computed independently from the paper's equations with mpmath at 40 + significant digits, plus far-field asymptote convergence and the + near-field divergence of the corrected Eq. 21 centerline temperature + quadrature. + subsystem: plume + category: verification + execution_mode: automated + development_status: implemented + collection_status: collected + reference: >- + Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Eqs. 13-19, 22-24 + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_verification_test_03.py + description: >- + Verification of the full collisionless model CollisionlessGasKinetics + against the paper's internal anchors: centerline reduction to the + Eq. 18/19 closed forms, vanishing W, the Eq. 30 far-field error + bound, monotonic centerline density decay and asymptote convergence. + subsystem: plume + category: verification + execution_mode: automated + development_status: implemented + collection_status: collected + reference: >- + Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Eqs. 5-12, 20, 30 + manual_command: null + collection_ignore_reason: null + + - path: tests/plume/plume_verification_test_04.py + description: >- + Reproduces Fig. 2: plume boundaries (the n/n0 = 0.001 contour of the + full analytical model) for exit speed ratios S0 = 1, 2, 3. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 2 + manual_command: python tests/plume/plume_verification_test_04.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_05.py + description: >- + Reproduces Fig. 3: normalized analytical centerline number density + (Eq. 18) versus X/D for S0 = 1, 2, 3. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 3 + manual_command: python tests/plume/plume_verification_test_05.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_06.py + description: >- + Reproduces Fig. 4: normalized analytical centerline U-velocity + (Eq. 19) versus X/D for S0 = 1, 2, 3. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 4 + manual_command: python tests/plume/plume_verification_test_06.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_07.py + description: >- + Reproduces Fig. 5: normalized analytical centerline temperature + (Eq. 21, exact quadrature) versus X/D for S0 = 1, 2, 3. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 5 + manual_command: python tests/plume/plume_verification_test_07.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_08.py + description: >- + Reproduces Fig. 6: normalized number-density contours at Kn = 100, + S0 = 2.0; analytical plus simplified in the upper half-plane, Simons + plus the DSMC overlay slot in the lower half-plane. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 6 + manual_command: python tests/plume/plume_verification_test_08.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_09.py + description: >- + Reproduces Fig. 7: normalized number-density contours at Kn = 0.1, + S0 = 2.0. Same analytic content as Fig. 6; the Kn distinction lives + in the DSMC overlay slot. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 7 + manual_command: python tests/plume/plume_verification_test_09.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_10.py + description: >- + Reproduces Fig. 8: normalized number-density contours at Kn = 0.01, + S0 = 2.0. Same analytic content as Fig. 6; the Kn distinction lives + in the DSMC overlay slot. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 8 + manual_command: python tests/plume/plume_verification_test_10.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_11.py + description: >- + Reproduces Fig. 9: relative density error between the analytical and + Simons cosine-law solutions, |n_A/n_Simons - 1| (Eq. 28), S0 = 2.0. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 9 + manual_command: python tests/plume/plume_verification_test_11.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_12.py + description: >- + Reproduces Fig. 10: relative density error between the analytical and + simplified analytical solutions, |n_A/n_As - 1| (Eq. 29), S0 = 2.0. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 10 + manual_command: python tests/plume/plume_verification_test_12.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_13.py + description: >- + Reproduces Fig. 11: normalized pressure contours + p1/p0 = (n1/n0)(T1/T0) at Kn = 100, S0 = 2.0. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 11 + manual_command: python tests/plume/plume_verification_test_13.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_14.py + description: >- + Reproduces Fig. 12: normalized pressure contours p1/p0 at Kn = 0.1, + S0 = 2.0. Same analytic content as Fig. 11. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 12 + manual_command: python tests/plume/plume_verification_test_14.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_15.py + description: >- + Reproduces Fig. 13: normalized temperature contours T1/T0 at + Kn = 100, S0 = 2.0 - the figure the paper uses to argue that + p = n k T0 is invalid because T1 < T0 everywhere downstream. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 13 + manual_command: python tests/plume/plume_verification_test_15.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_16.py + description: >- + Reproduces Fig. 14: normalized U-velocity contours U1*sqrt(beta0) at + Kn = 100, S0 = 2.0. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 14 + manual_command: python tests/plume/plume_verification_test_16.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_17.py + description: >- + Reproduces Fig. 15: normalized transverse-velocity contours at + Kn = 100, S0 = 2.0. In the plotted XOZ plane the y-component is zero + by axisymmetry, so the model's W (Eq. 7) is plotted. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 15 + manual_command: python tests/plume/plume_verification_test_17.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_18.py + description: >- + Reproduces Fig. 16: normalized radial-velocity Vr contours at + Kn = 100, S0 = 2.0. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 16 + manual_command: python tests/plume/plume_verification_test_18.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_19.py + description: >- + Reproduces Fig. 17: normalized radial-velocity Vr contours at + Kn = 0.1, S0 = 2.0. Same analytic content as Fig. 16. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 17 + manual_command: python tests/plume/plume_verification_test_19.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_20.py + description: >- + Reproduces Fig. 18: normalized radial-velocity Vr contours at + Kn = 0.01, S0 = 2.0. Same analytic content as Fig. 16. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 18 + manual_command: python tests/plume/plume_verification_test_20.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_21.py + description: >- + Reproduces Fig. 19: centerline density profiles at Kn = 100, + S0 = 2.0 - analytical (Eq. 18), simplified (Eq. 14), Simons + (exit-referenced cosine law) and the DSMC overlay slot. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 19 + manual_command: python tests/plume/plume_verification_test_21.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_22.py + description: >- + Reproduces Fig. 20: centerline density profiles at Kn = 0.1, + S0 = 2.0. Same analytic content as Fig. 19. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 20 + manual_command: python tests/plume/plume_verification_test_22.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_23.py + description: >- + Reproduces Fig. 21: centerline density profiles at Kn = 0.01, + S0 = 2.0. Same analytic content as Fig. 19. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 21 + manual_command: python tests/plume/plume_verification_test_23.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_24.py + description: >- + Reproduces Fig. 22: density profiles along r/D = 10 at Kn = 100, + S0 = 2.0 - analytical, simplified, Simons with kappa = 1.5/2/3 under + a single Boyton normalization, and the DSMC overlay slot. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 22 + manual_command: python tests/plume/plume_verification_test_24.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_25.py + description: >- + Reproduces Fig. 23: density profiles along r/D = 10 at Kn = 0.1, + S0 = 2.0. Same analytic content as Fig. 22. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 23 + manual_command: python tests/plume/plume_verification_test_25.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_26.py + description: >- + Reproduces Fig. 24: density profiles along r/D = 10 at Kn = 0.01, + S0 = 2.0. Same analytic content as Fig. 22. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 24 + manual_command: python tests/plume/plume_verification_test_26.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_27.py + description: >- + Reproduces Fig. 25: normalized mass flux along r/D = 10 versus theta + at S0 = 2.0, together with three DSMC overlay slots. The module + header records the flux-normalization convention adopted to match the + printed magnitudes. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Fig. 25 + manual_command: python tests/plume/plume_verification_test_27.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_28.py + description: >- + Reproduces Fig. 17: diffuse-plate surface pressure contours + Cp,d(s, tau) for the round argon jet on the inclined plate (Eq. 9), + overlaid with the current PyRPOD approximation. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 17 + manual_command: python tests/plume/plume_verification_test_28.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_29.py + description: >- + Reproduces Fig. 18: specular-plate surface pressure contours + Cp,s(s, tau) (Eq. 14), overlaid with the current PyRPOD + approximation at sigma = 0. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 18 + manual_command: python tests/plume/plume_verification_test_29.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_30.py + description: >- + Reproduces Fig. 19: diffuse-plate friction coefficient Cf1,d(s, tau) + along the inclined direction (Eq. 11), whose zero contour marks the + stagnation line, overlaid with the current PyRPOD approximation. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 19 + manual_command: python tests/plume/plume_verification_test_30.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_31.py + description: >- + Reproduces Fig. 20: diffuse-plate friction coefficient Cf2,d(s, tau) + along the horizontal direction (Eq. 12), antisymmetric in s, overlaid + with the current PyRPOD approximation. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 20 + manual_command: python tests/plume/plume_verification_test_31.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_32.py + description: >- + Reproduces Fig. 21: diffuse-plate heat-flux coefficient Cq,d(s, tau) + (Eq. 13), peaking just beneath the plate center, overlaid with the + current PyRPOD approximation. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 21 + manual_command: python tests/plume/plume_verification_test_32.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_33.py + description: >- + Reproduces Fig. 5: temperature contours T/T0 for the 2D slot jet + impinging on an inclined DIFFUSE planar plate, the combined free-jet + plus wall-emission field of Section 3. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 5 + manual_command: python tests/plume/plume_verification_test_33.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_34.py + description: >- + Reproduces Fig. 6: temperature contours T/T0 for the 2D slot jet + impinging on an inclined SPECULAR planar plate, using the paper's + virtual-nozzle construction (Eq. 7). + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 6 + manual_command: python tests/plume/plume_verification_test_34.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_35.py + description: >- + Reproduces Fig. 7: 2D diffuse-plate surface pressure profiles + Cp,d(s) (Eq. 2) for four (S0, alpha0) combinations at Tw/T0 = 1.5. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 7 + manual_command: python tests/plume/plume_verification_test_35.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_36.py + description: >- + Reproduces Fig. 8: 2D specular-plate surface pressure profiles + Cp,s(s) (Eq. 8) for four (S0, alpha0) combinations. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 8 + manual_command: python tests/plume/plume_verification_test_36.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_37.py + description: >- + Reproduces Fig. 9: 2D diffuse-plate surface friction profiles + Cf,d(s) (Eq. 3), including the near-s/(2H) = -2 zero crossing the + paper identifies as a possible flow-separation spot. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 9 + manual_command: python tests/plume/plume_verification_test_37.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_38.py + description: >- + Reproduces Fig. 10: 2D diffuse-plate surface heat-flux profiles + Cq,d(s) (Eq. 4) for four (S0, alpha0) combinations at Tw/T0 = 1.5. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 10 + manual_command: python tests/plume/plume_verification_test_38.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_39.py + description: >- + Reproduces Fig. 15: static-pressure contours p/p0 in the Y = 0 plane + for the round jet impinging on an inclined DIFFUSE rectangular plate. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 15 + manual_command: python tests/plume/plume_verification_test_39.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_test_40.py + description: >- + Reproduces Fig. 16: static-pressure contours p/p0 in the Y = 0 plane + for the round jet impinging on an inclined SPECULAR rectangular + plate, using the paper's 3D virtual-nozzle construction. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Fig. 16 + manual_command: python tests/plume/plume_verification_test_40.py + collection_ignore_reason: >- + Manual-run figure script (design decision D5); defines no pytest + tests and is excluded by tests/conftest.py collect_ignore. + + - path: tests/plume/plume_verification_error_summary.py + description: >- + Model-vs-model analog of Table 1: maximum relative differences in + density and mass flux between the analytical, simplified and Simons + models along the centerline and along r/D = 10 at S0 = 2.0. The DSMC + reference columns are emitted as "pending digitized data". + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: Cai & Wang 2012, JSR 49(1), DOI 10.2514/1.A32046, Table 1 + manual_command: python tests/plume/plume_verification_error_summary.py + collection_ignore_reason: >- + Manual-run summary generator; its filename does not match the pytest + python_files patterns, so pytest never collects it. + + - path: tests/plume/plume_impingement_error_summary.py + description: >- + Maximum and mean relative differences between the exact Cai 2016 + reference surface coefficients (Eqs. 9-14) and the current PyRPOD + approximation chain for the inclined-plate impingement study. + subsystem: plume + category: verification + execution_mode: manual + development_status: implemented + collection_status: manual + reference: >- + Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, + Figs. 17-21 conditions + manual_command: python tests/plume/plume_impingement_error_summary.py + collection_ignore_reason: >- + Manual-run summary generator; its filename does not match the pytest + python_files patterns, so pytest never collects it. + + # ------------------------------------------------------------------- rpod + - path: tests/rpod/rpod_unit_test_01.py + description: >- + Converts cylinder STL data to VTK and asserts the resulting file has + the proper VTK data format, cell counts and point counts. + subsystem: rpod + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_unit_test_02.py + description: >- + Reads a jet firing history and asserts that all required keys exist + and that every value has the expected data type. + subsystem: rpod + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_unit_test_03.py + description: >- + Captures the exact file output of the three JFH printing helpers in + pyrpod.util.io.file_print to a temporary directory and compares it + against the committed tests/rpod/jfh_outputs snapshot. + subsystem: rpod + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_unit_test_04.py + description: >- + Unit tests for the PR #116 plume-mesh consolidation: + VisitingVehicle.transform_plume_mesh, the cached thruster-id map, + multi-digit cluster ids, stl.compose_meshes and the reusable + stl.transform_mesh mesh-object API, pinned against hand-computed + golden values. + subsystem: rpod + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_01.py + description: >- + Asserts the expected number of cell strikes on a flat-plate STL for a + notional "sweep" trajectory above it across 20 distinct firings. This + is the established base case for RPOD plume-impingement analysis. + subsystem: rpod + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_02.py + description: >- + Asserts the expected number of cell strikes on a flat-plate STL as a + notional visiting vehicle approaches it along a 1D-physics trajectory, + across 15 distinct firings. + subsystem: rpod + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_03.py + description: >- + Analyzes keep-out-zone impingement and asserts the expected strike + counts against the committed expected-strikes fixture (module header + marks the case "WIP"). + subsystem: rpod + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_04.py + description: >- + Produces hollow-cube target data and asserts the expected strike + counts against the committed expected-strikes fixture. + subsystem: rpod + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_05.py + description: >- + Runs the plume gas-kinetic models through a JFH with multiple + thrusters per firing (case/rpod/multi_thrusters_square). Exercises + graph_jfh and jfh_plume_strikes end to end but makes no assertion. + subsystem: rpod + category: integration + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_06.py + description: >- + Performance benchmark for plume-strike computation (scalar vs + NumPy-vectorized geometry, serial vs process-parallel). Elapsed times + are printed but no speedup threshold is asserted; each benchmark + asserts only that the compared paths produce identical strike arrays. + subsystem: rpod + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_07.py + description: >- + Runs the Cai 2016 inclined-plate case through + PlumeStrikeEstimationStudy, converts per-face dimensional loads to + the paper's coefficient normalization and compares them face by face + against the exact reference functions evaluated at face centroids. + subsystem: rpod + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: >- + Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, + Eqs. 9-13 + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_integration_test_08.py + description: >- + End-to-end geometry-equivalence regressions for the PR #116 + plume-mesh consolidation: pins the visualization pipeline's produced + geometry, artifact counts and file names against an inline + reproduction of the legacy transform/compose sequence, and proves the + thruster transform is applied exactly once. + subsystem: rpod + category: integration + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_verification_test_04.py + description: >- + Visualizes STLs after decoupling the thruster configuration data and + is intended to verify that plume strikes line up with the thrusters. + Exercises graph_jfh and jfh_plume_strikes but makes no assertion, so + the alignment must still be checked by eye in ParaView. + subsystem: rpod + category: verification + execution_mode: automated + development_status: needs_review + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_verification_test_06.py + description: >- + Cai 2016 flat-plate SWEEP verification: runs the 95-firing sweep JFH + (19 approach angles x 5 stand-off distances) through the strike + pipeline, reduces each pose to the Eq.-15 plate-averaged + coefficients, checks mirror symmetry in +/- alpha and compares + against the exact reference envelope. + subsystem: rpod + category: verification + execution_mode: automated + development_status: implemented + collection_status: collected + reference: >- + Cai 2016, Aerospace 3(4), 43, doi:10.3390/aerospace3040043, Eq. 15 + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod/rpod_verification_test_07.py + description: >- + Cylinder-target sweep smoke test: exercises the full + PlumeStrikeEstimationStudy pipeline on a curved, closed target + (14036-face cylinder) over 19 approach angles x 5 orbit radii. + subsystem: rpod + category: verification + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null + + - path: tests/rpod_verification_test_05.py + description: >- + Visualizes the plume flow field using a series of target vehicles. + subsystem: rpod + category: verification + execution_mode: manual + development_status: blocked + collection_status: ignored + reference: null + manual_command: null + collection_ignore_reason: >- + Listed in tests/conftest.py collect_ignore: imports the flat + `pyrpod.` layout that no longer exists after the + rpod -> plume/vehicle refactor. Excluded from collection until it is + fixed or removed outright. + + - path: tests/test_case_25.py + description: >- + Tests the plume gas-kinetic models in JFH firings. + subsystem: rpod + category: integration + execution_mode: manual + development_status: blocked + collection_status: ignored + reference: null + manual_command: null + collection_ignore_reason: >- + Listed in tests/conftest.py collect_ignore: imports the flat + `pyrpod.` layout that no longer exists after the + rpod -> plume/vehicle refactor. Excluded from collection until it is + fixed or removed outright. + + # ---------------------------------------------------------- legacy (old/) + - path: tests/old/test_case_15.py + description: >- + Legacy test of the plume gas-kinetic models in JFH firings. + subsystem: rpod + category: integration + execution_mode: manual + development_status: archived + collection_status: archived + reference: null + manual_command: null + collection_ignore_reason: >- + tests/old is listed in tests/conftest.py collect_ignore. Imports the + flat `pyrpod.` layout removed by the rpod -> plume/vehicle + refactor, plus a `test_header` module that no longer exists. + + - path: tests/old/test_case_17.py + description: >- + Legacy test of STL-to-VTK conversion, checking the VTK data format. + Superseded by tests/rpod/rpod_unit_test_01.py. + subsystem: rpod + category: unit + execution_mode: manual + development_status: archived + collection_status: archived + reference: null + manual_command: null + collection_ignore_reason: >- + tests/old is listed in tests/conftest.py collect_ignore. Imports the + flat `pyrpod.` layout removed by the rpod -> plume/vehicle + refactor, plus a `test_header` module that no longer exists. + + - path: tests/old/test_case_19.py + description: >- + Legacy test producing hollow-cube data. Superseded by + tests/rpod/rpod_integration_test_04.py. + subsystem: rpod + category: integration + execution_mode: manual + development_status: archived + collection_status: archived + reference: null + manual_command: null + collection_ignore_reason: >- + tests/old is listed in tests/conftest.py collect_ignore. Imports the + flat `pyrpod.` layout removed by the rpod -> plume/vehicle + refactor, plus a `test_header` module that no longer exists. + + - path: tests/old/test_case_sweep_cants.py + description: >- + Legacy sweep building an array of cant-angle-swept thruster + configurations with symmetric pitch/yaw canting. Superseded by + tests/mdao/mdao_unit_test_02.py. + subsystem: mdao + category: unit + execution_mode: manual + development_status: archived + collection_status: archived + reference: null + manual_command: null + collection_ignore_reason: >- + tests/old is listed in tests/conftest.py collect_ignore. Imports the + flat `pyrpod.` layout removed by the rpod -> plume/vehicle + refactor, plus a `test_header` module that no longer exists. + + - path: tests/old/test_case_sweep_coords.py + description: >- + Legacy sweep building an array of axially swept thruster + configurations on a common ring x-coordinate. + subsystem: mdao + category: unit + execution_mode: manual + development_status: archived + collection_status: archived + reference: null + manual_command: null + collection_ignore_reason: >- + tests/old is listed in tests/conftest.py collect_ignore. Imports the + flat `pyrpod.` layout removed by the rpod -> plume/vehicle + refactor, plus a `test_header` module that no longer exists. + + # ---------------------------------------------------------------- tooling + - path: tests/tooling/tooling_unit_test_01.py + description: >- + Unit tests for the test-inventory tooling itself: manifest schema + validation, detection of missing and stale entries, and deterministic + rendering of the generated tests/README.md. + subsystem: tooling + category: unit + execution_mode: automated + development_status: implemented + collection_status: collected + reference: null + manual_command: null + collection_ignore_reason: null diff --git a/tests/tooling/tooling_unit_test_01.py b/tests/tooling/tooling_unit_test_01.py new file mode 100644 index 0000000..66f067e --- /dev/null +++ b/tests/tooling/tooling_unit_test_01.py @@ -0,0 +1,293 @@ +# ======================== +# PyRPOD: tests/tooling/tooling_unit_test_01.py +# ======================== +# Unit tests for the test-inventory tooling in +# scripts/generate_test_dashboard.py: manifest schema validation, detection of +# missing/stale entries against real pytest collection, deterministic rendering +# of tests/README.md, and the invariant that every manifest-declared +# placeholder is actually skipped by pytest. +# +# These exercise the tooling only; they assert nothing about PyRPOD's physics. + +import copy + +import pytest + +from scripts import generate_test_dashboard as dashboard + + +@pytest.fixture(scope="module") +def entries(): + """The real manifest, parsed once.""" + return dashboard.load_manifest() + + +def _entry(entries, **overrides): + """A valid copy of the first manifest entry, with fields overridden.""" + entry = copy.deepcopy(entries[0]) + entry.update(overrides) + return entry + + +# -------------------------------------------------------------------------- +# Manifest parsing and schema validation +# -------------------------------------------------------------------------- + + +def test_real_manifest_parses_and_validates(entries): + """The committed manifest must always satisfy its own schema.""" + assert entries + assert dashboard.validate_manifest(entries) == [] + + +def test_missing_manifest_raises_manifest_error(tmp_path): + with pytest.raises(dashboard.ManifestError, match="not found"): + dashboard.load_manifest(tmp_path / "absent.yaml") + + +def test_manifest_without_tests_key_raises(tmp_path): + path = tmp_path / "m.yaml" + path.write_text("other: []\n", encoding="utf-8") + with pytest.raises(dashboard.ManifestError, match="tests"): + dashboard.load_manifest(path) + + +def test_missing_required_field_is_reported(entries): + broken = _entry(entries) + del broken["description"] + problems = dashboard.validate_manifest([broken]) + assert any("missing required field 'description'" in p for p in problems) + + +def test_unknown_field_is_reported(entries): + problems = dashboard.validate_manifest([_entry(entries, status="passed")]) + assert any("unknown field 'status'" in p for p in problems) + + +@pytest.mark.parametrize( + "field, value", + [ + ("subsystem", "propulsion"), + ("category", "smoke"), + ("execution_mode", "semi-automated"), + ("development_status", "passed"), + ("collection_status", "failed"), + ], +) +def test_controlled_vocabulary_is_enforced(entries, field, value): + """Runtime outcomes such as 'passed'/'failed' are never valid metadata.""" + problems = dashboard.validate_manifest([_entry(entries, **{field: value})]) + assert any(f"{field} '{value}' is not one of" in p for p in problems) + + +def test_nonexistent_path_is_reported(entries): + broken = _entry(entries, path="tests/does/not/exist.py") + problems = dashboard.validate_manifest([broken]) + assert any("file does not exist" in p for p in problems) + + +def test_duplicate_path_is_reported(entries): + duplicated = [_entry(entries), _entry(entries)] + problems = dashboard.validate_manifest(duplicated) + assert any("duplicate path" in p for p in problems) + + +def test_automated_entry_must_be_collected(entries): + broken = _entry( + entries, + execution_mode="automated", + collection_status="manual", + collection_ignore_reason="because", + ) + problems = dashboard.validate_manifest([broken]) + assert any("requires collection_status 'collected'" in p for p in problems) + + +def test_uncollected_entry_requires_a_reason(entries): + broken = _entry( + entries, + execution_mode="manual", + collection_status="ignored", + manual_command=None, + development_status="blocked", + collection_ignore_reason=None, + ) + problems = dashboard.validate_manifest([broken]) + assert any("requires a collection_ignore_reason" in p for p in problems) + + +def test_review_required_description_forces_needs_review(entries): + broken = _entry( + entries, + description=f"{dashboard.REVIEW_REQUIRED}: test purpose is not " + "sufficiently documented.", + development_status="implemented", + ) + problems = dashboard.validate_manifest([broken]) + assert any("requires development_status 'needs_review'" in p for p in problems) + + +# -------------------------------------------------------------------------- +# Cross-checking the manifest against pytest collection +# -------------------------------------------------------------------------- + + +def test_files_from_nodes_deduplicates_parametrized_cases(): + nodes = [ + "tests/plume/plume_unit_test_02.py::test_q[0.1-0.5]", + "tests/plume/plume_unit_test_02.py::test_q[0.2-0.5]", + "tests/rpod/rpod_unit_test_01.py::Checks::test_stl", + ] + assert dashboard.files_from_nodes(nodes) == [ + "tests/plume/plume_unit_test_02.py", + "tests/rpod/rpod_unit_test_01.py", + ] + + +def test_collected_test_without_manifest_entry_is_reported(entries): + """A newly added test file must not be silently omitted.""" + problems = dashboard.cross_check( + entries, ["tests/rpod/rpod_unit_test_01.py", "tests/brand/new_test_01.py"] + ) + assert any( + "tests/brand/new_test_01.py" in p and "missing from the manifest" in p + for p in problems + ) + + +def test_stale_collected_entry_is_reported(entries): + """An entry claiming collection that pytest no longer collects is stale.""" + problems = dashboard.cross_check(entries, []) + stale = [p for p in problems if "stale entry" in p] + assert stale + assert all("collection_status 'collected'" in p for p in stale) + + +def test_manual_entry_that_pytest_collects_is_reported(entries): + """A manual script that starts defining pytest tests must be reclassified.""" + manual = next(e for e in entries if e["collection_status"] == "manual") + problems = dashboard.cross_check(entries, [manual["path"]]) + assert any( + manual["path"] in p and "the manifest says collection_status 'manual'" in p + for p in problems + ) + + +def test_real_manifest_is_in_sync_with_this_session(request, entries): + """Every test file collected in this very run is documented.""" + collected = dashboard.files_from_nodes( + item.nodeid for item in request.session.items + ) + documented = {e["path"] for e in entries} + assert set(collected) <= documented + + +# -------------------------------------------------------------------------- +# Placeholder bookkeeping +# -------------------------------------------------------------------------- + + +def test_placeholder_status_matches_pytest_skip_markers(request, entries): + """`development_status: placeholder` and an explicit skip must agree. + + Uses pytest's own collected items and their markers rather than parsing + source, so it cannot drift from what the HTML report actually shows. + """ + status_by_path = {e["path"]: e["development_status"] for e in entries} + + mismatched = [] + for item in request.session.items: + path = dashboard.files_from_nodes([item.nodeid])[0] + if path not in status_by_path: + continue + is_placeholder = status_by_path[path] == "placeholder" + is_skipped = any(m.name == "skip" for m in item.iter_markers()) + if is_placeholder != is_skipped: + mismatched.append((item.nodeid, is_placeholder, is_skipped)) + + assert not mismatched, ( + "placeholder metadata and skip markers disagree " + "(nodeid, manifest_says_placeholder, has_skip_marker): " + repr(mismatched) + ) + + +# -------------------------------------------------------------------------- +# tests/README.md rendering +# -------------------------------------------------------------------------- + + +def test_render_readme_is_deterministic(entries): + nodes = ["tests/rpod/rpod_unit_test_01.py::Checks::test_stl"] + assert dashboard.render_readme(entries, nodes) == dashboard.render_readme( + entries, nodes + ) + + +def test_render_readme_marks_the_file_as_generated(entries): + readme = dashboard.render_readme(entries, []) + assert readme.startswith("