Separate Ninja generation steps from runner reporting (#343)

docs/developers-guide.md

@@ -3672,6 +3672,48 @@ context rather than resolving output or process configuration again; tests
 should inject the program through `run_with_ninja_program` when they need a
 deterministic child executable.
 
+### Module: `runner::generation`
+
+`src/runner/generation.rs` owns the runner's reusable, in-memory generation
+pipeline. It separates manifest loading, IR construction, and Ninja bundle
+synthesis from command reporting and process execution. The read-only pipeline
+is `load_manifest` (optionally observing manifest stages), then `build_graph`,
+then `ninja_text`. Its final value is `GeneratedNinja`, including any dyndep
+sidecars, rather than a materialized file or a running Ninja process.
+
+`load_manifest` uses the manifest-query registration: it permits only its
+read-only helpers and rejects template access to the environment, filesystem,
+network, clock, and shell. `load_manifest_for_build` is a separate, explicitly
+effectful loader for build, clean, generate, and graph commands. It receives a
+network policy and enables the full build stdlib; it is not a dry-run or
+background-query primitive.
+
+#### Generation reuse boundary
+
+- **Ownership:** `runner::generation` is a private runner submodule. It owns
+  the three read-only generation steps, the explicitly effectful build loader,
+  their input/output hand-offs, and the manifest and IR error contexts. It
+  does not own `StatusReporter` updates, command dispatch, dyndep publication,
+  or Ninja execution.
+- **Permitted call-sites:** `runner::generate_ninja` composes the complete
+  build pipeline through `load_manifest_for_build` for build, clean, and
+  generate commands. `runner::graph::handle_graph` may stop after `build_graph`
+  to render the graph, and `runner::help_query` uses `load_manifest` for its
+  read-only target catalogue. Runner unit tests may compose the read-only
+  steps directly. New dry-run or background-generation work may use
+  `load_manifest`, `build_graph`, and `ninja_text` only within the runner
+  boundary; a public or cross-subsystem consumer requires an explicit
+  application boundary rather than widening these internal helpers.
+- **Composition rules:** command adapters report stages before or after the
+  relevant step and wrap `ninja_text` with runner-owned generation telemetry.
+  Only `load_manifest_with_stage_reporting` translates `StageObserver` events
+  into status updates and selects the effectful build loader. Consumers must
+  not call manifest parsing, IR generation, or `ninja_gen::generate_bundle`
+  directly in parallel with this pipeline. Before an adapter writes or
+  executes a returned bundle, it must use the existing capability-injected
+  dyndep-publication path to materialize its sidecars; the read-only steps
+  never write files, start processes, or invoke effectful template helpers.
+
 ### Module: `runner::reporter`
 
 `src/runner/reporter.rs` owns construction of the run's `StatusReporter` from

src/runner/generation.rs

@@ -0,0 +1,117 @@
+//! Query-style Ninja-generation steps and the build manifest loader.
+//!
+//! Generation decomposes into three query-style steps — load the manifest,
+//! build the graph, generate the Ninja text — none of which require a status
+//! reporter. Progress reporting stays in the thin orchestration wrappers in
+//! [`super`] (`generate_ninja`,
+//! `load_manifest_with_stage_reporting`), so generation can be reused as a
+//! pure operation (for example for dry runs or background generation).
+//!
+//! [`load_manifest`] is the read-only query step: it rejects template helpers
+//! that can access the environment, filesystem, network, clock, or shell.
+//! [`load_manifest_for_build`] is deliberately separate because command
+//! execution needs the full, effectful manifest stdlib.
+
+use anyhow::{Context, Result};
+use camino::Utf8Path;
+
+use crate::ast::NetsukeManifest;
+use crate::ir::BuildGraph;
+use crate::localization::{self, keys};
+use crate::stdlib::NetworkPolicy;
+use crate::{manifest, ninja_gen};
+
+/// Optional observer for manifest-loading stages.
+///
+/// Callers that want progress reporting pass a callback translating
+/// [`manifest::ManifestLoadStage`] values into their own reporting; passing
+/// `None` keeps the pipeline free of side effects.
+pub(super) type StageObserver<'a> = Option<&'a mut dyn FnMut(manifest::ManifestLoadStage)>;
+
+/// Load and render the Netsuke manifest at `path` without effectful helpers.
+///
+/// # Examples
+///
+/// ```rust,ignore
+/// let manifest = load_manifest(Utf8Path::new("Netsukefile"), None)?;
+/// // `manifest` is rendered and ready for `build_graph`.
+/// ```
+///
+/// # Errors
+///
+/// Returns an error when the manifest cannot be read, parsed, or rendered.
+pub(super) fn load_manifest(
+    path: &Utf8Path,
+    on_stage: StageObserver<'_>,
+) -> Result<NetsukeManifest> {
+    manifest::from_path_for_manifest_query(path.as_std_path(), on_stage).with_context(|| {
+        localization::message(keys::RUNNER_CONTEXT_LOAD_MANIFEST).with_arg("path", path.as_str())
+    })
+}
+
+/// Load and render a manifest with the full, effectful build stdlib.
+///
+/// This loader is only for command execution. Templates may use configured
+/// network, cache, environment, filesystem, clock, and shell helpers.
+///
+/// # Examples
+///
+/// ```rust,ignore
+/// let manifest = load_manifest_for_build(
+///     Utf8Path::new("Netsukefile"),
+///     NetworkPolicy::default(),
+///     None,
+/// )?;
+/// // `manifest` may use build-time template helpers before `build_graph`.
+/// ```
+///
+/// # Errors
+///
+/// Returns an error when the manifest cannot be read, parsed, or rendered.
+pub(super) fn load_manifest_for_build(
+    path: &Utf8Path,
+    policy: NetworkPolicy,
+    on_stage: StageObserver<'_>,
+) -> Result<NetsukeManifest> {
+    manifest::from_path_with_policy(path.as_std_path(), policy, on_stage).with_context(|| {
+        localization::message(keys::RUNNER_CONTEXT_LOAD_MANIFEST).with_arg("path", path.as_str())
+    })
+}
+
+/// Translate a manifest into the build graph intermediate representation.
+///
+/// # Examples
+///
+/// ```rust,ignore
+/// let graph = build_graph(&manifest)?;
+/// // `graph` contains the validated targets and actions for `ninja_text`.
+/// ```
+///
+/// # Errors
+///
+/// Returns an error when graph construction or validation fails (for example
+/// on circular dependencies or duplicate outputs).
+pub(super) fn build_graph(manifest: &NetsukeManifest) -> Result<BuildGraph> {
+    BuildGraph::from_manifest(manifest)
+        .context(localization::message(keys::RUNNER_CONTEXT_BUILD_GRAPH))
+}
+
+/// Generate the Ninja bundle for a build graph.
+///
+/// # Examples
+///
+/// ```rust,ignore
+/// let generated = ninja_text(&graph)?;
+/// let (text, sidecars) = generated.into_parts();
+/// assert!(text.contains("build hello:"));
+/// assert!(sidecars.is_empty());
+/// ```
+///
+/// # Errors
+///
+/// Returns an error when Ninja synthesis fails.
+pub(super) fn ninja_text(
+    graph: &BuildGraph,
+) -> Result<ninja_gen::GeneratedNinja, ninja_gen::NinjaGenError> {
+    ninja_gen::generate_bundle(graph)
+}

src/runner/graph.rs

@@ -16,15 +16,14 @@ use crate::graph_view::GraphView;
 use crate::graph_view::render::GraphRenderer;
 use crate::graph_view::render_dot::DotRenderer;
 use crate::graph_view::render_html::HtmlRenderer;
-use crate::ir::BuildGraph;
 use crate::localization::{self, keys};
 use crate::result_json;
 use crate::status::{LocalizationKey, PipelineStage, StatusReporter, report_pipeline_stage};
 
 use super::path_helpers::{
     ensure_manifest_exists_or_error, resolve_manifest_path, resolve_output_path,
 };
-use super::{load_manifest_with_stage_reporting, process};
+use super::{generation, load_manifest_with_stage_reporting, process};
 
 /// Render the build graph in-process and write the selected artefact.
 ///
@@ -53,8 +52,7 @@ pub(super) fn handle_graph(
         .context(localization::message(keys::RUNNER_CONTEXT_NETWORK_POLICY))?;
     let manifest = load_manifest_with_stage_reporting(&manifest_path, policy, reporter)?;
     report_pipeline_stage(reporter, PipelineStage::IrGenerationValidation, None);
-    let graph = BuildGraph::from_manifest(&manifest)
-        .context(localization::message(keys::RUNNER_CONTEXT_BUILD_GRAPH))?;
+    let graph = generation::build_graph(&manifest)?;
     let view = GraphView::from_build_graph(&graph);
 
     let status_key: LocalizationKey = if args.html {

src/runner/help_query.rs

@@ -10,6 +10,7 @@ use crate::localization::{self, keys};
 use crate::status::PipelineStage;
 
 use super::super::RunnerError;
+use super::super::generation;
 use super::super::path_helpers::{ensure_manifest_exists, resolve_manifest_path};
 use super::terminal_safe;
 
@@ -179,11 +180,7 @@ fn load_manifest_for_query(
     stages: &mut Vec<PipelineStage>,
 ) -> Result<NetsukeManifest> {
     let mut on_stage = |stage| stages.push(pipeline_stage(stage));
-    crate::manifest::from_path_for_manifest_query(manifest_path.as_std_path(), Some(&mut on_stage))
-        .with_context(|| {
-            localization::message(keys::RUNNER_CONTEXT_LOAD_MANIFEST)
-                .with_arg("path", manifest_path.as_str())
-        })
+    generation::load_manifest(manifest_path, Some(&mut on_stage))
 }
 
 /// Map manifest-loading events to data that the command boundary can report.

src/runner/mod.rs

@@ -13,7 +13,7 @@ use crate::localization::{self, keys};
 use crate::output_mode;
 use crate::output_prefs::OutputPrefs;
 use crate::status::{LocalizationKey, PipelineStage, StatusReporter, report_pipeline_stage};
-use crate::{ir::BuildGraph, manifest, ninja_gen};
+use crate::{manifest, ninja_gen};
 use anyhow::{Context, Result};
 use camino::Utf8PathBuf;
 pub use error::RunnerError;
@@ -24,6 +24,7 @@ use tracing::{debug, info};
 
 /// Default Ninja executable to invoke.
 pub const NINJA_PROGRAM: &str = "ninja";
+mod generation;
 /// Environment variable override for the Ninja executable.
 ///
 /// # Examples
@@ -304,45 +305,43 @@ fn generate_ninja(
     }
 
     report_pipeline_stage(reporter, PipelineStage::IrGenerationValidation, None);
-    let graph = BuildGraph::from_manifest(&manifest)
-        .context(localization::message(keys::RUNNER_CONTEXT_BUILD_GRAPH))?;
+    let graph = generation::build_graph(&manifest)?;
 
     report_pipeline_stage(
         reporter,
         PipelineStage::NinjaSynthesisAndExecution,
         tool_key,
     );
     dyndep_generation_telemetry::instrument_bundle_generation(&graph, || {
-        ninja_gen::generate_bundle(&graph)
+        generation::ninja_text(&graph)
     })
     .context(localization::message(keys::RUNNER_CONTEXT_GENERATE_NINJA))
 }
 
-/// Load the manifest, reporting each loading stage through `reporter`.
+/// Map manifest-loading stages onto the status reporter's pipeline stages.
+fn stage_reporting_callback(
+    reporter: &dyn StatusReporter,
+) -> impl FnMut(manifest::ManifestLoadStage) + '_ {
+    move |stage: manifest::ManifestLoadStage| {
+        let pipeline_stage = match stage {
+            manifest::ManifestLoadStage::ManifestIngestion => PipelineStage::ManifestIngestion,
+            manifest::ManifestLoadStage::InitialYamlParsing => PipelineStage::InitialYamlParsing,
+            manifest::ManifestLoadStage::TemplateExpansion => PipelineStage::TemplateExpansion,
+            manifest::ManifestLoadStage::FinalRendering => PipelineStage::FinalRendering,
+        };
+        report_pipeline_stage(reporter, pipeline_stage, None);
+    }
+}
+/// Load the manifest, translating loading stages into reporter updates.
+///
+/// Thin reporting wrapper over [`generation::load_manifest`].
 pub(super) fn load_manifest_with_stage_reporting(
     manifest_path: &Utf8PathBuf,
     policy: crate::stdlib::NetworkPolicy,
     reporter: &dyn StatusReporter,
 ) -> Result<crate::ast::NetsukeManifest> {
-    let mut on_stage = |stage: manifest::ManifestLoadStage| match stage {
-        manifest::ManifestLoadStage::ManifestIngestion => {
-            report_pipeline_stage(reporter, PipelineStage::ManifestIngestion, None);
-        }
-        manifest::ManifestLoadStage::InitialYamlParsing => {
-            report_pipeline_stage(reporter, PipelineStage::InitialYamlParsing, None);
-        }
-        manifest::ManifestLoadStage::TemplateExpansion => {
-            report_pipeline_stage(reporter, PipelineStage::TemplateExpansion, None);
-        }
-        manifest::ManifestLoadStage::FinalRendering => {
-            report_pipeline_stage(reporter, PipelineStage::FinalRendering, None);
-        }
-    };
-    manifest::from_path_with_policy(manifest_path.as_std_path(), policy, Some(&mut on_stage))
-        .with_context(|| {
-            localization::message(keys::RUNNER_CONTEXT_LOAD_MANIFEST)
-                .with_arg("path", manifest_path.as_str())
-        })
+    let mut on_stage = stage_reporting_callback(reporter);
+    generation::load_manifest_for_build(manifest_path, policy, Some(&mut on_stage))
 }
 
 #[cfg(test)]