Skip to content

refactor[next]: merge OTF definitions into stages, extract artifacts module - #2737

Open
egparedes wants to merge 1 commit into
otf-split-0-trivial-deletionsfrom
otf-split-1-stages-artifacts
Open

refactor[next]: merge OTF definitions into stages, extract artifacts module#2737
egparedes wants to merge 1 commit into
otf-split-0-trivial-deletionsfrom
otf-split-1-stages-artifacts

Conversation

@egparedes

@egparedes egparedes commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Description

Second PR of the otf-toolchain-split stack (on top of #2736). Pure module restructuring — no class is renamed, no behavior changes:

  • Merge otf.definitions into otf.stages: the definition-stage TypeVars/aliases (IRDefinitionT, ArgsDefinitionT, ConcreteProgramDef, CompilableProgramDef) and the step contracts (TranslationStep, CompilationStep) now live in stages, which becomes the single DSL-aware vocabulary module. otf/definitions.py is deleted.
  • Extract otf.artifacts: the DSL-agnostic artifact models (ProgramSource, BindingSource, ExtensionSource, BuildSystemProject, ExecutableProgram, CompilationArtifact), all code-spec classes (otf/code_specs.py is deleted), and format_source (moved out of binding.interface, which removes the interface -> code_specs import edge and with it the last cycle risk in the bottom layer).
  • Move the ConcreteArtifact pair type from otf.toolchain to the DSL-neutral bottom module otf.workflow, with no re-export left behind.
  • Drop nominal step-protocol bases from CPPCompiler, DaCeTranslator, and DaCeCompiler — the step protocols are structural (Protocol), so subclassing them was never required; explicit mixin bases are kept where .replace() is actually used.

Net effect: otf.workflow, otf.artifacts, otf.compilation, otf.runners, and otf.binding are now formally IR-free (no imports from ffront/iterator), preparing the later split of otf into toolchain core and build infrastructure. Persistent cache keys are unchanged for all cached steps (fingerprints use fully-qualified class names; no cached class moved).

Breaking changes

No compatibility re-exports are left behind — a clean break rather than a deprecation period:

  • gt4py.next.otf.definitions no longer exists; import those names from gt4py.next.otf.stages.
  • gt4py.next.otf.code_specs no longer exists; import the code-spec classes from gt4py.next.otf.artifacts.
  • The artifact models are no longer reachable as otf.stages.<Name> — use otf.artifacts.<Name>.
  • otf.toolchain.ConcreteArtifact moves to otf.workflow.ConcreteArtifact.

Requirements

  • All fixes and/or new features come with corresponding tests. (Import moves only; existing suites run against the new layout.)
  • Important design decisions have been documented in the appropriate ADR inside the docs/development/ADRs/ folder. (No decision changed here; the naming/pipeline ADR lands later in this stack.)

https://claude.ai/code/session_01R8zRtFMhdJ8c96XJYCXkRk

@egparedes
egparedes marked this pull request as ready for review July 30, 2026 17:58
@egparedes
egparedes force-pushed the otf-split-1-stages-artifacts branch from 7af9fe4 to 4f33611 Compare July 30, 2026 17:58
…module

Restructure the `gt4py.next.otf` modules so that the DSL-aware and
DSL-agnostic halves of the toolchain vocabulary live in separate modules,
without renaming any class:

- Merge `otf.definitions` into `otf.stages`: definition-stage TypeVars and
  aliases (`IRDefinitionT`, `ArgsDefinitionT`, `ConcreteProgramDef`,
  `CompilableProgramDef`) and the step contracts (`TranslationStep`,
  `CompilationStep`) now live in `stages`; `otf.definitions` is deleted.
- Extract `otf.artifacts`: the DSL-agnostic artifact models (`ProgramSource`,
  `BindingSource`, `ExtensionSource`, `BuildSystemProject`,
  `ExecutableProgram`, `CompilationArtifact`) plus all code-spec classes
  (`otf.code_specs` is deleted) and `format_source` (moved from
  `binding.interface`, breaking the interface->code_specs import edge).
- Move the `ConcreteArtifact` pair type from `otf.toolchain` to the
  DSL-neutral bottom module `otf.workflow`.
- Drop the step-protocol base classes from `CPPCompiler`, `DaCeTranslator`
  and `DaCeCompiler`; the protocols are structural, so nothing changes for
  isinstance/fingerprint purposes (fingerprints use qualified names only).

After this, `otf.compilation`, `otf.runners` and `otf.binding` no longer
import any IR/frontend module.

Breaking: `gt4py.next.otf.definitions` and `gt4py.next.otf.code_specs` are
gone, and the names they held must be imported from `gt4py.next.otf.stages`
and `gt4py.next.otf.artifacts` respectively; `otf.toolchain.ConcreteArtifact`
moves to `otf.workflow.ConcreteArtifact`. No compatibility re-exports are
left behind.

Claude-Session: https://claude.ai/code/session_01R8zRtFMhdJ8c96XJYCXkRk
@egparedes
egparedes force-pushed the otf-split-1-stages-artifacts branch from 4f33611 to fa8a324 Compare July 31, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant