System map

pySTAMPS is organized around the dataset directory as the source of truth.

The CLI, runtime scheduler, ported stages, optimized kernels, and verification tools all read from or write to the same StaMPS-style dataset tree.

Read stage code paths Read science guide Open function reference

Capability Map

The architecture splits user interfaces, dataset state, stage orchestration, native Rust execution, verification, and the local web console into separate responsibilities.

Runtime layers

Layer	Main modules	Responsibility
CLI	`pystamps.cli`	Parse commands, load config, print JSON reports.
Configuration	`pystamps.config`	Normalize runtime, kernel, tolerance, tool, and compatibility settings.
Dataset I/O	`pystamps.io.dataset`, `pystamps.io.mat`	Discover patches and read/write MATLAB-compatible artifacts.
Pipeline orchestration	`pystamps.pipeline.stages`	Select stages, skip existing artifacts, schedule patch or merged work.
Scientific stages	`pystamps.pipeline.ported`	Implement StaMPS-style stage behavior in Python.
Kernels	`pystamps.kernels`	Dispatch hot numerical kernels to Python, native Rust/CPU, or CUDA providers.
Native Rust core	`crates/pystamps-core`	Plan native runs, own the `pystamps-native` CLI, execute native stages, and report coverage.
Native stage registry	`crates/pystamps-stages`	Record stage-scope readiness and parity certification status.
Rust MAT support	`crates/pystamps-mat`	Write MATLAB v5 artifacts from Rust-owned stage code.
Rust web console	`crates/pystamps-web`	Serve the local run console and native coverage API.
Verification	`pystamps.verify`, `scripts/validate_audit.py`	Compare outputs against golden datasets and audit manifests.

Pipeline model

Stage	Scope	Pipeline name	Expected progress artifact
1	Patch	Initial load	`PATCH_*/ps1.mat`
2	Patch	Estimate gamma	`PATCH_*/pm1.mat`
3	Patch	Select PS pixels	`PATCH_*/select1.mat`
4	Patch	Weed adjacent pixels	`PATCH_*/weed1.mat`
5	Patch and merged	Correct phase and merge	`PATCH_*/ph2.mat`, root `ifgstd2.mat`
6	Merged	Unwrap phase	root `phuw2.mat`
7	Merged	Calculate SCLA	root `scla2.mat`
8	Merged	Filter SCN	root `uw_space_time.mat`

Patch-scoped stages run once per discovered PATCH_* directory. Merged stages run once at the dataset root. For implementation details by stage, use Stages and Code Paths.

Artifact-driven scheduling

Before running a stage, pystamps.pipeline.stages checks the expected artifact or merged stage bundle. If the artifacts already exist, the result status is skipped_existing.

Status	Meaning
`planned`	Dry-run selected the stage but did not execute it.
`completed`	Stage executed or strict reference replay copied the expected bundle.
`skipped_existing`	Expected artifacts were already present.
`skipped`	No artifact mapping exists for that scope.
`failed`	Stage raised an execution error.

Backend and kernel architecture

pySTAMPS separates runtime scheduling from numerical kernel selection. Runtime settings control thread/process/GPU-oriented scheduling. Kernel settings choose the reference Python implementation, compiled native Rust/CPU implementation, or CUDA provider where available.

uv run pystamps describe-backends

Runtime backends

auto, threads, processes, gpu, native.

Kernel backends

python, native, cuda, auto.

Optimized kernels

stage2_grid_accumulate
stage2_histogram
stage2_topofit
stage2_topofit_row_invariant
stage2_topofit_coh_row_invariant
stage4_edge_stats
stage7_scla
stage8_edge_noise

runtime:
  backend: auto
  stage2_kernel_backend: native
  stage2_native_threads: 0
  kernel_backend_overrides:
    stage2_grid_accumulate: native
    stage2_histogram: native
    stage2_topofit: native
    stage2_topofit_row_invariant: native
    stage2_topofit_coh_row_invariant: native
    stage4_edge_stats: native
    stage7_scla: native
    stage8_edge_noise: native
  io_workers: 8
  cpu_workers: 0
  stage7_chunk_ps: 100000
  stage8_chunk_edges: 200000

Workflow Graph

This graph focuses on the command flow around status, run, verify, and native gate reports.

Verification architecture

Single-run verification compares one run tree to one golden tree. Full parity audit is handled by make audit and reads the maintained manifest.

uv run pystamps verify --run RUN_DIR --golden GOLDEN_DIR
make audit

The audit dataset list is owned by pystamps/data/audited_workflow_manifest.json. Oracle precedence is owned by pystamps/data/oracle_contract.json.

Practical boundaries

The package implements stages 1 through 8 in pystamps.pipeline.ported.
The native Rust CLI owns parity-certified stage scopes across stages 1 through 8 plus Stage-5 merged aggregation.
The optimized Python extension accelerates selected hot kernels, while the Rust CLI can execute the full native chain directly.
External tools such as triangle and snaphu remain relevant for non-native fallback workflows.
Parity should be claimed from verify or audit evidence, not from command completion alone.
Speed should be claimed from make benchmark or scripts/benchmark_backends.py, not from a skipped pipeline run.