feat: PhysiCell substrate access + microenvironment analysis module - #5
Merged
Conversation
Close the main functional gap from the v0.2.0 defect report: the microenvironment .mat was parseable but unreachable from the high-level reader classes, so substrate-coupled analysis — the common case for tumour ABM work — could not be expressed through the API. Add two complementary views of "how much substrate": - Environmental concentration at a location. PhysiCellTimeStep gains substrates, voxel_positions, and substrate_at(name, x, y, z=None), which returns the concentration in the nearest voxel via a cached cKDTree. Verified exactly equal to a brute-force argmin nearest-voxel lookup on every cell of the example frame. - Per-cell internalized amount. internalized_substrates() returns PhysiCell's internalized_total_substrates field as a DataFrame keyed by substrate name, reflecting the framework's actual uptake dynamics rather than approximating uptake from the voxel the cell sits in. The microenvironment file is located from the XML's microenvironment/domain/data/filename element (falling back to the default naming), resolved lazily inside the timestep. discover_physicell_ timesteps keeps its 3-tuple return type, so no existing caller breaks. Adds tests/test_physicell_microenvironment.py (22 tests).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related pieces of the v0.4.0 PhysiCell/microenvironment work, plus the fixes from an in-house pair-review.
1. Substrate access on
PhysiCellTimeStep(commit 1)Closes the main functional gap from the original v0.2.0 defect report (issue #3): the microenvironment
.matwas parseable but unreachable from the high-level API. Exposes two distinct notions of "how much substrate," because they answer different questions:substrate_at(name, x, y)internalized_total_substratesinternalized_substrates()The internalized view reflects PhysiCell's actual uptake dynamics rather than approximating from the voxel a cell sits in. Also adds
substrates,voxel_positions,substrate_names.substrate_atuses a cachedcKDTreeand matches brute-forceargminexactly on all 1,035 cells of the example frame; ties are tested explicitly. Backward compatible —discover_physicell_timestepskeeps its 3-tuple return; the microenvironment file is resolved lazily inside the timestep.2.
microenvironmentanalysis module (commit 2)Implements the previously-empty stub with three composed, domain-framed analyses:
identify_niches(data, n_niches, radius=...)— spatial niches (recurring local cell-type compositions) by k-means over neighborhood composition vectors →NicheResult. Complementary to the LDA topic view.detect_boundaries(data, radius, labels=None)— interface cells scored by neighborhood "foreignness" →BoundaryResult. Generalizes the pairwisespatial.interface_cellsto any grouping; pass niche labels for niche interfaces.spatial_gradient/substrate_gradient/density_gradient— scalar-field gradients by local linear regression →GradientField(magnitude, direction). Validated against a finite-difference reference on the regular voxel grid (correlation > 0.97 on interior points; a globally linear field recovers its exact gradient).3. Pair-review fixes (commit 3)
I had a reviewer go over both pieces adversarially. It caught one real bug my own tests missed, plus two hardening items:
internalized_substrates()was broken for 1- and 3-substrate models. It hardcoded_0.._ncolumn suffixes, but the parser names a size-1 vector with the bare field name and a size-3 vector with_x/_y/_z. So oxygen-only sims (the most common PhysiCell setup) and 3-substrate sims raised a misleading "not recorded" error even though the data was present. The 5-substrate example fixture never exercised it. Now derives the column names from the field's own<labels>entry. Regression tests added for 1/3/5 substrates.spatial_gradientreturned a silent wrong answer for under-determined fits (fewer thanndim+1points, or collinear neighbors):lstsqgave a plausible minimum-norm gradient. Now clampsk ≥ ndim+1and returnsNaNfor rank-deficient fits. Tests added.substrate_atflattened multi-dimensional queries despite documenting shape preservation. Now broadcasts inputs and reshapes output. Tests added.Verification
ruff check .cleanNotes
internalized_total_substratesis all-zero — that field only accumulates when the run enablestrack_internalized_substrates_in_each_agent. Documented on the method; not a bug (columns match the raw matrix exactly).[Unreleased]in the changelog.microenvironment/module here covers niches/boundaries/gradients. It does not implement the separately-envisioned niche segmentation / spatial-domain partitioning beyond these; that can follow if wanted.