Conversation
Protocol (PromoterIntegrationExpressionPredictor + construct record), MytkBenchmark (per-promoter Spearman primary, pooled secondary), registry entry, data build script, spec, and mock-adapter tests. Adapters + sequences deferred (see spec open items). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_mytk_scaffold loads the 4-record payload FASTA (3 per-promoter payloads + mScarlet readout reference, located by substring) and builds the point-insertion context on the generic _cassette_scaffold. shorkie_mytk and yorzoi_mytk mirror the Hong adapters (absolute mScarlet-CDS readout, primary-only) and are registered under PromoterIntegrationExpressionPredictor. Declares the mytk_ints_promoter data artifact. CPU scaffold tests pass.
Standalone per-task config (Shorkie + Yorzoi) for mytk_ints_promoter, kept out of the canonical default.yaml while the task is WIP. Validated by a real Shorkie run on MPS.
dev renamed benchmarks/ → docs/benchmarks/; update the three mytk docstring references to match.
tdsone
self-requested a review
June 28, 2026 15:00
Member
|
@liv-soro Could you check if you can benchmark Shorkie and Yorzoi on modal on the mytk task? |
Lock the two runtime files (mytk_ints_promoter.tsv + mytk_payloads.fasta) and upload them to the HF mirror, so a fresh clone can fetch them with `ybench data get` instead of needing a local copy. The raw SI xlsx and the two build-input CSVs stay local — the manifest's `include` allowlist ships only the two files the benchmark reads at run time. Verified by deleting the local copies and re-fetching from the mirror: both come back byte-identical.
The authors reorganized seqnn-share: the weights moved from
seqnn-share/shorkie/f{i}/model_best.h5
to
seqnn-share/shorkie_models/shorkie/f{i}/model_best.h5
The old path 404s, so `ybench data get` could not fetch Shorkie on any machine
that didn't already have it — fresh clones and the Modal seed alike. Caught by a
Modal run, which was the first one to need those weights.
Bytes are unchanged (locked sha256 still matches), so this is a URL fix only —
no re-lock. Verified by deleting a local checkpoint and re-fetching.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A new benchmark,
mytk_ints_promoter, from the MYTK MoClo toolkit(ACS Synth. Biol. 2024, doi:10.1021/acssynbio.3c00423): predict mScarlet
reporter expression for a
[promoter]-mScarlet-tTDH1payload integrated at11 genomic sites × 3 promoters (pTDH3 / pRPL18B / pREV1) = 33 constructs.
Full design + rationale:
docs/benchmarks/mytk_ints_promoter.md.Design decisions
PromoterIntegrationExpressionPredictor(distinct from Hong'sIGRInsertionExpressionPredictorbecause the promoter — not just the site —varies).
effect). Secondary: pooled Spearman (promoter-strength-dominated, reported
for context). Per-site ρ is intentionally not reported (n=3, degenerate).
conserved (point insertion as in Hong et al.); readout over the mScarlet CDS
Contents
adapters/protocols.py)MytkBenchmark(benchmarks/mytk.py)adapters/{shorkie,yorzoi}_mytk.py,_mytk_scaffold.py), registered under the new protocolscripts/mytk/build_expression_table.py), manifest entry,config (
configs/mytk.yaml)tests/test_mytk.py+tests/test_mytk_scaffold.py(mock-adapter +synthetic-genome; no GPU/data needed) — all pass
Validation
Real Shorkie run on Apple MPS: pooled ρ = 0.86 (model captures promoter
strength) but mean per-promoter ρ ≈ −0.25 (no position-effect signal, in agreement with limited variation range observed in the real data)) —
Not included / follow-ups
mytk_ints_promoter.tsv,mytk_payloads.fasta) is declared inthe manifest but not yet in the checksum lock / on the mirror — runs need
--no-data-checkand the two files copied locally until published.configs/mytk.yaml, out of canonicaldefault.yaml,while WIP.