feat(sync): add Composio shopify memory-sync pipeline#112
Conversation
Add ShopifySyncPipeline implementing SyncPipeline + IncrementalSource, modeled on the document-shaped Notion/Linear providers. Ingests the Shopify product catalog via SHOPIFY_GET_PRODUCTS (products carry no customer PII, unlike orders) and keys each memory document on the stable numeric product id (document_id = "shopify:<id>", taint = external_sync) so re-syncing an edited product upserts in place instead of duplicating (openhuman#4953). Incremental depth rides updated_at_min server-side; timestamps are normalized to UTC for correct cursor ordering. Registered through providers/mod.rs, composio/mod.rs, and sync/mod.rs alongside the existing six pipelines. Adds 7 unit tests covering toolkit()/action(), extract_page(), argument shaping, dedupe key, UTC sort-cursor normalization, and a stable document_id from document(). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 55 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
Comment |
Summary
Adds a Composio memory-sync pipeline for the
shopifytoolkit — the tinycortex-side deliverable of #84. Connecting Shopify previously reportedACTIVEand then failed at sync withtinycortex sync does not support toolkit 'shopify'; this supplies the pipeline body so the slug becomes syncable.ShopifySyncPipelineimplements bothSyncPipelineandIncrementalSource, modeled on the document-shaped Notion/Linear providers (a Shopify catalog is a set of stable, addressable records, not a message stream).What it does
SHOPIFY_GET_PRODUCTS("Retrieves a list of products"). Products carry no customer PII, unlike orders — the cleaner memory source; orders can be a follow-up if wanted.document()helper:document_id = "shopify:<product_id>",metadata.taint = "external_sync". Re-syncing an edited product upserts in place rather than duplicating — the stable-document_idlesson from openhuman#4953. The dedupe key folds in the modification time (<id>@<updated_at>) so edits re-ingest while the document id stays stable.updated_at_minserver-side (server_side_depth() = true); Shopify's cursor paginationpage_infotravels alone as the engine requires. Timestamps are normalized to UTC so the engine's lexical cursor comparisons order correctly regardless of the store's timezone offset.tracing::debug!flow (toolkit + connection id only).Registration
Registered exactly like the existing six pipelines:
mod shopify;+pub useinproviders/mod.rs, re-exported throughcomposio/mod.rsandsrc/memory/sync/mod.rs.Reference followed
src/memory/sync/composio/providers/notion.rsandlinear.rs(document-shaped shape), against theIncrementalSource/run_incremental_synccontract inorchestrator.rsand theSyncPipeline/SkillDocumentcontracts intraits.rs.Tests run
cargo test --features sync --lib shopify— 7 new unit tests pass (toolkit/action slugs,extract_pageon a sample Composio JSON payload, first-page vs cursor-page argument shaping, dedupe key, UTC sort-cursor normalization, stabledocument_idfromdocument()).cargo test --features sync --test composio_sync_mock— 12 existing integration tests pass (no regressions).cargo fmt --all -- --checkclean;cargo clippy --features syncclean. File is 342 lines (< 500).Part of #84 (tinycortex pipeline body; openhuman wiring is the follow-up step)
🤖 Generated with Claude Code