Skip to content

feat(sync): add Composio shopify memory-sync pipeline#112

Closed
CodeGhost21 wants to merge 1 commit into
tinyhumansai:mainfrom
CodeGhost21:feat/composio-shopify-sync
Closed

feat(sync): add Composio shopify memory-sync pipeline#112
CodeGhost21 wants to merge 1 commit into
tinyhumansai:mainfrom
CodeGhost21:feat/composio-shopify-sync

Conversation

@CodeGhost21

Copy link
Copy Markdown
Collaborator

Summary

Adds a Composio memory-sync pipeline for the shopify toolkit — the tinycortex-side deliverable of #84. Connecting Shopify previously reported ACTIVE and then failed at sync with tinycortex sync does not support toolkit 'shopify'; this supplies the pipeline body so the slug becomes syncable.

ShopifySyncPipeline implements both SyncPipeline and IncrementalSource, modeled on the document-shaped Notion/Linear providers (a Shopify catalog is a set of stable, addressable records, not a message stream).

What it does

  • Fetch — the product catalog via the Composio action SHOPIFY_GET_PRODUCTS ("Retrieves a list of products"). Products carry no customer PII, unlike orders — the cleaner memory source; orders can be a follow-up if wanted.
  • Stable dedupe — each memory document is keyed on the stable numeric product id via the shared document() helper: document_id = "shopify:<product_id>", metadata.taint = "external_sync". Re-syncing an edited product upserts in place rather than duplicating — the stable-document_id lesson from openhuman#4953. The dedupe key folds in the modification time (<id>@<updated_at>) so edits re-ingest while the document id stays stable.
  • Incremental — depth rides updated_at_min server-side (server_side_depth() = true); Shopify's cursor pagination page_info travels alone as the engine requires. Timestamps are normalized to UTC so the engine's lexical cursor comparisons order correctly regardless of the store's timezone offset.
  • No PII/secrets logged — reuses the shared orchestrator's content-free tracing::debug! flow (toolkit + connection id only).

Registration

Registered exactly like the existing six pipelines: mod shopify; + pub use in providers/mod.rs, re-exported through composio/mod.rs and src/memory/sync/mod.rs.

Reference followed

src/memory/sync/composio/providers/notion.rs and linear.rs (document-shaped shape), against the IncrementalSource / run_incremental_sync contract in orchestrator.rs and the SyncPipeline / SkillDocument contracts in traits.rs.

Tests run

  • cargo test --features sync --lib shopify — 7 new unit tests pass (toolkit/action slugs, extract_page on a sample Composio JSON payload, first-page vs cursor-page argument shaping, dedupe key, UTC sort-cursor normalization, stable document_id from document()).
  • cargo test --features sync --test composio_sync_mock — 12 existing integration tests pass (no regressions).
  • cargo fmt --all -- --check clean; cargo clippy --features sync clean. File is 342 lines (< 500).

Part of #84 (tinycortex pipeline body; openhuman wiring is the follow-up step)

🤖 Generated with Claude Code

Add ShopifySyncPipeline implementing SyncPipeline + IncrementalSource,
modeled on the document-shaped Notion/Linear providers. Ingests the
Shopify product catalog via SHOPIFY_GET_PRODUCTS (products carry no
customer PII, unlike orders) and keys each memory document on the stable
numeric product id (document_id = "shopify:<id>", taint = external_sync)
so re-syncing an edited product upserts in place instead of duplicating
(openhuman#4953). Incremental depth rides updated_at_min server-side;
timestamps are normalized to UTC for correct cursor ordering.

Registered through providers/mod.rs, composio/mod.rs, and sync/mod.rs
alongside the existing six pipelines. Adds 7 unit tests covering
toolkit()/action(), extract_page(), argument shaping, dedupe key, UTC
sort-cursor normalization, and a stable document_id from document().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 55 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f20d057d-ed40-48db-b798-bb710195df15

📥 Commits

Reviewing files that changed from the base of the PR and between 9a0603a and 4b57390.

📒 Files selected for processing (4)
  • src/memory/sync/composio/mod.rs
  • src/memory/sync/composio/providers/mod.rs
  • src/memory/sync/composio/providers/shopify.rs
  • src/memory/sync/mod.rs

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant