Skip to content

Composio memory sync: add Salesforce (salesforce) sync pipeline #89

Description

@M3gA-Mind

Important

The pipeline body goes in tinyhumansai/tinycortex; the wiring goes here. This is a cross-repo deliverable.

How a Composio memory-source sync is actually built (Gmail is the worked example):

Piece Lives in Job
Enginerun_incremental_sync(source: &dyn IncrementalSource, …) (vendor/tinycortex/src/memory/sync/orchestrator.rs:113) tinycortex paging, incremental filtering, ingestion — generic, toolkit-agnostic
PipelineGmailSyncPipeline (vendor/tinycortex/src/memory/sync/composio/gmail.rs:18, ~229 lines) tinycortex implements SyncPipeline + IncrementalSource; answers toolkit-specific questions (toolkit()"gmail", action()GMAIL_FETCH_EMAILS, arguments(), extract_page()). Its tick() is one line — it hands itself to the engine
ProviderComposioProvider (src/openhuman/memory_sync/composio/providers/gmail/) openhuman account/profile, daily request budget, HTML→text post-processing, cursor/state; delegates ingestion to tinycortex
Selector — the match at src/openhuman/tinycortex/sync.rs:344 openhuman maps a toolkit slug → pipeline. A missing arm is what raises tinycortex sync does not support toolkit '<slug>'

Placement decision: the pipeline body lands in tinycortex, alongside the existing six (gmail, github, notion, linear, clickup, slack). To be precise about why: this is a deliberate consistency decision, not a technical constraintIncrementalSource is an unsealed pub trait over public types (orchestrator.rs:53-104) and the engine takes a trait object, so openhuman could legally implement a pipeline locally under the orphan rule. We are keeping all pipeline bodies in one repo instead.

Order of work:

  1. tinyhumansai/tinycortex — implement the pipeline (SyncPipeline + IncrementalSource), register it, cut a release.
  2. tinyhumansai/openhuman — bump the vendor/tinycortex submodule pointer (pinned at 9a0603af), add the sync.rs:344 arm, and register the ComposioProvider so memory_sources.supported_toolkits advertises the slug.

This issue stays on openhuman because step 2 is here and this is where the user-visible failure surfaces. Do not close it on the tinycortex release alone — it is done when the slug actually syncs end to end.

Dedupe (read before writing): set metadata.path_scope to a stable collection scope; per-item IDs are dedupe keys only. PR tinyhumansai/openhuman#4953 fixed exactly this bug class (stable document_id as the upsert key) — do not reintroduce it.

Coordinate with tinyhumansai/openhuman#4993, which adds a drift guard asserting the advertised set == the syncable set. As each pipeline registers, its slug moves from "advertised but not syncable" to both.


Summary

Composio memory-source sync has no pipeline for salesforce (Salesforce). Connecting it succeeds and reports ACTIVE, then fails at sync with tinycortex sync does not support toolkit 'salesforce' — so the source silently never ingests CRM records and notes.

Problem

src/openhuman/tinycortex/sync.rs:344-353 matches on the toolkit slug and only implements six: gmail, github, notion, linear, clickup, slack. salesforce falls to the _ arm:

tinycortex sync does not support toolkit 'salesforce'

But salesforce is advertised in CAPABILITY_TOOLKITS (src/openhuman/memory_sync/composio/providers/mod.rs:63), so the app offers it as connectable. The connect flow gives no indication sync is unimplemented — the user gets a source that looks healthy and produces nothing.

Solution

Implement a Salesforce sync pipeline and register it so memory_sources.supported_toolkits reports it:

  • Fetch accounts/opportunities/notes.
  • Map to memory documents with a stable dedupe key — set metadata.path_scope to a stable collection scope (per-item IDs are dedupe keys only; see AGENTS.md).
  • Register the provider so it appears in all_composio_sync_providers() and is selectable in tinycortex/sync.rs.
  • Follow the existing pipelines as the pattern (GmailSyncPipeline for message-shaped sources, NotionSyncPipeline for document-shaped ones).

Related: PR tinyhumansai/openhuman#4953 fixed a sync upsert-key defect (use a stable document_id) — apply that lesson here rather than reintroducing it.

Acceptance criteria

  • Pipeline implemented and registeredmemory_sources.supported_toolkits includes salesforce.
  • Sync works end to end — connecting Salesforce ingests real content; no does not support toolkit error.
  • Stable dedupe — re-syncing does not duplicate; path_scope set to a stable scope.
  • Regression coverage — pipeline unit test; diff coverage >= 80%.
  • No PII/secrets logged — debug logging present on the new flow, content-free.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions