feat(sync): add Composio asana memory-sync pipeline#113
Conversation
Implement AsanaSyncPipeline (SyncPipeline + IncrementalSource) so the `asana` toolkit ingests tasks instead of failing with "tinycortex sync does not support toolkit 'asana'". - Enumerate projects across every accessible workspace (ASANA_GET_MULTIPLE_WORKSPACES -> ASANA_GET_MULTIPLE_PROJECTS) as sync scopes, then page each project's tasks via ASANA_GET_MULTIPLE_TASKS. - Server-side incremental depth via Asana's `modified_since` filter (mirrors the GitHub pipeline), so fetched tasks are always newer than the persisted cursor and the engine's boundary check never truncates. - Stable dedupe: document_id = `asana:<gid>`, metadata.taint = "external_sync", metadata.path_scope = the project scope. Re-syncs upsert in place rather than duplicating (openhuman#4953 lesson). - Register through providers/mod.rs, composio/mod.rs and sync/mod.rs. - Content-free debug logging (scope counts only; no task titles/bodies). Tests: asana_tests.rs unit coverage (toolkit/action, extract_page, dedup/cursor, stable document_id, arguments) plus an end-to-end workspaces->projects->tasks mock in composio_sync_mock.rs asserting idempotent re-sync. `cargo fmt --all` clean; `cargo test --features sync sync::composio` green (23 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 54 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
Comment |
Summary
Adds
AsanaSyncPipeline— a Composio memory-sync pipeline for theasanatoolkit — implementing bothSyncPipelineandIncrementalSource, registered alongside the existing six (gmail,github,notion,linear,clickup,slack). Before this, connecting Asana reportedACTIVEbut sync failed withtinycortex sync does not support toolkit 'asana'and ingested nothing.Pipeline shape
Asana is task-shaped, so the pipeline follows GitHub (server-side incremental depth) combined with ClickUp (multi-scope discovery):
scopes()enumerates every accessible workspace viaASANA_GET_MULTIPLE_WORKSPACES, then lists each workspace's projects viaASANA_GET_MULTIPLE_PROJECTS.ASANA_GET_MULTIPLE_TASKSper project, paging on Asana'snext_page.offsetcursor, requestingopt_fieldssomodified_at(the incremental cursor) is present.modified_sincefilter (server_side_depth() -> true), mirroring GitHub'supdated:>cursor. Every fetched task is strictly newer than the persisted cursor, so the engine's boundary check never truncates an unordered page.tolerate_scope_errors() -> truekeeps one unreachable project from aborting the whole run.Stable dedupe (openhuman#4953 lesson)
document_id = asana:<gid>— the stable Asana task gid, never a per-run id, so re-syncs upsert in place.metadata.taint = "external_sync",metadata.path_scope = "asana/project/<project_gid>"(stable collection scope), plusproject_id/workspace_id.dedup_key = <gid>@<modified_at>so an edited task re-ingests a new version while keeping the same document id.Reference followed
src/memory/sync/composio/providers/github.rs(server-side depth) andclickup.rs(workspace→scope enumeration); shared helpers fromproviders/common.rs; engine contract inorchestrator.rs.Tests run
src/memory/sync/composio/providers/asana_tests.rs— unit coverage:toolkit()/action(),extract_page()on a sample Composio JSON payload (+ no-next_pagestop),dedup_key/sort_cursor,arguments()carryingproject/modified_since/offset, and a stableasana:<gid>document_idwithpath_scopefromdocument().tests/composio_sync_mock.rs::asana_enumerates_projects_and_stores_stable_task_documents— end-to-end workspaces→projects→tasks mock asserting stable document id, taint/path_scope/workspace_id metadata, cursor advance, and idempotent re-sync (0 duplicates).cargo fmt --allclean;cargo test --features sync sync::composiogreen (23 tests).Scope
Part of #83 (tinycortex pipeline body; openhuman wiring is the follow-up step). The openhuman-side work — bumping the
vendor/tinycortexsubmodule, adding thesync.rsselector arm, and registering theComposioProvidersomemory_sources.supported_toolkitsadvertises the slug — is intentionally not included here.🤖 Generated with Claude Code