feat: integrate asset count API for exact pagination totals (DX-9336)#295
Open
naman-contentstack wants to merge 1 commit into
Open
feat: integrate asset count API for exact pagination totals (DX-9336)#295naman-contentstack wants to merge 1 commit into
naman-contentstack wants to merge 1 commit into
Conversation
The assets/folders list responses cap their count field at 10,000,
which silently truncated exports of spaces with more than 10k items.
Pagination is now driven by the dedicated count endpoint
(GET /api/bff/spaces/{uid}/assets/count, is_dir toggle for folders):
- paginate() takes a required caller-resolved total (single flow, no
fallback to the capped response count)
- getAssetsCount/getFoldersCount fail the export when the exact total
is unavailable (paginating blind means silent data loss)
- permanently failed pages no longer vanish: streamWorkspaceAssets
returns { streamed, missing } and missing records surface as
failedAssets in the export global summary (previously hardcoded 0)
- spaces/fields/asset_types resolve totals via a limit=1 probe
- remove dead buffered getWorkspaceAssets (zero callers, unbounded
memory once the 10k cap is gone)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes silent truncation of AM exports for spaces with >10k assets/folders. The assets/folders list responses cap their
countfield at 10,000 server-side, so pagination planned from that count stopped at 10k and dropped everything beyond it.Pagination in
@contentstack/cli-asset-managementis now driven by the dedicated count endpoint:Changes
paginate(): takes a required, caller-resolvedtotal— never derives totals from list responses, no fallback path to the capped countgetAssetsCount/getFoldersCount: authoritative totals for assets/folders; a missing/unfetchable total hard-fails the export (paginating blind = silent data loss)streamWorkspaceAssetsreturns{ streamed, missing }; missing metadata records + failed binary downloads roll up asfailedAssetsinto the export global summary (previously hardcodedfailureCount = 0). Failed pages are recoverable via re-export/query-export, so they don't abort the runlimit=1probe resolves totals for spaces/fields/asset_types (no count endpoint exists for these)getWorkspaceAssets(buffered variant): zero callers, and unbounded memory once the 10k cap is gone.talismanrc: ignore entry for a false-positive secret pattern in a test titleTesting
contentstack-asset-management: 259 unit tests passing (12+ new: count API, single-flow pagination incl. beyond-cap planning, failed-page →missing,failedAssetspropagation)contentstack-export: 510 tests passingtsc -b)Jira: DX-9336
🤖 Generated with Claude Code