Preserve the blob codec across replication (harper#2443)

[kriszyp]

Preserve the blob codec across replication (harper#2443)

Rebased onto main at 72f1a604 without changing the advertised, backward-compatible deflate stored-body transfer: the capability negotiation and receiver-side inflater preserve a compressed body only for an opting-in peer. The merged core companion is already in main, so this PR has no core gitlink diff or outstanding companion dependency.

Where to look hardest: the bounded repair inflater and its codec-preserved receive path are the durability boundary; a malformed announced size must never permit an unbounded inflate.

For the human reviewer

• The receiver deliberately adopts the sender's codec even when its own storage.blobs.compression setting is off; reversing that choice would discard the feature's no-recompression benefit.
• The current rebase head has no independent-review receipt: the required full review ran Gemini successfully, but its graded Codex leg was terminated by the 900-second foreground deadline before producing an artifact. The preserved machine footers below belong to the pre-rebase 3fca85a head and do not cover 46f3cd0.

Verification

• Existing end-to-end route: the cluster codec-preservation test passed: 1 test, 0 failures (15.7s); it confirms the advertising peer receives the compressed bytes verbatim while the kill-switched peer converges uncompressed.
• npm run test:unit passed: 855 tests, 0 failures.
• Local npm run build could not complete after clean installs because this worktree's TypeScript resolution reports errors in analytics/profile.ts, core/server/threads/socketRouter.ts, and existing WebSocket/Error typings in replication/replicationConnection.ts; the corresponding GitHub Build Harper Pro jobs passed on Node 22, 24, and 26.5.

Refs HarperFast/harper#2443

🤖 Generated with GPT-5 Codex

_{Review-Coverage: authored=claude; ran=gemini,codex; adjudicated=domain; declined=cursor-grok,cursor-composer; rounds=9 @ 3fca85a}

_{Human-Review-Need: 4 (decisions: codec-crosses-node-policy, kill-switch-env-only, stored-body-verify-failure-permanent, tie-inflate-cost, announcement-in-sync-prefix, no-in-place-retry-for-stored-body) @ 3fca85a}

[gemini-code-assist]

Code Review

This pull request implements replication blob-codec preservation (harper#2443), allowing raw stored deflate-compressed blobs to be streamed directly to peers that advertise support, avoiding unnecessary inflation and recompression. It introduces capability negotiation, safe repair inflation with size limits to prevent compression bombs, and updates replication connection handling. Feedback on these changes highlights a potential resource leak if ws.send throws synchronously during the blob announcement before the main try-catch block is entered, and suggests logging teardown errors in the integration tests rather than silently swallowing them.

[gemini-code-assist]

If ws.send throws an error synchronously (for example, if the WebSocket connection is closed or in an invalid state), storedBody has already been opened and releaseBlobHold has been acquired, but neither will be closed or released because the error propagates before entering the main try block at line 6345. This can lead to a file descriptor and resource leak under network instability or high load. Wrapping the synchronous announcement in a try-catch block to clean up resources on failure prevents this leak.

Suggested change

	let storedBody: ReturnType<typeof openStoredBlobBody>;
	if (peerAcceptedBlobCodecs.has('deflate') && (blob as Blob & { storedCodec?: string }).storedCodec === 'deflate') {
	storedBody = openStoredBlobBody(blob);
	if (storedBody) {
	ws.send(
	encode([
	BLOB_CHUNK,
	{ fileId: id, transferId, size: storedBody.size, codec: storedBody.codec },
	Buffer.alloc(0),
	])
	);
	}
	}
	let storedBody: ReturnType<typeof openStoredBlobBody>;
	if (peerAcceptedBlobCodecs.has('deflate') && (blob as Blob & { storedCodec?: string }).storedCodec === 'deflate') {
	storedBody = openStoredBlobBody(blob);
	if (storedBody) {
	try {
	ws.send(
	encode([
	BLOB_CHUNK,
	{ fileId: id, transferId, size: storedBody.size, codec: storedBody.codec },
	Buffer.alloc(0),
	])
	);
	} catch (error) {
	storedBody.close();
	releaseBlobHold?.();
	blobsBeingSent.delete(blobSendKey);
	throw error;
	}
	}
	}

[gemini-code-assist]

According to the general rules, test cleanup hooks should log any failures with identifying information (such as hostnames) for easier diagnosis rather than silently swallowing them with .catch(() => null). Logging the error with the node's hostname helps troubleshoot teardown failures in CI/CD environments.

	after(async () => {
		await Promise.all(
			[ctx.nodeA, ctx.nodeB, ctx.nodeC].map((node) =>
				node && teardownHarper({ harper: node }).catch((error) => {
					console.error(`Failed to teardown Harper node ${node.hostname || 'unknown'}:`, error);
				})
			)
		);
	});

References

1. In test cleanup hooks, wrap individual process termination or cleanup steps in separate try-catch blocks to ensure they are attempted independently, and log any failures with identifying information (such as hostnames) for easier diagnosis.

[claude]

Reviewed; no blockers found.

[claude]

Suggestion (non-blocking): this catch branch (cleaning up storedBody/releaseBlobHold/blobsBeingSent when the codec announcement's ws.send throws synchronously) has no test exercising it — the commit message itself notes sendBlobs is internal to the connection with no isolated harness. Worth a regression test that drives replicateOverWS with a mock ws whose send throws on the announcement frame, then asserts the hold/stored-body/blobsBeingSent entry are released rather than stranded, so a future refactor of this cleanup can't silently reintroduce the leak this PR fixes.