Skip to content

Reuse HTTP connections for COPC ranges#64

Merged
rsasaki0109 merged 1 commit into
mainfrom
agent/http-copc-connection-reuse
Jul 16, 2026
Merged

Reuse HTTP connections for COPC ranges#64
rsasaki0109 merged 1 commit into
mainfrom
agent/http-copc-connection-reuse

Conversation

@rsasaki0109

Copy link
Copy Markdown
Owner

Summary

  • retain one ureq::Agent inside HttpByteSource
  • share the agent connection pool and TLS state across cloned sources, parallel range workers, and request batches
  • keep URL validation, query semantics, point decoding, and the default eight-range concurrency unchanged
  • record same-output public Autzen measurements

Root cause

The ureq::get and ureq::head convenience functions create a fresh agent. The previous COPC byte source used them for every hierarchy and point-chunk range, discarding connection/TLS state after each request. Parallel batches therefore repeatedly paid connection setup costs.

Real-data result

Public Autzen central ROI output remained exactly 889,058 points.

  • pre-change observation: 95.99 s
  • shared-pool runs: 59.69 s, 37.57 s, 61.83 s
  • shared-pool median: 59.69 s
  • dated same-workload improvement against the single pre-change run: 1.61× / 37.8% lower elapsed time

Network conditions are variable; this is not a portable latency guarantee.

Validation

  • cargo test -p spatialrust-io --features io-copc-http copc::http::tests
  • cargo test -p spatialrust-io --features io-copc-http --test copc_http_local
  • cargo clippy -p spatialrust-io --features io-copc-http --all-targets -- -D warnings
  • cargo clippy --workspace --all-targets -- -D warnings
  • RUSTFLAGS="-D warnings" cargo test --workspace --no-run
  • cargo test -p spatialrust --features mvp,mvp-http --test mvp_public_copc --release -- --ignored --nocapture mvp_http_autzen_copc_bounds_smoke

@rsasaki0109
rsasaki0109 merged commit 334967e into main Jul 16, 2026
88 checks passed
@rsasaki0109
rsasaki0109 deleted the agent/http-copc-connection-reuse branch July 16, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant