docs(fetch_winhttp): add design document for WinHTTP custom transport by sandersaares · Pull Request #595 · microsoft/oxidizer

sandersaares · 2026-07-22T13:35:16Z

[Copilot speaking]

Adds a placeholder fetch_winhttp crate whose sole content is a design document: crates/fetch_winhttp/docs/DESIGN.md. No transport code yet - this PR is to review the design before implementation.

fetch_winhttp is a Windows-only custom fetch transport that drives the OS WinHTTP API in asynchronous mode as a Tokio-free alternative to fetch_hyper. OS calls go through a bindings facade (real windows-crate impl plus a mock for unit tests), mirroring the pattern used in oxidizer_io.

The design covers the threading model (one shared OS session per built client, one transport instance per core, callbacks completing inline for processor affinity), the cancellation and FFI-ownership model (pooled RequestContext reclaimed on the final HANDLE_CLOSING), error handling, HTTP protocol negotiation, TLS (WinHTTP/Schannel-managed, with an insecure mode), compression, and request/response body streaming modelled as bytesbuf_io reads/writes. It also includes a test plan (mock-bindings unit tests plus localhost integration tests) and a chapter of feedback on the fetch API where its Hyper/Tokio-shaped assumptions force workarounds.

The document has been through several rounds of rubber-duck review; opening now for wider human review of the design.

Design-only phase: introduces the Windows-only fetch_winhttp custom transport crate as a placeholder (no implementation yet) carrying the DESIGN.md architecture document for the WinHTTP-based fetch transport. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

Apply editorial restructuring from the final rubber-duck pass: - add a concise "Architecture at a glance" (section 1.3) so the mental model precedes the detail sections - deduplicate the threading (4), context-ownership (5.4/11.3), and connection-lifetime (7.5) explanations, cross-referencing the canonical section instead of re-deriving - convert 7.5's rejected alternatives and 14.2's per-factor test list into scannable tables, keeping prose only for the sequenced factors - refine 17 to distinguish network-phase timers (transport concern) from end-to-end pipeline deadlines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

Fixes flagged by a second rubber-duck editorial pass: - Remove client-cert benefit claim from S1 that implied v1 mTLS (contradicts S9.1) - State the full session-level callback notification mask once (S5.3) instead of only naming FLAG_HANDLES; reference it from the S3 table and test plan - Correct the S14.2 session-lifetime test to match S5.3: the body reader's Arc keeps the session wrapper alive in-flight, and native parent refcounting (not the Rust Arc) carries the OS session through the final HANDLE_CLOSING - Fix atomic cross-ref S5.1 -> S5.5 - Distinguish dropping the execute future vs the response body in the cancellation hazard and test plan - Clarify one-session-per-built-client wording (S1.3) - Point WinHttpOptions comment at S12 as well - Drop misleading '64-bit Windows' qualifier on DWORD_PTR - Clarify S16 boundary: observable HTTP behaviors are contractual; only the machinery is implementation detail - Deduplicate the 'one transport-scheduled delay' claim across S12 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

github-actions · 2026-07-22T13:36:26Z

✅ Version increments look sufficient

cargo semver-checks compared the 1 crate(s) this PR publishes against their previous version-bump commit in git history. Every version increment is sufficient for the detected API changes.

Crate	Baseline	Baseline commit	This PR	Minimum required	Status
`fetch_winhttp`	new crate	—	0.1.0	0.1.0	✅ ok

This check is informational and does not block the merge.

View the check run

Copilot

Pull request overview

Introduces a new fetch_winhttp crate as a Windows-only fetch transport design artifact, allowing review of the proposed WinHTTP-based architecture before any implementation is added.

Changes:

Adds the fetch_winhttp crate with a placeholder lib.rs and a detailed docs/DESIGN.md describing the planned transport.
Wires the new crate into workspace metadata (workspace dependency list, root README and CHANGELOG, Cargo.lock).
Updates workspace spellcheck dictionary for WinHTTP/design terminology used in the new documentation.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
README.md	Adds `fetch_winhttp` to the top-level crate list.
crates/fetch_winhttp/src/lib.rs	Adds placeholder crate docs and rustdoc logo/favicon configuration.
crates/fetch_winhttp/README.md	Adds crate README describing scope/status and linking to the design doc.
crates/fetch_winhttp/logo.png	Adds crate logo asset via Git LFS pointer.
crates/fetch_winhttp/favicon.ico	Adds crate favicon asset via Git LFS pointer.
crates/fetch_winhttp/docs/DESIGN.md	Adds the WinHTTP transport design document and test plan.
crates/fetch_winhttp/CHANGELOG.md	Adds initial crate changelog stub.
crates/fetch_winhttp/Cargo.toml	Adds the new crate manifest and docs.rs metadata.
CHANGELOG.md	Adds `fetch_winhttp` to the workspace changelog index.
Cargo.toml	Adds `fetch_winhttp` to `[workspace.dependencies]`.
Cargo.lock	Adds the lockfile entry for the new crate.
.spelling	Adds WinHTTP- and design-related terms to the spellcheck allowlist.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The design-only placeholder crate had no tests, so `cargo nextest run` failed with "no tests to run" (exit 4) in the pr-test CI job. Add a trivial placeholder test until the real transport and its test suite land. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

… in AGENTS - Reword the double-verb "is ... is" sentence in DESIGN.md S1.2. - AGENTS.md: state that auto-generated crate READMEs must not be hand-edited or flagged for formatting/wording/casing in review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

martintmk · 2026-07-22T14:19:13Z

@@ -0,0 +1,1381 @@
+# `fetch_winhttp` design


nit: 1400 lines of design, can we shorten it?

Do you have specific suggestions for which sections are unnecessary or overly verbose? Hard to take action based on line count alone.

Since I had the same issue, I used an agent to help me outline my thoughts here. Please take some time go through them. If you think anything is unnecessary or verbose, I'd love to hear specific suggestions how to improve it!

fetch_winhttp_design_analysis.md

I read Ralf's analysis and more or less agree with that assessment pages. I would love to see a shortened section, max 3-4 pages with core tenets and requirements.

Many parts of this document read more as an implementation plan for agents.

What would really help is to see actual APIs and how these are used in fetch.

// do we need some arguments for win http? HttpClient::builder_winhttp(...) .winhttp(...) // is there win-http specific configuration .tls(TlsOptions::default()) // !!! THESE ARE IGNORED, we need to address this in fetch .build()

Maybe even highlight fetch knobs that are not applied.

martintmk · 2026-07-22T14:20:34Z

+
+- **No first-class way to plug in a downstream transport.** `HttpClient` and its
+  builder live in `fetch`, so a transport crate cannot add an inherent
+  `HttpClient::winhttp(..)` constructor (orphan rule). We expose free functions


it's better to expose these as extensions so you get a nice HttpClient::builder_win_http experience

martintmk · 2026-07-22T14:22:06Z

+  The slightly more verbose hello-world is worth the consistency.
+
+- **TLS configuration lives at the wrong layer.** `fetch`'s generic `TlsOptions`
+  carries rustls/native-tls material that only the Hyper transport understands;


TlsOptions are agnostic to hyper, the plumbing that consumes TlsOptions and adapts them to hypers lives in fetch_hyper

…ailable The `+ Send` bound on `bytesbuf_io`'s `ReadAsFuturesStream` in-flight read future is now merged, so state it as a provided guarantee rather than a merge-queue assumption (S11.2, S15). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

martintmk · 2026-07-22T14:26:02Z

+
+- **One shared OS session per built client**, opened at build time, `Arc`-shared by
+  all per-core instances, and immutable after setup (§4.2).
+- **One transport instance per core.** `fetch` clones and relocates the transport


we might leave this configurable, for clients with low-traffic, there is no need to create per-core instances

Might be worth noting down as a low-traffic client optimization pass (possibly deferred), though needs measurements/benchmarks to evaluate impact. Which of our current known customer scenarios would you classify as the most representative of this situation and how would you characterize their usage of fetch (i.e. what aspects would be important to replicate in a benchmark)?

…ttp-design # Conflicts: # .spelling

codecov · 2026-07-22T20:32:25Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (8ae7af6) to head (ad0ddd0).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #595   +/-   ##
=======================================
  Coverage   100.0%   100.0%           
=======================================
  Files         408      409    +1     
  Lines       36702    36703    +1     
=======================================
+ Hits        36702    36703    +1

Flag	Coverage Δ
linux	`100.0% <100.0%> (?)`
linux-arm	`100.0% <100.0%> (?)`
windows	`100.0% <100.0%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…action Both complaints were framed backwards. Restate them as fetch over-abstracting, not under-modeling: - TLS (S1.2, S17): different transports inherently support different TLS config models; that is a fact of life, not a fetch design choice. The shortcoming is fetch trying to configure TLS uniformly at the transport-agnostic level; it belongs on the transport. - Timeouts (S12.1, S17): fetch lacking fine-grained resolve/send timeouts is fine and expected - transports differ in which timers they support and what each includes/excludes. fetch should keep only pipeline-level deadlines and leave all network-phase timers to each transport, so exposing them via WinHttpOptions is the correct home, not a workaround. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

…refs - Rewrite S5.3 to lead with the one ownership rule in bold, then three tight phases (before/after SetOption, validity), instead of a dense wall of prose. - Reference plurality's Box::into_raw/from_raw as available; drop the now-merged PR #585 links in S5.3 and S15. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

Drop the pools' internal mechanics (EventPool's Arc-over-slab, MutexGuard-!Send reasoning, reference-counted backing details, thread-per-core aside) and keep only the contract-level facts: two transport-owned pools, what each rents, their Send/Sync properties, and why the plurality Pool needs a coarse uncontended Mutex. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

…traction Reframe the S17 connection-management bullet to match the TLS/timeout framing and lead with max_connections as the concrete example: it presumes a fetch-managed pool, but different transports pool connections differently (WinHTTP owns its own pool with its own knobs), so there is no single model to configure uniformly at the fetch level - it belongs on the transport. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

'Advisory' conflated two different cases. Replace with precise labels: - max_connections is 'Approximate' - the limit is applied but enforced per authority (WinHTTP) vs per pool (fetch), so a multi-host pool's effective cap differs. - connection_idle_timeout is 'Not honored' - WinHTTP exposes no idle-TTL knob, so the configured value has no effect and WinHTTP uses its own default. Also relabel the unhonored connection_lifetime rows and update the closing note. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

…example Keep-alive is a property of how each transport's stack maintains its own connections (WinHTTP applies its own per-protocol keep-alive with its own floors), so include ConnectionKeepAlive alongside max_connections/connection_lifetime/ connection_idle_timeout as connection-management config that over-abstracts at the fetch level and belongs on the transport. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

…olicy The connection_lifetime-specific warning paragraph described a general behavior. Promote it to a shared policy subchapter (S17.1) under the fetch API feedback: for any configured option the transport cannot faithfully honor, warn at build time (tracing + telemetry) and proceed, rather than silently ignoring (hides a broken guarantee) or hard-erroring (breaks drop-in use). Condense S7.5 to reference S17.1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

…rk in S11.2 The sentence described something that is not done; it adds nothing and is not emphasizing anything. Start the paragraph with the substantive pull-based/ backpressure point instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

…tive-timer backstop Rewrite S12.2 from a single dense paragraph into short, focused paragraphs and retitle it "The outer connect timeout" (it enforces a timeout, not a "delay"). Its core message is now up front: the native connect timer is per-attempt but fetch callers expect a total deadline, so the transport races the connect phase against one tick::Clock timer. Replace the unexplained "as backstop" note in the S12.1 table with a new S12.3 that states explicitly why the native receive timers are set: WinHTTP's own receive timers cannot be disabled, so we align them with the fetch value to keep the layers in agreement and to provide OS-level liveness if the caller's async executor stalls. Renumber the test subchapter to S12.4 and update cross-refs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

Telemetry will go through the observed crate rather than tracing. Add an observed::Sink field to WinHttpDeps (it is Clone/ThreadAware, so it relocates per core like the rest of the config) and note the transport emits telemetry through it (detailed in v1.1). Replace the opentelemetry/tracing dependencies with observed and reword the S17.1 warning to emit an observed event. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

Resolve Cargo.toml [workspace.dependencies] conflict: origin/main merged the separate local-dependency block into the single alphabetical dependency list. Re-apply our only delta - the fetch_winhttp local dependency entry - in its alphabetical position after fetch_tls. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8dbb7b85-e123-4586-8df2-a2a6a5b92194

ralfbiedert

Approved. The design is internally coherent, including FFI ownership, callback handoff, cancellation, and timeout reasoning. For implementation follow-up: keep the cross-crate design critique out of public rustdoc, verify docs.rs/fetch links, and hold the eventual unsafe FFI layer to the proposed mock/Miri validation.

sandersaares and others added 3 commits July 22, 2026 15:50

Copilot AI review requested due to automatic review settings July 22, 2026 13:35

Copilot started reviewing on behalf of sandersaares July 22, 2026 13:35 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Comment thread crates/fetch_winhttp/src/lib.rs

Comment thread crates/fetch_winhttp/README.md

Comment thread crates/fetch_winhttp/docs/DESIGN.md Outdated

Copilot AI review requested due to automatic review settings July 22, 2026 13:57

Copilot started reviewing on behalf of sandersaares July 22, 2026 13:57 View session

martintmk reviewed Jul 22, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into u/sasaares/fetch-winh…

4dbf204

…ttp-design # Conflicts: # .spelling

sandersaares and others added 11 commits July 23, 2026 09:03

ralfbiedert approved these changes Jul 24, 2026

View reviewed changes

Uh oh!

Conversation

sandersaares commented Jul 22, 2026

Uh oh!

github-actions Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Version increments look sufficient

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sandersaares Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ralfbiedert left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Jul 22, 2026 •

edited

Loading

sandersaares Jul 23, 2026 •

edited

Loading

codecov Bot commented Jul 22, 2026 •

edited

Loading