Skip to content

fix(fetchers): enforce policy for RFC source URLs#168

Open
chaliy wants to merge 1 commit into
mainfrom
2026-07-24-fix-ietf-rfc-fetcher-url-policy-bypass
Open

fix(fetchers): enforce policy for RFC source URLs#168
chaliy wants to merge 1 commit into
mainfrom
2026-07-24-fix-ietf-rfc-fetcher-url-policy-bypass

Conversation

@chaliy

@chaliy chaliy commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Motivation

  • A policy-bypass was discovered where IetfRfcFetcher constructs a canonical www.rfc-editor.org URL and calls transport_request without re-applying the caller's URL policy, allowing secondary outbound requests to evade allow_prefixes, block_prefixes, blocked_hosts, and allowed_ports checks.
  • The root cause is that registry-level validation only checked the original request URL before dispatch, not any derived/secondary URLs produced by fetchers.
  • The change closes that confused-deputy vector by ensuring derived outbound URLs are validated with the same configured policy before any transport call.

Description

  • Added a shared helper validate_url_policy(url: &Url, options: &FetchOptions) in crates/fetchkit/src/fetchers/mod.rs and switched registry validation to call it instead of duplicating logic.
  • IetfRfcFetcher::fetch now calls validate_url_policy on the derived canonical RFC Editor source URL before invoking transport_request to prevent outbound policy bypass.
  • Added regression tests in crates/fetchkit/src/fetchers/ietf_rfc.rs using a RecordingTransport to assert that blocked canonical URLs produce FetchError::BlockedUrl with no transport calls and that allowed canonical URLs still fetch normally.
  • Updated specs/fetchers.md to require that fetchers which derive secondary outbound URLs must apply the same configured URL policy to those derived URLs.

Testing

  • Ran formatting with cargo fmt --all which passed.
  • Ran unit tests including the new RFC policy tests via cargo test -p fetchkit canonical_source_url -- --nocapture and full test suite cargo test --workspace, both of which passed (new tests included and green).
  • Ran static checks and builds with cargo clippy --workspace --all-targets -- -D warnings, RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps, and cargo build --workspace --exclude fetchkit-python --release, all of which succeeded.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant