Skip to content

fix(fetchers): enforce GitLab API URL policy#172

Open
chaliy wants to merge 1 commit into
mainfrom
2026-07-24-fix-gitlab-fetcher-url-policy-bypass
Open

fix(fetchers): enforce GitLab API URL policy#172
chaliy wants to merge 1 commit into
mainfrom
2026-07-24-fix-gitlab-fetcher-url-policy-bypass

Conversation

@chaliy

@chaliy chaliy commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Motivation

  • A GitLab fetcher derived an https://gitlab.com/api/v4/... URL from a validated page URL and called transport without reapplying outbound policy, allowing an egress-policy bypass (port and prefix checks) for the hardcoded GitLab API.
  • The change closes this gap by ensuring any synthesized specialized-fetcher URLs are validated against the caller's FetchOptions before transport execution.

Description

  • Add enforce_url_policy(url, options) helper in crates/fetchkit/src/fetchers/mod.rs that consolidates options.validate_url plus allow/block prefix checks and is pub(crate) for reuse.
  • Replace the inline registry validation with a call to enforce_url_policy(&parsed_url, options) in validate_and_find_fetcher so initial dispatch behavior is unchanged but centralized.
  • Have GitLabFetcher::fetch call enforce_url_policy(&api_url, options)? before invoking transport_request, preventing API subrequests from bypassing allowed_ports, blocked_hosts, allow_prefixes, or block_prefixes.
  • Add a regression test gitlab_fetcher_enforces_policy_on_api_subrequest in crates/fetchkit/tests/transport.rs and update specs/fetchers.md to document that specialized-fetcher API subrequests must reapply URL policy.

Testing

  • Added integration test gitlab_fetcher_enforces_policy_on_api_subrequest which passed locally (ok).
  • Ran cargo test --workspace and all workspace tests passed (343 passed; 0 failed).
  • Ran cargo clippy --workspace --all-targets -- -D warnings, RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps, and cargo build --workspace --exclude fetchkit-python --release; all completed successfully.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant