Skip to content

OrcaRouter provider support for Wallbreaker #36

Description

@zsanig22-dotcom

Wallbreaker fills a real gap in the LLM red-teaming space: a Claude-Code-style harness that makes authorized testing repeatable and measurable. What stands out is the reliability-first design - standardized HarmBench batteries rather than hand-picked examples, validate re-firing runs to measure a real success rate, and a dual-protocol provider layer that can pin a backend for reproducible results instead of treating a one-shot COMPLIED as a bypass.

Because Wallbreaker already treats every backend as an OpenAI-/Anthropic-compatible base_url plus key (profiles, target, and judge are just config), users can point at OpenRouter, Z.AI, or a local Claude Code CLI today. Adding OrcaRouter as another optional backend would give red-teamers and safety researchers more model choices for the target and judge roles, with no change to existing providers.

I'm an engineer on the OrcaRouter team. OrcaRouter exposes an OpenAI-compatible API and uses standard API-key authentication, routing to many chat and reasoning models through a single endpoint. I'd like to propose adding it as an optional provider for Wallbreaker, not replacing or altering any backend you already support.

The integration seam fits your existing config surface: in config.toml, [profiles.*], [target], and [judge] each accept protocol = "openai", a base_url, an api_key, and a model, and OpenAIProvider already consumes any OpenAI-compatible /chat/completions stream, including reasoning channels. So this would mostly mean documenting an OrcaRouter base_url/api_key pair and, if you'd like it discoverable, registering a small alias in the provider factory - no new protocol. To be clear, nothing here is implemented or tested yet; this issue is a proposal to gauge interest.

A few OrcaRouter capabilities seem genuinely relevant to Wallbreaker's users:

  • One endpoint, many models - reach a broad set of chat and reasoning models through a single key, which fits sweeping many victim models as [target] or grading with [judge].
  • Automatic model routing and provider failover - aligns with the reliability-first ethos: if an upstream backend degrades mid-run, a run can route to a healthy provider instead of dying on a network error.
  • Usage tracking and budgets - useful for labs running many autonomous engagements, keeping per-project spend visible.

OrcaRouter is already integrated by several open-source projects (e.g. RAGFlow, Dify, goose, promptfoo, NocoBase, DB-GPT, models.dev / OpenCode); a running list is at https://www.orcarouter.ai/built-with.

Full disclosure: OrcaRouter has an optional open-source partner program in which approved OSS projects can receive a 5% revenue share from usage attributed to their integration. Participation is not a prerequisite for this integration - it is optional, and we would follow whatever disclosure or governance rules you prefer.

If maintainers are open to this direction, I'd be glad to submit an implementation PR (config docs, provider alias, and a small test). Thanks for considering it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions