Eval Engine API — Pay-per-call AI Evaluation

Score LLM outputs and agent trajectories against benchmark rubrics — pay per eval, no subscriptions.

Live at: https://eval.zuluworksai.com

$0.005 per evaluation  •  USDC on Base  •  x402 protocol  •  Workers AI Llama 3.1 8B

---

Quick Start

1. List available benchmarks (free)

curl https://eval.zuluworksai.com/benchmarks

2. Try an evaluation (get a 402 challenge)

curl -X POST https://eval.zuluworksai.com/eval \
  -H "Content-Type: application/json" \
  -d '{"benchmark_id":"binary-pass-fail","agent_trajectory":"{\"task\":\"Calculate 2+2\",\"steps\":[\"Add 2 and 2\"],\"result\":\"4\"}"}'

Returns a 402 Payment Required with a PAYMENT-REQUIRED header containing the x402 v2 challenge (nonce, expiry, payment details).

3. Sign and pay

Sign an EIP-3009 TransferWithAuthorization for 0.005 USDC and retry with the PAYMENT-SIGNATURE header:

curl -X POST https://eval.zuluworksai.com/eval \
  -H "Content-Type: application/json" \
  -H "PAYMENT-SIGNATURE: <base64-eip3009-payload>" \
  -H "PAYMENT-NONCE: <nonce-from-402>" \
  -d '{"benchmark_id":"binary-pass-fail","agent_trajectory":"..."}'

Returns a signed receipt with score (0-1), metric, and reasoning.

---

Benchmarks

ID	Name	Description	Metric
binary-pass-fail	Binary Pass/Fail	Did the agent complete the task correctly?	binary_pass
qa-accuracy	QA Accuracy	How accurate is the output on a 0-1 scale?	accuracy
reasoning-coherence	Reasoning Coherence	How coherent and logical is the reasoning?	coherence

---

MCP (Model Context Protocol)

MCP endpoint: https://eval.zuluworksai.com/mcp

Free methods (no payment): initialize, tools/list, resources/list, resources/read

Paid method: tools/call evaluate ($0.005)

{
  "mcpServers": {
    "eval-engine": {
      "url": "https://eval.zuluworksai.com/mcp"
    }
  }
}

---

Agent Discovery

Surface	URL	Purpose
OpenAPI Spec	/openapi.json	API contract
Agent Card	/.well-known/agent-card.json	A2A agent discovery
MCP Server Card	/.well-known/mcp/server-card.json	MCP server metadata
API Catalog	/.well-known/api-catalog	RFC 9727 discovery
Skills Index	/.well-known/agent-skills/index.json	Agent skills registry
x402 Manifest	/.well-known/x402.json	x402 payment discovery
llms.txt	/llms.txt	LLM crawler discovery

---

Registered On

Directory	Status
x402scan.com	✅ Live
tryponcho.com	✅ Indexed
AgentCash	✅ Indexed
MCP Registry	✅ Published
A2A Registry	✅ Registered
Smithery	✅ Published
awesome-mcp-servers	🟡 PR #10316
awesome-x402	🟡 PR #877

---

Architecture

Agent → POST /eval → 402 Challenge → EIP-3009 Signature → PAYMENT-SIGNATURE header
                                                              ↓
                                              CDP Facilitator (settlement)
                                                              ↓
                                              Workers AI Llama 3.1 8B (judge)
                                                              ↓
                                              Signed Receipt ← D1 (state)

Built on Cloudflare Workers with:

• D1 — Nonce tracking, spend guards, benchmark data
• Workers AI — LLM judge (Llama 3.1 8B)
• CDP — Coinbase Developer Platform x402 facilitator
• EIP-3009 — Gasless USDC transfer authorization

---

Pricing

• Per evaluation: $0.005 USDC
• Network: Base (eip155:8453)
• Asset: USDC (0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913)
• Wallet: 0xC70094bAA46Adc209EC56522b5922e5B5A2858c6
• Protocol: x402 v2 (EIP-3009 TransferWithAuthorization)

---

Contact

• Email: lee@zuluworksai.com
• Website: zuluworksai.com
• Issues: GitHub Issues