Skip to content

AI Gateway: Bedrock Application Inference Profile ARN fails via /compat/chat/completions #583

Description

@anthod0

A Bedrock Application Inference Profile ARN works via AI Gateway's Bedrock native endpoint, but fails via /compat/chat/completions.

Works

curl -X POST \
   "https://gateway.ai.cloudflare.com/v1/<CF_ACCOUNT_ID>/<GATEWAY_ID>/aws-bedrock/bedrock-runtime/us-east-2/model/arn%3Aaws%3Abedrock%3Aus-east-2%3A<AWS_ACCOUNT_ID>%3Aapplication-inference-profile%2F<PROFILE_ID>/invoke" \
    -H "cf-aig-authorization: Bearer <CF_API_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{
      "anthropic_version": "bedrock-2023-05-31",
      "max_tokens": 1024,
      "messages": [{
        "role": "user",
        "content": [{ "type": "text", "text": "Reply only OK" }]
      }]
}'

Response:

{
    "model": "claude-sonnet-...",
    "type": "message",
    "role": "assistant",
    "content": [{ "type": "text", "text": "OK" }],
    "stop_reason": "end_turn",
    "usage": { "input_tokens": 13, "output_tokens": 4 }
}

Fails: raw ARN

curl -X POST \
    "https://gateway.ai.cloudflare.com/v1/<CF_ACCOUNT_ID>/<GATEWAY_ID>/compat/chat/completions" \
    -H "cf-aig-authorization: Bearer <CF_API_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "aws-bedrock/arn:aws:bedrock:us-east-2:<AWS_ACCOUNT_ID>:application-inference-profile/<PROFILE_ID>",
      "messages": [{ "role": "user", "content": "Reply only OK" }],
      "max_tokens": 16
}'

Fails: URL-encoded ARN

{
    "model": "aws-bedrock/arn%3Aaws%3Abedrock%3Aus-east-2%3A<AWS_ACCOUNT_ID>%3Aapplication-inference-profile%2F<PROFILE_ID>",
    "messages": [{ "role": "user", "content": "Reply only OK" }],
    "max_tokens": 16
}

Also tried:

aws-bedrock/application-inference-profile/<PROFILE_ID>
aws-bedrock/<PROFILE_ID>
aws-bedrock/us-east-2/arn:aws:bedrock:us-east-2:<AWS_ACCOUNT_ID>:application-inference-profile/<PROFILE_ID>

Question: is Bedrock Application Inference Profile ARN supported through /compat/chat/completions? If yes, what is the correct model format?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions