Skip to content

Probe Bedrock and Vertex provider keys (ADR-0003 follow-up) #15

Description

@johnnyhuy

Why

The current probe layer (src/main/probes/) only knows Anthropic. The catalog has 18 entries but probeProvider() returns error: No probe implemented for "<id>" for any provider whose probeKind is aws, gcp, or azure.

Bedrock and Vertex don't expose a cheap "validate this credential" HTTP endpoint the way OpenAI/Anthropic do — there's no /v1/models you can hit. The standard check is to call a tiny STS-shaped API and parse the response. Three axes:

  1. Bedrock — call bedrock:ListFoundationModels (or bedrock:ListCustomModels) with the user's AWS credentials (default chain). 403/expired token ≠ working.
  2. Vertex — call aiplatform.googleapis.com/v1/projects/<project>/locations/global/publishers/google/models with the configured ADC + project + region. The envKeys[] for Vertex is empty today so we need to surface a "Project + Region required" step in the wizard before this can be wired.
  3. AzureGET <endpoint>/openai/deployments?api-version=2024-02-01 with the user's AZURE_OPENAI_API_KEY. Or just probe the deployment the user names.

Scope

  • Wire aws probe kind: uses SDK's default credential provider chain; minimal IAM sigv4 against bedrock-runtime or bedrock control plane; returns valid | error with the AWS error code surfaced in detail.
  • Wire gcp probe kind: takes a project and region from wizard config; uses @google-cloud/aiplatform or just an authenticated REST GET; requires gcloud auth application-default login upstream.
  • Wire azure probe kind (and azure-openai): simple GET against the deployment list with the configured key.
  • Vault entry for Vertex needs to carry {project, region} alongside the credential. Either widen SecretWriteOptions or surface project/region as separate UI fields in the Keys step.
  • Update PROVIDER_CATALOG notes for cloud_creds rows to indicate "wizard will ask for project/region first".
  • Renderer Keys step: when user picks Vertex, ask for project id + region before "Save"; persist into vault alongside the credential (or as a sibling secret).
  • App + CLI both pick up the same probes via the shared runProbe() dispatcher.

Acceptance criteria

  • runProbe({ providerId: 'bedrock' }) returns { valid: true, status: 'ok' } against a working AWS account with Bedrock access.
  • Same returns { valid: false, status: 'invalid', detail: 'AWS error: ...' } for an unconfigured credential chain.
  • Vertex probe requires a saved project + region; errors sensibly without one.
  • Azure probe works against an actual deployment.
  • All three appear in the Validate button in the renderer without code changes beyond the probe files.
  • Tests (or fixture-mocked HTTP) for at least the error paths.
  • npm run typecheck && npm run lint && npm run build clean.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions