Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 17 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
[![CI](https://github.com/runcaptain/compass/actions/workflows/ci.yml/badge.svg)](https://github.com/runcaptain/compass/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-1.88%2B-orange.svg)](https://www.rust-lang.org)
[![Image](https://img.shields.io/badge/ghcr.io-runcaptain%2Fcompass-2496ED?logo=docker&logoColor=white)](https://github.com/runcaptain/compass/pkgs/container/compass)
[![Deploy](https://img.shields.io/badge/%E2%98%81%20one--command%20deploy-Kubernetes%20%2F%20AWS-326CE5?logo=kubernetes&logoColor=white)](#deploy-to-kubernetes--aws-one-command)
</div>

Built by [Captain](https://runcaptain.com) for high-throughput retrieval in on-prem enterprise deployments where customer data cannot leave their VPC.
Expand Down Expand Up @@ -42,6 +44,10 @@ cargo build --release
# Listening on http://localhost:4001
```

Or skip the build entirely: `docker run -p 4001:4001 ghcr.io/runcaptain/compass:latest`
— and for the full serverless topology on a Kubernetes cluster, see
[Deploy to Kubernetes / AWS](#deploy-to-kubernetes--aws-one-command) (one command, no clone).

Environment variables: `PORT` (default 4001), `DATA_DIR` (default ./data), `COMPASS_API_KEY` (bearer-token auth; unauthenticated when unset), `COMPASS_STORAGE` (see [Object storage](#object-storage-s3--gcs--azure)). See [`.env.example`](.env.example) for the full list.

## Examples
Expand Down Expand Up @@ -421,19 +427,21 @@ docker run -p 4001:4001 -v ./data:/app/data compass

## Deploy to Kubernetes / AWS (one command)

[`deploy/`](deploy/README.md) ships the full serverless topology as code: a
kustomize tree (serving StatefulSet with a PVC per replica, stateless writer
Deployment, optional cold tier) plus Terraform for the AWS storage half
(private encrypted S3 bucket + least-privilege IAM, IRSA-ready). Try the whole
thing on any local cluster:
One command deploys the full serverless topology — serving nodes on
persistent volumes, stateless writers, an optional cold tier, and a bundled
dev S3 — onto **any** Kubernetes cluster, no clone required:

```bash
kubectl apply -k deploy/kubernetes/overlays/minio-dev
kubectl apply -k "https://github.com/runcaptain/compass//deploy/kubernetes/overlays/minio-dev?ref=main"
kubectl -n compass-dev get pods # serving-0, writer, cold, minio
```

Production: `terraform apply` in [`deploy/terraform/aws`](deploy/terraform/aws),
point [`deploy/kubernetes/overlays/aws`](deploy/kubernetes/overlays/aws) at the
bucket, `kubectl apply -k`. Details and operational notes: [deploy/README.md](deploy/README.md).
Production on AWS is two steps: `terraform apply` in
[`deploy/terraform/aws`](deploy/terraform/aws) (private encrypted S3 bucket +
least-privilege IAM, IRSA-ready), then point
[`deploy/kubernetes/overlays/aws`](deploy/kubernetes/overlays/aws) at the
bucket and `kubectl apply -k` it. Full walkthrough and operational notes:
[deploy/README.md](deploy/README.md).

## Object storage (S3 / GCS / Azure)

Expand Down
4 changes: 3 additions & 1 deletion deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,11 @@ overlays/minio-dev self-contained dev stack (in-cluster MinIO) — try it on
overlays/aws production: real S3, IRSA or access-key auth
```

Try the whole topology on a local cluster:
Try the whole topology on any cluster — no clone needed:

```bash
kubectl apply -k "https://github.com/runcaptain/compass//deploy/kubernetes/overlays/minio-dev?ref=main"
# or, from a checkout:
kubectl apply -k deploy/kubernetes/overlays/minio-dev
kubectl -n compass-dev get pods # serving-0, writer, cold, minio
kubectl -n compass-dev port-forward svc/compass-read 4001:4001
Expand Down
17 changes: 10 additions & 7 deletions deploy/kubernetes/overlays/minio-dev/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,10 @@ resources:

images:
- name: compass
newName: compass # expects a locally imported image in dev clusters
# Pulls the published image so `kubectl apply -k <github url>` works on
# any cluster with no clone. For offline dev, import your own build under
# this exact name (docker save | ctr -n k8s.io images import / kind load).
newName: ghcr.io/runcaptain/compass
newTag: latest

secretGenerator:
Expand All @@ -25,30 +28,30 @@ secretGenerator:
disableNameSuffixHash: true

patches:
# The dev image is imported into the cluster (kind load / ctr images
# import); :latest defaults to pullPolicy Always, which would try (and
# fail) to pull from a registry. Never = use the imported image only.
# IfNotPresent: pulls ghcr on a fresh cluster, but prefers a locally
# imported image of the same name (offline dev), instead of :latest's
# default Always which would re-pull over it.
- target:
kind: StatefulSet
name: compass-serving
patch: |-
- op: add
path: /spec/template/spec/containers/0/imagePullPolicy
value: Never
value: IfNotPresent
- target:
kind: Deployment
name: compass-writer
patch: |-
- op: add
path: /spec/template/spec/containers/0/imagePullPolicy
value: Never
value: IfNotPresent
- target:
kind: Deployment
name: compass-cold
patch: |-
- op: add
path: /spec/template/spec/containers/0/imagePullPolicy
value: Never
value: IfNotPresent
- patch: |-
apiVersion: v1
kind: ConfigMap
Expand Down
Loading