StreamHouse

S3-native event streaming. One binary replaces Kafka.

StreamHouse stores everything in S3 — no broker fleets, no disk replication, no JVM. Kafka protocol compatible. 1 TB of retention costs ~$23/month.

Demo

Stream events to Postgres in 5 commands. No Kafka, no Flink, no glue code.

Get Started

Cloud (fastest)

Sign up at streamhouse.app, create an org, grab your API key.

CLI

brew install gbram1/tap/stm
stm auth login

# Create a topic
stm topic create events --partitions 3

# Produce
stm produce events --key user-1 --value '{"event":"signup","user":"alice"}'

# Consume
stm consume events --partition 0

# SQL
stm sql query "SELECT * FROM events LIMIT 10"

# Metrics
stm metrics overview

Python SDK

pip install streamhouse
# On macOS, use a venv if pip is blocked:
# python3 -m venv .venv && source .venv/bin/activate && pip install streamhouse

from streamhouse import StreamHouse

sh = StreamHouse(api_key="sk_live_...")

# Create a topic
sh.admin.create_topic("events", partitions=3)

# Produce
producer = sh.producer()
producer.send("events", key="user-1", value='{"event":"signup"}')

# Consume
consumer = sh.consumer()
records = consumer.poll("events", partition=0, offset=0)
for record in records:
    print(record.value_str)

TypeScript SDK

npm install streamhouse

import { StreamHouse } from "streamhouse";

const sh = new StreamHouse({ apiKey: "sk_live_..." });

// Create a topic
await sh.admin.createTopic("events", { partitions: 3 });

// Produce
const producer = sh.producer();
await producer.send("events", { event: "signup", user: "alice" }, { key: "user-1" });

// Consume
const consumer = sh.consumer();
for await (const record of consumer.subscribe("events")) {
  console.log(record.value);
}

SQL Queries

Query your streams with SQL — no external engine needed.

stm sql query "SELECT value->>'event' as event, COUNT(*) as cnt FROM events GROUP BY 1"

result = sh.admin.get_metrics()  # or use the SQL endpoint

const result = await sh.query('SELECT * FROM events WHERE value->>\'event\' = \'signup\' LIMIT 10');
console.log(result.rows);

Pipelines & Sink Connectors

Stream data from topics to external systems with optional SQL transforms.

# Create a Postgres sink target
stm pipeline target create my-postgres \
  --target-type postgres \
  --url "postgres://user:pass@host:5432/mydb" \
  --table events_sink

# Create a pipeline with a SQL transform
stm pipeline create signup-pipeline \
  --source-topic events \
  --target my-postgres \
  --transform "SELECT value->>'user' as user_id, value->>'event' as event_type, timestamp FROM events WHERE value->>'event' = 'signup'"

# Start it
stm pipeline start signup-pipeline

# Check status
stm pipeline list

Supported sinks: PostgreSQL, S3 (Parquet/JSON/CSV), Elasticsearch.

Kafka Protocol

Works with any Kafka client — kcat, kafka-python, confluent-kafka, etc.

# kcat
echo '{"event":"signup"}' | kcat -P -b localhost:9092 -t events -k user-1
kcat -C -b localhost:9092 -t events -o beginning

Self-Hosted

docker compose up -d

Service	URL
REST API	http://localhost:8080
gRPC	localhost:50051
Kafka	localhost:9092
Swagger UI	http://localhost:8080/swagger-ui/
Grafana	http://localhost:3001

Why StreamHouse?

Cost. Kafka replicates data 3x on broker disks. StreamHouse stores data in S3 — retention is nearly free.

Simplicity. No JVM, no ZooKeeper, no KRaft, no broker fleet. One Rust binary. SQLite for dev, PostgreSQL for prod.

Flexibility. Choose durability per-write:

Mode	Latency	Best for
acks=buffered	~1ms	High-throughput ingestion
acks=durable	~150ms	Production workloads needing S3 durability

Features

• Kafka protocol — 23 APIs, consumer groups, transactions, SASL auth
• REST + gRPC APIs — OpenAPI docs at /swagger-ui/
• SQL engine — Query streams with SQL, window aggregations, JSON operators
• Schema Registry — JSON Schema, Avro, Protobuf with compatibility checking
• Pipelines — Stream processing with SQL transforms, sink to Postgres/S3/Elasticsearch
• Multi-tenancy — Org-scoped isolation, API keys with permissions and topic scopes
• Log compaction — Tombstone handling, background compaction
• Observability — Prometheus metrics, Grafana dashboards, real-time WebSocket metrics
• Disaster recovery — S3 metadata snapshots, self-healing reconciliation

Performance

Metric	Throughput
WAL writes	2.21M records/sec
Full path (WAL → S3)	769K records/sec
gRPC ProduceBatch	100K+ messages/sec
Segment read (LZ4)	3.10M records/sec

Docs

Getting Started	Installation, first topic, Docker
Architecture	Segments, WAL, leases, rebalancing
API Reference	REST, gRPC, Kafka endpoints
Configuration	Env vars, tuning, durability
Authentication	API keys, SASL, multi-tenancy

Compared to Alternatives

	StreamHouse	Kafka	WarpStream
Storage	S3	Broker disks (3x)	S3
Runtime	Single Rust binary	JVM + ZK/KRaft	Go agents
Retention (1 TB)	~$23/mo	~$3K+/mo	~$23/mo
Built-in SQL	Yes	No	No
Schema Registry	Built-in	Separate	No

Contributing

Issues and PRs welcome. See CONTRIBUTING.md.

License

Apache License, Version 2.0