Distill

Distill crawls a set of Atom/RSS feeds, stores their entries in a local SQLite database, and exposes a RESTful HTTP API for managing feeds and reading back their entries — including keyword search across an entry's title and summary.

Running

go run .

This opens (creating if necessary) feeds.db in the working directory and starts the API server on :8080. Feeds aren't crawled automatically; use the /feeds/refresh endpoints below to trigger a crawl.

To generate daily briefs (see below), copy .env.example to .env and fill in the API key(s) for whichever LLM provider(s) — and, optionally, email provider — you want to use. .env is gitignored and loaded automatically on startup.

See api.http for runnable examples (VS Code REST Client extension) covering the full flow: registering feeds, refreshing them, querying entries, and generating a digest.

API

Feeds support full CRUD. Entries are read-only, since they're populated by crawling rather than authored directly.

Method	Path	Purpose
POST	/feeds	create a feed (409 on duplicate)
GET	/feeds	list feeds
GET	/feeds/{id}	get a feed
PUT	/feeds/{id}	update a feed's URL
DELETE	/feeds/{id}	delete a feed + its entries
DELETE	/feeds	delete every feed + their entries
POST	/feeds/refresh	crawl all feeds, persist entries
POST	/feeds/{id}/refresh	crawl one feed
GET	/feeds/{id}/entries	entries for one feed
GET	/entries	list entries, ?keyword= or ?after= filter
GET	/entries/{id}	single entry with full content
POST	/digest	generate (and optionally email) a daily brief from the last 24h of entries, ?provider=claude|openai|deepseek (default claude)

?keyword= matches entries whose title or summary contains the given keyword (case-insensitive substring match). ?after= filters to entries published after the given RFC3339 timestamp.

/digest asks the selected LLM provider to summarize everything published in the last 24 hours into a brief aimed at a lead technical engineer. The provider's API key (and optional model override) is read from the environment — see .env.example.

If a recipient is available — ?to= or EMAIL_TO in .env — the brief is also emailed, rendered as simple HTML (headings/bold/links/lists) instead of raw markdown. ?email_provider=webde|gmail (default webde, or EMAIL_PROVIDER) picks the SMTP account to send from; email delivery failure is reported in the response (email_sent/email_error) but doesn't fail the request, since the brief itself was still generated.

Project layout

• main.go — loads .env, opens the database, and starts the API server.
• internal/feeds — fetches and parses Atom/RSS feeds.
• internal/database — SQLite persistence and queries for feeds and entries.
• internal/api — the HTTP API handlers built on top of internal/database.
• internal/llm — minimal clients for the Claude, OpenAI, and DeepSeek APIs.
• internal/digest — builds the daily-brief prompt from recent entries, calls an internal/llm.Client, and renders the result as HTML for email.
• internal/email — sends HTML email over SMTP, configured per provider (web.de, Gmail) via environment variables.
• internal/config — tiny .env file loader.