Modulo as front-facing server: Site abstraction, native TLS, native logging

[hugithordarson]

Summary

Modulo currently lives behind a web server (Apache/Caddy/nginx) whose primary remaining job is TLS termination + ACME cert renewal. Everything else — host routing, request forwarding, header rewriting — modulo already does. The web server layer has become a redundant hop maintained mostly out of inertia.

The README has flagged this direction since day one: "Jetty is really just a web server… it could be easier to just use Jetty as the front-facing server, meaning no double-proxying. But that's for later." This issue is the formal articulation of "later."

Absorbing TLS into modulo is the headline change, but it forces two adjacent concerns into the open:

1. Configuration model. Domains are currently a hardcoded Map<String,String> (see Modulo.java:246). Once modulo owns TLS, domains need a real conceptual home — they're tied to certificates, routing, and logging.
2. Access logging. When the web server goes away, modulo inherits responsibility for the access log. This is a sneakily-large surface (formats, rotation, per-site separation, structured sinks).

Proposed conceptual model

A three-layer separation between what's configured by the operator and what's discovered at runtime:

Site (configured)  →  App (referenced by ID)  →  Instance (discovered per app type)

Site

The first-class operator-facing object. Owns:

• Hostnames (one or many, e.g. lidamot.is + www.lidamot.is)
• TLS configuration (reference to a Cert; mode: ACME / manual / none)
• Routing rules (path → app, with a default)
• Logging configuration (destination, format, rotation)
• Policies (HTTP→HTTPS redirect, HSTS, body size limits, etc.)

App (logical, type-aware)

A stable identity that bridges Site references to the discovery layer. Owns:

• ID (used in logs/metrics)
• Type (wo, ng-objects, future types — static, etc.)
• Instance source (strategy: wotaskd poll, ng-objects registry, static directory…)
• Load-balancing strategy

Instance (discovered, ephemeral)

What modulo currently gets from wotaskd. Host, port, health, type-specific metadata (e.g. WO instance number). Discovery is pluggable per app type.

Why this shape

• Routing becomes uniform. Today there are two code paths: path-prefix matching for /Apps/WebObjects/*, then a domain-fallback map. Modeling routing rules explicitly per Site collapses both into one mechanism and naturally supports "this hostname serves a WO app at /admin and something else at /."
• App type is a strategy, not a branch. Adding ng-objects (and future types) shouldn't mean adding parallel discovery code in the proxy core — it should mean implementing an InstanceSource.
• Sites unify what's currently scattered. TLS cert, hostname list, log file, redirect policy — they all naturally hang off Site. The "conceptual container" the README has been missing.

Open design questions

These are the decisions I think are worth making explicitly, not in passing:

• Cert ownership. Are certs owned by Sites, or a separate object Sites reference? Leaning separate — cleaner when sites share domains (wildcards) and when migrating providers.
• Config storage. File-per-site directory (à la nginx sites-enabled)? Single config file? Embedded store? Leaning file-per-site — composes with git, scp, config management tools, and matches operator mental models.
• ACME providers. ACME is one cert provider; manual upload and self-signed (dev) are others. Site says "I need a cert for these hostnames"; the cert subsystem decides how to obtain/renew. Keeps the door open for DNS-01 / wildcards / corporate CAs.
• HTTP-01 challenge routing. /.well-known/acme-challenge/... must be served by modulo itself, not proxied — a reserved route that has to exist before the cert does (chicken-and-egg on first issuance).
• Hot reload. Adding/removing sites and renewing certs must not require restart. The existing adaptor-config reload timer (Modulo.java:166) generalizes into a config-watcher abstraction.
• Logging defaults. Per-site CLF-format file by default (existing log tooling just works) + optional structured JSON sink. Rotation should be logrotate-friendly (HUP to reopen).
• Management interface. Modulo's admin/status surface (currently a hardcoded self-app at Modulo.java:156) should probably live on a separate management port — keeps ACLs simple and avoids reserving hostnames/paths on user-facing sites.
• Renewal observability. Cert renewal failures fail silently in the worst way — discovered when the cert expires. Need metrics + alerting on "expires in N days, renewal failing."
• Backwards compatibility. Existing deployments have Apache in front. Modulo must keep working as a plain-HTTP reverse proxy; the TLS layer is opt-in per site.

Likely child issues (to be split out as the design solidifies)

Tentative — keeping this as one issue while the model is still fluid.

1. Site / routing model + config storage (refactor; ships value standalone).
2. App-type pluggability via InstanceSource (unblocks ng-objects and future types).
3. Native TLS termination + ACME (the headline feature).
4. Access logging.
5. Operability: renewal monitoring, cert expiry warnings, management interface separation.

Non-goals (for now)

• Replacing the web server for users who want Apache/nginx in front. Modulo as front-facing server is opt-in.
• HTTP/3, h2c on the backend (separately discussed and rejected — HTTP/1.1 keep-alive on the upstream leg is the right default).
• A web admin UI. File-based config first; UI can sit on top later.

[hugithordarson]

Related: undur/wo-adaptor-jetty#2 — the same front-facing concerns (Site, Cert, ACME, access logging) viewed from the single-app embedded side. Proposes extracting them into a shared library (modulo-frontend working name) consumed by both projects, so the design here is informed by both single-app and multi-app deployments.