Proposal: migration path from YAML accounts to database-backed accounts (v0.17.x)

[Kilombino]

Background

ThunderHub v0.16.0 introduced an optional database backend (PostgreSQL or SQLite via Drizzle) with a multi-user team model: users → user_nodes → nodes, with credentials stored encrypted at rest. Prior to that, node access was configured entirely through thubConfig.yaml. v0.17.0 shipped the first DB-dependent feature visible to end users: per-channel notes (#744).

These two systems are currently disjoint at runtime:

• YAML accounts are loaded into memory at startup by AccountsService.onModuleInit, indexed by a SHA-256 hash derived from the account fields.
• Database accounts live in the DB, queried via Drizzle, keyed by UUID. The JWT payload for YAML sessions carries yaml:{hash}; for DB sessions, user:{uuid}.
• A YAML-authenticated user has no user_id and therefore cannot be associated with any DB-backed feature.
• A DB-authenticated user has no path to the credentials already on disk.

This means features introduced alongside or after the DB system — starting with channel notes in v0.17.0 — are silently unavailable to users whose setup predates v0.16.0. The UI surfaces a notice, but the only resolution is the "Connect a Node" wizard, which asks for the LND socket, hex-encoded admin macaroon, and hex-encoded TLS certificate. All three values are already present on disk and referenced by thubConfig.yaml; the user ends up re-entering data they never needed to touch before.

For operators who followed distribution guides (RaspiBolt, MiniBolt, Umbrel-style setups), the YAML is often autogenerated. The manual hex-encode-and-paste step is friction the project could remove.

Reproduction context

Reproduced on v0.17.0 with a fresh DB enabled (Postgres; applies equally to SQLite). YAML at ~/thunderhub/thubConfig.yaml with one account pointing at LND. After enabling DB, login via YAML works but channel notes are unavailable. Login via DB requires going through the "Connect a Node" wizard and manually pasting hex-encoded admin macaroon and cert, despite both being on disk and referenced by the YAML.

(Note: on Postgres specifically, the wizard currently also hits a SUBSTR-on-UUID type error — tracked separately in #750. That is a bug fix, not related to this design proposal.)

Goals

1. Existing YAML users can adopt v0.17.x DB features without re-entering credentials they have already configured.
2. No regression for users who do not enable a database — DB remains fully optional.
3. No regression for users who set up DB from scratch via the wizard.
4. Migration is observable and reversible — the user knows it happened and can identify what changed.

Non-goals

• Replacing the YAML system. YAML should remain a valid, first-class configuration source.
• Silently uploading credentials to the DB without explicit user consent.
• Forcing a migration — users who never want DB features should not be prompted.

Design options

I want to present three sketches and invite the maintainer to pick a direction (or push back entirely). These are not implementation proposals; they are frames for the discussion.

---

Option A: Auto-promotion at login

When a YAML user logs in successfully and a DB is configured, check whether a users row already exists that maps to this YAML account. If not, create the users, nodes, and user_nodes rows transparently, reading macaroon and cert from the disk paths already in the YAML. Subsequent logins issue a user: JWT instead of yaml:.

Pros:

• Zero user action required; existing operators get DB features on next login.
• No documentation burden; nothing to communicate to distro guide authors.

Cons:

• Implicit. The user has no moment of consent and may not realize credentials have been persisted to a new store.
• Hard to communicate or undo — the user doesn't know what happened.
• Requires the server to read raw credential files at login time with the right filesystem permissions.
• If the YAML changes later (e.g., macaroon rotated), the DB copy silently diverges.

---

Option B: Explicit operator CLI / npm script

An npm run migrate:yaml-to-db (or similar) script, run by the operator outside the web UI. It reads thubConfig.yaml, creates the corresponding DB rows, prints a summary of what it wrote, and exits. The operator then restarts ThunderHub, and YAML accounts can log in as DB users going forward.

Pros:

• Fully explicit and scriptable; easy to document in a single line in distro guides.
• The operator runs it once with intent; no hidden state changes.
• Straightforward to test in isolation.

Cons:

• Requires an extra manual step; users who don't know it exists won't benefit.
• Needs a way to handle the encryption key consistently with what the wizard uses.
• Adds a script that must be maintained alongside schema changes.

---

Option C: UI-driven import on first login after DB is enabled

When a YAML-authenticated user logs in and a DB is configured but no corresponding users row exists, surface a dismissible banner or modal: "Import this account into the database to enable channel notes and team features?" with a single action button. On confirmation, the server creates the DB rows using the same encryption path as the wizard. Future logins use user: JWTs.

Pros:

• Discoverable — meets the user where they already are.
• Consent-explicit — user actively chooses to import.
• No distro guide changes needed beyond "click the banner if you want DB features."

Cons:

• More frontend surface area (banner, confirmation, progress/error state).
• The YAML session and the newly-created DB session are initially different JWT types; the transition needs to be handled cleanly (redirect, re-auth, or token swap).
• Same filesystem-read concern as Option A if reading macaroon/cert from disk.

---

Open questions for the maintainer

• Is a migration path within scope for the project, or do you consider DB users and YAML users permanently separate user populations (i.e., if you want DB features, you start from scratch with the wizard)?

• If in scope, which of the three options above feels most aligned with where the project is heading? Or is there a fourth option I haven't considered?

• Encryption: when the wizard stores macaroon and cert, it encrypts them with a key from database.encryptionKey. Is that key derived from JWT_SECRET, a separate env var, or something else? A CLI or UI migration path needs to use exactly the same scheme.

• Password mapping: the YAML masterPassword is hashed with a thunderhub- prefix and stored in memory. Should an imported users row mirror that password (so the operator keeps logging in with the same credential), or should they be prompted to choose a new one?

• YAML-only fields: thubConfig.yaml supports fields with no DB equivalent (healthCheckPingEnabled, channelPushEnabled, backupsEnabled, twofaSecret, authToken). Should an import preserve them somehow, or is it a clean-slate import of connection credentials only?

• Scope: given that the DB model is still fresh, is this something you'd want to address now while the code is easy to change, or after a few more releases have stabilized it?

---

Happy to sketch any of these in more detail if it helps. The main goal of this issue is to give you a clear problem statement and three concrete options to react to — I don't want to waste your time reconstructing the problem from scratch.