Adopt appliances by port redirection instead of SoftAP provisioning

[3735943886]

This started from @kkqq9320's fork — https://github.com/kkqq9320/rethink — which is where the certificate, /route and bridge-registration work came from. Five of the commits here are his, included with his agreement.

What this adds

Two capabilities that together let an appliance be adopted and kept, rather than taken over:

An appliance can be added without SoftAP provisioning. Today an appliance has to be put through SoftAP setup to reach rethink. That is not always possible or wanted — it means resetting a unit that is working, and on a multi-split installation it means doing that to every indoor unit. With this branch an appliance can instead be pointed at rethink purely by redirecting its traffic at the router (port DNAT on 443 and 8883). It is never taken out of setup mode, nothing on it is changed, and removing the redirection hands it straight back to the manufacturer.

Bridge mode no longer destroys the LG registration. Enabling a bridge used to delete the appliance from the owner's home and register it again, which renames it, announces a removal to every app on the account, and leaves the appliance unable to reach LG on its own. It now keeps the registration it has.

Running together, this gives local control in Home Assistant while the official app, Google Home and the ThinQ integration keep working.

What changed

• Serve a certificate per requested TLS server name, signed by the CA the appliance already pinned, so units that ask for different LG hostnames all connect.
• Let /route echo back the name the appliance asked for, so a redirected appliance keeps using an address it already resolves and stores nothing that outlives the redirection (advertise_requested_host, off by default).
• Keep an appliance's existing LG registration when enabling a bridge, instead of deleting it and building a new one.
• Introduce a bridged appliance to the cloud with its real deploy appInfo/platformInfo instead of fixed placeholders.
• One implementation of the openssl calls in util/pki.ts: failures are reported instead of passed on as empty output, serials are unique, names are validated before reaching the command line, and no shell is needed.
• Answer a failed CSR signing with a failure, rather than resultCode 0000 and an empty certificate.
• Fix the monitor page's websocket address, which pointed at a path nothing serves.
• Fix the management panel showing every bridge switch as off after any update.

Adding an appliance

The flow has nothing in common with SoftAP provisioning — the appliance is never put into setup mode and never leaves the state it is already in:

1. Set the appliance up as normal, with the official ThinQ app. It ends up registered to the owner's account, which is where it stays.
2. Start rethink, with advertise_requested_host enabled so /route leaves the appliance on a name it already resolves.
3. Add the DNAT rules for that appliance's address: tcp/443 and tcp/8883 to the rethink host. DNS is left alone, so the appliance goes on resolving the manufacturer's real addresses.
4. Break the appliance's existing connection to the cloud. The nat table is consulted only for the first packet of a connection, and the appliance holds a long-lived MQTT session, so until that session drops it keeps talking straight past the new rule. Dropping its conntrack entries does it; so does power-cycling the appliance.
5. It reconnects and lands on rethink. Nothing was changed on the appliance and nothing was stored on it that outlives the rule.
6. Turn on bridge mode and it keeps working in the official app, and in anything attached to that account, while rethink drives it locally.

What this buys, and what it costs

Against SoftAP provisioning:

• Simpler. A couple of router rules and a reconnect, instead of putting the appliance into setup mode and driving the SoftAP handshake through to the end.
• More likely to work. SoftAP setup has to succeed at a sequence the appliance controls, and that is the step most likely to fail. Redirection only needs the appliance to open the connection it was going to open anyway.
• Nothing on the appliance is reconfigured. No wifi reset, no re-entering credentials, no losing the registration it already has.

What it asks for in return:

• A router that can DNAT, and access to configure it.
• A fixed address for the appliance, since the rules match on its source address. A DHCP reservation is enough.
• While the rule is in place, everything that appliance does on 443 goes to rethink, not only ThinQ traffic.
• ThinQ1 appliances are not covered. They reach the cloud on different ports and have no /route step to redirect them with, so the flow above is ThinQ2 only. The certificate work applies to their listeners too, but nothing here has been tried against a ThinQ1 unit.

Why this works

The appliance is never told about rethink. It rebuilds its trust in the cloud every time it attaches, and the step it rebuilds it from is not authenticated:

1. It opens HTTPS to the API port, and does not check the certificate there. Whatever answers is the cloud.
2. Over that connection it asks for /route/certificate and pins what comes back. That is now rethink's CA.
3. /route tells it which servers to use from here on.
4. It opens MQTT — and here it does check the certificate. What it gets is a leaf carrying the name it asked for, signed by the CA it pinned a moment earlier, so the check passes.
5. From there it runs its ordinary provisioning exchange, and rethink answers it.

Steps 1 and 2 are what make the rest possible: the appliance will hand out its own trust anchor on request, over a channel that authenticates nobody. Everything after that is the appliance doing exactly what it does against the manufacturer's servers.

Why the redirection is done by port, not by DNS

Pointing the LG hostnames at rethink in DNS looks like the obvious way to do this, and it does not hold up. An appliance does not talk to one name: it opens connections to a spread of *.lgthinq.com hosts on 443 and 8883 that differ by region and by model. Enumerating that set means keeping up with names the appliance has not asked for yet, and getting it wrong fails silently — the appliance simply carries on to the manufacturer.

The names also do not stop at LG. Several are CNAMEs onward into AWS, so what an appliance actually resolves is an Amazon endpoint that is shared with everything else hosted there and does not stay put. Overriding by address is therefore out, and following the chain means intercepting names that have nothing to do with LG. There is no version of this that is a tidy list.

And there is no way back: the appliance caches what it resolved, so removing a DNS entry does not release it.

That is the failure reported in #58 #104: redirecting common.lgthinq.com was enough for an air conditioner, a washer and a dehumidifier, while an air purifier and a cooktop on the same network went on registering with the manufacturer. They were not asking for that name, and there is no list of the names they do ask for. Redirected by port, both of them attach — as does every other appliance on that network.

That is also why one certificate is not enough. Because the appliance still asks for whichever LG name it was going to use, rethink has to be able to answer to all of them.

Why the redirection path needed work

An appliance that arrives by redirection behaves differently from one that was set up against rethink, in two ways that both had to be handled:

It asks for whatever LG hostname its firmware carries, and units of the same model do not agree on which. Observed across four CST_570004_WW units on one installation: some ask for kic-common.lgthinq.com, others for kic-mclip.lgthinq.com, with common.iot.kic.lgthinq.com on the MQTT port. rethink presented one certificate named after config.hostname, so pinning it to one of those names dropped the others. The API port does not check the certificate, so this only showed up on the MQTT port — as a handshake that failed for some units and worked for others. rethink now mints a leaf per requested name, signed by the CA the appliance already pinned when it fetched /route/certificate, and hands it out through SNICallback. Connections asking for config.hostname, or asking for nothing, are unaffected.

/route told it to use config.hostname from then on. That is right for an appliance set up over SoftAP, but wrong for a redirected one: it now needs a DNS record it did not need before, and because the appliance stores what /route tells it, it keeps asking for that name long after the redirection is gone. Removing the redirection then no longer hands the appliance back — which was the one property that made redirection attractive. advertise_requested_host (off by default) makes /route echo the name the appliance asked for instead.

Commits

Bug fixes, independent of the rest:

• Fix the monitor page's websocket address — the page built its socket address by appending to its own path, so opening /monitor asked for /monitordevice and the connection never came up.
• panel: keep the bridge switches showing what is actually bridged — toggling one appliance's bridge made every switch in the table read as off. Nothing had turned off; the row markup is rebuilt from a string and the row's state was never re-applied.
• bridge: keep the existing LG registration when enabling a bridge — the destructive-registration fix described above.

Redirected-appliance support:

• Serve a certificate per requested TLS server name — the SNI work described above. Names are checked against a hostname pattern before reaching openssl and the number of distinct certificates is capped, so a peer on the network cannot drive unbounded process spawning.
• Note why the provisioning routes keep the CA rather than the SNI options — comment only; those routes hand out the CA itself, so they must not serve a per-name leaf.
• Add an option to leave a redirected appliance on the name it already uses — advertise_requested_host, off by default.

Certificate handling:

• pki: one implementation of the openssl calls — five places shelled out to openssl and disagreed on the parts that matter: whether a failure is noticed, whether the serial is unique, whether a name reaches the command line unchecked. util/pki.ts is now the one implementation. Everything openssl reads or writes goes through a temporary file, which removes the sh -c 'cat | openssl ... /dev/stdin' workaround and with it the need for a shell in the image (the alpine runtime has no bash, so the monitor's path could not have run in a container). The default TLS context becomes a leaf for config.hostname rather than the CA itself, so it carries a subjectAltName; the CA is left untouched on disk.
• provisioning: answer a failed signing with a failure — the route that signs an appliance's CSR discarded stderr, ignored the exit status, and answered resultCode 0000 with whatever reached stdout. A failed signing arrived at the appliance as an empty certificate delivered as a success.
• bridge, monitor: generate keys through util/pki — moves the two remaining call sites over; bridge/util.ts is no longer used by anything and goes.
• bridge: forward the device's real deploy appInfo/platformInfo upstream — a bridged appliance was introduced to the cloud with fixed placeholders, including protocolVer: '1'. Told that, the cloud framed its reservation ("service") polls in a legacy encoding that a protocolVer 7 firmware ignores, so the official app's reservation screen reported the appliance as unreachable. Basic state polls use a version-independent framing, which is why control worked throughout and only that one screen was broken.

Compatibility

The CA on disk is not touched. Verified on an existing installation: the CA fingerprint is identical before and after startup, and no regeneration is logged. The trust anchor appliances have pinned is unchanged, so nothing needs re-provisioning.

Two changes are worth calling out:

• The certificate served for config.hostname is now a CA-signed leaf rather than the CA itself. Verified with strict verification: a client pinning the CA completes the handshake in all three cases — config.hostname, an LG hostname over SNI, and no SNI at all.
• The CA is no longer regenerated when its subject does not match config.hostname. That check treated every failure alike, so an unreadable file or a changed hostname silently replaced a CA that appliances had already pinned. Since the served certificate is now a leaf, the CA's subject no longer has to match anything. Only "there is no CA yet" creates one.

The key generation moved from openssl ec -pubout to openssl pkey -pubout. The public key PEM is byte-identical for both EC and RSA, so the hash that goes into the pairing ciphertext is unchanged.

advertise_requested_host is optional and defaults to false; existing config files need no changes.

Testing

341 tests pass, including new coverage for the certificate issuer (unit and a real TLS handshake against a validating client), the openssl wrapper, hostname validation, the CSR signing route's failure paths, bridge registration, and the deploy-info fallback.

Attribution

Five of these commits are by @kkqq9320, from https://github.com/kkqq9320/rethink, submitted here with his agreement. Authorship is preserved in the commits.

Closes #104.

[3735943886]

A branch that lets a bridged appliance take a firmware update is ready:
https://github.com/3735943886/rethink/tree/pr/ota — 2 commits, +650, of which 296 are tests.

Two things were in the way. The cloud asks whether the appliance is there before it offers
anything, and the bridge dropped that message, so the appliance looked offline. Once that is
carried across, the download itself fails: the image sits on a public CDN, and the appliance
checks that certificate against its built-in roots rather than the CA it pinned from rethink,
so it drops the connection during the handshake — before there is any request to answer.

The fix is for rethink to stop answering those connections. The ClientHello already says which
host is wanted, so names rethink serves are handled as before, and firmware downloads are
passed through to the real server for the appliance to validate itself. Nothing needs to be
excluded from the redirection, and rethink is not told where firmware lives — the cloud names
it in startFota, which goes through the bridge on the way down.

Verified on one appliance, a 1WPU4CIGCR__2: the update ran to completion, and the checksum
it reports for that micom afterwards is the one startFota announced. One device, one
successful update — no second data point yet.

It relies on what this PR adds, so I have not opened it as a PR of its own yet.

[anszom]

This sounds very interesting, but also a bit suspicious. Redirecting already-provisioned devices would depend on two conditions (unless I misunderstood the scheme you proposed):

1. The devices not pinning the LG servers' certificates. As far as I recall, my one did - after the initial response to /route/certificate, it would refuse unrelated trust roots.
2. The LG backend not verifying the devices' certificates. Each device gets a client certificate, which identifies it to the backend. If the certificate wasn't actually validated, I could easily launch a fake device which would impersonate yours, as long as I knew the UUID.

1 - I will need to verify if this is actually the case, I may not recall things correctly.

2 - I would be mildly surprised if LG's backend had a flaw of this magnitude. But if they had - It would be responsible to report it to LG instead of building infrastructure depending on the flaw never being fixed.

[3735943886]

I've rewritten this with AI help to express my point more clearly — the findings and the
reasoning are mine. I went back through my logs to check both of your conditions, so most of
what follows is observed rather than assumed.

1. Certificate pinning.

You're right that the device pins — but it pins what it is told, and it asks again.
The two ports behave differently, which is the part that matters:

• The API port (HTTPS) does not validate the server certificate. Over that connection the
  appliance fetches GET /route and GET /route/certificate?name=aws-iot, and pins whatever
  CA comes back. rethink returns its own CA here.
• The MQTT port does validate — against the CA it just pinned. rethink satisfies it by
  minting a leaf per requested SNI name from that same CA.

The pin isn't fixed at manufacture; it's re-acquired from /route/certificate each time the
appliance runs its provisioning bootstrap. Here is one of mine doing exactly that against
rethink, still carrying its LG-issued AWS IoT topics:

10:14:28  incoming clip/provisioning/devices/<uuid> {"cmd":"undeploy"}
10:14:28  publish <uuid> availability offline
          ... 8 minutes ...
10:22:21  status Issued a certificate for kic-mclip.lgthinq.com
10:22:34  status Issued a certificate for kic-common.lgthinq.com
10:22:34  HTTPS kic-common.lgthinq.com /route/certificate
10:22:35  HTTPS kic-common.lgthinq.com /route/certificate?name=aws-iot
10:22:36  incoming $aws/rules/clip_provisioning_rule/clip/provisioning/devices/<uuid>
          {"cmd":"deploy","data":{"appInfo":{"appVersion":"clip_ble_v1.9.237", ...

Two things to note. The hostnames it asks for are kic-common and kic-mclip.lgthinq.com —
its factory LG names, not the hostname rethink is configured with. And the deploy arrives on
$aws/rules/clip_provisioning_rule/..., a topic it can only know from its original LG
provisioning. So this is precisely the already-provisioned case you're asking about, and it
re-fetched the CA and carried on.

That's also why rethink mints a certificate per requested SNI name: units of the same model
don't agree on which LG hostname their firmware was built with, and a single certificate named
after our own hostname would satisfy the API port and fail the MQTT one.

My guess at why it's built this way: an appliance with a ten-year service life can't carry a
hardcoded trust anchor without eventually bricking its own connectivity when that anchor
rotates or expires, and re-fetching it at bootstrap is the cheap way out. Speculation, but it
fits.

2. Client certificate validation.

I think there's a premise worth correcting first: "non-destructive bridge" doesn't mean rethink skips
server-side enrolment. It means it doesn't tear down the existing one.

Those are two separate operations in the ThinQ API:

• Home registration (POST /service/homes/{homeId}/devices) — rethink passes
  initDevice: false and keeps the existing alias, so the device is never dropped and
  re-announced. It stays in your app, under its own name, still reachable by the official app.
• Certificate enrolment (POST /device/<uuid>/certificate) — this always runs. rethink
  generates a fresh EC keypair and CSR and has LG issue a new client certificate for it.

So rethink is not presenting a forged or borrowed certificate. It presents one LG issued to it,
through the same OTP-authorised path the official app uses:

1. POST /service/devices/otp/certificate returns the OTP. It requires x-emp-token and
   x-user-no from an authenticated account — and fails outright unless
   POST /service/users/client was called first.
2. rethink generates its own EC keypair and CSR.
3. POST /device/<uuid>/certificate carries the OTP, the CSR, the public key, and a blob
   RSA-encrypted under the OTP's public key containing
   nonce ‖ otp ‖ sha256(deviceId) ‖ sha256(csr) ‖ sha256(publicKey), binding the OTP, the
   device id and the key material together.
4. addDevice submits nonce ‖ sha256(deviceId), encrypted the same way, as proof of
   possession, scoped to the homeId of that authenticated account.

I checked which of those two calls the backend actually treats as binding, and it's step 3, not
step 4. On all five of my bridged appliances addDevice fails outright — they're already in
the home, so it returns 0125 (ERROR_ALREADY_DEVICES_REGISTERED_IN_HOME) and rethink keeps
the existing registration — and the MQTT connection succeeds anyway, seconds later, with the
certificate issued in step 3:

MGMT <ip> /bridge/<uuid>/enable
Fetching API urls
Fetching CA cert
Trying to generate a certificate with otp <otp>
POST https://kic-service.lgthinq.com:46030/v1/service/homes/<homeId>/devices
  -> { resultCode: '0125' }
Device already registered, keeping the existing registration
bridge <uuid> connecting to ssl://<...>.iot.ap-northeast-2.amazonaws.com:8883
bridge <uuid> connected

So the existing home registration really is untouched — the call that would modify it doesn't
even succeed — and AWS IoT accepted the connection under mutual TLS with the LG-issued
certificate. If anything that's a small point in the backend's favour: the certificate had to
chain correctly to be let in at all.

Which is why your second condition doesn't apply. Nothing here depends on the backend failing
to validate client certificates — it can validate as strictly as it likes and the bridge still
works, because the certificate is genuine.

The impersonation scenario doesn't follow either. Knowing the UUID gets you nothing: step 1 is
account-gated, and without an OTP you can't get past step 3. What you'd need isn't a UUID, it's
the victim's LG credentials — at which point the certificate is the least of their problems.

That’s my reading of it, anyway — but I could certainly be wrong. I'd welcome your thoughts.

[anszom]

I'll need some more time to better understand this. Meanwhile, I've cherry-picked the two initial commits (management panel fixes).

[anszom]

I've previously assumed that the device pins the certificates at provisioning time until it's re-provisioned. But I've re-checked this now and it turns out that it re-pins them on each boot, making this scenario possible.

The other way around I haven't tested yet. Tell me if I understand this correctly. The device obtains a valid certificate from LG's backend. When redirected to rethink, it simply uses that certificate (no harm done), while rethink obtains another valid certificate from LG, without invalidating the old one, and without ejecting the device from the user's home. Is that right?

[3735943886]

That's my understanding too, except for one part I can't confirm.

The only thing this PR changes there is the ejection. addDevice already sent initDevice: false and retried with initDevice: true on ERROR_ALREADY_DEVICES_REGISTERED_IN_HOME — that retry is what deleted and rebuilt the registration, and it's what I dropped. The certificate enrolment is untouched.

Whether the old certificate stays valid, I don't know. What makes me suspect it does is how little it took: the bridge was already there, and all that was needed was to stop deleting the device and to pass its real deploy info through instead of the placeholders. So my guess is that LG's side already accommodates a connection like this — but it is a guess, and I can't see the backend to check it.

If that's not something you'd want to rely on, it doesn't have to hold up the rest. The redirection work stands on its own without it — SNI certificates, advertise_requested_host, the pki consolidation and the signing-failure fix, six commits with no dependency on the registration change. I can reduce this PR to those and open the bridge change separately, or drop it entirely; the existing delete-and-re-register path keeps working either way.

[variousred]

Independent live validation of the server-leaf certificate portion of this PR with a newer ThinQ2 Wi-Fi module.

Test device (household identifiers removed):

• Product model: LW1224IVSM
• ThinQ model ID: WIN_056905_WW
• Appliance software: 487100
• Wi-Fi module: clip_ble_v1.9.225 / RTK_RTL8720cm

Behavior with current master was repeatable:

1. The appliance joined the 2.4 GHz LAN and reached the local server.
2. It requested /route, /route/certificate, and /route/certificate?name=common-server.
3. It then stopped: no device CSR POST and no MQTT connection.

The generated CA certificate was also being presented directly as the HTTPS/MQTTS server certificate (CA:TRUE, no hostname SAN). Applying the proper CA-signed rethink.lan server-leaf approach from this PR, while preserving the existing CA, changed the behavior immediately.

The local server leaf used:

• CA:FALSE
• SAN DNS:rethink.lan
• serverAuth extended key usage
• signature from the existing rethink CA

Strict hostname/chain verification then passed for both ports 443 and 8883. On the next setup attempt the appliance continued past common-server, POSTed its device CSR, requested the aws-iot certificate, completed provisioning, and established its TLS MQTT connection. It has remained online and bidirectional with no LG account or cloud access.

So this is an independent confirmation that newer clip_ble firmware can reject the old CA-as-server-certificate arrangement without surfacing a useful error, and that the separate hostname-valid server leaf is necessary for enrollment on at least this firmware/model combination.