generate connect clients and handlers alongside twirp - #75
Merged
Conversation
Every consumer of these declarations needs the Connect clients and handlers before it can go dual stack, so this is the release the rest of the migration waits on. The generated `<package>connect` packages hold connect-go's own client and handler plus the adapters that put them on the plain protobuf service interface, which is the contract implementations and Go callers already use — so switching a client is a constructor call and switching a server is a mount line, with no handler signatures touched. Generation moves from protoc in the elephant-twirptools image to buf with plugins pinned in ttab/mage and run as "go run <module>@<version>", so it needs no Docker and installs nothing. The targets are renamed to match, and newsdoc/newsdoc.proto declares no service, so the rpc targets do not discover it and the Newsdoc target compiles it itself before regenerating the services that embed its messages. The generator move rewrites every .pb.go, because protoc-gen-go v1.36.12 emits the embedded descriptor as a string constant where v1.36.2 emitted a byte slice, and rewrites the gzipped descriptor blob in every service.twirp.go because Go's gzip encodes differently from protoc's. The compiled descriptors themselves are byte for byte identical to the ones the image produced, and the OpenAPI specifications come out unchanged, so nothing in the API moved. The module gains connectrpc.com/connect and stays free of elephantine: a declarations module that every service and the non-Go tooling depends on must not drag pgx, prometheus, oauth2 and vault behind it. Error helpers, header propagation and the ErrorMeta detail live in elephantine/rpc instead. The ttab/mage requirement is pinned to a local checkout while the rpc namespace is unreleased.
The README described `mage rpc:generate` as writing the `<package>connect/service.elephant.go` adapters, but ttab/mage skips protoc-gen-elephant-rpc while it has no released version to pin, and does so without an error. Anyone editing a .proto and regenerating today would get fresh messages, Connect and Twirp code beside adapters that were never rerun, and nothing would say so — CI does not regenerate, so the drift would reach a tag. Say what the override is, and record that a release waits for the elephantine release that ships the plugin and the ttab/mage release that pins it, which is the sequencing the migration already calls for.
The specifications under docs/ described the Twirp paths and Twirp's error schema only, nobody generated a client from them, and the generator that wrote them cannot run under buf. The .proto files are the declaration a non-Go consumer generates from. With nothing left to stamp a version into, a release is a plain git tag: there is no rpc:release target and no "bump to vX.Y.Z" commit.
elephant-docs renders each API's README.md as the introduction of its page, and repositorysocket had none. It is the one API that is neither Twirp nor Connect, so its page says so: the messages travel over the repository's websocket, and the error codes are the RPC code strings.
The replace directive pointed at a sibling checkout while the rpc namespace was unpushed. It is pushed now and pins protoc-gen-elephant-rpc itself, so the requirement becomes the pseudo-version of that commit and the documentation stops telling people to regenerate with the plugin override.
The Configuration service arrived on main in v0.24.2 while the Connect branch was in flight. The rebase regenerated its adapters; this pins them the way the other services' are pinned, and lists the constructor.
Decision 11 of the Connect migration puts every module in this round on the latest Go 1.27 patch. For a declarations module the directive is not a build detail: it forces the floor of every consumer that imports the message types, so it is recorded as its own outermost CHANGELOG lead-in rather than as a sentence inside the generation paragraph. google.golang.org/protobuf moves to v1.36.12, the runtime version the pinned protoc-gen-go already names in the generated headers, so go.mod and the CHANGELOG agree with what the code says. ttab/mage is re-pinned at the rpc namespace commit that runs protoc-gen-twirp from a module it carries, under a pinned Go toolchain, so the committed output no longer depends on the ambient toolchain. Regenerating under the new pins reproduces the committed files byte for byte.
The adversarial review found three promises in the shipped documentation that the code does not keep. A Connect JSON response is not a Twirp one with a different path. Twirp marshals with UseProtoNames, Connect with protojson's defaults, so every multi-word field is spelled lowerCamelCase on the Connect paths. Decision 9 keeps the standard encoding rather than installing a UseProtoNames codec, so the difference is documented instead: it is the whole cost for a caller that reads JSON by hand, and nothing at all for a generated client. gRPC and gRPC-Web are served on the Connect paths, but the platform's ingress speaks HTTP/1.1 to its targets, so they stop at the cluster edge. Decision 14 leaves it that way, so the README and CHANGELOG say in-cluster rather than implying an API a customer can call. The mage and elephantine branches pin each other, so the release order in the README was impossible as written. It now names the four steps, and the reason mage has to be tagged first.
The module moved to go 1.27.1 and the v2.11 binary the lint job used is built with an older Go, which refuses a target newer than itself. v2.13 is built with Go 1.27 and the code lints clean under it.
The rpc namespace and the plugin it pins are released, so the requirement moves from a pseudo-version of the feature branch to the tag. The generated code is unchanged, and the README describes the generators as released rather than listing a tagging order.
danijelvukoje
approved these changes
Sep 7, 2026
danijelvukoje
left a comment
danijelvukoje
left a comment
Contributor
There was a problem hiding this comment.
Looks great. It will make the transition painless. 🙂
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The platform is moving from Twirp to Connect with a long dual-stack period, and this module declares the APIs the core services serve. Every service now ships Connect clients and handlers next to the Twirp ones, generated with the released
ttab/magev0.13.1, which pinsprotoc-gen-elephant-rpcto elephantine v0.29.0.New protocol surface. Per service a
<package>connectsubpackage (repository/repositoryconnect,index/indexconnect,spell/spellconnect,user/userconnect,replicant/replicantconnect) withNew<Service>ServiceClient(httpClient, baseURL), returning the same plain service interfaceNew<Service>ProtobufClientreturns, andNew<Service>ServiceHandler(svc, opts...), returning the mount path and handler for elephantine'sRegisterConnect. connect-go's ownNew<Service>ClientandNew<Service>Handlerare generated too; theServiceinfix tells them apart. The Connect paths are the unprefixed/<package>.<Service>/<Method>; the Twirp clients, servers and/twirp/paths are unchanged.repositorysocketdeclares no service and gains a README describing the websocket protocol instead.Wire facts a consumer will notice. Connect error bodies are
{"code","message","details"}with the key/value metadata as anelephantine.rpc.ErrorMetadetail; the codes and messages are identical on both stacks, and three HTTP statuses differ (failed_precondition400 against 412,canceled499 against 408,deadline_exceeded504 against 408). Connect JSON responses spell field names in lowerCamelCase where Twirp uses the proto names; requests accept both. gRPC and gRPC-Web are served on the Connect paths inside the cluster only.Removed. The OpenAPI specifications under
docs/: nobody generated a client from them, they described the Twirp surface only, and the generator cannot run under buf. A release is now a plain tag with no bump commit.Build. Generation runs through the
rpcmage namespace with buf and pinned plugins, no Docker;mage rpc:generatereproduces the committed output byte for byte. The module requiresconnectrpc.com/connectv1.20.0 and does not depend on elephantine. Go 1.27.1. The first regeneration rewrote every.pb.gofor the newerprotoc-gen-go; the embedded descriptors were verified identical tomain's.Rebased onto v0.24.2, so the user
Configurationservice added there has its adapters and tests. The unreleased CHANGELOG section has the detail per lead-in.The fleet reference for how Connect is served, called, tested and generated is elephantine's docs/connect.md; the playbooks for moving a service and a client are docs/migration-service.md and docs/migration-client.md.