Skip to content

[WIP] Physical to logical replica support #521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
rasifr wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

[WIP] Physical to logical replica support #521

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f5c80bc
Add spock_create_subscriber — bootstrap a subscriber from a physical …
rasifr Jun 22, 2026
d688e21
spock_create_subscriber: drop legacy paths and extension usage
rasifr Jun 22, 2026
16821b2
Added usage document for spock_create_subscriber
rasifr Jun 22, 2026
051450c
spock_apply: advance forwarded origin using pre-created disabled subs…
rasifr Jul 22, 2026
d3b5370
spock_create_subscriber address review feedback
rasifr Jul 22, 2026
80578d2
spock_create_subscriber: add bidirectional join plumbing (PR2)
rasifr Jul 9, 2026
401476f
spock_create_subscriber: add TAP test for --bidirectional plumbing
rasifr Jul 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
results
regression_output
tmp_check
spock_create_subscriber
.vimrc
*.o
*.so
Expand Down
1 change: 1 addition & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ EXTENSION = spock
PGFILEDESC = "spock - multi-master replication"

MODULES = spock_output
SUBDIRS = utils/spock_create_subscriber

# Lookup source directory
vpath % src src/compat/$(PGVER)
Expand Down
38 changes: 38 additions & 0 deletions docs/creating_subscriber_nodes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
## Creating a Subscriber Node with pg_basebackup

Spock supports creating a subscriber node by cloning the provider with [`pg_basebackup`](https://www.postgresql.org/docs/current/app-pgbasebackup.html) and starting it as a Spock subscriber. Use the `spock_create_subscriber` utility (located in the `bin` directory of your pgEdge platform installation) to register the node.

### Synopsis:

`spock_create_subscriber [OPTION]...`

**Options**

Specify the following options as needed.

| Option | Description
|----------|-------------
| `-D`, `--pgdata=DIRECTORY` | The `data` directory to be used for new node. This can be either an empty/non-existing directory, or a directory populated using the `pg_basebackup -X stream` command.
| `--databases` | An optional list of databases to replicate.
| `-n`, `--subscriber-name=NAME` | The name of the newly created subscriber.
| `--subscriber-dsn=CONNSTR` | A connection string to the newly created subscriber.
| `--provider-dsn=CONNSTR` | A connection string to the provider.
| `--replication-sets=SETS` | A comma-separated list of replication set names.
| `--apply-delay=DELAY` | The apply delay in seconds (by default 0).
| `--drop-slot-if-exists` | Drop replication slot of conflicting name.
| `-s`, `--stop` | Stop the server once the initialization is done.
| `-v` | Increase logging verbosity.
| `--extra-basebackup-args` | Additional arguments to pass to `pg_basebackup`. Safe options are: `-T`, `-c`, `--xlogdir`/`--waldir`
Comment on lines +9 to +25

@coderabbitai coderabbitai Bot Jul 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Document the --text-types option.

The --text-types CLI option (line 211 in spock_create_subscriber.c, which sets force_text_transfer) is neither listed in this options table nor in the usage() output (lines 578–603). Users have no way to discover it.

Add a row to the options table and a corresponding printf in usage().

📝 Proposed additions 
 | `--extra-basebackup-args` | Additional arguments to pass to `pg_basebackup`. Safe options are: `-T`, `-c`, `--xlogdir`/`--waldir`
+| `--text-types` | Force text transfer for all columns.
 printf(_("  --extra-basebackup-args     additional arguments to pass to pg_basebackup.\n"));
 printf(_("                              Safe options: -T, -c, --xlogdir/--waldir\n"));
+printf(_("  --text-types                 force text transfer for all columns\n"));
🧰 Tools
🪛 LanguageTool

[grammar] ~20-~20: Use a hyphen to join words.
Context: ...r. | --replication-sets=SETS | A comma separated list of replication set names....

(QB_NEW_EN_HYPHEN)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/creating_subscriber_nodes.md` around lines 9 - 25, The `--text-types`
CLI option is missing from both the subscriber creation documentation and the
CLI help output, so users cannot discover it. Add a new row for `--text-types`
in the options table in the subscriber docs, and add a matching `printf` entry
in `usage()` in `spock_create_subscriber.c` alongside the existing option
descriptions, referencing the `force_text_transfer` behavior.

| `--text-types` | Transfer all column values as text rather than binary during initial sync. Use this when provider and subscriber differ in endianness or type representation.

**Configuration files overrides**

You can use the following options to override the location of the configuration files.

| Option | Description
|----------|-------------
|`--hba-conf` | path to the new `pg_hba.conf`
| `--postgresql-conf` | path to the new `postgresql.conf`
| `--recovery-conf` | path to the template recovery configuration

Unlike `spock.sub_create`'s other data sync options, this method of cloning ignores replication sets and copies all tables on all databases. However, it's often much faster, especially over high-bandwidth connections.
Loading
Loading