Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,106 @@ Oracle examples:
- Disable network encryption negotiation: `oracle.net.encryption_client=REJECTED`
- Set session program name: `v$session.program=DMSAgent`

## Postgres Large object migration parameters

### LO_SNAPSHOT_ENABLED

Feature gate for the large object snapshot. When false (the default), no large object related
operations run: the source is not scanned, no partitions are computed, and nothing is uploaded
or published. Set to true to opt into the large object snapshot.

### LO_SNAPSHOT_PARALLELISM

Number of parallel workers used for the large object snapshot.

For example the LO_SNAPSHOT_PARALLELISM = 20

If the agent mode is reader, there will be 20 workers COPY OUT the large object from source database and upload to object storage in parallel.

If the agent mode is writer, there will be 20 workers download the large object and COPY IN to the target database.

#### Recommandation:

**Our performance benchmark test environment setting:**

Deploy Source postgres database(8vCPU, 32G Mem) and Reader mode agent(16vCPU, 16G Mem) in one i4i.8xlarge EC2

Deploy Target epas database(16vCPU, 32G Mem) and Writer mode agent(4vCPU, 16G Mem) in another one i4i.8xlarge EC2

**Tables setting:**

Large objects counts: 6400

Large objects size: 32MB

Large objects expected partion size: 1024MB

Forign key paraent table size: 100000rows

Forign key child table size: 10000000rows

Integer Non-unique index size: 10000000rows

**Large object agent setting:**

Reader mode agent parallelism: 24

Writer mode agent parallelism: 32

**How to calculate large object snapshot duration:**

Search log in reader mode agent:
```
T1: 2026-08-27 06:25:44,632 INFO [com.enterprisedb.transporter.cdcreader.largeobject.LargeObjectManager] (pool-11-thread-1) [LO snapshot] Start large object snapshot
T2: 2026-08-27 06:26:14,491 INFO [com.enterprisedb.transporter.cdcreader.largeobject.LargeObjectManager] (pool-11-thread-1) [LO snapshot] Published large object manifest completion for topic org_beaconator4Appli_prj_xGQSUwMKDJZCxvhx.67590cea-a1df-11f1-a5fa-f66ef5321c73.large-object-manifest (partitionCount=16)
```
calculte the large object COPY OUT and upload time = T2 - T1

Search log in writer mode agent:
```
T3: 2026-08-27 06:25:44,725 INFO [com.enterprisedb.transporter.cdcwriter.connector.Connector] (pool-9-thread-1) [LO snapshot] Launching large object manifest consumer for edb
T4: 2026-08-27 06:26:59,315 INFO [com.enterprisedb.transporter.cdcwriter.connector.Connector] (lo-manifest-consumer-edb) [LO snapshot] Large object snapshot finished for edb
```
calculte the end to end time = T4 - T1

**Result:**

| E2E duration | Speed (MB/s) | Reader Peak CPU | Reader Peak Mem | Writer Peak CPU | Writer Peak Mem | Source Peak CPU | Source Peak Mem | Target Peak CPU | Target Peak Mem |
|--------------|--------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|
| 8min30s | 401.57 | 47.90% | 35.10% | 49% | 12.80% | 38% | 75.10% | 39.50% | 25% |

### LO_SNAPSHOT_PARTITION_TARGET_MB

All the large object will be splited to different partition. This option is the your expected partion size in MB.

### LO_SNAPSHOT_PARTITION_TARGET_LO_COUNT

All the large object will be splited to different partition. This option is the your expected large object count for each partion.
Agent will compute the partition number using both LO_SNAPSHOT_PARTITION_TARGET_MB and LO_SNAPSHOT_PARTITION_TARGET_LO_COUNT then
chose the larger one.

### LO_SNAPSHOT_COMPRESSION_RATIO

Compression ratio used to estimate logical (decompressed) volume. Intentionally unset by default:
when empty, the logical volume is estimated by sampling large objects at startup. When pinned, the
value is used directly with no sampling.

### LO_SNAPSHOT_SAMPLE_LO_COUNT

Number of large objects to decompress when estimating the compression ratio. A fixed count
(not a percentage) bounds the sampling cost regardless of total LO volume (default: 200)
Larger LO_SNAPSHOT_SAMPLE_LO_COUNT will increase the estimating time.

### LO_SNAPSHOT_SAMPLE_TIMEOUT_SECONDS

Statement timeout (seconds) for the sampling query for estimating the compression ratio (default: 30)
Increase this when agnet timeout during estimating the compression ratio

### LO_SNAPSHOT_HIGH_VOLUME_WARN_LOGICAL_GB

Estimated logical (decompressed) large object volume, in GiB, at or above which the snapshot logs a
high-volume WARN at startup. Set to 0 to disable the warning (default: 5120)

## Optional parameters

### METRICS_TASK_ENABLED
Expand Down
Loading