diff --git a/CHANGELOG.md b/CHANGELOG.md index f11804b7..6fb52a60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ All notable changes to this project will be documented in this file. +## [1.58.0] (31/07/2026) +Added the `company-data-copier` command, which triggers the platform Data Copier to copy a source company's data (account values incl. adjustments, text properties, people/company drop and configuration) into a brand-new company in a destination development firm. Intended for BSO developers to reproduce a client's situation in a dev firm without touching the production firm. Only *data* is copied, not template *code* — templates must already exist in the destination firm to be populated. + ## [1.57.2] (30/07/2026) Cap the liquid-test polling interval at 5 seconds. The delay between result polls grew 5% per poll without a limit, so long test runs (~10 minutes) were only checked every 30-60 seconds and a finished run could sit unnoticed for up to a minute. diff --git a/README.md b/README.md index 068b43b9..2662f843 100644 --- a/README.md +++ b/README.md @@ -265,6 +265,38 @@ silverfin development-mode --handle silverfin development-mode --update-templates ``` +### Copy company data (Data Copier) + +The `company-data-copier` command triggers the platform's Data Copier to copy a source company's data into a brand-new company in a destination (development) firm. This lets you reproduce a client's situation in a dev firm to debug templates against realistic data, without ever touching the production firm. + +It copies **data only** — account values (including adjustments), text properties, people/company drop and configuration. It does **not** copy template code: a template is only populated if it already exists in the destination firm. + +```bash +silverfin company-data-copier --source-company-id --source-ledger-ids --firm +``` + +- `--source-company-id` / `-c`: the company id to copy data from (in the source firm). +- `--source-ledger-ids` / `-l`: one or more period ids to copy, space-separated. You can find each id in the source company URL between `ledgers/` and `/workflows`. +- `--firm` / `-f`: the destination firm whose token authenticates the request and where the copied company will be created (defaults to your configured default firm). + +**This is fire-and-forget.** The command only *requests* the copy: the platform returns `202 enqueued` immediately and the copy then runs asynchronously in a background job. The command does **not** wait for it, and there is **no way to poll the status**. After a few minutes, check the destination firm: + +- A new company named like `Company__` (e.g. `Company_42_a3f1b9d4`) means the copy succeeded. +- A company whose name ends in `_FAILED` (e.g. `Company_42_a3f1b9d4_FAILED`) means the background copy failed. The job does not retry — delete the `_FAILED` company and run the command again. + +#### Prerequisites and gotchas + +The backend rejects or silently skips several situations. To avoid confusion: + +- **The destination firm must be a demo firm.** Trial, customer, junk and churn firms are rejected (`422`). +- **The source firm must differ from the destination firm** (`422`). +- **All `--source-ledger-ids` must belong to `--source-company-id`** (`422`). +- **Templates must already exist in the destination firm.** Reconciliation texts are matched by `handle` and account detail templates by their Dutch name (`name_nl`). If a template's handle / ADT Dutch name does not already exist in the destination firm, its data is **silently skipped** — no error. So the template you want to debug must already be present in the destination firm before you run the copier. +- **You must have the right to run it:** either a Silverfin support user, or a firm admin in the source firm. +- **One copy per destination firm at a time.** The backend holds a mutex (5-minute TTL) per destination firm. If you fire the command again at the same destination firm while a copy is still in flight, the second request **silently does nothing** (no new company, no `_FAILED`). Don't run it repeatedly — wait and check the destination firm. + +> **Note:** the Data Copier is deployed to specific environments. Point the CLI at the right host first with `silverfin config --set-host ` (or the `SF_HOST` env var). + ## Contributing If you find any bug or you have any suggestion, please feel free to open an issue in this repository. diff --git a/bin/cli.js b/bin/cli.js index cdcc004f..e3dfb6da 100755 --- a/bin/cli.js +++ b/bin/cli.js @@ -827,6 +827,39 @@ program await generator.generateAndOpenFile(); }); +// COPY company data (Data Copier) +program + .command("company-data-copier") + .description("Copy a source company's data into a new company in a destination (development) firm. Copies data only (account values, text properties, people/company drop, configuration) — not template code.") + .requiredOption("-c, --source-company-id ", "Company id to copy data from (source firm)") + .requiredOption("-l, --source-ledger-ids ", "One or more period ids to copy, space-separated (the id in the source company URL between 'ledgers/' and '/workflows')") + .requiredOption("-f, --firm ", "Destination firm where the copied company will be created", firmIdDefault) + .action(async (options) => { + cliUtils.checkDefaultFirm(options.firm, firmIdDefault); + + const sourceCompanyId = Number(options.sourceCompanyId); + if (!Number.isInteger(sourceCompanyId) || sourceCompanyId <= 0) { + consola.error(`Invalid source company id: "${options.sourceCompanyId}". It must be a positive integer.`); + process.exit(1); + } + + // Commander collects the variadic option into an array of strings (e.g. "-l 123 456"). + const sourceLedgerIds = options.sourceLedgerIds.map((value) => Number(value)); + + if (sourceLedgerIds.some((id) => !Number.isInteger(id) || id <= 0)) { + consola.error(`Invalid source ledger (period) id in: "${options.sourceLedgerIds.join(" ")}". Each id must be a positive integer.`); + process.exit(1); + } + + // copyCompanyData returns false on failure (e.g. a 404/400 that responseErrorHandler swallows + // without exiting), so the exit code has to be set here or scripts/CI read a failed copy as success. + const result = await toolkit.copyCompanyData(options.firm, sourceCompanyId, sourceLedgerIds); + + if (!result) { + process.exit(1); + } + }); + // Update the CLI if (pkg.repository && pkg.repository.url) { program diff --git a/index.js b/index.js index 2a310437..7eddd025 100644 --- a/index.js +++ b/index.js @@ -1346,6 +1346,58 @@ async function updateFirmName(firmId) { } } +/** + * Copy a source company's data into a brand-new company in a destination (development) firm using the + * platform's Data Copier. Intended for BSO developers to reproduce a client's situation in a dev firm + * without touching the production firm. Only *data* is copied (account values incl. adjustments, text + * properties, people/company drop and configuration) — not template *code*, which must already exist + * in the destination firm to be populated. + * @param {Number} destinationFirmId - Firm where the copied company will be created + * @param {Number} sourceCompanyId - Company id to copy data from (source/production firm) + * @param {Array} sourceLedgerIds - Period ids to copy (the `ledgers/{period_id}/workflows` id in the source company URL) + * @returns {Object|Boolean} - The API response payload, or false on failure + */ +async function copyCompanyData(destinationFirmId, sourceCompanyId, sourceLedgerIds) { + try { + if (!sourceCompanyId) { + consola.error("A source company id is required (--source-company-id)."); + return false; + } + if (!Array.isArray(sourceLedgerIds) || sourceLedgerIds.length === 0) { + consola.error("At least one source ledger (period) id is required (--source-ledger-ids)."); + return false; + } + + const attributes = { + source_company_id: sourceCompanyId, + source_ledger_ids: sourceLedgerIds, + }; + + consola.info(`Requesting data copy into firm ${destinationFirmId} (source company ${sourceCompanyId}, period id(s): ${sourceLedgerIds.join(", ")})`); + + const response = await SF.runCompanyDataCopier("firm", destinationFirmId, attributes); + + if (!response || !response.data) { + consola.error(`Data Copier request failed for firm ${destinationFirmId}. Verify the source company id and period ids exist, and that you are an admin of the source firm.`); + return false; + } + + consola.success(`Data copy requested (enqueued) for firm ${destinationFirmId}.`); + consola.info( + [ + `The copy runs asynchronously on the platform — this command does not wait for it and there is no way to poll its status.`, + `Check the destination firm after a few minutes for a new copied company (named like "Company_${sourceCompanyId}_", e.g. "Company_${sourceCompanyId}_a3f1b9d4").`, + `A company whose name ends in "_FAILED" means the background copy failed — delete it and retry.`, + ].join("\n") + ); + consola.debug(`Data Copier response: ${JSON.stringify(response.data)}`); + return response.data; + } catch (error) { + errorUtils.errorHandler(error); + return false; + } +} + module.exports = { fetchReconciliationByHandle, fetchReconciliationById, @@ -1389,4 +1441,5 @@ module.exports = { getTemplateId, getAllTemplatesId, updateFirmName, + copyCompanyData, }; diff --git a/lib/api/sfApi.js b/lib/api/sfApi.js index 56e2697a..61eedb80 100644 --- a/lib/api/sfApi.js +++ b/lib/api/sfApi.js @@ -741,6 +741,34 @@ async function readSamplerRun(partnerId, samplerId) { } } +/** + * Trigger the Data Copier for a destination firm. It copies a source company's data (account values, + * text properties, people/company drop and configuration) into a brand-new company in the destination + * firm. The copy runs as a fire-and-forget async Sidekiq job: a `202 {"status":"enqueued"}` response + * only means the copy was accepted, not that the new company already exists. There is no status endpoint + * — the only failure signal is a destination company whose name ends in `_FAILED`. + * + * The endpoint is firm-scoped on the v4 firm API (`/api/v4/f/:id/company_data_copier/run`): the + * destination firm is the `:id` in the path. The firm axios instance's baseURL is already + * `/api/v4/f/:id`, so we POST the relative `company_data_copier/run` path and inherit that baseURL + * along with the destination firm's token, staging Basic-auth and token-refresh interceptor. + * @param {String} type - Environment type (only "firm" is supported) + * @param {Number} envId - Destination firm id — the `:id` in the URL path and whose token authenticates the request + * @param {Object} attributes - { source_company_id, source_ledger_ids } + * @returns {Object} - The axios response + */ +async function runCompanyDataCopier(type, envId, attributes) { + const instance = AxiosFactory.createInstance(type, envId); + try { + const response = await instance.post("company_data_copier/run", attributes); + apiUtils.responseSuccessHandler(response); + return response; + } catch (error) { + const response = await apiUtils.responseErrorHandler(error); + return response; + } +} + module.exports = { authorizeFirm, refreshFirmTokens, @@ -797,4 +825,5 @@ module.exports = { getExportFileInstance, createSamplerRun, readSamplerRun, + runCompanyDataCopier, }; diff --git a/lib/utils/apiUtils.js b/lib/utils/apiUtils.js index 5069698e..20f9a5cf 100644 --- a/lib/utils/apiUtils.js +++ b/lib/utils/apiUtils.js @@ -43,7 +43,7 @@ async function responseErrorHandler(error) { } // Unprocessable Entity if (error.response.status === 422) { - consola.error(`Response Error (422): ${JSON.stringify(error.response.data)}`, "\n", `You don't have the rights to update the previous parameters`); + consola.error(`Response Error (422): ${JSON.stringify(error.response.data)}`); process.exit(1); } if (error.response.status === 401) { diff --git a/package-lock.json b/package-lock.json index b442eb71..45fac338 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "silverfin-cli", - "version": "1.57.2", + "version": "1.58.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "silverfin-cli", - "version": "1.57.2", + "version": "1.58.0", "license": "MIT", "dependencies": { "adm-zip": "^0.6.0", diff --git a/package.json b/package.json index 1ca95c47..860a8f68 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "silverfin-cli", - "version": "1.57.2", + "version": "1.58.0", "description": "Command line tool for Silverfin template development", "main": "index.js", "license": "MIT", diff --git a/tests/TESTS.md b/tests/TESTS.md index 824d550b..f61f75e3 100644 --- a/tests/TESTS.md +++ b/tests/TESTS.md @@ -31,7 +31,8 @@ tests/ │ ├── import-shared-part.test.js │ ├── update-shared-part.test.js │ ├── create-shared-part.test.js -│ └── get-shared-part-id.test.js +│ ├── get-shared-part-id.test.js +│ └── company-data-copier.test.js └── lib/ ├── api/ │ ├── axiosFactory.test.js @@ -149,6 +150,25 @@ Source: `bin/cli.js` (Commander program) | silverfin update-reconciliation --help output contains --handle option | Verifies `--handle` is declared for the update-reconciliation command. | | silverfin update-reconciliation --help output contains --id option | Verifies `--id` is declared. | | silverfin update-reconciliation --help output contains --all option | Verifies `--all` is declared. | +| silverfin --help output contains company-data-copier | Verifies the `company-data-copier` subcommand is registered. | +| silverfin company-data-copier --help output contains --firm option | Verifies `--firm` is declared for the company-data-copier command. | +| silverfin company-data-copier --help output contains --source-company-id option | Verifies `--source-company-id` is declared. | +| silverfin company-data-copier --help output contains --source-ledger-ids option | Verifies `--source-ledger-ids` is declared. | +| silverfin company-data-copier exit codes exits 1 on an invalid source company id | Verifies a non-integer `--source-company-id` terminates the process with exit code 1. | +| silverfin company-data-copier exit codes exits 1 on an invalid source ledger id | Verifies a non-integer `--source-ledger-ids` value terminates the process with exit code 1. | +| silverfin company-data-copier exit codes exits 1 when a required option is missing | Verifies Commander's required-option check terminates the process with exit code 1. | + +--- + +### `tests/bin/cli/company-data-copier.test.js` +Source: `index.js` → `lib/api/sfApi.js` + +| Function | Test | Description | +|---|---|---| +| `copyCompanyData` | should call the Data Copier with firm type and the correct attributes and log success | Verifies that the API is called with `"firm"`, the destination firm id, and the `{ source_company_id, source_ledger_ids }` body, a success message is logged, and the response data is returned. | +| `copyCompanyData` | should error and return false when no source company id is given | Verifies that the API is not called, an error is logged, and `false` is returned when the source company id is missing. | +| `copyCompanyData` | should error and return false when no source ledger ids are given | Verifies that the API is not called, an error is logged, and `false` is returned when the ledger id list is empty. | +| `copyCompanyData` | should error and return false when the API returns no data | Verifies that an error is logged and `false` is returned when the Data Copier response has no data. | --- @@ -422,6 +442,8 @@ Source: `lib/api/sfApi.js` | `updateAccountTemplate` | should POST to update account template and return response | Verifies that a POST to `account_templates/:id` returns the updated account template. | | `findAccountTemplateByName` | should find account template by name_nl | Verifies that the matching account template is returned when its `name_nl` is found in the list. | | `findAccountTemplateByName` | should return null when list is empty | Verifies that `null` is returned when the list API returns an empty array. | +| `runCompanyDataCopier` | should POST to the firm-scoped company_data_copier/run route with the attributes and return the response | Verifies that the Data Copier attributes are POSTed to the relative `company_data_copier/run` path on the firm-scoped `/api/v4/f/:id` baseURL (not an absolute public-v3 URL) and the response is returned. | +| `runCompanyDataCopier` | should delegate to the error handler on failure | Verifies that a non-2xx response is routed through `responseErrorHandler`. | --- diff --git a/tests/bin/cli.test.js b/tests/bin/cli.test.js index c83f60b7..b5ae2126 100644 --- a/tests/bin/cli.test.js +++ b/tests/bin/cli.test.js @@ -15,6 +15,21 @@ function runCli(args) { } } +// Runs the CLI and returns only the exit code, discarding output. Used to assert the process +// exit-code contract: a failing command must not exit 0, or scripts/CI read failure as success. +function runCliExitCode(args) { + try { + execSync(`node bin/cli.js ${args}`, { + cwd: repoRoot, + stdio: "ignore", + env: { ...process.env, NODE_ENV: "test", SF_API_CLIENT_ID: "test", SF_API_SECRET: "test" }, + }); + return 0; + } catch (err) { + return err.status; + } +} + describe("bin/cli.js Commander wiring", () => { describe("silverfin --help", () => { let helpOutput; @@ -42,6 +57,10 @@ describe("bin/cli.js Commander wiring", () => { it("output contains import-account-template", () => { expect(helpOutput).toMatch(/import-account-template/); }); + + it("output contains company-data-copier", () => { + expect(helpOutput).toMatch(/company-data-copier/); + }); }); describe("silverfin import-reconciliation --help", () => { @@ -87,4 +106,38 @@ describe("bin/cli.js Commander wiring", () => { expect(helpOutput).toMatch(/--all/); }); }); + + describe("silverfin company-data-copier --help", () => { + let helpOutput; + + beforeAll(() => { + helpOutput = runCli("company-data-copier --help"); + }); + + it("output contains --firm option", () => { + expect(helpOutput).toMatch(/--firm/); + }); + + it("output contains --source-company-id option", () => { + expect(helpOutput).toMatch(/--source-company-id/); + }); + + it("output contains --source-ledger-ids option", () => { + expect(helpOutput).toMatch(/--source-ledger-ids/); + }); + }); + + describe("silverfin company-data-copier exit codes", () => { + it("exits 1 on an invalid source company id", () => { + expect(runCliExitCode("company-data-copier -c abc -l 33417839 -f 13692")).toBe(1); + }); + + it("exits 1 on an invalid source ledger id", () => { + expect(runCliExitCode("company-data-copier -c 1224550 -l xyz -f 13692")).toBe(1); + }); + + it("exits 1 when a required option is missing", () => { + expect(runCliExitCode("company-data-copier -f 13692")).toBe(1); + }); + }); }); diff --git a/tests/bin/cli/company-data-copier.test.js b/tests/bin/cli/company-data-copier.test.js new file mode 100644 index 00000000..d7986c9c --- /dev/null +++ b/tests/bin/cli/company-data-copier.test.js @@ -0,0 +1,91 @@ +const fsPromises = require("fs").promises; +const path = require("path"); +const os = require("os"); + +jest.mock("consola"); +jest.mock("../../../lib/api/sfApi"); + +const SF = require("../../../lib/api/sfApi"); +const consola = require("consola"); +const toolkit = require("../../../index"); + +describe("company-data-copier", () => { + let tempDir; + let originalCwd; + let originalExit; + + const destinationFirmId = 13692; + const sourceCompanyId = 1224550; + const sourceLedgerIds = [33417839, 32116688]; + + beforeEach(async () => { + jest.clearAllMocks(); + + tempDir = await fsPromises.mkdtemp(path.join(os.tmpdir(), "sf-cli-test-")); + + originalCwd = process.cwd(); + process.chdir(tempDir); + + originalExit = process.exit; + process.exit = jest.fn(); + + consola.success = jest.fn(); + consola.error = jest.fn(); + consola.info = jest.fn(); + consola.log = jest.fn(); + consola.warn = jest.fn(); + consola.debug = jest.fn(); + }); + + afterEach(async () => { + process.chdir(originalCwd); + process.exit = originalExit; + await fsPromises.rm(tempDir, { recursive: true, force: true }); + }); + + describe("copyCompanyData", () => { + it("should call the Data Copier with firm type and the correct attributes and log success", async () => { + const mockResponse = { data: { status: "enqueued" } }; + SF.runCompanyDataCopier.mockResolvedValue(mockResponse); + + const result = await toolkit.copyCompanyData(destinationFirmId, sourceCompanyId, sourceLedgerIds); + + expect(SF.runCompanyDataCopier).toHaveBeenCalledWith("firm", destinationFirmId, { + source_company_id: sourceCompanyId, + source_ledger_ids: sourceLedgerIds, + }); + // Success is reported as "requested/enqueued", not "completed": the copy is fire-and-forget. + expect(consola.success).toHaveBeenCalledWith(expect.stringMatching(/requested|enqueued/i)); + // The follow-up guidance must flag the async nature and the _FAILED failure signal. + const infoMessages = consola.info.mock.calls.map((call) => call.join(" ")).join("\n"); + expect(infoMessages).toMatch(/asynchronously/i); + expect(infoMessages).toMatch(/_FAILED/); + expect(result).toEqual(mockResponse.data); + }); + + it("should error and return false when no source company id is given", async () => { + const result = await toolkit.copyCompanyData(destinationFirmId, undefined, sourceLedgerIds); + + expect(SF.runCompanyDataCopier).not.toHaveBeenCalled(); + expect(consola.error).toHaveBeenCalled(); + expect(result).toBe(false); + }); + + it("should error and return false when no source ledger ids are given", async () => { + const result = await toolkit.copyCompanyData(destinationFirmId, sourceCompanyId, []); + + expect(SF.runCompanyDataCopier).not.toHaveBeenCalled(); + expect(consola.error).toHaveBeenCalled(); + expect(result).toBe(false); + }); + + it("should error and return false when the API returns no data", async () => { + SF.runCompanyDataCopier.mockResolvedValue(undefined); + + const result = await toolkit.copyCompanyData(destinationFirmId, sourceCompanyId, sourceLedgerIds); + + expect(consola.error).toHaveBeenCalled(); + expect(result).toBe(false); + }); + }); +}); diff --git a/tests/lib/api/sfApi.test.js b/tests/lib/api/sfApi.test.js index 67bba82c..2cfef194 100644 --- a/tests/lib/api/sfApi.test.js +++ b/tests/lib/api/sfApi.test.js @@ -587,4 +587,33 @@ describe("sfApi", () => { expect(result.data).toEqual(periodsData); }); }); + + // ─── runCompanyDataCopier ───────────────────────────────────────────────── + + describe("runCompanyDataCopier", () => { + const attributes = { source_company_id: 1224550, source_ledger_ids: [33417839, 32116688] }; + + it("should POST to the firm-scoped company_data_copier/run route with the attributes and return the response", async () => { + const responseData = { status: "enqueued" }; + axiosMock.onPost("company_data_copier/run").reply(202, responseData); + + const result = await SF.runCompanyDataCopier("firm", 100, attributes); + + expect(result.data).toEqual(responseData); + // The Data Copier is firm-scoped: it must hit the firm instance's /api/v4/f/:id baseURL via the + // relative "company_data_copier/run" path, NOT an absolute public-v3 URL. The destination firm is + // the :id in that baseURL. + expect(axiosMock.history.post[0].url).toBe("company_data_copier/run"); + expect(JSON.parse(axiosMock.history.post[0].data)).toEqual(attributes); + }); + + it("should delegate to the error handler on failure", async () => { + axiosMock.onPost("company_data_copier/run").reply(422, { error: "invalid" }); + + const result = await SF.runCompanyDataCopier("firm", 100, attributes); + + // responseErrorHandler is mocked to resolve undefined + expect(result).toBeUndefined(); + }); + }); });