Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
170 changes: 85 additions & 85 deletions bun.lock

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions docs/build-pieces/misc/bundling-pieces.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,21 @@ If a dependency must not be inlined (for example a native module), add it to a `
}
```

### Files loaded at runtime (forked processes)

A file your piece loads by path at runtime — for example `child_process.fork(path.join(__dirname, 'runner.js'))` — is invisible to the bundler's import graph, so by default it would not exist in the published package. Declare it in `bundleForkedEntries`:

```jsonc
{
"name": "@activepieces/piece-oracle-database",
"bundleForkedEntries": ["src/lib/common/oracle-runner.ts"]
}
```

Each declared entry is bundled on its own and emitted **next to the main bundle** at `src/<name>.js` — which is where `path.join(__dirname, '<name>.js')` resolves at runtime, since the bundled parent code lives at `src/index.js`. Dependencies that only the forked file imports (e.g. `oracledb`) are still captured into the published `dependencies`.

The bundler **fails the build** if piece code uses `__dirname` without declaring any `bundleForkedEntries`, because such code breaks silently after publishing.

### Building and publishing

Bundling happens automatically when you build or publish a piece:
Expand Down
2 changes: 0 additions & 2 deletions docs/embedding/navigation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,6 @@ Here is the list for routes the sdk can navigate to:
| `/connections` | Connections table
| `/tables` | Tables table
| `/tables/{tableId}` | Opens up a table
| `todos` | Todos table
| `todos/{todoId}` | Opens up a todo


## Navigate to Initial Route
Expand Down
8 changes: 8 additions & 0 deletions docs/install/reference/breaking-changes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ icon: "hammer"

### What has changed?

#### Piece builds fail when piece code uses `__dirname` without declaring `bundleForkedEntries`

The piece bundler now emits files a piece loads by path at runtime (for example a `child_process.fork` target) beside the main bundle, when they are declared in a `bundleForkedEntries` array in the piece's `package.json`, and it keeps dependencies that only those files import in the published manifest. Because an undeclared `__dirname`-relative file access always breaks after publishing — this is exactly how `@activepieces/piece-oracle-database` 0.1.11/0.1.12 shipped with every new connection failing — the build now fails loudly when a piece's source references `__dirname` and declares no forked entries. Previously such a piece built successfully and shipped broken.

#### What you need to do

Nothing for catalog pieces — oracle-database is the only piece that forks a sibling file, and this change fixes it (0.1.13). If you build custom pieces and one references `__dirname`, either declare the runtime-loaded file in `bundleForkedEntries` (see [Bundling Pieces](/build-pieces/misc/bundling-pieces)) or remove the `__dirname` usage. Already-published pieces keep working; the check runs at build time only.

#### MCP OAuth: revoking a token requires a client identity, and registration issues a usable secret

Two changes to the MCP OAuth endpoints:
Expand Down
94 changes: 93 additions & 1 deletion packages/cli/src/lib/utils/bundle-piece-utils.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, readFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
Expand Down Expand Up @@ -82,3 +82,95 @@ describe('bundlePiece — external dependency capture', () => {
expect(result.inlined).not.toContain('esm-dep')
})
})

describe('bundlePiece — forked sibling entries (GIT-1772)', () => {
let root: string | undefined

afterEach(() => {
if (root) {
rmSync(root, { recursive: true, force: true })
root = undefined
}
})

function setupForkingPiece({ declareEntry }: { declareEntry: boolean }): string {
root = mkdtempSync(join(tmpdir(), 'ap-bundle-'))

const dbDir = join(root, 'node_modules', 'oracledb')
mkdirSync(dbDir, { recursive: true })
writeFileSync(join(dbDir, 'package.json'), JSON.stringify({ name: 'oracledb', version: '6.10.0', main: 'index.js' }))
writeFileSync(join(dbDir, 'index.js'), 'module.exports = {};\n')

const piecePath = join(root, 'piece')
mkdirSync(join(piecePath, 'src', 'lib'), { recursive: true })
writeFileSync(join(piecePath, 'package.json'), JSON.stringify({
name: 'piece-forking',
version: '0.0.1',
dependencies: { oracledb: '6.10.0' },
...(declareEntry ? { bundleForkedEntries: ['src/lib/runner.ts'] } : {}),
}))
writeFileSync(join(piecePath, 'src', 'index.ts'), 'import { start } from \'./lib/pool\'\nexport const piece = { start }\n')
writeFileSync(join(piecePath, 'src', 'lib', 'pool.ts'), 'import { fork } from \'child_process\'\nimport { join } from \'path\'\nexport const start = () => fork(join(__dirname, \'runner.js\'))\n')
writeFileSync(join(piecePath, 'src', 'lib', 'protocol.ts'), 'export const READY_MESSAGE = \'runner-ready\'\n')
writeFileSync(join(piecePath, 'src', 'lib', 'runner.ts'), 'import db from \'oracledb\'\nimport { READY_MESSAGE } from \'./protocol\'\nconsole.log(READY_MESSAGE, db)\n')
mkdirSync(join(piecePath, 'dist'), { recursive: true })
return piecePath
}

it('emits a declared forked entry beside the main bundle and keeps its native dep external', async () => {
const piecePath = setupForkingPiece({ declareEntry: true })

const result = await bundlePieceUtils.bundlePiece({ piecePath, distPath: join(piecePath, 'dist'), repoRoot: root! })

expect(result.extraBundleFiles).toEqual(['src/runner.js'])
const runnerBundle = readFileSync(join(piecePath, 'dist', 'src', 'runner.js'), 'utf-8')
expect(runnerBundle).toContain('runner-ready')
expect(runnerBundle).toContain('require("oracledb")')
expect(result.external).toContain('oracledb')
})

it('fails loudly when piece source uses __dirname without declaring a forked entry', async () => {
const piecePath = setupForkingPiece({ declareEntry: false })

await expect(bundlePieceUtils.bundlePiece({ piecePath, distPath: join(piecePath, 'dist'), repoRoot: root! }))
.rejects.toThrow(/bundleForkedEntries/)
})

it('fails loudly on a declared entry that does not exist', async () => {
const piecePath = setupForkingPiece({ declareEntry: true })
rmSync(join(piecePath, 'src', 'lib', 'runner.ts'))

await expect(bundlePieceUtils.bundlePiece({ piecePath, distPath: join(piecePath, 'dist'), repoRoot: root! }))
.rejects.toThrow(/no file at/)
})

it('fails loudly when two declared entries collide on the same output name', async () => {
const piecePath = setupForkingPiece({ declareEntry: true })
mkdirSync(join(piecePath, 'src', 'lib', 'other'), { recursive: true })
writeFileSync(join(piecePath, 'src', 'lib', 'other', 'runner.ts'), 'export const other = true\n')
writeFileSync(join(piecePath, 'package.json'), JSON.stringify({
name: 'piece-forking',
version: '0.0.1',
dependencies: { oracledb: '6.10.0' },
bundleForkedEntries: ['src/lib/runner.ts', 'src/lib/other/runner.ts'],
}))

await expect(bundlePieceUtils.bundlePiece({ piecePath, distPath: join(piecePath, 'dist'), repoRoot: root! }))
.rejects.toThrow(/collides with another declared entry/)
})

it('fails loudly when two declared entries collide on output name differing only by case', async () => {
const piecePath = setupForkingPiece({ declareEntry: true })
mkdirSync(join(piecePath, 'src', 'lib', 'other'), { recursive: true })
writeFileSync(join(piecePath, 'src', 'lib', 'other', 'RUNNER.ts'), 'export const other = true\n')
writeFileSync(join(piecePath, 'package.json'), JSON.stringify({
name: 'piece-forking',
version: '0.0.1',
dependencies: { oracledb: '6.10.0' },
bundleForkedEntries: ['src/lib/runner.ts', 'src/lib/other/RUNNER.ts'],
}))

await expect(bundlePieceUtils.bundlePiece({ piecePath, distPath: join(piecePath, 'dist'), repoRoot: root! }))
.rejects.toThrow(/collides with another declared entry/)
})
})
99 changes: 97 additions & 2 deletions packages/cli/src/lib/utils/bundle-piece-utils.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { statSync, existsSync, readFileSync } from 'node:fs'
import { builtinModules } from 'node:module'
import { join, resolve, isAbsolute } from 'node:path'
import { join, resolve, isAbsolute, basename, sep } from 'node:path'
import * as esbuild from 'esbuild'

async function bundlePiece({ piecePath, distPath, repoRoot }: BundlePieceParams): Promise<BundleResult> {
Expand Down Expand Up @@ -46,13 +46,85 @@ async function bundlePiece({ piecePath, distPath, repoRoot }: BundlePieceParams)
}
}

assertDirnameUsageDeclared({ piecePath, metafile: pass.result.metafile, manifest })
const forked = await bundleForkedEntries({ piecePath, distPath, repoRoot, manifest, inlineAll, inlineList, excludeList })
for (const dep of forked.externalized) {
pass.externalized.add(dep)
}

const bundleBytes = statSync(outfile).size
const rawBytes = totalInputBytes(pass.result.metafile)
const external = [...pass.externalized].filter((dep) => !dep.startsWith('@activepieces/') && !BUNDLE_HELPER_DEPS.has(dep))

enforceSizeGate({ piecePath, bundleBytes })

return { bundleFile: outfile, bundleBytes, rawBytes, external, inlined: [...pass.inlined] }
return { bundleFile: outfile, bundleBytes, rawBytes, external, inlined: [...pass.inlined], extraBundleFiles: forked.files }
}

async function bundleForkedEntries({ piecePath, distPath, repoRoot, manifest, inlineAll, inlineList, excludeList }: ForkedEntriesParams): Promise<ForkedEntriesResult> {
const files: string[] = []
const externalized = new Set<string>()
for (const entry of manifest.bundleForkedEntries ?? []) {
const entryFile = join(piecePath, entry)
if (!existsSync(entryFile)) {
throw new Error(`[bundlePiece] bundleForkedEntries: no file at ${entryFile}`)
}
const outRel = `src/${basename(entry).replace(/\.ts$/, '.js')}`
if (outRel === BUNDLE_FILENAME) {
throw new Error(`[bundlePiece] bundleForkedEntries: "${entry}" collides with the main bundle at ${BUNDLE_FILENAME}`)
}
if (files.some((file) => file.toLowerCase() === outRel.toLowerCase())) {
throw new Error(`[bundlePiece] bundleForkedEntries: "${entry}" collides with another declared entry at ${outRel}`)
}
const outfile = join(distPath, outRel)
let pass = await runEsbuild({ entryFile, outfile, repoRoot, inlineAll, inlineList, external: new Set(excludeList) })
const unsafe = new Set([
...unsafePackages({ metafile: pass.result.metafile, warnings: pass.result.warnings }),
...importMetaPackages(pass.result.metafile),
])
if (unsafe.size > 0) {
pass = await runEsbuild({ entryFile, outfile, repoRoot, inlineAll, inlineList, external: new Set([...excludeList, ...unsafe]) })
}
const issues = gateBundle({ metafile: pass.result.metafile, warnings: pass.result.warnings })
if (issues.length > 0) {
throw new Error(`[bundlePiece] ${piecePath} forked entry "${entry}" failed the safety gate:\n - ${issues.join('\n - ')}`)
}
enforceSizeGate({ piecePath: `${piecePath} (${entry})`, bundleBytes: statSync(outfile).size })
files.push(outRel)
for (const dep of pass.externalized) {
externalized.add(dep)
}
}
return { files, externalized }
}

function assertDirnameUsageDeclared({ piecePath, metafile, manifest }: DirnameGateParams): void {
if ((manifest.bundleForkedEntries ?? []).length > 0) {
return
}
const pieceRoot = resolve(piecePath)
for (const input of Object.keys(metafile.inputs)) {
const abs = resolve(process.cwd(), input)
if (!abs.startsWith(pieceRoot + sep) || abs.includes(`${sep}node_modules${sep}`)) {
continue
}
if (/\b__dirname\b/.test(safeReadFile(abs))) {
throw new Error(
`[bundlePiece] ${input} uses __dirname but the piece declares no bundleForkedEntries. `
+ 'The published piece is a single bundled src/index.js, so __dirname-relative file access breaks after publish. '
+ 'Declare the runtime-loaded file in package.json "bundleForkedEntries" (it will be emitted beside the bundle), or remove the __dirname usage.',
)
}
}
}

function safeReadFile(file: string): string {
try {
return readFileSync(file, 'utf-8')
}
catch {
return ''
}
}

async function runEsbuild({ entryFile, outfile, repoRoot, inlineAll, inlineList, external }: RunEsbuildParams): Promise<EsbuildPass> {
Expand Down Expand Up @@ -337,6 +409,7 @@ export type BundleResult = {
rawBytes: number
external: string[]
inlined: string[]
extraBundleFiles: string[]
}

type InlineConfig = {
Expand All @@ -363,6 +436,28 @@ type RunEsbuildParams = {
type PieceManifest = {
dependencies?: Record<string, string>
bundleDeps?: boolean | string[]
bundleForkedEntries?: string[]
}

type ForkedEntriesParams = {
piecePath: string
distPath: string
repoRoot: string
manifest: PieceManifest
inlineAll: boolean
inlineList: Set<string>
excludeList: Set<string>
}

type ForkedEntriesResult = {
files: string[]
externalized: Set<string>
}

type DirnameGateParams = {
piecePath: string
metafile: esbuild.Metafile
manifest: PieceManifest
}

type ExternalizeParams = {
Expand Down
10 changes: 10 additions & 0 deletions packages/cli/src/lib/utils/prepare-piece-utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,16 @@ describe('rewriteManifestForBundle', () => {
expect(manifestOf(distPath).dependencies).toEqual({ pg: '8.11.3' })
})

it('publishes forked entry bundles alongside the main bundle', () => {
const { distPath, repoRoot } = setup({ oracledb: '6.10.0' })

rewriteManifestForBundle({ distPath, external: ['oracledb'], repoRoot, extraBundleFiles: ['src/oracle-runner.js'] })

const manifest = manifestOf(distPath)
expect(manifest.files).toEqual(['src/index.js', 'src/oracle-runner.js', 'package.json', 'src/i18n'])
expect(manifest.bundleForkedEntries).toBeUndefined()
})

it('leaves an optional peer dependency out instead of failing on it', () => {
const { distPath, repoRoot } = setup({ pg: '8.11.3' })

Expand Down
12 changes: 7 additions & 5 deletions packages/cli/src/lib/utils/prepare-piece-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,21 +39,22 @@ async function preparePieceDistForPublish(piecePath: string): Promise<void> {
copyPackageJson(paths)
copyI18nAssets(paths)

const { bundleBytes, rawBytes, external } = await bundlePieceUtils.bundlePiece({ ...paths, repoRoot })
const { bundleBytes, rawBytes, external, extraBundleFiles } = await bundlePieceUtils.bundlePiece({ ...paths, repoRoot })

rewriteManifestForBundle({ distPath, external, repoRoot })
rewriteManifestForBundle({ distPath, external, repoRoot, extraBundleFiles })
pruneDistToPublishedFiles({ distPath })

const ratio = rawBytes > 0 ? (rawBytes / bundleBytes).toFixed(1) : '—'
const extNote = external.length ? ` external=[${external.join(', ')}]` : ''
console.info(`[preparePiece] bundled ${piecePath} → ${(bundleBytes / 1024).toFixed(0)} KB (${ratio}x smaller than ${(rawBytes / 1024).toFixed(0)} KB raw inputs)${extNote}`)
const forkNote = extraBundleFiles.length ? ` forked=[${extraBundleFiles.join(', ')}]` : ''
console.info(`[preparePiece] bundled ${piecePath} → ${(bundleBytes / 1024).toFixed(0)} KB (${ratio}x smaller than ${(rawBytes / 1024).toFixed(0)} KB raw inputs)${extNote}${forkNote}`)
}

// The published artifact inlines @activepieces/* workspace code AND third-party deps into the
// self-contained bundle by default. Only deps that cannot be safely inlined (native addons,
// dynamic require) stay external and are kept here so the runtime installer resolves them.
// A piece can force a dep external via bundleDeps in its package.json (escape hatch).
function rewriteManifestForBundle({ distPath, external, repoRoot }: { distPath: string, external: string[], repoRoot: string }): void {
function rewriteManifestForBundle({ distPath, external, repoRoot, extraBundleFiles = [] }: { distPath: string, external: string[], repoRoot: string, extraBundleFiles?: string[] }): void {
const distPackageJsonPath = join(distPath, 'package.json')
const json = JSON.parse(readFileSync(distPackageJsonPath, 'utf-8'))

Expand All @@ -79,7 +80,8 @@ function rewriteManifestForBundle({ distPath, external, repoRoot }: { distPath:
delete json.scripts
delete json.types
delete json.bundleDeps
json.files = [bundlePieceUtils.BUNDLE_FILENAME, 'package.json', 'src/i18n']
delete json.bundleForkedEntries
json.files = [bundlePieceUtils.BUNDLE_FILENAME, ...extraBundleFiles, 'package.json', 'src/i18n']

writeFileSync(distPackageJsonPath, JSON.stringify(json, null, 2) + '\n')
}
Expand Down
2 changes: 1 addition & 1 deletion packages/pieces/community/activecampaign/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@activepieces/piece-activecampaign",
"version": "0.1.7",
"version": "0.1.8",
"main": "./dist/src/index.js",
"types": "./dist/src/index.d.ts",
"scripts": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { CreateAccountRequest } from '../../common/types';
export const createAccountAction = createAction({
auth: activeCampaignAuth,
name: 'activecampaign_create_account',
classification: 'WRITE',
displayName: 'Create Account',
description: 'Creates a new account.',
audience: 'both',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { CreateAccountRequest } from '../../common/types';
export const updateAccountAction = createAction({
auth: activeCampaignAuth,
name: 'activecampaign_update_account',
classification: 'WRITE',
displayName: 'Update Account',
description: 'Updates an account.',
audience: 'both',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { activecampaignCommon, makeClient } from '../../common';
export const addContactToAccountAction = createAction({
auth: activeCampaignAuth,
name: 'activecampaign_add_contact_to_account',
classification: 'WRITE',
displayName: 'Add Contact to Account',
description: 'Adds a contact to an ActiveCampaign account.',
audience: 'both',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { activecampaignCommon, makeClient } from '../../common';
export const addTagToContactAction = createAction({
auth: activeCampaignAuth,
name: 'activecampaign_add_tag_to_contact',
classification: 'WRITE',
displayName: 'Add Tag to Contact',
description: 'Adds a tag to contact.',
audience: 'both',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { CreateContactRequest } from '../../common/types';
export const createContactAction = createAction({
auth: activeCampaignAuth,
name: 'activecampaign_create_contact',
classification: 'WRITE',
displayName: 'Create Contact',
description: 'Creates a new contact.',
audience: 'both',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { activecampaignCommon, makeClient } from '../../common';
export const subscribeOrUnsubscribeContactFromListAction = createAction({
auth: activeCampaignAuth,
name: 'activecampaign_subscribe_or_unsubscribe_contact_from_list',
classification: 'WRITE',
displayName: 'Subscribe or Unsubscribe Contact From List',
description:
'Subscribes a Contact to a List it is not currently associated with, or Unsubscribes a Contact from a list is currently associated with.',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { CreateContactRequest } from '../../common/types';
export const updateContactAction = createAction({
auth: activeCampaignAuth,
name: 'activecampaign_update_contact',
classification: 'WRITE',
displayName: 'Update Contact',
description: 'Updates an existing contact.',
audience: 'both',
Expand Down
Loading
Loading