Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@ All notable changes to BeatDesign are documented in this file.

## Unreleased

### Added

- Added a versioned built-in Skill catalog exposed through MCP Resources and two
read-only compatibility tools, with WorkBuddy runtime packaging and probes.

### Changed

- Delegated account-level generation concurrency entirely to the active Provider,
removing BeatDesign's cross-Project mutex and per-Project running-task limit.
- Updated the asset-first generation request to v2: image attachments remain
generic references, while canonical `@ImageN` prompt directives express first-
and last-frame intent without separate persisted frame roles.

## [0.2.3] - 2026-09-05

### Added
Expand Down
2 changes: 1 addition & 1 deletion README.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ pnpm dev

## AgentからBeatDesignを使う

BeatDesignはProject、Asset、Canvas、生成、Editor操作をカバーする27個のローカルMCPツールを提供し、正式なTimelineからのMP4書き出しにも対応します。Agentによる変更は同じProjectサービスを通り、ブラウザーのワークスペースに表示されます。
BeatDesignはSkill、Project、Asset、Canvas、生成、Editor操作をカバーする29個のローカルMCPツールを提供し、内蔵Skillの検出と正式なTimelineからのMP4書き出しにも対応します。Agentによる変更は同じProjectサービスを通り、ブラウザーのワークスペースに表示されます。

MCP Hostへ接続した後は、次のように依頼できます。

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Add your own [BeatAPI API key](https://beatapi.io/dashboard/apikeys) only when y

## Use BeatDesign with an Agent

BeatDesign exposes 27 local MCP tools for Projects, Assets, Canvas, generation, and Editor operations, including authoritative MP4 timeline rendering. Agent changes use the same project services and become visible in the browser workspace.
BeatDesign exposes 29 local MCP tools for Skills, Projects, Assets, Canvas, generation, and Editor operations, including built-in Skill discovery and authoritative MP4 timeline rendering. Agent changes use the same project services and become visible in the browser workspace.

After connecting your MCP host, you can ask:

Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ pnpm dev

## 让 Agent 操作 BeatDesign

BeatDesign 提供 27 个本地 MCP 工具,覆盖 Project、Asset、Canvas、生成和 Editor 操作,包括从权威时间线导出 MP4。Agent 修改会经过同一套项目服务,并显示在浏览器工作空间中。
BeatDesign 提供 29 个本地 MCP 工具,覆盖 Skill、Project、Asset、Canvas、生成和 Editor 操作,包括发现内置 Skill 目录以及从权威时间线导出 MP4。Agent 修改会经过同一套项目服务,并显示在浏览器工作空间中。

连接 MCP Host 后,可以直接提出这样的要求。

Expand Down
2 changes: 2 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ The shared product model is asset-first:

The browser calls local `/api` routes. Server routes validate input and resolve a logical model through the active Generation Provider contract. BeatAPI is the built-in/default provider; forks can register another source-level provider without changing Canvas, Editor, MCP, or the asset-first request contract.

BeatDesign does not implement account concurrency policy. Projects may submit generations independently, and the active Generation Provider is the sole authority for account-level admission. The local submission lock protects SQLite intent/history consistency only; it does not reserve a long-running generation slot or reject a different Project because another task is active.

Provider and optional R2/S3 credentials are encrypted in the local `config` table. Browser components never receive raw credentials.

Upload storage is a separate adapter boundary. File selection remains browser-local. A successful generation precheck creates a short-lived, one-time SQLite intent that binds the project, model, exact upload count, uploaded URLs, and final generation submission. Required references are promoted only after that point and immediately before task submission; they become project assets only after the provider accepts the task. The default path uploads supported references to BeatAPI Files. Users may instead configure a public R2/S3-compatible bucket; those credentials remain local and are used only for confirmed generation inputs.
Expand Down
24 changes: 20 additions & 4 deletions docs/MCP.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,10 @@ installation shapes.

## Tool groups

There are **27** tools:
There are **29** tools:

- Skill (2): list the bundled official Skill catalog and read one complete
versioned Skill definition.
- Project (5): list, get, create, target the current MCP session, and open a
workspace review surface.
- Asset (4): list, get by project membership, import a local file, and extract a
Expand All @@ -117,6 +119,14 @@ There are **27** tools:
- Editor (8): get, incremental edit, SRT import, authoritative MP4 render,
semantic snapshot, diagnostics, deep-link view, and command history.

The same bundled Skill catalog is also available through the read-only MCP
Resource `beatdesign://skills`. Individual Skill instructions use the resource
template `beatdesign://skills/{skillId}`. The two Skill tools are compatibility
fallbacks for Agent hosts that do not surface MCP Resources. Until an official
creative Skill passes prompt evaluation and a visible MCP workflow test, the
catalog intentionally returns an empty `skills` array and no Showcase entry is
published.

MCP writes use `origin=mcp` assigned inside the server. `canvas.apply` and
`editor.apply` accept stable IDs, revisions, and idempotency keys. The server
does not expose full Canvas or Timeline replacement.
Expand Down Expand Up @@ -185,9 +195,11 @@ For a newly connected Canvas node, append a `place_card` operation after its
listed in `sourceCardIds`, or to the right of the card's `referenceCardIds` when
that field is omitted. This is an explicit initial-layout action, not a live
auto-layout system: later drag positions and large manually arranged graphs stay
saved until a caller explicitly places the card again. References are passed to
generation independently of prompt text; BeatDesign does not insert synthetic
`@Image1` or `@Image2` tokens into a user's prompt.
saved until a caller explicitly places the card again. BeatDesign does not infer
first or last frames from attachment order. Image references remain generic,
while explicit workflows such as tail-frame continuation may insert canonical
prompt directives like `Use @Image1 as the first frame.`. The BeatAPI adapter
translates only those explicit `@ImageN` directives into provider frame fields.

## Current boundaries

Expand Down Expand Up @@ -216,6 +228,10 @@ generation independently of prompt text; BeatDesign does not insert synthetic

## Skill and MCP

- `skills/official/` is the future single source of truth for portable,
versioned creative Skills bundled with BeatDesign. The loader validates the
manifest schema, directory identity, minimum BeatDesign version, and required
MCP tools; it reads instructions as data and never executes Skill scripts.
- The host Skills are the workflow layer: they tell the Agent when to select a
project, which MCP tools to combine, where user authorization is required, and
which Canvas or Editor view must remain open for review.
Expand Down
10 changes: 7 additions & 3 deletions docs/PRODUCT_PLAN_AND_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ BeatDesign 是一个免费、开源、本地优先的 AI 图片/视频创作工

- 远程 AI 模型能力。
- 异步任务、模型路由和服务计费。
- 账号级并发限制,所有 BeatDesign Project 共享同一 BeatAPI 账号额度。
- 用户通过自己的 BeatAPI API Key 使用服务。

### 不属于本仓库
Expand Down Expand Up @@ -93,6 +94,7 @@ Codex / Claude Code / Other Agent
- 生成图片、视频和封面在展示前本地化。
- 项目 Assets 和生成历史索引。
- BeatAPI 与存储凭证保留在服务端。
- BeatDesign 不会因另一个 Project 有运行中任务而拒绝新生成,也不设本地每 Project 并发上限;账号并发由当前 Generation Provider 返回的结果决定。

### Studio / Canvas

Expand Down Expand Up @@ -142,9 +144,10 @@ Codex / Claude Code / Other Agent
- 幂等键会绑定首次 command;复用到另一个 command 会返回 `INVALID_COMMAND`,不会静默冒用旧结果。
- UI 内部允许 revision-checked `editor.replace_document` 支撑 undo/redo 和本地自动保存;MCP 被明确禁止整份替换,只能调用 `editor.apply`。
- Generation 的 `AssetFirstGenerationRequest` 已成为服务端权威输入:适配器媒体参数由 Asset ID 和当前 generation intent 编译,旧的客户端 URL 字段不再决定引用事实。
- `AssetFirstGenerationRequest` v2 不保存首帧或尾帧角色;图片保持普通引用,明确的 `@ImageN` Prompt 指令是首尾帧语义的唯一来源,Adapter 仅负责翻译 Provider 所需字段和校验硬性模型限制。
- UI 命令入口不再接受客户端 `origin`;服务端固定写入 `ui`,MCP 入口在内核边界固定写入 `mcp`。
- Provider Contract 已将逻辑模型目录与 BeatAPI effectId、上传路径和上游模型名拆开;BeatAPI 是默认实现,Fork 可在源码扩展点注册其他 Provider。
- 本地 stdio MCP Server 提供 27 个工具(Project / Asset / Canvas / Generation / Editor),模型和参数通过 capability discovery 暴露;Canvas / Editor 增量操作使用完整 JSON Schema,Agent 可直接发现操作类型和参数。MCP 生成直接调用当前 Provider,本地产品不重复实现 API Key、余额、计费或限流策略,只透传 Provider 的结果与错误。`bdesign_project_target` 绑定当前会话项目,Project/Canvas/Editor view 工具返回 Codex Browser handoff;`bdesign_asset_import` 把本地文件导入项目 Asset 库;`bdesign_asset_extract_frame` 与 `bdesign_canvas_continue_from_tail` 负责抽帧续写;Editor MCP 可导入 SRT、放置和替换任意项目图片 Overlay、调整叠层与单条字幕参数,并通过 `bdesign_editor_render` 将权威时间线导出为项目内 MP4 Asset。
- 本地 stdio MCP Server 提供 29 个工具(Skill / Project / Asset / Canvas / Generation / Editor),模型和参数通过 capability discovery 暴露;Canvas / Editor 增量操作使用完整 JSON Schema,Agent 可直接发现操作类型和参数。MCP 生成直接调用当前 Provider,本地产品不重复实现 API Key、余额、计费或限流策略,只透传 Provider 的结果与错误。`bdesign_skill_list` / `bdesign_skill_get` 是不显示 MCP Resources 的宿主兼容入口;`bdesign_project_target` 绑定当前会话项目,Project/Canvas/Editor view 工具返回 Codex Browser handoff;`bdesign_asset_import` 把本地文件导入项目 Asset 库;`bdesign_asset_extract_frame` 与 `bdesign_canvas_continue_from_tail` 负责抽帧续写;Editor MCP 可导入 SRT、放置和替换任意项目图片 Overlay、调整叠层与单条字幕参数,并通过 `bdesign_editor_render` 将权威时间线导出为项目内 MP4 Asset。
- Codex、Claude Code 与 WorkBuddy 接入包内含 `beatdesign-workspace` Skill,负责项目选择、字幕/续写工具编排、付费生成停点和可视化复核;三者共用同一 MCP 与本地 Project 数据,其中 Claude Code 和 WorkBuddy 使用本机 HTTP MCP。

## 6. v0.2 Phase 1 本地已实现
Expand All @@ -159,7 +162,8 @@ Codex / Claude Code / Other Agent
- Canvas -> Timeline Node -> Editor 连续工作流。
- Editor 自动保存接入命令入口,并补齐冲突三方合并、重复操作保护和稳定播放头时间。
- 图片 Clip、时间线拖拽调整持续时间与图片/视频统一视觉轨。
- 本地 MCP Server 提供 27 个 Project、Asset、Canvas、Generation、Editor 工具;支持会话项目绑定、Canvas/Editor 可视化交接、从绝对路径导入本地素材、抽取尾帧续写、导入和精调 SRT 字幕、放置/替换/调整图片 Overlay,以及权威时间线 MP4 导出。
- 本地 MCP Server 提供 29 个 Skill、Project、Asset、Canvas、Generation、Editor 工具;支持内置 Skill 目录发现、会话项目绑定、Canvas/Editor 可视化交接、从绝对路径导入本地素材、抽取尾帧续写、导入和精调 SRT 字幕、放置/替换/调整图片 Overlay,以及权威时间线 MP4 导出。
- 已加入只读 `beatdesign://skills` MCP Resource、单 Skill Resource Template、schema v1 清单校验和宿主兼容工具;当前官方创作 Skill 目录有意保持为空,待具体 Skill 完成提示词评测和可见工作流联调后再加入,并在那时开放 Showcase 入口。
- MCP 增量 Canvas/Editor 命令在短暂 revision 竞争时会基于最新权威文档限次自动重放;持续冲突返回最新 revision 和明确重试提示。
- Canvas 与 Editor 每 2 秒并在页面重新聚焦时检查 MCP 写入的新 revision。

Expand All @@ -186,7 +190,7 @@ Codex / Claude Code / Other Agent

### Agent / MCP 后续边界

- MCP Resources 和更完整的 schema versioning。
- 除内置 Skill 目录之外的更多 MCP Resources,以及 Canvas / Editor 等现有合同更完整的 schema versioning。
- Agent Activity、命令审计和实时 UI 事件桥。
- 外部市场正式审核与上架。仓库已提供 Codex 本地插件、可直接添加的 Claude Code 仓库插件市场,以及符合目录结构的 WorkBuddy MCP + Skill Connector;这些本地接入包不等于已通过第三方市场审核。
- 独立的 headless 像素预览与后台媒体 Worker;当前 MCP MP4 导出在本地 MCP Server 进程中完成。
Expand Down
4 changes: 4 additions & 0 deletions docs/PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

BeatAPI is the built-in and default generation/analysis provider. The official BeatAPI adapter keeps its upstream URL fixed to `https://api.beatapi.io`; users only provide their own BeatAPI API key.

BeatAPI owns account-level generation concurrency. BeatDesign does not impose a separate cross-Project or per-Project generation limit; it submits each confirmed request and surfaces the provider's admission error when the connected account has no remaining capacity.

Configure it in the Provider dialog in the workspace header. The key is encrypted in the local `config` table with a per-install key stored under `data/`.

The adapter uses:
Expand All @@ -14,6 +16,8 @@ The adapter uses:

User-facing model capabilities are defined in `src/core/effects/effect-registry.ts`. Provider bindings live in `src/core/generation-providers/`; MCP and UI use the logical model id and capability schema, not BeatAPI `effectId` or raw upstream fields.

Image attachments remain generic references in the BeatDesign request contract. First-frame and last-frame intent exists only in explicit prompt directives such as `Use @Image1 as the first frame.` and `Use @Image2 as the last frame.`; attachment order alone never assigns a frame role. The BeatAPI adapter recognizes this canonical syntax and still enforces each upstream model's hard media limits.

Kling 2.6 and Kling 3.0 Motion Control are exposed as BeatAPI models. Each run requires exactly one character image and one MP4/MOV motion video uploaded through the connected BeatAPI account. The Workspace never asks users for a KIE key; BeatAPI owns the upstream provider route, billing, polling, and output persistence.

Video Analysis is exposed as a stable BeatAPI workflow with Standard and Deep depth controls. The Workspace uploads one MP4/MOV input, submits the analysis task, polls `GET /v1/tasks/:id`, and stores the returned report text and usage in the local project history. Provider-specific Gemini routing remains private to BeatAPI.
Expand Down
2 changes: 1 addition & 1 deletion docs/RELEASE_SCOPE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Release verification requires a clean install, schema creation, production build

- [x] Application, Codex, Claude Code, WorkBuddy, Claude marketplace, and LobeHub metadata agree on `0.2.3`.
- [x] The WorkBuddy Connector archive passes deterministic structure validation.
- [x] The packaged WorkBuddy runtime installs in an empty directory, starts the local workspace, completes an MCP handshake, exposes all 27 tools, and keeps data outside the package directory.
- [x] The packaged WorkBuddy runtime installs in an empty directory, starts the local workspace, completes an MCP handshake, exposes all 29 tools plus the bundled Skill catalog Resource, and keeps data outside the package directory.
- [x] Run `pnpm typecheck`, `pnpm test`, `pnpm i18n:check`, and `pnpm build` on the release candidate.
- [ ] Publish `@beatapi/beatdesign-workbuddy@0.2.3` to npm and verify the public registry artifact.
- [ ] Upload the Connector archive to the WorkBuddy Open Platform and submit it for review.
Expand Down
4 changes: 2 additions & 2 deletions docs/prd/BEATDESIGN_V0_2.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Canvas 对 Asset、Generation 配置或 Timeline 的可视化引用。节点位
### 3.6 两类关系

1. Canvas Edge:用户排列、连接和关注关系。
2. Generation Lineage:真实的 `derived_from`、`first_frame`、`last_frame`、`continuation_of`、`redo_of`、`timeline_render` 等来源关系。
2. Generation Lineage:真实的 `derived_from`、`continuation_of`、`redo_of`、`timeline_render` 等来源关系;首尾帧用途只由明确的 `@ImageN` Prompt 指令表达,不作为独立持久化状态

本轮先在派生 Asset metadata 与 Canvas 引用中记录尾帧和时间线关系;后续升级为独立可查询的 lineage contract。

Expand Down Expand Up @@ -138,7 +138,7 @@ Editor Clip 必须引用具体 `assetId`,不能动态跟随 Canvas Generation
- 本地 MCP Server。`已完成基础版`
- Project、Canvas、Editor、Assets、Generation 工具组。`已完成基础版`
- 模型 capability discovery。`已完成`
- 只读 MCP Resources。`未完成`
- 只读 MCP Resources。`已完成 Skill 目录基础版;更多资源后续补充`
- 付费生成确认、幂等键、结构化结果和操作历史。
- Agent 修改后 UI 实时刷新。
- Codex 优先接入,Claude Code 和其他宿主使用同一协议。
Expand Down
2 changes: 1 addition & 1 deletion integrations/codex/beatdesign/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Canvas or Editor URL to Codex's in-app Browser for visible review.

```text
Browser: pnpm dev → http://127.0.0.1:3020 (Canvas/Editor review surface)
Agent: MCP stdio → pnpm mcp (Agent calls 27 tools)
Agent: MCP stdio → pnpm mcp (Agent calls 29 tools)
Both processes share the same local SQLite + project files.
```

Expand Down
4 changes: 4 additions & 0 deletions scripts/build-workbuddy-runtime-package.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ const runtimePackage = {
'bin',
'drizzle',
'mcp',
'skills',
'LICENSE',
'README.md',
],
Expand Down Expand Up @@ -87,6 +88,9 @@ await Promise.all([
resolve(outputRoot, 'drizzle', 'sqlite'),
{ recursive: true }
),
cp(resolve(repositoryRoot, 'skills'), resolve(outputRoot, 'skills'), {
recursive: true,
}),
cp(
resolve(repositoryRoot, 'integrations', 'workbuddy', 'runtime', 'launcher.mjs'),
resolve(outputRoot, 'bin', 'beatdesign-workbuddy.mjs')
Expand Down
24 changes: 24 additions & 0 deletions scripts/probe-workbuddy-runtime-package.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,19 @@ try {
send({
jsonrpc: '2.0',
id: 3,
method: 'resources/read',
params: { uri: 'beatdesign://skills' },
});
const skillCatalog = await readJsonLine(15_000);
const catalogText = (
skillCatalog.result as { contents?: Array<{ text?: string }> }
).contents?.[0]?.text;
assert.ok(catalogText);
assert.ok(Array.isArray((JSON.parse(catalogText) as { skills?: unknown[] }).skills));

send({
jsonrpc: '2.0',
id: 4,
method: 'tools/call',
params: { name: 'bdesign_project_list', arguments: { limit: 5 } },
});
Expand Down Expand Up @@ -169,6 +182,17 @@ try {
assert.match(conflictError, /Port 3020 is already serving another BeatDesign/);

await access(resolve(dataDirectory, 'local.db'));
await access(
resolve(
probeRoot,
'node_modules',
'@beatapi',
'beatdesign-workbuddy',
'skills',
'official',
'README.md'
)
);
await assert.rejects(
access(
resolve(
Expand Down
Loading
Loading