diff --git a/.env.example b/.env.example index 42a2702..9391b37 100644 --- a/.env.example +++ b/.env.example @@ -6,7 +6,7 @@ VITE_APP_NAME=BeatDesign VITE_APP_DESCRIPTION=The open-source, local-first AI canvas for image and video creation. VITE_APP_LOGO=/logo.png VITE_DEFAULT_LOCALE=en -VITE_GENERATION_PROVIDER=beatapi +GENERATION_PROVIDER=beatapi # Local persistence DATABASE_PROVIDER=sqlite diff --git a/.github/workflows/quality.yaml b/.github/workflows/quality.yaml index e6e1a65..047251e 100644 --- a/.github/workflows/quality.yaml +++ b/.github/workflows/quality.yaml @@ -23,6 +23,9 @@ jobs: node-version: 22 cache: pnpm - run: pnpm install --frozen-lockfile + # The MCP stdio probe exercises real Project queries, so CI needs the + # same local SQLite schema created by the documented quick-start flow. + - run: pnpm db:push # The Vite build generates the gitignored Paraglide and TanStack Router # artifacts that TypeScript needs in a clean checkout. - run: pnpm build diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 0000000..f8ac110 --- /dev/null +++ b/.mcp.json @@ -0,0 +1,8 @@ +{ + "mcpServers": { + "beatdesign": { + "command": "pnpm", + "args": ["--silent", "mcp"] + } + } +} diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ed5e8da..f8903df 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -9,28 +9,59 @@ Projects + Home + Studio + Canvas + + Editor + Assets + Generation history + Provider configuration + Storage configuration ``` -Studio, Canvas, and Assets are three views over the same project, generation, and asset services. Switching modes never creates a second project or backend. +Studio, Canvas, Editor, and Assets are views over the same project, generation, asset, and history services. Switching modes never creates a second project or backend. + +The shared product model is asset-first: + +- A generation creates an `Asset`; it does not own a single UI placement. +- A Canvas node references an Asset, Generation, or Timeline. +- A Timeline clip pins a concrete Asset and never silently follows a Canvas generation's latest output. +- A Take is an alternate Asset for one clip; the original source remains recoverable. +- Canvas edges describe visual organization. Generation lineage describes real derivation and is stored separately. ## Runtime flow -The browser calls local `/api` routes. Server routes validate input and call the BeatAPI adapter. The adapter submits image/video generation tasks or the stable `/v1/video-analysis/tasks` workflow, then checks `/v1/tasks/:id`. Generation state and analysis text are stored locally, while provider result URLs are indexed as project assets. +The browser calls local `/api` routes. Server routes validate input and resolve a logical model through the active Generation Provider contract. BeatAPI is the built-in/default provider; forks can register another source-level provider without changing Canvas, Editor, MCP, or the asset-first request contract. Provider credentials are read from environment variables or the local `config` table. Browser components never receive the raw API key. Upload storage is a separate adapter boundary. File selection remains browser-local. A successful generation precheck creates a short-lived, one-time SQLite intent that binds the project, model, exact upload count, uploaded URLs, and final generation submission. Required references are promoted only after that point and immediately before task submission; they become project assets only after BeatAPI accepts the task. The built-in provider is fixed to the official `https://api.beatapi.io` endpoint. Official `BEATAPI_MANAGED_R2_*` secrets and self-hosted `R2_*` credentials are deliberately separate and never fall through to each other. +## Command boundary + +UI actions, the local MCP server, and a future CLI call the same Command Kernel. The kernel owns validation and pure document operations for Canvas and Editor. Persistence boundaries remain responsible for revision checks, durable writes, and returning the authoritative document. + +```text +UI / MCP / CLI + | +Command envelope + validation + | +Canvas / Editor / Generation commands + | +revision-checked project persistence + | +SQLite + project media files +``` + +MCP does not write SQLite directly or replace an entire Canvas or Timeline document. Its tools call `persistBeatDesignCommand` with a transport-controlled `origin=mcp`. The public UI route assigns `origin=ui` server-side and rejects an `origin` request field. UI-only timeline replacement exists for local autosave and undo/redo, remains revision-checked, and is rejected for MCP/CLI origins. Generation commands are asset-first and intentionally have no `placement` field; the server compiles provider media URLs from project-owned Asset IDs and the one-time generation intent before submission. + +Canvas layout persistence is the deliberate exception on the UI side: drag, resize, viewport, and the complete visual arrangement are saved as a revision-checked snapshot. Semantic Canvas operations are also exposed through `canvas.apply`, and external agents must use those operations rather than snapshot replacement. + ## Persistence -The SQLite/D1 schema contains ten tables: +The SQLite/D1 schema contains twelve tables: - `project` - `project_canvas_state` +- `project_timeline_state` +- `project_command_receipt` - `project_workflow_state` - `generation_history` - `generation_upload_intent` @@ -40,11 +71,13 @@ The SQLite/D1 schema contains ten tables: - `project_asset_membership` - `config` +`project_command_receipt` stores bounded, short-lived command results for retry idempotency and powers the current MCP history tool. It is not a permanent audit log; long-term Agent Activity still needs a separate durable event contract. + No user, session, role, order, subscription, payment, credit, API-key, ticket, or CMS table belongs in this repository. ## Model catalog -`src/core/effects/effect-registry.ts` is the canonical user-facing catalog. `src/core/adapters/beatapi-adapter.ts` maps that catalog to BeatAPI request bodies. Do not add a second database-backed model registry. +`src/core/effects/effect-registry.ts` is the canonical user-facing logical catalog. `src/core/generation-providers/` maps those logical IDs to provider bindings and adapters. `src/core/adapters/beatapi-adapter.ts` contains BeatAPI request mapping. Do not leak upstream effect IDs or field names into MCP tools, and do not add a second database-backed model registry. ## Deployment boundary diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..743d64e --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,32 @@ +# Changelog + +All notable changes to BeatDesign are documented in this file. + +## [0.2.0] - 2026-08-30 + +### Added + +- A local stdio MCP server with 20 Project, Asset, Canvas, Generation, and Editor tools. +- A shared Command Kernel with runtime schemas, revision checks, stable IDs, idempotent receipts, and project Asset validation. +- Canvas-to-Editor workflows: tail-frame extraction, continuation generation nodes, Timeline nodes, and multi-asset timeline creation. +- Local image, video, and audio import through both the UI and MCP. +- Image clips, audio tracks, non-destructive Takes, split/trim/move/delete, undo/redo, timeline diagnostics, and browser-native MP4 export. +- BeatAPI model capability discovery and asset-first generation contracts. +- Codex plugin packaging plus generic MCP configuration for Claude Code, Cursor, OpenCode, and other stdio hosts. + +### Changed + +- Canvas and Editor now check for Agent/MCP revisions every two seconds and whenever the page regains focus. +- Canvas rebases pending local layout edits onto newer Agent revisions instead of blocking visible MCP updates; snapshot restoration also removes stale shapes before rebuilding the current document. +- Editor autosave adopts the canonical saved document and ignores timestamp-only differences, preventing repeated save echoes from blocking MCP updates. +- Generated and imported media are persisted in project-owned local storage and indexed in SQLite. +- External Canvas media writes require project-owned Asset IDs, and local image/video/audio imports validate type and size before reading file contents. +- GitHub documentation now includes a product-first English/Chinese README and a BeatDesign release cover. + +### Current boundaries + +- Editor snapshots are semantic rather than pixel-rendered. +- MP4 export remains browser-driven and is not yet available as a headless MCP tool. +- Captions, transitions, speed controls, multiple named timelines, and native desktop packaging remain follow-up work. + +[0.2.0]: https://github.com/BeatAPI/BeatDesign/releases/tag/v0.2.0 diff --git a/PROVIDERS.md b/PROVIDERS.md index 4020e0f..246b782 100644 --- a/PROVIDERS.md +++ b/PROVIDERS.md @@ -1,6 +1,6 @@ # Provider configuration -BeatAPI is the fixed built-in generation and analysis provider implemented by this repository. The upstream URL is intentionally fixed to `https://api.beatapi.io`; users only provide their own BeatAPI API key. +BeatAPI is the built-in and default generation/analysis provider. The official BeatAPI adapter keeps its upstream URL fixed to `https://api.beatapi.io`; users only provide their own BeatAPI API key. Configure it in either place: @@ -17,13 +17,15 @@ The adapter uses: - `GET /v1/tasks/:id` - `POST /v1/files` for supported reference files -Model support is defined in `src/core/effects/effect-registry.ts`, not discovered dynamically. This keeps Canvas and Studio behavior deterministic. When adding a model, update the registry, request mapping, media capability rules, and tests together. +User-facing model capabilities are defined in `src/core/effects/effect-registry.ts`. Provider bindings live in `src/core/generation-providers/`; MCP and UI use the logical model id and capability schema, not BeatAPI `effectId` or raw upstream fields. Kling 2.6 and Kling 3.0 Motion Control are exposed as BeatAPI models. Each run requires exactly one character image and one MP4/MOV motion video uploaded through the connected BeatAPI account. The Workspace never asks users for a KIE key; BeatAPI owns the upstream provider route, billing, polling, and output persistence. Video Analysis is exposed as a stable BeatAPI workflow with Standard and Deep depth controls. The Workspace uploads one MP4/MOV input, submits the analysis task, polls `GET /v1/tasks/:id`, and stores the returned report text and usage in the local project history. Provider-specific Gemini routing remains private to BeatAPI. -An API with a different request or polling contract still needs its own adapter. This repository does not ship placeholder KIE, Vidu, Evolink, Gemini, Fal, Replicate, or payment-provider integrations. +An API with a different request or polling contract needs its own adapter. Forks can register one in `src/config/generation-providers.ts`, bind only the logical models they support, and set server-side `GENERATION_PROVIDER=`. BeatAPI is the default only when no custom provider is selected; an unknown configured id fails explicitly so it cannot accidentally submit a task to another provider. This repository does not ship placeholder KIE, Vidu, Evolink, Gemini, Fal, Replicate, or payment-provider integrations. + +A custom provider definition owns adapter construction, readiness checks, parameter validation, model bindings, upstream model names, and upload paths. Provider credentials must stay server-side. Switching providers does not change Canvas nodes, Editor clips, Asset IDs, or MCP requests; each submitted task also records its provider/model identity so polling does not silently follow a later default-provider change. ## Storage diff --git a/README.md b/README.md index 421a814..8db9c7d 100644 --- a/README.md +++ b/README.md @@ -1,30 +1,62 @@ -# BeatDesign +

+ BeatDesign — one local creative workspace, controlled by any MCP-capable agent +

-**Beat complexity. Design freely.** +

BeatDesign

-The open-source, local-first AI canvas for image and video creation and analysis. BeatDesign brings Projects, a guided Studio, a node-based Canvas, shared assets, generation history, configurable storage, and an official BeatAPI connection into one focused workspace. +

+ One local creative workspace. Any agent.
+ Generate on a Canvas, edit on a timeline, and let an MCP-capable Agent operate the same project beside you. +

-There is no login, account system, payment flow, subscription, local credit ledger, admin panel, or API-key issuing service in this repository. +

+ Quick start · + MCP · + Product status · + 中文 +

-[中文说明](./README.zh-CN.md) +BeatDesign is a free, open-source, local-first AI media workbench. It combines a guided Studio, an infinite Canvas, a short-form video Editor, and one shared Asset library inside a single local Project. -## What is included +The browser is your visual workspace. An Agent such as Codex, Claude Code, Cursor, or another MCP stdio host can read and modify the same Project through BeatDesign's 20 local tools. There is no account system, subscription, local credit ledger, or admin panel in this repository. -- Studio for prompt-first image and video generation, plus Standard and Deep video analysis. -- React Flow Canvas for connected, multi-step creative workflows. -- A project-scoped Assets view shared by Studio and Canvas. -- Local projects, snapshots, generation history, and asset indexing. -- One code-defined catalog of supported BeatAPI image and video models. -- BeatAPI task submission and status polling. -- BeatAPI as the fixed built-in provider; users supply their own BeatAPI API key. -- BeatAPI managed R2 by default, with optional bring-your-own R2/S3 storage. -- Local connection configuration, with optional encryption at rest. -- English and Chinese UI. -- Local SQLite by default and Cloudflare D1 as an optional deployment target. +## The creative loop + +```text +Prompt or local media + ↓ +Studio / Canvas ── generate, branch, connect, reuse + ↓ +Shared Assets ─── image, video, audio, extracted clip, render + ↓ +Editor ────────── trim, split, move, mix, AI redo, export MP4 + ↑ +Any MCP Agent ─── inspect and apply project-level commands +``` + +The result is one continuous workflow instead of separate AI generators, download folders, and editing tools. + +## One Project, four views + +| View | What it is for | +| --- | --- | +| **Studio** | Prompt-first image/video generation and Standard or Deep video analysis. | +| **Canvas** | Connected media nodes, generation history, references, branching, and reusable outputs. | +| **Editor** | A local short-form timeline for video, still images, and audio, with non-destructive editing and MP4 export. | +| **Assets** | The project-scoped source of truth shared by Studio, Canvas, Editor, and MCP. | + +## Why it feels different + +- **Local by default.** Projects, Canvas snapshots, timelines, generation history, and imported media stay in the local workspace. +- **Agent-native, not Agent-locked.** BeatDesign exposes a standard stdio MCP server instead of embedding one proprietary chatbot. +- **Asset-first.** Generated and imported media become stable project Assets before they are placed on Canvas or Editor. +- **One command path.** UI and MCP writes pass through the same Command Kernel, revision checks, and persisted receipts. +- **No system FFmpeg setup.** Preview and MP4 export use browser-native WebCodecs and Mediabunny. +- **Provider boundary is explicit.** BeatAPI is the built-in generation adapter; forks can register their own adapter without rewriting Canvas or Editor. ## Quick start -Requirements: Node.js 22+ and pnpm 10+. +Requirements: Node.js 22+, pnpm 10+, and current Chrome on macOS or Windows. ```bash pnpm install @@ -33,73 +65,116 @@ pnpm db:push pnpm dev ``` -Open `http://localhost:3020`. Home is the default route. Opening Studio or Canvas does not create a database record until you confirm that you want to start a project. Use the connection button in a workspace header to add your own [BeatAPI API key](https://beatapi.io/dashboard/apikeys) and choose storage, or configure both through `.env.development`. +Open [http://127.0.0.1:3020](http://127.0.0.1:3020). -## How data flows +Local import, Canvas work, timeline editing, preview, and MP4 export do not require an API key. Add your own [BeatAPI API key](https://beatapi.io/dashboard/apikeys) only when you want to generate, analyze media, or use AI redo. -```text -Studio / Canvas / Assets - -> local server routes - -> imported files saved under data/project-assets and indexed in SQLite - -> generation precheck and a one-time SQLite generation intent - -> just-in-time provider upload of the durable local reference - -> BeatAPI image or video task API - -> provider status polling - -> generated media copied into the local project asset directory - -> local generation history and asset index -``` +## Agent control through MCP -The API key and storage credentials remain server-side. Project state and history live in the local SQLite database. In local SQLite mode, imported image and video files are copied immediately into the project-owned `data/project-assets//` directory and indexed in SQLite before a card is added to the Canvas. Video analysis uploads MP4/MOV input through BeatAPI's file endpoint and stores the resulting text in the same project generation history. Generated images, videos, and video covers are downloaded into the same project-owned directory before the Canvas receives their URLs; provider URLs remain metadata only. +BeatDesign runs as two local processes over the same SQLite database: -Canvas state is saved as a complete project snapshot after changes, checked again every five seconds while dirty, and flushed when the page is hidden, refreshed, or closed. A populated snapshot cannot be replaced by an unconfirmed empty snapshot. +| Process | Command | Role | +| --- | --- | --- | +| Visual workspace | `pnpm dev` | Opens Studio, Canvas, Editor, and Assets in the browser. | +| Agent control plane | `pnpm --silent mcp` | Exposes project-level tools over stdio. | -Selecting or dragging in a local file does not send it to a provider, but it is persisted immediately on the user's own machine so a refresh, restart, or development hot reload cannot lose it. When the user confirms Generate, the server validates the project, model, prompt, concurrency, and BeatAPI connection, then creates a short-lived, one-time generation intent in SQLite. The durable local reference is uploaded to the configured provider storage only for that confirmed generation. The intent fixes the project, model, and exact upload count, and is consumed when the billable task is submitted. +The repository includes a root `.mcp.json` for compatible hosts and a thin Codex launcher under `integrations/codex/beatdesign`. -The `data/` directory is gitignored and contains both the SQLite database and local project assets. Back up or move the directory as one unit when migrating a workspace to another machine. +The 20 tools are grouped by durable product concepts: -Storage entitlement follows BeatAPI billing. The built-in provider is fixed to the official `https://api.beatapi.io` endpoint and uses the user's BeatAPI API key. Supported image, audio, and subtitle inputs go through BeatAPI Files; an official hosted deployment can preconfigure managed R2 for video inputs that Files does not accept through the separate `BEATAPI_MANAGED_R2_*` secrets. Self-hosters may instead provide their own Cloudflare R2 or other S3-compatible endpoint, bucket, credentials, and public URL through `R2_*`. The two credential sets never fall through to each other. +- **Project (3):** list, read, create. +- **Asset (3):** list, read, import a local image/video/audio file. +- **Canvas (3):** read, search, apply incremental operations. +- **Generation (5):** discover models and parameters, submit, refresh status, read history. +- **Editor (6):** read, edit, inspect a semantic snapshot, diagnose, deep-link a view, read command history. -No shared R2 secret is committed to this repository. “Preconfigured” means the official deployment supplies its storage credentials as deployment secrets, while a source checkout uses BeatAPI managed Files for supported inputs or the operator's own bucket. +MCP does not automate UI pixels. The Agent reads the Project, applies stable commands, and BeatDesign refreshes the visible Canvas or Editor from the same local state. -## Supported model catalog +See [MCP setup and tool boundaries](./docs/MCP.md). -The canonical catalog lives in `src/core/effects/effect-registry.ts`. The current workspace exposes four image models and seven video models through one BeatAPI adapter, including Kling 2.6 and Kling 3.0 Motion Control. Update the registry and adapter together when BeatAPI's public contract changes. +## What works today -## Commands +### Canvas and generation -| Command | Purpose | -| --- | --- | -| `pnpm dev` | Start the local app on `127.0.0.1:3020` | -| `pnpm typecheck` | Check TypeScript contracts | -| `pnpm test` | Run unit and contract tests | -| `pnpm i18n:check` | Validate English and Chinese messages | -| `pnpm media:localize` | Copy provider-hosted media in existing project snapshots into local project storage | -| `pnpm build` | Build the production app | -| `pnpm db:push` | Apply the schema during local development | -| `pnpm db:generate` | Generate a reviewable D1/SQLite migration | -| `pnpm cf:build` | Build the Cloudflare Workers artifact | - -## Project structure +- Image, video, audio, timeline, and text-oriented Canvas nodes. +- Prompt/model/parameter configuration from one code-defined BeatAPI catalog. +- Asset references, connected workflows, generation history, and locally persisted outputs. +- Image/video generation plus Standard and Deep video analysis. +- Canvas-to-Editor handoff through shared Assets and timeline nodes. + +### Local Editor + +- Video, still-image, and audio clips on a shared timeline. +- Drag-to-trim, split, move, delete, ripple delete, and still-image duration editing. +- Undo/redo, playback, source range selection, audio volume and fades. +- Selected-range AI redo as non-destructive Takes. +- Timeline diagnostics for gaps, overlaps, missing media, duration mismatches, and tiny clips. +- Browser-side H.264/AAC MP4 export; completed renders return to project Assets. + +### Local persistence + +- SQLite project data under `data/`. +- Project-owned media under `data/project-assets//`. +- Revision-aware Canvas and Editor saving. +- English and Chinese UI. +- Optional Cloudflare D1 deployment target. + +## Data and provider boundary ```text -src/components/beatcanvas Canvas UI and interactions -src/components/studio Guided Studio UI -src/core/adapters BeatAPI generation adapter -src/core/effects Model registry and generation lifecycle -src/core/projects Projects and snapshots -src/core/workspace-lib Shared asset and workspace utilities -src/routes/api Local server API -src/config/db SQLite/D1 schema +Browser UI / MCP + ↓ +Command Kernel + Project services + ↓ +SQLite + data/project-assets + ↓ only after a confirmed generation request +Generation adapter (BeatAPI by default) + ↓ +Output copied back into the local Asset library ``` -See [ARCHITECTURE.md](./ARCHITECTURE.md), [PROVIDERS.md](./PROVIDERS.md), and [WORKSPACE_MODES.md](./WORKSPACE_MODES.md) for contributor details. +API keys and storage credentials remain server-side. Selecting or dragging a local file does not send it to a provider. BeatDesign persists the file locally first and uploads only the durable project Asset required by a confirmed generation request. + +BeatDesign does not reproduce provider billing, balance, or rate-limit logic. It returns the provider's result or error to the UI/MCP caller. The upstream repository ships the official BeatAPI adapter; a fork can implement `BaseAdapter` and register it in `src/config/generation-providers.ts`. + +Read [provider architecture](./PROVIDERS.md) and [system architecture](./ARCHITECTURE.md) for the full contract. + +## Current v0.2 boundaries + +BeatDesign v0.2 is focused on local, short-form AI video workflows: + +- Current Chrome on macOS and Windows. +- Common browser-decodable images, MP4/MOV video, and MP3/WAV/M4A/AAC/OGG audio. +- One active timeline per Project and browser-memory-constrained export. +- Canvas and Editor poll for Agent revisions every two seconds; a realtime event bus is not implemented yet. +- Editor MCP snapshot is semantic inspection, not a rendered pixel frame. +- Captions, transitions, speed controls, waveforms, multiple named timelines, and native desktop packaging remain follow-up work. + +The complete completed/planned boundary lives in [Product plan and status](./docs/PRODUCT_PLAN_AND_STATUS.md). + +## Developer commands + +| Command | Purpose | +| --- | --- | +| `pnpm dev` | Start the local app on `127.0.0.1:3020`. | +| `pnpm --silent mcp` | Start the local MCP server over stdio. | +| `pnpm typecheck` | Check TypeScript contracts. | +| `pnpm test` | Run unit and contract tests. | +| `pnpm i18n:check` | Validate English and Chinese messages. | +| `pnpm build` | Build the production app. | +| `pnpm db:push` | Apply the local SQLite schema during development. | +| `pnpm media:localize` | Copy provider-hosted media in existing snapshots into local project storage. | -## Security and deployment +## Contributing -The default development server binds to `127.0.0.1`. Connection settings are writable from the trusted local workspace only. If you expose this application to the internet, configure provider and storage credentials through deployment secrets and add your own network access control in front of the app. +Start with [CONTRIBUTING.md](./CONTRIBUTING.md), then use: -Never commit `.env.development`, local databases, or provider credentials. +- [ARCHITECTURE.md](./ARCHITECTURE.md) for system boundaries. +- [WORKSPACE_MODES.md](./WORKSPACE_MODES.md) for product surfaces. +- [docs/MCP.md](./docs/MCP.md) for Agent integration. +- [DESIGN.md](./DESIGN.md) for the BeatDesign visual language. +- [SECURITY.md](./SECURITY.md) for local and deployment safety. ## License -Licensed under the [Apache License 2.0](./LICENSE). The license covers the code, not BeatAPI trademarks, logos, service access, model-provider rights, or third-party assets. See [TRADEMARKS.md](./TRADEMARKS.md). +BeatDesign is licensed under [Apache License 2.0](./LICENSE). The license covers the code, not BeatAPI trademarks, service access, model-provider rights, or third-party assets. The timeline contract is derived from MIT-licensed OpenReel and the browser media pipeline uses MPL-2.0 Mediabunny; see [`third_party/`](./third_party) and [TRADEMARKS.md](./TRADEMARKS.md). diff --git a/README.zh-CN.md b/README.zh-CN.md index 6925b08..4c7e678 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -1,28 +1,62 @@ -# BeatDesign +

+ BeatDesign——一个可被任意 MCP Agent 操控的本地创作工作台 +

-**化繁为简,自由创作。** +

BeatDesign

-开源、本地优先的 AI 图片、视频创作与分析画布。BeatDesign 将项目、Studio、节点式 Canvas、共享素材、生成历史、可配置存储和 BeatAPI 官方连接整合在一个专注的工作台中。 +

+ 一个本地创作工作台,连接任意 Agent。
+ 在 Canvas 上生成,在时间线上编辑,让支持 MCP 的 Agent 在旁边操控同一个项目。 +

-本仓库不包含登录、账号、支付、订阅、本地积分、后台管理、RBAC 或 API Key 发放系统。 +

+ 快速开始 · + MCP · + 产品状态 · + English +

-## 包含什么 +BeatDesign 是一个免费、开源、本地优先的 AI 媒体工作台。它把引导式 Studio、无限 Canvas、短视频 Editor 和共享 Asset 素材库放进同一个本地 Project。 -- Studio:以提示词和参数为主的快速生成体验,并支持 Standard 与 Deep 视频分析。 -- Canvas:基于 React Flow 的多步骤节点画布。 -- Assets:Studio 与 Canvas 共用的项目素材库。 -- 本地项目、画布快照、生成历史和素材索引。 -- 由代码统一维护的 BeatAPI 图片/视频模型目录。 -- BeatAPI 任务提交、状态轮询和结果回填。 -- 内置 Provider 固定使用 BeatAPI,用户填写自己的 BeatAPI API Key。 -- 默认使用 BeatAPI 托管 R2,也允许切换到自己的 Cloudflare R2/S3。 -- Provider 与存储配置保留在服务端,可选静态加密。 -- 中英文界面。 -- 默认 SQLite,可选部署到 Cloudflare D1。 +浏览器是用户可见的创作空间。Codex、Claude Code、Cursor 或其他支持 MCP stdio 的 Agent,可以通过 BeatDesign 的 20 个本地工具读取和修改同一个 Project。本仓库不包含账号、订阅、本地积分账本、支付或管理后台。 + +## 连续创作闭环 + +```text +提示词或本地素材 + ↓ +Studio / Canvas ── 生成、分支、连接、复用 + ↓ +共享 Assets ───── 图片、视频、音频、截取片段、成片 + ↓ +Editor ────────── 裁剪、切分、移动、混音、AI 重做、导出 MP4 + ↑ +任意 MCP Agent ── 读取项目并执行项目级指令 +``` + +它把原本分散在 AI 生成器、下载文件夹和剪辑软件里的步骤,收敛成一条连续工作流。 + +## 一个 Project,四个视图 + +| 视图 | 用途 | +| --- | --- | +| **Studio** | 用提示词完成图片/视频生成,以及 Standard / Deep 视频分析。 | +| **Canvas** | 组织媒体节点、生成历史、引用关系、分支与可复用输出。 | +| **Editor** | 在本地时间线上编辑视频、图片和音频,并导出 MP4。 | +| **Assets** | Studio、Canvas、Editor 和 MCP 共享的项目级素材事实层。 | + +## BeatDesign 的核心区别 + +- **本地优先。** Project、Canvas 快照、时间线、生成历史和导入媒体都保存在本地工作区。 +- **Agent 原生,但不绑定某个 Agent。** BeatDesign 暴露标准 stdio MCP Server,而不是内置一个封闭聊天机器人。 +- **Asset-first。** 生成和导入的媒体先成为稳定的 Project Asset,再进入 Canvas 或 Editor。 +- **统一命令路径。** UI 与 MCP 写操作共用 Command Kernel、revision 检查和持久化 receipt。 +- **不要求安装系统 FFmpeg。** 预览和 MP4 导出使用浏览器 WebCodecs 与 Mediabunny。 +- **Provider 边界清楚。** BeatAPI 是默认内置生成适配器;fork 可以增加自己的适配器,不需要重写 Canvas 或 Editor。 ## 快速开始 -需要 Node.js 22+ 与 pnpm 10+。 +需要 Node.js 22+、pnpm 10+,以及 macOS 或 Windows 上的新版 Chrome。 ```bash pnpm install @@ -31,54 +65,116 @@ pnpm db:push pnpm dev ``` -打开 `http://localhost:3020` 会直接进入 Home。访问 Studio 或 Canvas 不会立即写入项目数据库,只有确认开始项目后才会创建记录。点击工作台顶部的连接配置按钮,可以填写你自己的 [BeatAPI API Key](https://beatapi.io/dashboard/apikeys) 并选择存储;也可以通过 `.env.development` 配置。 +打开 [http://127.0.0.1:3020](http://127.0.0.1:3020)。 + +本地导入、Canvas 操作、时间线剪辑、预览和 MP4 导出都不需要 API Key。只有在生成、分析媒体或使用 AI 重做时,才需要填写自己的 [BeatAPI API Key](https://beatapi.io/dashboard/apikeys)。 + +## 通过 MCP 让 Agent 操控 + +BeatDesign 本地运行两个进程,它们读取同一个 SQLite 数据库: + +| 进程 | 命令 | 作用 | +| --- | --- | --- | +| 可视工作台 | `pnpm dev` | 在浏览器中打开 Studio、Canvas、Editor 和 Assets。 | +| Agent 控制面 | `pnpm --silent mcp` | 通过 stdio 暴露项目级工具。 | -## 数据逻辑 +仓库根目录已经包含兼容宿主可读取的 `.mcp.json`;Codex 的轻量启动封装位于 `integrations/codex/beatdesign`。 + +20 个工具按照稳定的产品概念分组: + +- **Project(3):** 列表、读取、创建。 +- **Asset(3):** 列表、读取、导入本地图片/视频/音频。 +- **Canvas(3):** 读取、搜索、增量修改。 +- **Generation(5):** 获取模型与参数、提交生成、刷新状态、读取历史。 +- **Editor(6):** 读取、编辑、语义快照、诊断、视图深链、命令历史。 + +MCP 不会模拟点击界面像素。Agent 读取 Project、提交稳定命令,BeatDesign 再从同一份本地状态刷新用户可见的 Canvas 或 Editor。 + +详细接入方式见 [MCP 设置与工具边界](./docs/MCP.md)。 + +## 当前已经支持 + +### Canvas 与生成 + +- 图片、视频、音频、时间线及文本向 Canvas 节点。 +- 由一份代码目录统一定义的 BeatAPI 模型、参数与引用能力。 +- Asset 引用、连接工作流、生成历史和本地持久化输出。 +- 图片/视频生成,以及 Standard / Deep 视频分析。 +- 通过共享 Asset 与时间线节点完成 Canvas-to-Editor 交接。 + +### 本地 Editor + +- 视频、静态图片和音频共用一条时间线。 +- 拖动裁剪、切分、移动、删除、波纹删除,以及静态图片时长调整。 +- Undo/redo、播放、源素材范围选择、音量和淡入淡出。 +- 把选中区间交给 AI 重做,并以非破坏性 Take 保存。 +- 检查空隙、重叠、素材缺失、时长不一致和过短片段。 +- 在浏览器中导出 H.264/AAC MP4;成片会回到 Project Assets。 + +### 本地持久化 + +- Project 数据保存在 `data/` 下的 SQLite。 +- 项目媒体保存在 `data/project-assets//`。 +- Canvas 与 Editor 使用 revision-aware 保存。 +- 中英文界面。 +- 可选 Cloudflare D1 部署目标。 + +## 数据与 Provider 边界 ```text -Studio / Canvas / Assets - -> 本地服务端接口 - -> 导入文件立即保存到 data/project-assets,并写入 SQLite 素材索引 - -> 生成预检与 SQLite 一次性生成意图 - -> 仅在确认生成后,把已持久化的本地引用上传到 Provider - -> BeatAPI 图片、视频生成任务或视频分析接口 - -> 轮询 Provider 状态 - -> 将生成媒体复制到本地项目素材目录 - -> 写入本地生成历史与素材索引 +浏览器 UI / MCP + ↓ +Command Kernel + Project services + ↓ +SQLite + data/project-assets + ↓ 仅在用户确认生成后 +Generation adapter(默认 BeatAPI) + ↓ +输出复制回本地 Asset 素材库 ``` -API Key 与存储密钥只保留在服务端。项目和历史数据保存在本地 SQLite。在本地 SQLite 模式下,导入的图片和视频会先复制到项目自己的 `data/project-assets//` 目录,并写入 SQLite 素材索引,然后才加入 Canvas。视频分析会把 MP4/MOV 输入上传到 BeatAPI Files,并把返回的分析文本写入同一份项目生成历史。生成的图片、视频和视频封面都会先下载到同一个项目素材目录,Canvas 再使用对应的本地地址;Provider 地址只保留为元数据。 +API Key 和存储凭据只留在服务端。选择或拖入本地文件不会把它发送给 Provider;BeatDesign 会先把它持久化为本地 Project Asset,只有用户确认生成后,才上传该次生成真正需要的素材。 + +BeatDesign 不重复实现 Provider 的余额、计费或限流逻辑,只把 Provider 的结果或错误返回给 UI / MCP。上游仓库内置官方 BeatAPI 适配器;fork 可以实现 `BaseAdapter`,并在 `src/config/generation-providers.ts` 中注册。 -Canvas 内容变化后会保存一份完整项目快照;存在未保存变化时,每 5 秒再检查一次,并在页面切到后台、刷新或关闭时强制落盘。已有内容的快照不会被未经确认的空快照覆盖。 +完整约定见 [Provider 架构](./PROVIDERS.md) 与 [系统架构](./ARCHITECTURE.md)。 -用户选择文件或从 Finder 拖入文件时,不会立刻发送给 Provider,但会立即持久化在用户自己的电脑上,因此刷新、重启或开发热更新都不会丢失。用户确认“生成”后,服务端先检查项目、模型、提示词、并发限制和 BeatAPI 连接,再在 SQLite 中创建短时、一次性的生成意图;只有这时,已经持久化的本地引用才会上传到配置好的 Provider 存储。意图会绑定项目、模型和准确上传数量,并在提交付费任务时被消费。 +## v0.2 当前边界 -`data/` 目录已被 Git 忽略,其中同时包含 SQLite 数据库和本地项目素材。把工作台迁移到另一台电脑时,应将整个目录一起备份或复制。 +BeatDesign v0.2 聚焦本地 AI 短视频工作流: -存储权益跟随 BeatAPI 计费:内置 Provider 固定通过官方 `https://api.beatapi.io` 接口,并使用用户自己的 BeatAPI API Key。图片、音频和字幕通过 BeatAPI Files 进入托管存储;官方部署通过独立的 `BEATAPI_MANAGED_R2_*` Secret 预配置 R2,处理 Files 暂不支持的视频输入。源码用户也可以通过 `R2_*` 配置自己的 Cloudflare R2 或其他 S3 兼容存储。两组凭据不会互相回退。 +- 支持 macOS / Windows 上的新版 Chrome。 +- 支持常见浏览器可解码图片、MP4/MOV 视频,以及 MP3/WAV/M4A/AAC/OGG 音频。 +- 每个 Project 当前只有一条活动时间线;导出仍受浏览器内存约束。 +- Canvas 和 Editor 每两秒轮询 Agent revision;暂未实现实时事件总线。 +- Editor MCP snapshot 是语义检查,不是像素截图。 +- 字幕、转场、变速、波形、多时间线和原生桌面封装仍属于后续阶段。 -仓库不会内置或泄露共享 R2 密钥。所谓“默认配置好”,指官方部署通过部署 Secret 注入托管 R2;源码用户对受支持输入使用 BeatAPI 托管 Files,或自行接入自己的存储。 +完整的已完成/待完成边界见 [产品规划与当前状态](./docs/PRODUCT_PLAN_AND_STATUS.md)。 -## 主要命令 +## 开发命令 -| 命令 | 用途 | +| 命令 | 作用 | | --- | --- | -| `pnpm dev` | 在 `127.0.0.1:3020` 启动本地应用 | -| `pnpm typecheck` | 类型检查 | -| `pnpm test` | 单元与契约测试 | -| `pnpm i18n:check` | 检查中英文文案 | -| `pnpm media:localize` | 将现有项目快照中的 Provider 媒体复制到本地项目存储 | -| `pnpm build` | 生产构建 | -| `pnpm cf:build` | 构建 Cloudflare Workers 产物 | -| `pnpm db:push` | 本地同步数据库结构 | -| `pnpm db:generate` | 生成可审阅的 SQLite/D1 迁移 | +| `pnpm dev` | 在 `127.0.0.1:3020` 启动本地应用。 | +| `pnpm --silent mcp` | 通过 stdio 启动本地 MCP Server。 | +| `pnpm typecheck` | 检查 TypeScript 契约。 | +| `pnpm test` | 运行单元与契约测试。 | +| `pnpm i18n:check` | 校验中英文消息。 | +| `pnpm build` | 构建生产版本。 | +| `pnpm db:push` | 在本地开发中应用 SQLite schema。 | +| `pnpm media:localize` | 把旧 Project 快照中的 Provider 媒体复制到本地素材目录。 | -## 安全提示 +## 参与贡献 -开发服务默认只监听 `127.0.0.1`,连接配置只允许在可信本地环境修改。如果部署到公网,请通过部署平台 Secret 配置 Provider 与存储凭据,并在应用前增加网络访问控制。 +先阅读 [CONTRIBUTING.md](./CONTRIBUTING.md),再按需查看: -不要提交 `.env.development`、本地数据库或任何 Provider 凭据。 +- [ARCHITECTURE.md](./ARCHITECTURE.md):系统边界。 +- [WORKSPACE_MODES.md](./WORKSPACE_MODES.md):产品视图。 +- [docs/MCP.md](./docs/MCP.md):Agent 接入。 +- [DESIGN.md](./DESIGN.md):BeatDesign 视觉语言。 +- [SECURITY.md](./SECURITY.md):本地与部署安全。 -## 许可证 +## License -代码采用 [Apache License 2.0](./LICENSE)。许可证不授予 BeatAPI 商标、Logo、服务访问、模型供应商权利或第三方素材的使用权,详见 [TRADEMARKS.md](./TRADEMARKS.md)。 +BeatDesign 使用 [Apache License 2.0](./LICENSE)。许可证覆盖代码,不覆盖 BeatAPI 商标、服务访问、模型 Provider 权利或第三方素材。时间线契约派生自 MIT 许可的 OpenReel,浏览器媒体管线使用 MPL-2.0 Mediabunny;详见 [`third_party/`](./third_party) 与 [TRADEMARKS.md](./TRADEMARKS.md)。 diff --git a/RELEASE_SCOPE.md b/RELEASE_SCOPE.md index ecd1e58..5edee36 100644 --- a/RELEASE_SCOPE.md +++ b/RELEASE_SCOPE.md @@ -6,4 +6,16 @@ Included: homepage, projects, Studio, Canvas, provider configuration, supported Excluded: authentication, login, accounts, payments, subscriptions, credits, API-key issuing, invitations, RBAC, admin, support tickets, CMS, email delivery, and unrelated AI-provider adapters. -Release verification requires a clean install, schema creation, typecheck, test, i18n check, production build, and local route smoke test. A real paid BeatAPI generation is a separate credentialed end-to-end check. +Release verification requires a clean install, schema creation, production build (which generates Paraglide and route types), typecheck, test, i18n check, MCP stdio handshake, and local route smoke test. A real paid BeatAPI generation is a separate credentialed end-to-end check. + +## v0.2.0 release gate + +- [x] Package, MCP server, and Codex plugin versions agree on `0.2.0`. +- [x] Canvas, Editor, and MCP contracts have automated coverage. +- [x] MCP exposes Project, Asset, Canvas, Generation, and Editor groups without full-document replacement tools. +- [x] Local media import and image clips are documented as shipped capabilities. +- [x] Verify a clean checkout with `pnpm install --frozen-lockfile`, schema creation, `pnpm build`, typecheck, tests, i18n, and MCP handshake—in that order. +- [x] Run a visible Canvas/Editor + MCP smoke test: Canvas prompt changes and Editor duration changes appeared without a page refresh, then the QA fixture was restored through MCP. +- [x] Prepare the local release branch and release documentation; GitHub push, merge, tag, and Release remain explicit external actions performed only with user authorization. + +Release notes are maintained in [`CHANGELOG.md`](./CHANGELOG.md). diff --git a/SECURITY.md b/SECURITY.md index 56e2b84..6a4040a 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -10,4 +10,4 @@ Include the affected route or component, reproduction steps, impact, and any sug BeatDesign is a single-user, local-first application and does not include authentication. The development server binds to localhost. Operators who expose it to a network must add access control in front of the application and provide provider and storage credentials through deployment secrets. -The maintainers support the latest released `0.1.x` version. Security fixes are published in the next patch release when possible. +The maintainers support the latest released `0.2.x` version. Security fixes are published in the next patch release when possible. diff --git a/WORKSPACE_MODES.md b/WORKSPACE_MODES.md index e0c160b..135db2c 100644 --- a/WORKSPACE_MODES.md +++ b/WORKSPACE_MODES.md @@ -1,15 +1,17 @@ # Workspace modes -Each project can be opened in three modes: +Each project can be opened in four views: - **Studio** is a guided, form-first surface for image/video generation and Standard or Deep video analysis. - **Canvas** is a node-based surface for connected references, generation, and durable video-analysis reports. -- **Assets** is the project-scoped library shared by Studio and Canvas. +- **Editor** is the local timeline for arranging, trimming, splitting, previewing, and exporting short-form video. +- **Assets** is the project-scoped media library shared by Studio, Canvas, Editor, and MCP. -All three modes share the same project ID, assets, model catalog, provider connection, generation and analysis history, and persistence layer. The last opened mode is stored on the project so the project list can resume in place. +All four views share the same project ID, Assets, model catalog, provider connection, generation and analysis history, and persistence layer. The last opened view is stored on the project so the project list can resume in place. Routes: - `/studio/:projectId` - `/canvas/:projectId` +- `/editor/:projectId` - `/assets/:projectId` diff --git a/docs/MCP.md b/docs/MCP.md new file mode 100644 index 0000000..3718420 --- /dev/null +++ b/docs/MCP.md @@ -0,0 +1,94 @@ +# BeatDesign MCP + +BeatDesign exposes one local stdio MCP server. It uses the same project, +Asset, Command Kernel, provider, and SQLite services as the browser UI; it is +not a second backend. + +## Start + +```bash +pnpm install +pnpm db:push +pnpm --silent mcp +``` + +For a generic MCP host, register: + +```json +{ + "mcpServers": { + "beatdesign": { + "command": "/absolute/path/to/Beat Design/node_modules/.bin/tsx", + "args": ["/absolute/path/to/Beat Design/scripts/mcp-server.ts"] + } + } +} +``` + +The Codex plugin source is under `integrations/codex/beatdesign`. When a host +copies that plugin outside this repository, pass `BEATDESIGN_ROOT` with the +absolute repository path. + +## Mental model + +BeatDesign is two processes sharing one local database: + +1. `pnpm dev` is the visual workbench in the browser. +2. `pnpm mcp` is the Agent control plane over stdio. + +An Agent never "opens the Canvas as a sidebar" inside Codex or Claude Code. +It calls tools. You watch the same project at `http://127.0.0.1:3020`. + +The Codex plugin is optional packaging. Claude Code, OpenCode, Cursor, and any +other MCP stdio host can skip it and start `pnpm --silent mcp` directly. A +marketplace application is not required for local use. + +Repo-root `.mcp.json` is the Claude Code / generic host config. See +`integrations/codex/beatdesign/README.md` for Codex folder install and +OpenCode `mcp` config. + +## Tool groups + +There are **20** tools: + +- Project (3): list, get, create. +- Asset (3): list, get by project membership, and import a local file. +- Canvas (3): get, search, incremental apply. +- Generation (5): list model capabilities, read one model, submit an + asset-first request, refresh status, and list history. +- Editor (6): get, incremental edit, semantic snapshot, diagnostics, deep-link + view, and command history. + +MCP writes use `origin=mcp` assigned inside the server. `canvas.apply` and +`editor.apply` accept stable IDs, revisions, and idempotency keys. The server +does not expose full Canvas or Timeline replacement. + +Generation tools accept a logical `modelId`, generic `parameters`, and Asset +references. Call `bdesign_generation_models` or +`bdesign_generation_model_get` before choosing parameters. Outputs are Assets; +placing them on Canvas or Editor remains a separate command. +`bdesign_generation_submit` submits once through the configured generation +provider. BeatDesign does not duplicate API-key validity, balance, billing, or +rate-limit policy. BeatAPI (or a fork's selected provider) remains authoritative, +and its response or error details are returned to the MCP caller. + +`bdesign_canvas_apply` and `bdesign_editor_edit` advertise every supported +incremental operation as a concrete JSON Schema. Agents can discover required +IDs, time fields, media roles, card parameters, Takes, and render fields from +the MCP tool contract instead of guessing an opaque operation object. + +## Current boundaries + +- `bdesign_editor_snapshot` resolves active clips and source times; it does not + rasterize a pixel frame yet. +- Browser-only MP4 export is not exposed as a headless MCP tool yet. +- `bdesign_asset_import` copies a local image, video, or audio file into the + project Asset library from an absolute path. It does not place the Asset on + Canvas or Editor; use `bdesign_canvas_apply` or `bdesign_editor_edit` after. +- `bdesign_asset_import` is the local-file bridge. Generation still accepts + project Asset IDs rather than arbitrary file paths; the shared generation + path performs any provider upload that is required after submission. +- Command history is a bounded retry receipt log, not a permanent audit trail. +- Live UI event push is not implemented. Canvas and Editor poll project + revisions every two seconds while idle and check again on focus/visibility, + so MCP writes become visible without a full event bus. diff --git a/docs/PRODUCT_PLAN_AND_STATUS.md b/docs/PRODUCT_PLAN_AND_STATUS.md new file mode 100644 index 0000000..65a37a1 --- /dev/null +++ b/docs/PRODUCT_PLAN_AND_STATUS.md @@ -0,0 +1,274 @@ +# BeatDesign 产品规划与当前完成状态 + +- 文档用途:给新加入的开发者、Codex、Claude Code 和其他 AI Agent 提供统一产品上下文 +- 当前基线:BeatDesign v0.2 Phase 1 本地实现 +- 最后核对:2026-08-30 +- 事实边界:只有代码、测试和本文“已完成”栏目共同证明的能力才算完成;规划项不等于已发布 +- 发布状态:当前工作树已实现并完成本地工程验证,尚未据此声明 GitHub Release 或桌面端发布 + +## 1. BeatDesign 是什么 + +BeatDesign 是一个免费、开源、本地优先的 AI 图片/视频创作工作台。它把 Studio、节点 Canvas、项目 Assets、本地视频 Editor、生成历史和 BeatAPI 连接放在同一个 Project 中。 + +长期产品形态: + +> One local creative workspace. Any agent. + +用户可以独立使用 BeatDesign,也可以让 Codex、Claude Code 或其他通用 Agent 通过 MCP 操控画布和编辑器。BeatDesign 不在画布里重新内置一个通用 Agent。 + +## 2. 产品边界 + +### 开源 BeatDesign 负责 + +- 本地项目、素材、快照和时间线。 +- Studio、Canvas、Editor、Assets 和 Generation History。 +- 图片/视频生成、视频分析和后续音频能力入口。 +- 本地媒体处理、预览、剪辑、诊断和导出。 +- Command Kernel、MCP、CLI 和可视化交接。 +- BeatAPI 官方连接和 BYOK。 + +### BeatAPI 服务负责 + +- 远程 AI 模型能力。 +- 异步任务、模型路由和服务计费。 +- 用户通过自己的 BeatAPI API Key 使用服务。 + +### 不属于本仓库 + +- SaaS 账户、登录、支付、订阅、积分和管理后台。 +- 通用聊天 Agent、通用 Router/Planner/Executor 平台。 +- 社交平台发布、飞书连接和其他外部应用自动化。 + +## 3. 产品架构 + +```text +Studio ─────┐ +Canvas ─────┼── Project / Assets / Generation / History +Editor ─────┘ │ + ▼ + Local SQLite + Files + │ + BeatAPI(确认后调用) +``` + +- Studio:引导式单次生成和分析。 +- Canvas:复杂生成、参考素材、分支和血缘。 +- Editor:时间组织、剪辑、音频、字幕和交付。 +- Assets:媒体事实来源,所有工作空间共享。 + +未来 Agent 架构: + +```text +Codex / Claude Code / Other Agent + │ + BeatDesign MCP + │ + Command Kernel + │ + Studio / Canvas / Editor / Assets / Generation +``` + +## 4. 统一术语 + +| 名称 | 含义 | +|---|---| +| Asset | 本地真实媒体文件及其元数据 | +| Canvas Node | Asset、Generation 或 Timeline 在画布上的引用 | +| Generation | 产生 Asset 的异步任务,不绑定单一 placement | +| Timeline | 项目级剪辑文档 | +| Clip | Asset 在 Timeline 上的一次使用 | +| Take | 同一 Clip 的备选媒体版本,可恢复 Original | +| Canvas Edge | 用户视觉组织关系 | +| Generation Lineage | 真实生成或派生来源关系 | +| Command Kernel | UI、MCP、CLI 共用的唯一业务操作层 | + +## 5. 已完成并有代码支撑的能力 + +### Project / Local-first + +- 项目创建、打开、重命名和本地 SQLite 持久化。 +- Canvas Snapshot 自动保存、页面隐藏时刷新和版本冲突保护。 +- 本地上传图片、视频、音频后立即复制到项目目录。 +- 生成图片、视频和封面在展示前本地化。 +- 项目 Assets 和生成历史索引。 +- BeatAPI 与存储凭证保留在服务端。 + +### Studio / Canvas + +- Studio 图片、视频生成和 Standard/Deep 视频分析。 +- React Flow 节点 Canvas。 +- 图片、视频、音频 Asset Node。 +- 图片/视频 Generation Node、Generation Output 与多个版本。 +- 模型、提示词、画幅、时长、质量和参考素材配置。 +- Canvas 节点连接、分组、复制、删除、撤销/重做。 +- 音频节点和 Timeline Node。 +- Asset、Canvas、Editor 之间的基础导航。 + +### Editor + +- 项目级 TimelineDocument、Video/Audio Track、Clip 和 Take。 +- 视频和音频导入。 +- 非破坏式 trim、split、move、delete 和 ripple delete。 +- undo/redo。 +- 音量、静音、淡入和淡出。 +- 浏览器本地预览。 +- 选区派生 MP4。 +- BeatAPI 选区 AI redo。 +- Redo 结果作为 Take 保存、激活和恢复 Original。 +- 完整 Timeline H.264/AAC MP4 导出。 +- 导出写回 Assets 和 Canvas Timeline Node。 +- 缺失素材、Clip 重叠和视频轨空隙诊断。 +- Editor 自动保存统一走 Command Kernel;并发写入使用 revision/CAS 和三方合并,不再用旧文档覆盖新版本。 +- 同一次加入时间线使用稳定 Clip ID 和幂等键,重复点击、网络重试不会重复插入同一批 Clip。 +- Take 必须覆盖 Clip 时长;过短 Take 不可激活,诊断会阻止可能产生冻结尾帧的导出。 + +### 媒体技术 + +- 使用 WebCodecs + Mediabunny 在浏览器完成媒体检查、编码和 MP4 封装。 +- 不要求用户安装系统 FFmpeg。 +- OpenReel 只作为时间线术语、文档模型和非破坏式编辑行为的参考;未打包其完整 UI 和应用外壳。 +- 媒体 metadata 采用限并发队列并带超时释放;单个损坏媒体不会永久阻塞后续卡片。 + +### Command Kernel / Agent-ready 基础 + +- `POST /api/app/projects/:id/commands` 是 Canvas/Editor 业务命令的统一写入口。 +- 命令请求使用严格运行时 Schema,拒绝未知命令、缺少稳定 ID 的 Clip/Take 操作和越界参数。 +- 命令中的 Asset 会按当前 Project membership 重新验证,并由服务端写入权威媒体 URL。 +- `project_command_receipt` 保存短期幂等回执;同一项目和幂等键只执行一次,并有限时、限量清理。 +- 幂等键会绑定首次 command;复用到另一个 command 会返回 `INVALID_COMMAND`,不会静默冒用旧结果。 +- UI 内部允许 revision-checked `editor.replace_document` 支撑 undo/redo 和本地自动保存;MCP/CLI 被明确禁止整份替换,只能调用 `editor.apply`。 +- Generation 的 `AssetFirstGenerationRequest` 已成为服务端权威输入:适配器媒体参数由 Asset ID 和当前 generation intent 编译,旧的客户端 URL 字段不再决定引用事实。 +- UI 命令入口不再接受客户端 `origin`;服务端固定写入 `ui`,MCP 入口在内核边界固定写入 `mcp`。 +- Provider Contract 已将逻辑模型目录与 BeatAPI effectId、上传路径和上游模型名拆开;BeatAPI 是默认实现,Fork 可在源码扩展点注册其他 Provider。 +- 本地 stdio MCP Server 提供 20 个工具(Project / Asset / Canvas / Generation / Editor),模型和参数通过 capability discovery 暴露;Canvas / Editor 增量操作使用完整 JSON Schema,Agent 可直接发现操作类型和参数。MCP 生成直接调用当前 Provider,本地产品不重复实现 API Key、余额、计费或限流策略,只透传 Provider 的结果与错误。`bdesign_asset_import` 把本地文件导入项目 Asset 库。 + +## 6. v0.2 Phase 1 本地已实现 + +主 PRD:[BEATDESIGN_V0_2.md](./prd/BEATDESIGN_V0_2.md) + +- 统一 asset-first 产品模型。 +- Command Kernel 基础、严格请求 Schema、Project Asset 边界和短期幂等回执。 +- Canvas 视频尾帧提取。 +- 使用尾帧创建下一段视频生成节点。 +- 多个 Canvas 视频一次创建/追加时间线。 +- Canvas -> Timeline Node -> Editor 连续工作流。 +- Editor 自动保存接入命令入口,并补齐冲突三方合并、重复操作保护和稳定播放头时间。 +- 图片 Clip、时间线拖拽调整持续时间与图片/视频统一视觉轨。 +- 本地 MCP Server 提供 20 个 Project、Asset、Canvas、Generation、Editor 工具;支持从绝对路径导入本地素材。 +- Canvas 与 Editor 每 2 秒并在页面重新聚焦时检查 MCP 写入的新 revision。 + +本阶段的本地验收边界:类型检查、自动测试、国际化检查、生产构建和浏览器关键路径。Git 提交、远端合并和公开 Release 属于后续独立状态。 + +2026-08-30 浏览器实操证据:保留了本地项目 `v0.2 QA – Canvas-to-Editor`。一段 10.01 秒本地 MP4 已完成“尾帧提取 -> 派生图片 Asset -> 下一段视频 Generation Node -> Timeline Node -> 打开 Editor -> 裁取 3.00–7.00 秒 -> 得到 4.00 秒 Clip -> 导出 H.264/AAC MP4 -> Render 写回 Assets”的闭环。实操同时发现并修复了开发服务器把 `Sec-Fetch-Dest: video/image/audio` 错当成静态文件、导致本地媒体 404 的问题。 + +2026-08-30 本轮内核验收:未知 command 返回 `INVALID_COMMAND`;伪造 Asset 返回 `NOT_FOUND`;相同幂等键连续提交两次只产生一条回执且返回同一 revision;`origin=mcp` 的整份 Timeline 替换被拒绝。本地 MCP 工具目录为 20 个;Canvas / Editor 操作的完整 JSON Schema、未知 command、伪造 Asset、幂等回执、MCP 本地素材导入和 `origin=mcp` 禁止整份 Timeline 替换均有自动测试覆盖。发布门禁的最新测试数量以仓库当前 `pnpm test` 结果为准。 + +2026-08-30 MCP 可见联调证据:在已打开的本地 Canvas 中,通过 stdio MCP 修改生成节点提示词,页面在轮询周期内自动显示新内容;在已打开的 Editor 中,通过 `bdesign_editor_edit` 把图片 Clip 从 4.00 秒改为 3.50 秒,时间线和素材信息同步更新,无需刷新。联调完成后,Canvas 提示词与 Editor 时长均通过 MCP 恢复原值。该联调同时修复了 Canvas 在存在待保存本地布局时跳过 Agent revision 的问题。 + +## 7. 尚未完成 + +### 产品 P1 + +- 字幕轨、SRT 导入、字幕样式和转录入口。 +- 时间线吸附、缩放、音频波形和多选。 +- 变速和音频变速策略。 +- 交叉溶解、淡入淡出、黑场转场。 +- 合成 Preview Frame、短区间预览和更完整 Diagnostics。 +- AI Bridge 衔接片段。 +- 多个命名 Timeline。 +- 更广的浏览器/编码兼容矩阵和代理媒体。 + +### Agent / MCP 后续边界 + +- MCP Resources 和更完整的 schema versioning。 +- Agent Activity、命令审计和实时 UI 事件桥。 +- Claude Code 等宿主的一键安装体验;Codex 当前只有仓库内薄插件封装。 +- headless 预览/导出本地媒体 Worker。 + +当前可以称为“已支持本地 MCP 基础版”,但不能称为完整 Agent 编辑环境:像素级 Snapshot、headless 导出、实时 UI 事件和永久审计仍未实现。本地文件导入桥已完成;UI 当前使用 2 秒 revision 轮询和聚焦检查,而不是实时事件推送。 + +Canvas 的拖拽、缩放、视口和完整布局仍使用 revision-checked Snapshot 自动保存;Canvas/Timeline 业务 operation 已有 Command 合同,Timeline Node 回写也已接入 `/commands`。后续做 MCP parity 时,应继续把可语义化的 Canvas UI 动作迁移为 `canvas.apply`,不能让外部 Agent 调用完整 Snapshot 覆盖。 + +### 桌面端 + +- Electron/Tauri 壳。 +- 内置 Node runtime、MCP 和媒体 Worker。 +- 原生文件选择、系统集成、签名、公证和自动更新。 +- 需要时再评估内置原生 FFmpeg;当前开源 localhost 版本不要求它。 + +## 8. 路线图 + +### v0.2:Canvas-to-Editor Creative Loop + +- 统一模型和 Command Kernel。 +- 尾帧续写。 +- 多片段创建时间线。 +- 人工操作闭环稳定。 + +### v0.3:Short-form Completion + +- 图片、字幕、变速、基础转场。 +- 波形、吸附和预览诊断。 +- AI Bridge。 + +### v0.4:Agent Control + +- 本地 MCP Server 基础版已前置完成。 +- UI/MCP parity tests。 +- Codex 优先接入。 +- Agent 操作实时可见、可撤销、可审计。 + +### v0.5:Desktop Experience + +- 桌面壳、一键安装、后台媒体 Worker。 +- 更稳定的大文件和长任务处理。 + +## 9. 必须保持的设计原则 + +1. 没有 Agent 时,BeatDesign 仍然是完整可用的产品。 +2. 每次生成先形成 Asset,再由 Canvas 或 Editor 引用。 +3. Editor 锁定具体 Asset,不自动跟随 Canvas 切换版本。 +4. UI、MCP 和 CLI 必须调用同一个 Command Kernel。 +5. MCP 不直接写 SQLite,不盲目覆盖完整 Snapshot。 +6. 本地操作不自动把素材上传到外部服务。 +7. 付费生成必须经过 precheck 和用户确认。 +8. Agent 操作必须可见、可撤销、可验证。 +9. 生成操作和确定性剪辑/转码必须区分。 +10. 开源仓库不混入 SaaS 账户、支付和积分系统。 + +## 10. 新 AI 接手时的阅读顺序 + +1. `AGENTS.md` +2. 本文 `docs/PRODUCT_PLAN_AND_STATUS.md` +3. `docs/prd/BEATDESIGN_V0_2.md` +4. `README.md` +5. `ARCHITECTURE.md` +6. `PROVIDERS.md` +7. `docs/prd/VIDEO_TIMELINE_PHASE_1.md` + +修改产品前先运行: + +```text +git status --short +pnpm typecheck +pnpm test +pnpm i18n:check +pnpm build +``` + +不要把“本地测试通过”“代码存在”“GitHub 已推送”“桌面端已发布”混为同一种完成状态。 + +## 11. 交给其他 AI 的检查清单 + +请先保持本文的产品边界,不要把 SaaS Template、内置通用 Agent 或桌面壳提前混入本阶段。然后重点检查: + +1. `docs/prd/BEATDESIGN_V0_2.md` 的每条 Phase 1 验收标准,是否有对应实现和测试证据。 +2. Canvas 与 Editor 的写操作是否真正复用 `src/core/commands/`,有没有 UI 独有且 MCP 未来无法复用的第二套业务逻辑。 +3. Timeline Clip 是否始终引用具体 Asset;Canvas 切换 Generation pinned output 时,是否可能静默改变已经剪好的片段。 +4. 多素材加入 Timeline 的顺序、重复调用、版本冲突重试和部分失败,是否会产生重复 Clip 或丢失更新。 +5. 尾帧提取面对无效历史 URL、跨域媒体、零时长/极短视频和不支持解码的编码格式时,错误是否可理解且不留下半成品节点。 +6. Canvas Snapshot 与 Timeline revision/CAS 是否覆盖 UI 与未来 MCP 并发写入;MCP 是否能避免整份覆盖文档。 +7. 新增依赖、OpenReel 派生合同和 Mediabunny 使用是否满足许可证归档与分发要求。 +8. 使用一段本地视频完整走通:Canvas 选中视频 -> 尾帧续写节点 -> 多选片段 -> 创建 Timeline -> trim/split -> 导出 MP4 -> Render 回到 Assets/Canvas。 + +审查输出请区分:阻断问题、Phase 1 应修问题、v0.3 建议和纯偏好;不要把 v0.3 功能缺失当成 v0.2 回归。 diff --git a/docs/assets/beatdesign-readme-cover-v3.jpg b/docs/assets/beatdesign-readme-cover-v3.jpg new file mode 100644 index 0000000..d7a2558 Binary files /dev/null and b/docs/assets/beatdesign-readme-cover-v3.jpg differ diff --git a/docs/prd/BEATDESIGN_V0_2.md b/docs/prd/BEATDESIGN_V0_2.md new file mode 100644 index 0000000..c3ac7d0 --- /dev/null +++ b/docs/prd/BEATDESIGN_V0_2.md @@ -0,0 +1,261 @@ +# BeatDesign v0.2 主 PRD:Canvas-to-Editor Creative Loop & Agent Control + +- 状态:Phase 1 与 MCP v0.2 adapter implemented locally +- 目标版本:v0.2 +- 产品范围:BeatDesign 开源本地工作台 +- 更新时间:2026-08-30 +- 关联文档:[产品规划与完成状态](../PRODUCT_PLAN_AND_STATUS.md) + +## 1. 产品结论 + +BeatDesign v0.2 的目标不是在画布里增加一个内置聊天 Agent,而是把 BeatDesign 建成通用 Agent 可操控的本地视觉创作运行环境。 + +产品有三个用户可见工作空间: + +1. Studio:最快完成一次生成或分析的引导式入口。 +2. Canvas:组织素材、生成节点、版本分支和生成血缘。 +3. Editor:组织时间、精确剪辑、音频、字幕、预览和导出。 + +三者共享同一个 Project、Assets、Generation、Media 和 History。Generation 是共享能力,不是第四个工作空间。 + +```text +通用 Agent / 用户界面 + │ + ▼ +BeatDesign Command Kernel + │ + ├── Project / Assets + ├── Canvas + ├── Generation + └── Editor + │ + ▼ + SQLite + 本地媒体 + BeatAPI +``` + +## 2. v0.2 主链路 + +本版本必须让用户先手动、后由 Agent 完成同一条创作链路: + +```text +在 Canvas 生成视频 A + -> 从 A 提取尾帧 + -> 以尾帧为下一段首帧创建视频生成节点 B + -> 继续生成 B / C + -> 选择 A、B、C 创建或追加 Timeline + -> 在 Editor trim / split / move / delete + -> 选择局部范围执行 BeatAPI AI redo + -> 在 Original 与多个 Take 之间切换 + -> 诊断时间线 + -> 导出 MP4 + -> Render 回到 Assets 与 Canvas Timeline Node +``` + +## 3. 统一产品模型 + +### 3.1 Project + +唯一项目边界。Studio、Canvas、Editor 和 Assets 都是同一 Project 的视图,不创建第二套后端或媒体副本。 + +### 3.2 Asset + +本地媒体事实来源。每次上传、生成、帧提取、片段截取和时间线导出都先形成 Asset。 + +Asset 可以被 Canvas Node、Timeline Clip、Generation Reference 同时引用。删除视图中的引用不等于删除 Asset。 + +### 3.3 Canvas Node + +Canvas 对 Asset、Generation 配置或 Timeline 的可视化引用。节点位置和连线表达用户当前的组织意图,不自动等价于生成血缘。 + +### 3.4 Generation + +产生 Asset 的异步任务。核心请求不包含单一 `placement` 字段;生成完成后由 Canvas、Editor 或 Studio 再引用结果 Asset。 + +### 3.5 Timeline / Clip / Take + +- Timeline:项目级剪辑文档。 +- Clip:一个 Asset 在时间线上的一次使用,保存 start、in/out、duration 和音频参数。 +- Take:同一 Clip 时间槽的备选媒体版本。激活 Take 不改变原 Clip 的位置和原始 Asset,可随时恢复 Original。 + +### 3.6 两类关系 + +1. Canvas Edge:用户排列、连接和关注关系。 +2. Generation Lineage:真实的 `derived_from`、`first_frame`、`last_frame`、`continuation_of`、`redo_of`、`timeline_render` 等来源关系。 + +本轮先在派生 Asset metadata 与 Canvas 引用中记录尾帧和时间线关系;后续升级为独立可查询的 lineage contract。 + +## 4. Canvas 与 Editor 的结合 + +### 4.1 Canvas -> Editor + +- 单个视频或音频:加入当前时间线。 +- 多个视频:按照显式选择顺序创建/追加时间线。 +- 创建 Timeline Node,展示时长、Clip 数量和最近 Render。 +- Timeline Node 打开 `/editor/:projectId`。 + +### 4.2 Editor -> Canvas + +- 将 Timeline Node 写回 Canvas。 +- 导出 Render 更新 Timeline Node。 +- 截取片段保存为派生 Asset。 +- AI redo 结果保存为 Asset 并挂为 Clip Take。 +- 后续支持从 Editor 展开某次生成在 Canvas 中的完整血缘。 + +### 4.3 稳定引用规则 + +Editor Clip 必须引用具体 `assetId`,不能动态跟随 Canvas Generation Node 的当前 pinned output。Canvas 切换版本时,Editor 只有在用户明确确认后才能替换 Clip。 + +## 5. v0.2 功能范围 + +### P0:本轮 + +- 统一产品术语与 asset-first Generation contract。 +- Command Kernel 基础合同。 +- Editor 现有核心操作接入统一 command executor。 +- 视频尾帧本地提取并保存为派生 Asset。 +- 尾帧节点与下一段视频 Generation Node 自动创建。 +- 多选 Canvas 视频创建/追加时间线。 +- Timeline Node 在 Canvas 中可见并可打开 Editor。 +- revision/CAS 保持现有保护。 +- Command 请求运行时 Schema、Project Asset 校验、稳定 ID 和短期幂等回执。 +- Editor 并发保存三方合并;MCP/CLI 只能执行增量 `editor.apply`,不能整份替换时间线。 +- Generation 服务端从 Asset ID 与 generation intent 编译权威媒体输入。 +- 图片 Clip 与可拖动的图片持续时间。 +- MCP 本地素材导入桥与 2 秒 UI revision 同步。 +- 单元测试、类型检查、i18n 和生产构建通过。 + +### P1:产品成片能力 + +- 字幕轨和 SRT 导入。 +- 时间线吸附、缩放、波形和多选。 +- 变速。 +- 交叉溶解、淡入淡出和黑场过渡。 +- 合成帧 Preview 与更完整 Diagnostics。 +- AI Bridge:使用前后帧生成衔接片段。 + +### P2:Agent Control + +- 本地 MCP Server。`已完成基础版` +- Project、Canvas、Editor、Assets、Generation 工具组。`已完成基础版` +- 模型 capability discovery。`已完成` +- 只读 MCP Resources。`未完成` +- 付费生成确认、幂等键、结构化结果和操作历史。 +- Agent 修改后 UI 实时刷新。 +- Codex 优先接入,Claude Code 和其他宿主使用同一协议。 + +## 6. Command Kernel + +Command Kernel 是 UI、MCP 和未来 CLI 的唯一写入入口。MCP 不直接修改 SQLite,也不整份覆盖 Canvas Snapshot。 + +统一结果目标: + +```ts +type CommandResult = { + ok: boolean; + projectId: string; + commandId: string; + changedIds: string[]; + revision?: number; + warnings: string[]; + jobId?: string; + editorUrl?: string; +}; +``` + +第一阶段命令域: + +```text +project.* +asset.* +canvas.apply +generation.prepare / submit / status / cancel +editor.apply / validate / export +``` + +所有写命令必须支持稳定 ID、预期 revision、幂等和可审计 origin。API Key、余额、计费和限流由所选 Provider 后端负责;BeatDesign 只提交请求并呈现返回结果或错误。外部上传、不可恢复删除和覆盖导出仍保留明确确认边界。 + +## 7. MCP 工具目录 + +MCP 是 Command Kernel 的薄适配层,不重新实现业务逻辑。 + +```text +Project: list / create / get / open +Canvas: get / apply / search / focus +Editor: get / apply / preview / validate / focus / history / export +Assets: list / import / inspect +Generation: models / prepare / submit / status / cancel +``` + +工具返回 Asset ID、本地 URL、节点 ID、Clip ID、revision 和 job status,不通过 MCP 传输大型视频二进制。 + +## 8. 交互要求 + +### 8.1 视频节点“继续生成” + +1. 用户选择一个有本地 URL 的视频 Asset 或 Generation Output。 +2. 点击“尾帧续写”。 +3. BeatDesign 在浏览器本地解码最后一个可用画面。 +4. PNG 保存到项目 Assets,metadata 标记 `video_tail_frame`。 +5. Canvas 在原视频右侧创建尾帧 Asset Node。 +6. Canvas 在尾帧右侧创建视频 Generation Node,并把尾帧作为首帧参考。 +7. Composer 自动聚焦到新节点,用户填写下一段提示词后确认生成。 + +### 8.2 多片段创建时间线 + +1. 用户按顺序选择多个可用视频结果。 +2. 点击“创建时间线”。 +3. BeatDesign 一次读取并持久化时间线,避免逐个写入产生冲突。 +4. Clip 按选择顺序首尾排列。 +5. Canvas 创建或更新 Timeline Node,并连接所选视频。 +6. 用户点击 Timeline Node 打开 Editor。 + +## 9. 安全与失败边界 + +- 文件选择和帧提取保持本地,不因选择或预览自动上传给 BeatAPI。 +- 尾帧只在用户明确触发后写入项目本地 Assets。 +- 生成仍走现有 precheck 和一次性 intent;Provider 后端是 API Key、余额、计费与限流状态的唯一事实来源。 +- Timeline 与 Canvas 保存继续使用 revision/CAS。 +- 多操作命令必须原子应用到内存文档,再执行一次持久化。 +- 页面未打开时,浏览器预览与导出不可假装完成;未来由本地媒体 Worker 承接 headless 执行。 + +## 10. 验收标准 + +### 人工闭环 + +- 从本地视频或生成结果成功提取尾帧。 +- 尾帧成为可复用 Asset,并在刷新后仍存在。 +- 新视频 Generation Node 正确引用尾帧。 +- 两个以上视频可以一次创建连续 Timeline。 +- Timeline Node 可打开 Editor。 +- Clip 顺序、时长和 Asset 引用正确。 +- 原有 trim、split、move、delete、AI redo、Take、导出无回归。 + +### Agent-ready 内核 + +- Editor operations 可由纯 command executor 调用。 +- Canvas operations 有稳定 contract 和纯文档 executor。 +- Generation request 是 asset-first 且不绑定单一 placement。 +- 命令结果结构可被后续 MCP 直接包装。 +- 未知命令、非本项目 Asset 和 MCP 整份 Timeline 替换会被拒绝。 +- 相同幂等键的重试返回同一持久化结果,不重复创建 Clip。 +- 幂等键复用于不同 command 时必须失败,不能返回不相关的旧结果。 + +### 工程门禁 + +```text +pnpm typecheck +pnpm test +pnpm i18n:check +pnpm build +``` + +当前基础版不暴露整份 Snapshot 替换;`editor.snapshot` 是语义时间点快照,像素预览、浏览器导出、本地文件导入和 UI 实时事件桥仍未完成。 + +## 11. 本版本非目标 + +- 不内置通用聊天 Agent。 +- 不实现多个宿主的完整一键安装器与 headless 媒体 Worker。 +- 不引入完整 OpenReel 应用外壳。 +- 不要求系统安装 FFmpeg。 +- 不做桌面签名、公证和自动更新。 +- 不做多人协作、云项目同步、支付和账户系统。 diff --git a/docs/prd/VIDEO_TIMELINE_PHASE_1.md b/docs/prd/VIDEO_TIMELINE_PHASE_1.md new file mode 100644 index 0000000..a77efde --- /dev/null +++ b/docs/prd/VIDEO_TIMELINE_PHASE_1.md @@ -0,0 +1,826 @@ +# BeatDesign 视频时间线与音频画布第一阶段 PRD + +- 状态:Draft for implementation +- 目标版本:Phase 1 / localhost open-source release +- 产品范围:BeatDesign 开源仓库,不包含 SaaS Template +- 主要参考:Updream 的画布—编辑器闭环、MiniMax Design 的 OpenReel 编辑器集成 +- 更新时间:2026-08-29 + +## 1. 产品结论 + +下一阶段将 BeatDesign 从“本地 AI 图片/视频生成画布”升级为“本地优先的 AI 视频创作工作台”。 + +产品由两个互补工作空间组成: + +1. Canvas:负责素材、提示词、生成任务、版本和来源关系。 +2. Editor:负责时间线、精确裁剪、音频、字幕、预览和 MP4 导出。 + +第一阶段只完成 GitHub 开源版和 localhost 工作流。Electron 桌面壳、通用 Plugin SDK、MCP/CLI 和内置 Agent 均不在本阶段范围内。 + +第一阶段必须跑通的唯一主链路: + +```text +BeatAPI 生成 10 秒视频 + -> 视频本地化为 Asset + -> 添加到时间线 + -> 截取其中 3-4 秒 + -> 导出 MP4 + -> 将选区交给 BeatAPI 重做 + -> 新结果作为 Take 返回原 Clip + -> 替换并再次导出 + -> 导出结果回到 Canvas +``` + +## 2. 为什么现在做 + +当前 BeatDesign 已具备: + +- Project、Studio、Canvas、Assets 和 Generation History。 +- 图片与视频生成、视频分析和 BeatAPI 连接。 +- 本地 SQLite、项目快照和本地项目素材目录。 +- 生成结果自动本地化和素材来源记录。 +- 底层 `AssetType` 已包含 `audio`。 + +当前关键缺口: + +- Canvas 只能正式表达图片、视频和分析结果,不能表达音频素材。 +- 没有 TimelineDocument、Track、Clip、Take 和 Render Job。 +- 不能从生成视频中精确选择一个范围。 +- 不能组合多个生成片段并导出一个成片。 +- 不能将时间线选区重新送入 BeatAPI 并无损替换。 + +## 3. 产品目标与非目标 + +### 3.1 目标 + +1. 用户克隆仓库后,只需安装项目依赖并启动 localhost,不需要单独安装 FFmpeg。 +2. 将 OpenReel 编辑能力作为第一方模块集成到 BeatDesign。 +3. 支持视频、图片和音频素材进入同一时间线。 +4. 支持非破坏式 trim、split、move、delete 和 undo/redo。 +5. 支持 1080p MP4/H.264/AAC 导出。 +6. 支持导出到本地下载,也支持保存为 BeatDesign 项目素材并创建 Canvas 节点。 +7. 支持选区生成派生素材并调用 BeatAPI 重做。 +8. 支持一个 Clip 保留多个 Take,并允许预览、切换和替换。 +9. 增加 Canvas 音频素材节点,并打通音频到时间线的路径。 +10. 保持当前 BeatDesign 视觉系统和本地优先边界。 + +### 3.2 非目标 + +- 不做 Electron、macOS 签名、公证和自动更新。 +- 不分发原生 FFmpeg 二进制。 +- 不做通用 Plugin SDK。 +- 不做 MCP、CLI 或画布内聊天 Agent。 +- 不做完整剪映、Premiere 或 DaVinci 功能对齐。 +- 不承诺 4K 长视频、ProRes、多机位和电影级调色。 +- 不做多人协作、云项目同步和素材市场。 +- 不做完整 Skill 商店;可在后续独立维护示例 Skill 集合。 + +## 4. 第一阶段用户范围 + +主要用户: + +- 使用 BeatAPI 生成图片和短视频的独立创作者。 +- 希望在本机管理素材、选择片段并快速重做的 AI 视频开发者。 +- 希望通过 GitHub 直接运行一个完整 AI 视频工作台的开源用户。 + +建议第一阶段媒体边界: + +- 项目时长:5-120 秒。 +- 默认输出:1920x1080、30 FPS、MP4、H.264/AAC。 +- 首发浏览器:Chrome 稳定版;Edge 作为兼容目标。 +- 素材:MP4/MOV 视频、PNG/JPEG/WebP 图片、MP3/WAV/M4A/AAC 音频。 +- 一个主视频轨、最多两个附加视频/图片轨、最多三个音频轨、一个字幕轨。 + +## 5. 信息架构 + +现有 AppShell 模式从: + +```text +Studio | Canvas | Assets +``` + +扩展为: + +```text +Studio | Canvas | Edit | Assets +``` + +`Edit` 是项目级视图,不创建第二套 Project。进入方式包括: + +- 顶部 WorkspaceSwitcher 的 Edit。 +- 视频节点上下文工具栏中的“添加到时间线”。 +- Timeline Node 中的“打开编辑器”。 +- Assets 中素材菜单的“添加到时间线”。 + +项目层级: + +```text +Project +├── CanvasSnapshot +├── Assets +├── Generations +└── Timelines + ├── Timeline A + └── Timeline B +``` + +## 6. Canvas 新增节点 + +### 6.1 Audio Asset Node + +第一阶段必须新增音频素材节点。它代表可连接、可播放、可送入时间线或模型的音频 Asset,不承担精确音频编辑。 + +字段: + +```text +id +assetId +type: audio +audioRole: music | voice | sound_effect | source_audio | reference +name +url +durationSec +waveformPeaks +sourceGenerationId? +sourceAssetId? +``` + +节点能力: + +- 播放、暂停、跳转。 +- 显示波形、时长和音频用途。 +- 添加到当前时间线。 +- 作为 BeatAPI reference/driving audio。 +- 从视频节点“提取音频”后生成。 +- 展示本地化、缺失和失败状态。 + +节点视觉: + +- 推荐尺寸 320x104。 +- 使用 `--beat-surface`、`--beatcanvas-line` 和 8px 圆角。 +- 波形使用冷蓝 `--beat-graph`,不使用新的紫色或绿色品牌色。 +- Play 按钮使用中性高对比色;只有生成、重做、导出等主动作使用橙色。 +- 时间码使用 Geist Mono 和 tabular numerals。 +- 被选中时沿用 Canvas 冷蓝边框和外发光。 + +### 6.2 Timeline Node + +Timeline Node 是 Canvas 对一个 TimelineDocument 的入口和结果摘要,不把完整时间线塞进无限画布。 + +字段: + +```text +id +timelineId +name +durationSec +aspectRatio +lastRenderAssetId? +coverAssetId? +clipCount +updatedAt +``` + +节点能力: + +- 打开编辑器。 +- 接收图片、视频、音频 Asset 连接。 +- 显示最新成片封面、项目时长和 Clip 数量。 +- 导出成功后更新最新 Render。 +- 将 Render 拖出/连接到下游生成节点。 + +节点视觉: + +- 推荐尺寸 360x220。 +- 上部为 16:9/9:16 预览区域,下部为简化时间线条带。 +- 节点标签使用 `Timeline` 或本地化名称,不命名为 Plugin。 +- “打开编辑器”是中性导航动作;“导出”与“AI 重做”才使用橙色主动作。 + +### 6.3 Derived Clip Asset + +截取选区后创建新的 Video Asset Node,而不是创建新的特殊节点类型。 + +通过 metadata 表达: + +```text +assetClass: derived +parentAssetId +sourceInSec +sourceOutSec +operation: timeline_extract +timelineId +clipId +``` + +节点显示“截取 3.2-6.8s”等来源徽标。 + +### 6.4 本阶段不新增的节点 + +- 不新增独立 Music、Voice、Sound Effect 节点类型;统一为 Audio Asset + audioRole。 +- 不新增 Subtitle Canvas Node;字幕第一阶段属于 TimelineDocument。 +- 不新增 Agent Node、MCP Node 和 Skill Node。 +- 不新增 FFmpeg Node;媒体引擎是内部实现细节。 + +## 7. Editor 界面设计 + +### 7.1 总体布局 + +Editor 使用当前 AppShell 的 56px 顶栏和全屏生产工作区,不增加永久全局侧边栏。 + +```text +┌──────────────────────────────────────────────────────────────┐ +│ Back to Canvas | Timeline name | Saved | Ratio | Export │ +├──────────────┬───────────────────────────────┬───────────────┤ +│ │ │ │ +│ Project │ Preview │ Inspector │ +│ Assets │ │ │ +│ │ │ │ +├──────────────┴───────────────────────────────┴───────────────┤ +│ Toolbar: Undo Redo Split Delete Snap Zoom │ +├──────────────────────────────────────────────────────────────┤ +│ Video 2 ───────────────────────────────────────────── │ +│ Video 1 ───── Clip A ───── Clip B ─────────────────── │ +│ Audio 1 ───────── Waveform ────────────────────────── │ +│ Captions ───── Cue ───── Cue ───── Cue ─────────────── │ +└──────────────────────────────────────────────────────────────┘ +``` + +布局建议: + +- 左侧素材区:260-300px,可折叠。 +- 右侧 Inspector:280-320px,无选择时折叠或显示项目设置。 +- Timeline:默认 260px 高,可上下拖动。 +- Preview:占据剩余空间。 +- 小于 960px 时,素材区和 Inspector 改为抽屉;时间线仍保持可操作。 + +### 7.2 与 BeatDesign UI 匹配 + +必须直接复用现有 Token: + +- 背景:`--beat-bg`。 +- 主表面:`--beat-surface`。 +- 抬升表面:`--beat-surface-2`。 +- 边线:`--beat-border` / `--beatcanvas-line`。 +- 文本:`--beat-text-1/2/3`。 +- 品牌动作:`--beat-accent`。 +- 选择、吸附、轨道焦点:`--beat-graph`。 +- 成功、警告、错误:现有 Canvas semantic tokens。 + +字体: + +- UI:Figtree / CJK fallback。 +- 时间码、FPS、时长、导出数据:Geist Mono。 + +形状: + +- 控件 4-8px 圆角。 +- Panel 12px 圆角。 +- Preview/媒体 16px 圆角。 +- 不把 OpenReel 默认皮肤直接搬入 BeatDesign。 +- 不引入第二套紫色、渐变主色和高饱和多色按钮。 + +交互: + +- 80ms/160ms 微交互,不给拖动、播放头或 trim handle 增加延迟感。 +- 所有 Timeline 操作支持键盘和可见焦点。 +- 状态不能只靠颜色表达。 +- 所有触控目标尽可能达到 44px。 + +### 7.3 第一阶段显示的工具 + +左侧: + +- Project Assets。 +- Videos / Images / Audio 筛选。 +- 搜索。 +- 添加到时间线。 + +顶部: + +- 返回 Canvas。 +- Timeline 名称。 +- 自动保存状态。 +- 画幅。 +- Export。 + +Preview: + +- 播放、暂停。 +- 前后跳转。 +- 当前时间/总时长。 +- 静音。 +- 预览缩放。 +- 适应画布。 + +Timeline: + +- Select、Split、Delete。 +- Trim handles。 +- Drag move。 +- Snap。 +- Undo/Redo。 +- Timeline zoom。 +- Track lock/mute/solo。 +- Add track。 + +Inspector: + +- Transform:position、scale、rotation、fit。 +- Audio:volume、mute、fade in/out。 +- Speed:第一阶段只支持 0.5x、1x、2x;如实现风险高可移至 Phase 1.1。 +- Clip metadata:素材名称、source in/out、timeline start/end。 + +### 7.4 默认隐藏的 OpenReel 功能 + +- Graphics、SVG、Stickers、Backgrounds。 +- 高级 Effects 和模板。 +- LUT、Scopes、复杂 Color Grading。 +- Keyframes、Speed Ramp、Optical Flow、Stabilization。 +- Recording、Webcam、Screen Capture。 +- Multicam、3D、Motion Tracking。 +- AI Agent、Chat、MCP、任意第三方模型 Provider。 +- Templates Marketplace、Stock Media、Music Marketplace。 + +代码可以保留在 Fork 中,但不进入首发导航和默认 Bundle;无法按需拆包的模块应在 Fork 中删除或延迟加载。 + +## 8. OpenReel 引入方案 + +### 8.1 引入方式 + +OpenReel 以固定 commit 的 Fork 形式进入 BeatDesign,不使用 Git Submodule。 + +推荐先作为同仓第一方 micro-frontend: + +```text +packages/openreel-editor + -> 独立 Vite build + -> 同源挂载在 /editor-runtime/ + -> BeatDesign /editor/:timelineId 路由承载 +``` + +理由: + +- OpenReel 当前是完整应用而不是稳定的 UI npm 包。 +- 隔离 OpenReel 的 React、Zustand、Worker、WASM 和样式依赖。 +- 避免立即将 OpenReel 大量状态耦合到 Canvas 组件。 +- 后续同步上游更新更清晰。 +- 可以通过同源 API 和 typed bridge 与 BeatDesign 通信。 + +第一阶段 bridge 不是通用 Plugin SDK,只定义本编辑器需要的固定协议: + +```text +host.ready +host.getTimeline +host.saveTimeline +host.listAssets +host.resolveAsset +host.saveDerivedAsset +host.submitGeneration +host.getGenerationStatus +editor.selectionChanged +editor.exportProgress +editor.projectChanged +``` + +### 8.2 保留 + +- `packages/core` 中 timeline、actions、undo/redo、media、playback。 +- WebCodecs Export Backend。 +- MediaBunny 容器读写。 +- WebGPU/Canvas2D Preview。 +- Web Audio、waveform 和基础音频能力。 +- Trim、split、ripple delete、move、snap。 +- Text/caption 基础模型。 +- 项目 JSON schema 和版本迁移能力。 +- 缺失素材检查所需能力。 + +### 8.3 删除 + +- OpenReel 内置 Agent、Chat 和模型配置界面。 +- KieAI 或其他与 BeatAPI 重复的生成 Provider。 +- OpenReel 自带账户、云同步、订阅或远程项目功能。 +- Desktop、mobile 和原生安装器代码,不进入 BeatDesign Phase 1 构建。 +- OpenReel Welcome、Recent Projects 和独立项目首页。 +- 重复的导航、主题切换和品牌外壳。 +- 与 BeatDesign Assets/Generations 重复的云素材来源。 + +### 8.4 替换 + +- OpenReel IndexedDB 项目保存 -> BeatDesign Timeline service + SQLite/D1。 +- OpenReel 最近项目 -> BeatDesign Project/Timeline 列表。 +- OpenReel 媒体导入 -> BeatDesign Asset Adapter。 +- OpenReel 下载导出 -> BeatDesign Render service;仍保留“下载文件”选项。 +- OpenReel 生成能力 -> BeatAPI Generation service。 +- OpenReel UI tokens -> BeatDesign `--beat-*` 和 `--beatcanvas-*`。 +- OpenReel toast/dialog -> BeatDesign 现有组件和 i18n。 + +### 8.5 暂时保留但关闭入口 + +- 字幕样式、基础转场、基础特效、图形、关键帧。 +- 后续可以逐项打开,不在第一阶段重写。 + +## 9. FFmpeg 与浏览器媒体方案 + +### 9.1 第一阶段 + +用户不安装系统 FFmpeg。 + +默认媒体链路: + +```text +Preview -> WebCodecs + WebGPU/Canvas2D +Audio -> Web Audio +MP4 mux -> MediaBunny +Export -> OpenReel WebCodecsBackend +Fallback -> ffmpeg.wasm(可选、短素材) +``` + +第一阶段不把原生 macOS/Windows/Linux FFmpeg 二进制提交到 Git 仓库。 + +### 9.2 ffmpeg.wasm 策略 + +- 不作为 P0 导出主引擎。 +- 如必须启用,以依赖方式安装并从 localhost 自托管 Core,不从 CDN 动态获取。 +- 只用于不兼容素材转码、短选区截取或浏览器能力 Fallback。 +- 默认先采用单线程版本,避免第一阶段强制 COOP/COEP;多线程与长视频性能留到后续。 +- 在 `THIRD_PARTY_NOTICES.md` 中单独记录 wrapper、core 和 FFmpeg license。 +- 在许可证方案正式确认前,可通过 feature flag 关闭 wasm fallback,不阻塞 WebCodecs 主链路。 + +### 9.3 桌面阶段 + +Electron 阶段再把合规的原生 FFmpeg/ffprobe 作为 Sidecar 放入安装包。届时替换 Export Backend,不重写 Timeline、Preview、项目格式和 Canvas 闭环。 + +## 10. 核心数据模型 + +### 10.1 TimelineDocument + +```ts +type TimelineDocument = { + schemaVersion: 1; + id: string; + projectId: string; + name: string; + width: number; + height: number; + frameRate: number; + sampleRate: number; + channels: number; + tracks: TimelineTrack[]; + markers: TimelineMarker[]; + durationSec: number; + createdAt: string; + updatedAt: string; +}; +``` + +### 10.2 Track + +```ts +type TimelineTrack = { + id: string; + kind: 'video' | 'audio' | 'subtitle' | 'text' | 'graphics'; + name: string; + order: number; + locked: boolean; + hidden: boolean; + muted: boolean; + solo: boolean; + clips: TimelineClip[]; +}; +``` + +### 10.3 Clip + +```ts +type TimelineClip = { + id: string; + assetId: string; + trackId: string; + timelineStartSec: number; + sourceInSec: number; + sourceOutSec: number; + durationSec: number; + speed: number; + volume: number; + fadeInSec: number; + fadeOutSec: number; + activeTakeId: string | null; + takeIds: string[]; +}; +``` + +### 10.4 Take + +```ts +type ClipTake = { + id: string; + clipId: string; + assetId: string; + generationId: string | null; + source: 'original' | 'derived' | 'beatapi'; + prompt: string | null; + modelId: string | null; + status: 'ready' | 'pending' | 'failed'; + createdAt: string; +}; +``` + +### 10.5 RenderJob + +```ts +type RenderJob = { + id: string; + projectId: string; + timelineId: string; + status: 'queued' | 'rendering' | 'saving' | 'succeeded' | 'failed' | 'cancelled'; + progress: number; + format: 'mp4'; + width: number; + height: number; + frameRate: number; + resultAssetId: string | null; + error: string | null; +}; +``` + +### 10.6 数据库范围 + +建议新增: + +- `timeline_document` +- `timeline_take` +- `render_job` + +TimelineDocument 第一阶段可以整体 JSON 保存并使用 revision/CAS,避免过早把每个 Clip 拆成关系表。Take 与 RenderJob 单独建表以便关联 Generation 和 Asset。 + +Canvas snapshot 升级时必须兼容旧项目;旧项目打开后不得丢失图片或视频节点。 + +## 11. 功能需求 + +### FR-1 创建与打开时间线 + +- 一个 Project 可以有多个 Timeline。 +- 第一次添加素材时可以自动创建“Timeline 1”。 +- 用户可以从 Canvas、Assets 或 Edit 进入。 +- 重启 localhost 后时间线可恢复。 + +### FR-2 素材进入时间线 + +- 图片、视频、音频均可添加。 +- 同一 Asset 重复添加时复用 Asset,不重复复制文件。 +- 默认添加到兼容轨道;没有轨道时自动创建。 +- OpenReel 内部只保存 `assetId` 和相对引用,不保存临时 Blob URL 作为真相。 + +### FR-3 非破坏式剪辑 + +- 支持 trim、split、move、delete、ripple delete。 +- 原始 Asset 不改变。 +- Clip 的 sourceIn/sourceOut 不得越界。 +- 同轨 Clip 默认不允许重叠。 +- 所有 P0 剪辑操作必须进入 Undo/Redo。 + +### FR-4 音频 + +- Canvas 可创建和恢复 Audio Asset Node。 +- Timeline 可添加音频轨和音频 Clip。 +- Preview 可同步播放音视频。 +- 支持 volume、mute、fade in/out。 +- 第一阶段不要求自动字幕、Beat Sync 和降噪。 + +### FR-5 选区派生素材 + +- 用户可选择一个 Clip 或时间范围。 +- “截取为新素材”生成新的本地 Video Asset。 +- 派生 Asset 记录 parent、sourceIn/out、timeline、clip 和 operation。 +- 新 Asset 可出现在 Assets 和 Canvas。 + +### FR-6 BeatAPI 重做 + +- 用户选择 Clip/选区并点击“AI 重做”。 +- 弹出与现有 BeatDesign Composer 一致的轻量生成面板。 +- 显示模型、提示词、参考输入和付费确认。 +- 使用派生视频或首尾帧作为 BeatAPI 输入。 +- 返回结果必须先本地化,再进入 Take。 +- 失败不改变当前 Clip。 +- 成功后用户可预览新 Take 并选择替换。 + +第一阶段只保证一种已验证的 BeatAPI 视频编辑/视频转视频路径。模型能力由现有 effect registry 和 adapter 决定,不在 Editor 内建立第二套模型目录。 + +### FR-7 MP4 导出 + +- 默认 1080p/30fps/H.264/AAC。 +- 支持下载文件和保存到项目。 +- 保存到项目后创建 Render Asset,并允许创建 Canvas Output Node。 +- 显示进度、取消、成功和失败状态。 +- 导出期间不冻结整个界面。 +- 导出结果不得包含未解释黑帧、重叠 Clip 或丢失素材。 + +### FR-8 自动保存与恢复 + +- 时间线变更采用 dirty state + debounce 自动保存。 +- 页面隐藏、刷新和离开前执行 flush。 +- 使用 revision/CAS 防止旧状态覆盖新状态。 +- 保存失败要明确提示,不得伪装为 Saved。 +- 空文档不得覆盖已有非空文档。 + +## 12. 关键状态和错误处理 + +必须处理: + +- Asset 文件丢失。 +- 不支持的 Codec。 +- 浏览器不支持 WebCodecs/WebGPU。 +- 导出内存不足。 +- 导出取消。 +- BeatAPI 任务失败或超时。 +- 派生素材保存失败。 +- 页面刷新时仍在保存。 +- 时间线文档版本冲突。 +- Audio duration/metadata 读取失败。 +- Clip 超出源时长。 +- 同轨重叠和可见视频空隙。 + +Editor 必须提供只读 Diagnostics: + +- `timeline_gap` +- `clip_overlap` +- `media_missing` +- `caption_overlap` +- `caption_out_of_video` +- `tiny_clip` +- `empty_track` + +P0 导出前自动运行 Diagnostics;Error 级问题阻止导出,Warning 级问题允许用户确认后继续。 + +## 13. 测试与验收计划 + +### 13.1 单元测试 + +- Clip trim 数学和边界。 +- Split 后 sourceIn/sourceOut 连续。 +- Ripple delete 后轨道位置。 +- Move 和 snap 计算。 +- Take 创建、切换、替换和回滚。 +- Audio 节点序列化、反序列化和旧快照兼容。 +- Timeline schema normalize/migrate。 +- RenderJob 状态机。 +- revision/CAS 冲突。 +- 空时间线保护。 + +### 13.2 集成测试 + +- BeatDesign Asset -> OpenReel MediaItem。 +- OpenReel Project JSON -> Timeline service -> reload。 +- Export Blob -> localhost -> project-assets -> Asset index。 +- 选区 -> Derived Asset -> generation intent。 +- 模拟 BeatAPI 成功、失败、超时和取消。 +- Take 替换后 Timeline 保存和恢复。 +- 音频 Asset 添加到音频轨并恢复。 + +### 13.3 E2E 主路径 + +使用一个固定 10 秒测试视频: + +1. 打开 Canvas。 +2. 将视频添加到 Timeline。 +3. Trim 为 3.2-6.8 秒。 +4. 保存并刷新页面。 +5. 验证 Clip 仍为 3.6 秒。 +6. 导出 MP4。 +7. 本地 probe 输出:分辨率、时长、FPS、音频流。 +8. 选择该 Clip 并创建 Derived Asset。 +9. 使用 mock BeatAPI 返回新视频。 +10. 新视频成为 Take。 +11. 切换并替换当前 Clip。 +12. 再次导出。 +13. 保存到项目并在 Canvas 创建 Output Node。 + +### 13.4 媒体测试矩阵 + +- 横屏 MP4、有音频。 +- 横屏 MP4、无音频。 +- 竖屏 MP4。 +- MOV 输入。 +- MP3、WAV、M4A。 +- 图片作为定长 Clip。 +- 两个连续视频 Clip。 +- 视频 + 背景音乐。 +- 30 秒、60 秒、120 秒时间线。 +- 缺失文件和损坏文件。 + +### 13.5 视觉与音频 QA + +- Preview 与导出画面比例一致。 +- Trim 后第一帧、最后一帧符合选区。 +- 无意外黑帧。 +- 转场未启用时是硬切。 +- 音画同步。 +- 音频无意外静音、爆音和截断。 +- Canvas Output Node 展示真实导出封面。 +- 音频节点波形、时长和播放状态一致。 + +### 13.6 回归门槛 + +每个合并批次必须通过: + +```text +pnpm typecheck +pnpm i18n:check +pnpm test +pnpm build +``` + +同时保留 Canvas 的: + +- Zoom、pan、connect、selection、undo/redo。 +- 项目快照与空状态保护。 +- 生成任务和本地化。 +- Assets/Studio/Canvas 模式切换。 +- 当前 Start Here 空状态视觉。 + +## 14. 实施里程碑 + +### M0:技术验证 + +- Fork 并固定 OpenReel commit。 +- 在 BeatDesign 仓库内完成独立构建。 +- 加载一个本地 10 秒 MP4。 +- Preview、trim、split、MP4 export 可用。 +- 不要求保存、Canvas 或 BeatAPI。 + +退出条件:用户不安装系统 FFmpeg即可完成 10 秒 -> 3.6 秒 -> MP4。 + +### M1:本地项目闭环 + +- Timeline service 和数据库。 +- Asset Adapter。 +- 自动保存、刷新恢复。 +- Canvas Timeline Node。 +- Audio Asset Node。 +- Export 保存到项目并返回 Canvas。 + +退出条件:重启 localhost 后项目、视频、音频、时间线和导出节点都可恢复。 + +### M2:BeatAPI 重做闭环 + +- 选区派生素材。 +- 现有 effect registry 接入。 +- 付费确认。 +- Generation -> Take。 +- Take 预览、替换和回滚。 + +退出条件:选区可重做,失败不破坏原 Clip,成功可替换并导出。 + +### M3:可靠性与发布准备 + +- Diagnostics。 +- Export progress/cancel。 +- 不支持格式提示与 fallback。 +- 许可证和 Third Party Notices。 +- 中英文文案。 +- README、架构文档、示例项目和演示素材。 + +退出条件:主路径、媒体矩阵、回归门槛全部通过。 + +## 15. 开源用户体验 + +目标启动方式: + +```bash +pnpm install +cp .env.example .env.development +pnpm db:push +pnpm dev +``` + +要求: + +- 不要求 `brew install ffmpeg`。 +- 不要求 Git Submodule。 +- 不从 CDN 动态加载 OpenReel、WASM、字体或编辑器核心。 +- 没有 BeatAPI key 时仍可导入、剪辑和导出本地视频。 +- 用户只在调用 BeatAPI 时配置 key。 +- README 明确支持的浏览器、格式、时长和已知限制。 +- 提供一个体积可控、许可证清楚的测试素材和示例时间线。 + +## 16. 开源与许可证 + +- BeatDesign 主项目继续使用 Apache-2.0。 +- OpenReel Fork 保留 MIT License、版权声明和上游 commit。 +- Mediabunny、FFmpeg wrapper/core、字体、图标和测试素材单独记录。 +- 新增 `THIRD_PARTY_NOTICES.md`。 +- 如启用 ffmpeg.wasm core,必须明确其实际 license,不将其声明为 Apache-2.0。 +- 第一阶段不复制 MiniMax Design 打包文件、其 FFmpeg 二进制或专有 Host 代码。 + +## 17. 发布判定 + +第一阶段被认为完成,当且仅当: + +1. 新用户从 GitHub 克隆后无需安装系统 FFmpeg即可剪辑和导出。 +2. 10 秒生成视频可在 Timeline 中精确截取 3-4 秒。 +3. 时间线、音频节点和本地素材可在刷新/重启后恢复。 +4. MP4 可下载,也可保存到项目并回到 Canvas。 +5. 选区可调用 BeatAPI 重做并作为 Take 返回。 +6. 原 Clip 和原 Asset 始终可以恢复。 +7. 主 E2E、媒体 QA 和项目回归全部通过。 +8. 不引入侧边 Agent、Electron、通用 Plugin SDK 或 MCP。 diff --git a/drizzle.config.ts b/drizzle.config.ts index a6bb1bd..6388880 100644 --- a/drizzle.config.ts +++ b/drizzle.config.ts @@ -14,6 +14,6 @@ export default defineConfig({ out: './drizzle/d1', dialect: 'sqlite', dbCredentials: { - url: process.env.DATABASE_URL || 'file:data/local.db', + url: process.env.DATABASE_URL || 'file:data/workspace.db', }, }); diff --git a/drizzle/d1/0002_lethal_micromacro.sql b/drizzle/d1/0002_lethal_micromacro.sql new file mode 100644 index 0000000..7c964d8 --- /dev/null +++ b/drizzle/d1/0002_lethal_micromacro.sql @@ -0,0 +1,9 @@ +CREATE TABLE `project_timeline_state` ( + `project_id` text PRIMARY KEY NOT NULL, + `document_json` text NOT NULL, + `version` integer DEFAULT 1 NOT NULL, + `updated_at` integer DEFAULT (cast((julianday('now') - 2440587.5)*86400000 as integer)) NOT NULL, + FOREIGN KEY (`project_id`) REFERENCES `project`(`id`) ON UPDATE no action ON DELETE cascade +); +--> statement-breakpoint +CREATE INDEX `project_timeline_state_updated_at_idx` ON `project_timeline_state` (`updated_at`); \ No newline at end of file diff --git a/drizzle/d1/0003_slow_the_phantom.sql b/drizzle/d1/0003_slow_the_phantom.sql new file mode 100644 index 0000000..8b5a653 --- /dev/null +++ b/drizzle/d1/0003_slow_the_phantom.sql @@ -0,0 +1,15 @@ +CREATE TABLE `project_command_receipt` ( + `id` text PRIMARY KEY NOT NULL, + `project_id` text NOT NULL, + `idempotency_key` text NOT NULL, + `command_id` text NOT NULL, + `origin` text NOT NULL, + `command_type` text NOT NULL, + `result_json` text NOT NULL, + `created_at` integer DEFAULT (cast((julianday('now') - 2440587.5)*86400000 as integer)) NOT NULL, + FOREIGN KEY (`project_id`) REFERENCES `project`(`id`) ON UPDATE no action ON DELETE cascade +); +--> statement-breakpoint +CREATE INDEX `project_command_receipt_project_idx` ON `project_command_receipt` (`project_id`);--> statement-breakpoint +CREATE INDEX `project_command_receipt_command_idx` ON `project_command_receipt` (`command_id`);--> statement-breakpoint +CREATE UNIQUE INDEX `project_command_receipt_idempotency_unique` ON `project_command_receipt` (`project_id`,`idempotency_key`); \ No newline at end of file diff --git a/drizzle/d1/meta/0002_snapshot.json b/drizzle/d1/meta/0002_snapshot.json new file mode 100644 index 0000000..5105701 --- /dev/null +++ b/drizzle/d1/meta/0002_snapshot.json @@ -0,0 +1,1352 @@ +{ + "version": "6", + "dialect": "sqlite", + "id": "330a5db6-48ea-498b-ac5d-790bada59f0f", + "prevId": "7cf09fc2-c7d9-4243-b0f8-fb0ff41a485c", + "tables": { + "config": { + "name": "config", + "columns": { + "name": { + "name": "name", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "value": { + "name": "value", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_asset_link": { + "name": "generation_asset_link", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "generation_id": { + "name": "generation_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "asset_id": { + "name": "asset_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "generation_asset_link_generation_idx": { + "name": "generation_asset_link_generation_idx", + "columns": [ + "generation_id" + ], + "isUnique": false + }, + "generation_asset_link_asset_idx": { + "name": "generation_asset_link_asset_idx", + "columns": [ + "asset_id" + ], + "isUnique": false + }, + "generation_asset_link_unique": { + "name": "generation_asset_link_unique", + "columns": [ + "generation_id", + "asset_id", + "role" + ], + "isUnique": true + } + }, + "foreignKeys": { + "generation_asset_link_generation_id_generation_history_id_fk": { + "name": "generation_asset_link_generation_id_generation_history_id_fk", + "tableFrom": "generation_asset_link", + "tableTo": "generation_history", + "columnsFrom": [ + "generation_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "generation_asset_link_asset_id_asset_id_fk": { + "name": "generation_asset_link_asset_id_asset_id_fk", + "tableFrom": "generation_asset_link", + "tableTo": "asset", + "columnsFrom": [ + "asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_history": { + "name": "generation_history", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "effect_id": { + "name": "effect_id", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "provider_task_id": { + "name": "provider_task_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "lifecycle_phase": { + "name": "lifecycle_phase", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "last_provider_sync_at": { + "name": "last_provider_sync_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "execution_mode": { + "name": "execution_mode", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'create_new'" + }, + "submitted_prompt": { + "name": "submitted_prompt", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "submitted_params": { + "name": "submitted_params", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "result_asset_id": { + "name": "result_asset_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "input": { + "name": "input", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "output": { + "name": "output", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "error": { + "name": "error", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "started_at": { + "name": "started_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "completed_at": { + "name": "completed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "failed_at": { + "name": "failed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "generation_history_project_idx": { + "name": "generation_history_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "generation_history_effect_idx": { + "name": "generation_history_effect_idx", + "columns": [ + "effect_id" + ], + "isUnique": false + }, + "generation_history_status_idx": { + "name": "generation_history_status_idx", + "columns": [ + "status" + ], + "isUnique": false + }, + "generation_history_provider_task_idx": { + "name": "generation_history_provider_task_idx", + "columns": [ + "provider_task_id" + ], + "isUnique": false + }, + "generation_history_lifecycle_idx": { + "name": "generation_history_lifecycle_idx", + "columns": [ + "lifecycle_phase" + ], + "isUnique": false + }, + "generation_history_result_asset_idx": { + "name": "generation_history_result_asset_idx", + "columns": [ + "result_asset_id" + ], + "isUnique": false + } + }, + "foreignKeys": { + "generation_history_project_id_project_id_fk": { + "name": "generation_history_project_id_project_id_fk", + "tableFrom": "generation_history", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "generation_history_result_asset_id_asset_id_fk": { + "name": "generation_history_result_asset_id_asset_id_fk", + "tableFrom": "generation_history", + "tableTo": "asset", + "columnsFrom": [ + "result_asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_intent_upload": { + "name": "generation_intent_upload", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "intent_id": { + "name": "intent_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'reserved'" + }, + "storage_provider": { + "name": "storage_provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "bucket": { + "name": "bucket", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "object_key": { + "name": "object_key", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "public_url": { + "name": "public_url", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "completed_at": { + "name": "completed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + } + }, + "indexes": { + "generation_intent_upload_intent_idx": { + "name": "generation_intent_upload_intent_idx", + "columns": [ + "intent_id" + ], + "isUnique": false + }, + "generation_intent_upload_status_idx": { + "name": "generation_intent_upload_status_idx", + "columns": [ + "status" + ], + "isUnique": false + } + }, + "foreignKeys": { + "generation_intent_upload_intent_id_generation_upload_intent_id_fk": { + "name": "generation_intent_upload_intent_id_generation_upload_intent_id_fk", + "tableFrom": "generation_intent_upload", + "tableTo": "generation_upload_intent", + "columnsFrom": [ + "intent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_upload_intent": { + "name": "generation_upload_intent", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "effect_id": { + "name": "effect_id", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'pending'" + }, + "expected_upload_count": { + "name": "expected_upload_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 0 + }, + "reserved_upload_count": { + "name": "reserved_upload_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 0 + }, + "completed_upload_count": { + "name": "completed_upload_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 0 + }, + "generation_id": { + "name": "generation_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "expires_at": { + "name": "expires_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "consumed_at": { + "name": "consumed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "generation_upload_intent_project_idx": { + "name": "generation_upload_intent_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "generation_upload_intent_status_expiry_idx": { + "name": "generation_upload_intent_status_expiry_idx", + "columns": [ + "status", + "expires_at" + ], + "isUnique": false + } + }, + "foreignKeys": { + "generation_upload_intent_project_id_project_id_fk": { + "name": "generation_upload_intent_project_id_project_id_fk", + "tableFrom": "generation_upload_intent", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "generation_upload_intent_generation_id_generation_history_id_fk": { + "name": "generation_upload_intent_generation_id_generation_history_id_fk", + "tableFrom": "generation_upload_intent", + "tableTo": "generation_history", + "columnsFrom": [ + "generation_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project": { + "name": "project", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "cover_asset_id": { + "name": "cover_asset_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'active'" + }, + "current_state_version": { + "name": "current_state_version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "last_workspace_mode": { + "name": "last_workspace_mode", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'canvas'" + }, + "last_opened_at": { + "name": "last_opened_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "archived_at": { + "name": "archived_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "deleted_at": { + "name": "deleted_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_status_idx": { + "name": "project_status_idx", + "columns": [ + "status" + ], + "isUnique": false + }, + "project_updated_at_idx": { + "name": "project_updated_at_idx", + "columns": [ + "updated_at" + ], + "isUnique": false + }, + "project_last_opened_at_idx": { + "name": "project_last_opened_at_idx", + "columns": [ + "last_opened_at" + ], + "isUnique": false + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_asset_membership": { + "name": "project_asset_membership", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "asset_id": { + "name": "asset_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "source_run_id": { + "name": "source_run_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "category": { + "name": "category", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "workflow_type": { + "name": "workflow_type", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "workflow_instance_id": { + "name": "workflow_instance_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "slot_id": { + "name": "slot_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "metadata": { + "name": "metadata", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_asset_membership_project_idx": { + "name": "project_asset_membership_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "project_asset_membership_asset_idx": { + "name": "project_asset_membership_asset_idx", + "columns": [ + "asset_id" + ], + "isUnique": false + }, + "project_asset_membership_unique": { + "name": "project_asset_membership_unique", + "columns": [ + "project_id", + "asset_id", + "category" + ], + "isUnique": true + } + }, + "foreignKeys": { + "project_asset_membership_project_id_project_id_fk": { + "name": "project_asset_membership_project_id_project_id_fk", + "tableFrom": "project_asset_membership", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "project_asset_membership_asset_id_asset_id_fk": { + "name": "project_asset_membership_asset_id_asset_id_fk", + "tableFrom": "project_asset_membership", + "tableTo": "asset", + "columnsFrom": [ + "asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "project_asset_membership_source_run_id_generation_history_id_fk": { + "name": "project_asset_membership_source_run_id_generation_history_id_fk", + "tableFrom": "project_asset_membership", + "tableTo": "generation_history", + "columnsFrom": [ + "source_run_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_canvas_state": { + "name": "project_canvas_state", + "columns": { + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "document_json": { + "name": "document_json", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_canvas_state_updated_at_idx": { + "name": "project_canvas_state_updated_at_idx", + "columns": [ + "updated_at" + ], + "isUnique": false + } + }, + "foreignKeys": { + "project_canvas_state_project_id_project_id_fk": { + "name": "project_canvas_state_project_id_project_id_fk", + "tableFrom": "project_canvas_state", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_timeline_state": { + "name": "project_timeline_state", + "columns": { + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "document_json": { + "name": "document_json", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_timeline_state_updated_at_idx": { + "name": "project_timeline_state_updated_at_idx", + "columns": [ + "updated_at" + ], + "isUnique": false + } + }, + "foreignKeys": { + "project_timeline_state_project_id_project_id_fk": { + "name": "project_timeline_state_project_id_project_id_fk", + "tableFrom": "project_timeline_state", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_workflow_state": { + "name": "project_workflow_state", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "workflow_type": { + "name": "workflow_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "workflow_instance_id": { + "name": "workflow_instance_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "template_slug": { + "name": "template_slug", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'draft'" + }, + "form_json": { + "name": "form_json", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "layout_json": { + "name": "layout_json", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "selection_json": { + "name": "selection_json", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_workflow_state_project_idx": { + "name": "project_workflow_state_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "project_workflow_state_unique": { + "name": "project_workflow_state_unique", + "columns": [ + "project_id", + "workflow_type", + "workflow_instance_id" + ], + "isUnique": true + } + }, + "foreignKeys": { + "project_workflow_state_project_id_project_id_fk": { + "name": "project_workflow_state_project_id_project_id_fk", + "tableFrom": "project_workflow_state", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "asset": { + "name": "asset", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "type": { + "name": "type", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "asset_class": { + "name": "asset_class", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'original'" + }, + "storage_provider": { + "name": "storage_provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "bucket": { + "name": "bucket", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'beatapi'" + }, + "object_key": { + "name": "object_key", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "public_url": { + "name": "public_url", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "sha256": { + "name": "sha256", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "width": { + "name": "width", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "height": { + "name": "height", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "duration_ms": { + "name": "duration_ms", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "origin_project_id": { + "name": "origin_project_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "thumbnail_asset_id": { + "name": "thumbnail_asset_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "metadata": { + "name": "metadata", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "asset_type_idx": { + "name": "asset_type_idx", + "columns": [ + "type" + ], + "isUnique": false + }, + "asset_class_idx": { + "name": "asset_class_idx", + "columns": [ + "asset_class" + ], + "isUnique": false + }, + "asset_origin_project_idx": { + "name": "asset_origin_project_idx", + "columns": [ + "origin_project_id" + ], + "isUnique": false + }, + "asset_created_at_idx": { + "name": "asset_created_at_idx", + "columns": [ + "created_at" + ], + "isUnique": false + }, + "asset_bucket_object_key_unique": { + "name": "asset_bucket_object_key_unique", + "columns": [ + "bucket", + "object_key" + ], + "isUnique": true + } + }, + "foreignKeys": { + "asset_origin_project_id_project_id_fk": { + "name": "asset_origin_project_id_project_id_fk", + "tableFrom": "asset", + "tableTo": "project", + "columnsFrom": [ + "origin_project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "asset_thumbnail_asset_id_asset_id_fk": { + "name": "asset_thumbnail_asset_id_asset_id_fk", + "tableFrom": "asset", + "tableTo": "asset", + "columnsFrom": [ + "thumbnail_asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + } + }, + "views": {}, + "enums": {}, + "_meta": { + "schemas": {}, + "tables": {}, + "columns": {} + }, + "internal": { + "indexes": {} + } +} \ No newline at end of file diff --git a/drizzle/d1/meta/0003_snapshot.json b/drizzle/d1/meta/0003_snapshot.json new file mode 100644 index 0000000..4fc272f --- /dev/null +++ b/drizzle/d1/meta/0003_snapshot.json @@ -0,0 +1,1456 @@ +{ + "version": "6", + "dialect": "sqlite", + "id": "6622e3e9-3968-4f55-a0a6-cf8538d1a594", + "prevId": "330a5db6-48ea-498b-ac5d-790bada59f0f", + "tables": { + "config": { + "name": "config", + "columns": { + "name": { + "name": "name", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "value": { + "name": "value", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_asset_link": { + "name": "generation_asset_link", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "generation_id": { + "name": "generation_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "asset_id": { + "name": "asset_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "generation_asset_link_generation_idx": { + "name": "generation_asset_link_generation_idx", + "columns": [ + "generation_id" + ], + "isUnique": false + }, + "generation_asset_link_asset_idx": { + "name": "generation_asset_link_asset_idx", + "columns": [ + "asset_id" + ], + "isUnique": false + }, + "generation_asset_link_unique": { + "name": "generation_asset_link_unique", + "columns": [ + "generation_id", + "asset_id", + "role" + ], + "isUnique": true + } + }, + "foreignKeys": { + "generation_asset_link_generation_id_generation_history_id_fk": { + "name": "generation_asset_link_generation_id_generation_history_id_fk", + "tableFrom": "generation_asset_link", + "tableTo": "generation_history", + "columnsFrom": [ + "generation_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "generation_asset_link_asset_id_asset_id_fk": { + "name": "generation_asset_link_asset_id_asset_id_fk", + "tableFrom": "generation_asset_link", + "tableTo": "asset", + "columnsFrom": [ + "asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_history": { + "name": "generation_history", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "effect_id": { + "name": "effect_id", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "provider_task_id": { + "name": "provider_task_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "lifecycle_phase": { + "name": "lifecycle_phase", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "last_provider_sync_at": { + "name": "last_provider_sync_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "execution_mode": { + "name": "execution_mode", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'create_new'" + }, + "submitted_prompt": { + "name": "submitted_prompt", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "submitted_params": { + "name": "submitted_params", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "result_asset_id": { + "name": "result_asset_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "input": { + "name": "input", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "output": { + "name": "output", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "error": { + "name": "error", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "started_at": { + "name": "started_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "completed_at": { + "name": "completed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "failed_at": { + "name": "failed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "generation_history_project_idx": { + "name": "generation_history_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "generation_history_effect_idx": { + "name": "generation_history_effect_idx", + "columns": [ + "effect_id" + ], + "isUnique": false + }, + "generation_history_status_idx": { + "name": "generation_history_status_idx", + "columns": [ + "status" + ], + "isUnique": false + }, + "generation_history_provider_task_idx": { + "name": "generation_history_provider_task_idx", + "columns": [ + "provider_task_id" + ], + "isUnique": false + }, + "generation_history_lifecycle_idx": { + "name": "generation_history_lifecycle_idx", + "columns": [ + "lifecycle_phase" + ], + "isUnique": false + }, + "generation_history_result_asset_idx": { + "name": "generation_history_result_asset_idx", + "columns": [ + "result_asset_id" + ], + "isUnique": false + } + }, + "foreignKeys": { + "generation_history_project_id_project_id_fk": { + "name": "generation_history_project_id_project_id_fk", + "tableFrom": "generation_history", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "generation_history_result_asset_id_asset_id_fk": { + "name": "generation_history_result_asset_id_asset_id_fk", + "tableFrom": "generation_history", + "tableTo": "asset", + "columnsFrom": [ + "result_asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_intent_upload": { + "name": "generation_intent_upload", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "intent_id": { + "name": "intent_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'reserved'" + }, + "storage_provider": { + "name": "storage_provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "bucket": { + "name": "bucket", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "object_key": { + "name": "object_key", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "public_url": { + "name": "public_url", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "completed_at": { + "name": "completed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + } + }, + "indexes": { + "generation_intent_upload_intent_idx": { + "name": "generation_intent_upload_intent_idx", + "columns": [ + "intent_id" + ], + "isUnique": false + }, + "generation_intent_upload_status_idx": { + "name": "generation_intent_upload_status_idx", + "columns": [ + "status" + ], + "isUnique": false + } + }, + "foreignKeys": { + "generation_intent_upload_intent_id_generation_upload_intent_id_fk": { + "name": "generation_intent_upload_intent_id_generation_upload_intent_id_fk", + "tableFrom": "generation_intent_upload", + "tableTo": "generation_upload_intent", + "columnsFrom": [ + "intent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "generation_upload_intent": { + "name": "generation_upload_intent", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "effect_id": { + "name": "effect_id", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'pending'" + }, + "expected_upload_count": { + "name": "expected_upload_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 0 + }, + "reserved_upload_count": { + "name": "reserved_upload_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 0 + }, + "completed_upload_count": { + "name": "completed_upload_count", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 0 + }, + "generation_id": { + "name": "generation_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "expires_at": { + "name": "expires_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "consumed_at": { + "name": "consumed_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "generation_upload_intent_project_idx": { + "name": "generation_upload_intent_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "generation_upload_intent_status_expiry_idx": { + "name": "generation_upload_intent_status_expiry_idx", + "columns": [ + "status", + "expires_at" + ], + "isUnique": false + } + }, + "foreignKeys": { + "generation_upload_intent_project_id_project_id_fk": { + "name": "generation_upload_intent_project_id_project_id_fk", + "tableFrom": "generation_upload_intent", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "generation_upload_intent_generation_id_generation_history_id_fk": { + "name": "generation_upload_intent_generation_id_generation_history_id_fk", + "tableFrom": "generation_upload_intent", + "tableTo": "generation_history", + "columnsFrom": [ + "generation_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project": { + "name": "project", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "cover_asset_id": { + "name": "cover_asset_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'active'" + }, + "current_state_version": { + "name": "current_state_version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "last_workspace_mode": { + "name": "last_workspace_mode", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'canvas'" + }, + "last_opened_at": { + "name": "last_opened_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "archived_at": { + "name": "archived_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "deleted_at": { + "name": "deleted_at", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_status_idx": { + "name": "project_status_idx", + "columns": [ + "status" + ], + "isUnique": false + }, + "project_updated_at_idx": { + "name": "project_updated_at_idx", + "columns": [ + "updated_at" + ], + "isUnique": false + }, + "project_last_opened_at_idx": { + "name": "project_last_opened_at_idx", + "columns": [ + "last_opened_at" + ], + "isUnique": false + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_asset_membership": { + "name": "project_asset_membership", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "asset_id": { + "name": "asset_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "source_run_id": { + "name": "source_run_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "category": { + "name": "category", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "workflow_type": { + "name": "workflow_type", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "workflow_instance_id": { + "name": "workflow_instance_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "slot_id": { + "name": "slot_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "role": { + "name": "role", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "metadata": { + "name": "metadata", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_asset_membership_project_idx": { + "name": "project_asset_membership_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "project_asset_membership_asset_idx": { + "name": "project_asset_membership_asset_idx", + "columns": [ + "asset_id" + ], + "isUnique": false + }, + "project_asset_membership_unique": { + "name": "project_asset_membership_unique", + "columns": [ + "project_id", + "asset_id", + "category" + ], + "isUnique": true + } + }, + "foreignKeys": { + "project_asset_membership_project_id_project_id_fk": { + "name": "project_asset_membership_project_id_project_id_fk", + "tableFrom": "project_asset_membership", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "project_asset_membership_asset_id_asset_id_fk": { + "name": "project_asset_membership_asset_id_asset_id_fk", + "tableFrom": "project_asset_membership", + "tableTo": "asset", + "columnsFrom": [ + "asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "project_asset_membership_source_run_id_generation_history_id_fk": { + "name": "project_asset_membership_source_run_id_generation_history_id_fk", + "tableFrom": "project_asset_membership", + "tableTo": "generation_history", + "columnsFrom": [ + "source_run_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_canvas_state": { + "name": "project_canvas_state", + "columns": { + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "document_json": { + "name": "document_json", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_canvas_state_updated_at_idx": { + "name": "project_canvas_state_updated_at_idx", + "columns": [ + "updated_at" + ], + "isUnique": false + } + }, + "foreignKeys": { + "project_canvas_state_project_id_project_id_fk": { + "name": "project_canvas_state_project_id_project_id_fk", + "tableFrom": "project_canvas_state", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_command_receipt": { + "name": "project_command_receipt", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "idempotency_key": { + "name": "idempotency_key", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "command_id": { + "name": "command_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "origin": { + "name": "origin", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "command_type": { + "name": "command_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "result_json": { + "name": "result_json", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_command_receipt_project_idx": { + "name": "project_command_receipt_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "project_command_receipt_command_idx": { + "name": "project_command_receipt_command_idx", + "columns": [ + "command_id" + ], + "isUnique": false + }, + "project_command_receipt_idempotency_unique": { + "name": "project_command_receipt_idempotency_unique", + "columns": [ + "project_id", + "idempotency_key" + ], + "isUnique": true + } + }, + "foreignKeys": { + "project_command_receipt_project_id_project_id_fk": { + "name": "project_command_receipt_project_id_project_id_fk", + "tableFrom": "project_command_receipt", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_timeline_state": { + "name": "project_timeline_state", + "columns": { + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "document_json": { + "name": "document_json", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_timeline_state_updated_at_idx": { + "name": "project_timeline_state_updated_at_idx", + "columns": [ + "updated_at" + ], + "isUnique": false + } + }, + "foreignKeys": { + "project_timeline_state_project_id_project_id_fk": { + "name": "project_timeline_state_project_id_project_id_fk", + "tableFrom": "project_timeline_state", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "project_workflow_state": { + "name": "project_workflow_state", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "project_id": { + "name": "project_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "workflow_type": { + "name": "workflow_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "workflow_instance_id": { + "name": "workflow_instance_id", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "template_slug": { + "name": "template_slug", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'draft'" + }, + "form_json": { + "name": "form_json", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "layout_json": { + "name": "layout_json", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "selection_json": { + "name": "selection_json", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": 1 + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "project_workflow_state_project_idx": { + "name": "project_workflow_state_project_idx", + "columns": [ + "project_id" + ], + "isUnique": false + }, + "project_workflow_state_unique": { + "name": "project_workflow_state_unique", + "columns": [ + "project_id", + "workflow_type", + "workflow_instance_id" + ], + "isUnique": true + } + }, + "foreignKeys": { + "project_workflow_state_project_id_project_id_fk": { + "name": "project_workflow_state_project_id_project_id_fk", + "tableFrom": "project_workflow_state", + "tableTo": "project", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + }, + "asset": { + "name": "asset", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true, + "autoincrement": false + }, + "type": { + "name": "type", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "asset_class": { + "name": "asset_class", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'original'" + }, + "storage_provider": { + "name": "storage_provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "bucket": { + "name": "bucket", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "'beatapi'" + }, + "object_key": { + "name": "object_key", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "public_url": { + "name": "public_url", + "type": "text", + "primaryKey": false, + "notNull": true, + "autoincrement": false + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "sha256": { + "name": "sha256", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "width": { + "name": "width", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "height": { + "name": "height", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "duration_ms": { + "name": "duration_ms", + "type": "integer", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "origin_project_id": { + "name": "origin_project_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "thumbnail_asset_id": { + "name": "thumbnail_asset_id", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "metadata": { + "name": "metadata", + "type": "text", + "primaryKey": false, + "notNull": false, + "autoincrement": false + }, + "created_at": { + "name": "created_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + }, + "updated_at": { + "name": "updated_at", + "type": "integer", + "primaryKey": false, + "notNull": true, + "autoincrement": false, + "default": "(cast((julianday('now') - 2440587.5)*86400000 as integer))" + } + }, + "indexes": { + "asset_type_idx": { + "name": "asset_type_idx", + "columns": [ + "type" + ], + "isUnique": false + }, + "asset_class_idx": { + "name": "asset_class_idx", + "columns": [ + "asset_class" + ], + "isUnique": false + }, + "asset_origin_project_idx": { + "name": "asset_origin_project_idx", + "columns": [ + "origin_project_id" + ], + "isUnique": false + }, + "asset_created_at_idx": { + "name": "asset_created_at_idx", + "columns": [ + "created_at" + ], + "isUnique": false + }, + "asset_bucket_object_key_unique": { + "name": "asset_bucket_object_key_unique", + "columns": [ + "bucket", + "object_key" + ], + "isUnique": true + } + }, + "foreignKeys": { + "asset_origin_project_id_project_id_fk": { + "name": "asset_origin_project_id_project_id_fk", + "tableFrom": "asset", + "tableTo": "project", + "columnsFrom": [ + "origin_project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "asset_thumbnail_asset_id_asset_id_fk": { + "name": "asset_thumbnail_asset_id_asset_id_fk", + "tableFrom": "asset", + "tableTo": "asset", + "columnsFrom": [ + "thumbnail_asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "checkConstraints": {} + } + }, + "views": {}, + "enums": {}, + "_meta": { + "schemas": {}, + "tables": {}, + "columns": {} + }, + "internal": { + "indexes": {} + } +} \ No newline at end of file diff --git a/drizzle/d1/meta/_journal.json b/drizzle/d1/meta/_journal.json index 6d3a0b5..4b1fca2 100644 --- a/drizzle/d1/meta/_journal.json +++ b/drizzle/d1/meta/_journal.json @@ -15,6 +15,20 @@ "when": 1787227875042, "tag": "0001_tiresome_xavin", "breakpoints": true + }, + { + "idx": 2, + "version": "6", + "when": 1788020112960, + "tag": "0002_lethal_micromacro", + "breakpoints": true + }, + { + "idx": 3, + "version": "6", + "when": 1788060451792, + "tag": "0003_slow_the_phantom", + "breakpoints": true } ] } \ No newline at end of file diff --git a/integrations/codex/beatdesign/.codex-plugin/plugin.json b/integrations/codex/beatdesign/.codex-plugin/plugin.json new file mode 100644 index 0000000..e7724ad --- /dev/null +++ b/integrations/codex/beatdesign/.codex-plugin/plugin.json @@ -0,0 +1,27 @@ +{ + "name": "beatdesign", + "version": "0.2.0", + "description": "Control a local BeatDesign Canvas, generation workspace, and video Editor through MCP.", + "author": { + "name": "BeatAPI" + }, + "homepage": "https://github.com/BeatAPI/BeatDesign", + "repository": "https://github.com/BeatAPI/BeatDesign", + "license": "Apache-2.0", + "keywords": ["beatdesign", "canvas", "video-editor", "generation", "mcp"], + "interface": { + "displayName": "BeatDesign", + "shortDescription": "Create and edit local media projects", + "longDescription": "BeatDesign connects Codex to a free local-first Canvas and video Editor. Read projects and Assets, generate media with logical model capabilities, and apply safe incremental Canvas or timeline edits.", + "developerName": "BeatAPI", + "category": "Design", + "capabilities": ["Read", "Write"], + "defaultPrompt": [ + "Open my latest BeatDesign project and summarize the canvas", + "Turn these project assets into a rough video timeline", + "Diagnose the current timeline and fix safe editing issues" + ], + "brandColor": "#111111" + }, + "mcpServers": "./.mcp.json" +} diff --git a/integrations/codex/beatdesign/.mcp.json b/integrations/codex/beatdesign/.mcp.json new file mode 100644 index 0000000..fd0aa3e --- /dev/null +++ b/integrations/codex/beatdesign/.mcp.json @@ -0,0 +1,17 @@ +{ + "mcpServers": { + "beatdesign": { + "command": "./scripts/launch_beatdesign_mcp", + "cwd": ".", + "env_vars": [ + "BEATDESIGN_ROOT", + "NODE_ENV", + "DATABASE_PROVIDER", + "DATABASE_URL", + "DATABASE_AUTH_TOKEN", + "GENERATION_PROVIDER" + ], + "tool_timeout_sec": 3600 + } + } +} diff --git a/integrations/codex/beatdesign/README.md b/integrations/codex/beatdesign/README.md new file mode 100644 index 0000000..c1aafff --- /dev/null +++ b/integrations/codex/beatdesign/README.md @@ -0,0 +1,74 @@ +# BeatDesign Codex plugin + +This folder is only a **Codex packaging wrapper**. It starts the same local +stdio MCP server as `pnpm mcp`. It does not embed the Canvas UI. + +## What actually happens + +```text +Browser: pnpm dev → http://127.0.0.1:3020 (you look at Canvas/Editor) +Agent: MCP stdio → pnpm mcp (Agent calls 20 tools) +Both processes share the same local SQLite + project files. +``` + +The Agent does **not** open a BeatDesign sidebar inside Codex. You keep the +browser open; the Agent writes through MCP; the UI polls and refreshes. + +You do **not** need this plugin for every Agent. Any MCP host that can start a +local stdio command can use BeatDesign. The Codex plugin is only the convenient +install shape for Codex. + +## Local Codex install (no marketplace) + +1. `pnpm install && pnpm db:push && pnpm dev` in the BeatDesign repo. +2. In Codex, install a **local plugin from folder**: + `integrations/codex/beatdesign` +3. If Codex copies the plugin elsewhere, set + `BEATDESIGN_ROOT=/absolute/path/to/Beat Design` +4. Start a new Codex thread and check that `bdesign_project_list` exists. + +Marketplace submission is later. Local folder install is enough to test. + +## Other Agents + +Same MCP command, different config files: + +| Host | Needs Codex plugin? | Config | +|---|---|---| +| Codex | Yes, this folder | Local plugin install | +| Claude Code | No | Repo-root `.mcp.json` | +| OpenCode | No | `opencode.json` `mcp.beatdesign` local command | +| Cursor / generic MCP | No | stdio `pnpm --silent mcp` | +| Pi / WorkBuddy / Hermes | Only if that host speaks MCP stdio | Same command | + +Generic stdio config: + +```json +{ + "mcpServers": { + "beatdesign": { + "command": "pnpm", + "args": ["--silent", "mcp"] + } + } +} +``` + +OpenCode: + +```json +{ + "mcp": { + "beatdesign": { + "type": "local", + "command": ["pnpm", "--silent", "mcp"], + "enabled": true + } + } +} +``` + +Run the host **from the BeatDesign repository** so `pnpm mcp` can find +`package.json`. If the host starts in another directory, point `command` at +this repo's `node_modules/.bin/tsx` and `scripts/mcp-server.ts`, or set +`BEATDESIGN_ROOT`. diff --git a/integrations/codex/beatdesign/scripts/launch_beatdesign_mcp b/integrations/codex/beatdesign/scripts/launch_beatdesign_mcp new file mode 100755 index 0000000..f2c298e --- /dev/null +++ b/integrations/codex/beatdesign/scripts/launch_beatdesign_mcp @@ -0,0 +1,23 @@ +#!/bin/sh +set -eu + +launcher_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +plugin_root=$(CDPATH= cd -- "$launcher_dir/.." && pwd) +workspace_root=${BEATDESIGN_ROOT:-} + +if [ -z "$workspace_root" ]; then + workspace_root=$(CDPATH= cd -- "$plugin_root/../../.." && pwd) +fi + +if [ ! -f "$workspace_root/package.json" ]; then + echo "BeatDesign root not found. Set BEATDESIGN_ROOT to the cloned repository path." >&2 + exit 1 +fi + +tsx_bin="$workspace_root/node_modules/.bin/tsx" +if [ ! -x "$tsx_bin" ]; then + echo "BeatDesign dependencies are missing. Run pnpm install in BEATDESIGN_ROOT." >&2 + exit 1 +fi + +exec "$tsx_bin" "$workspace_root/scripts/mcp-server.ts" diff --git a/messages/en.json b/messages/en.json index 3bd7619..b81ab17 100644 --- a/messages/en.json +++ b/messages/en.json @@ -137,8 +137,10 @@ "recentAssetsTitle": "Recent assets", "recentImages": "Recent images", "recentVideos": "Recent videos", + "recentAudios": "Recent audio", "recentImageAlt": "Recent image", "recentVideoAlt": "Recent video", + "recentAudioAlt": "Recent audio", "loading": "Loading...", "loadFailed": "Failed to load assets", "emptyAssets": "No assets yet" @@ -557,6 +559,129 @@ "emptyDescription": "Create your first project to start building a canvas-driven pipeline for product images and videos.", "updatedAt": "Updated {value}" }, + "editor": { + "media": { + "eyebrow": "Local media", + "title": "Project source", + "drop": "Drop an image or video here", + "localOnly": "The original stays on this device", + "stillDuration": "Still image · {duration}s" + }, + "empty": { + "title": "Import an image or video to start editing", + "description": "Drop a local image or MP4. Arrange, trim, preview, and export without installing system FFmpeg." + }, + "status": { + "ready": "Ready", + "reading": "Reading media…", + "saving": "Saving…", + "saved": "Saved locally", + "canvasAdded": "Timeline added to Canvas" + }, + "actions": { + "import": "Import image or video", + "importAudio": "Import audio", + "play": "Play selection", + "pause": "Pause", + "split": "Split", + "undo": "Undo edit", + "redoEdit": "Redo edit", + "redo": "AI redo", + "redoHint": "Redo the selected range with BeatAPI", + "extract": "Extract asset", + "addTimelineToCanvas": "Add timeline to Canvas", + "export": "Export MP4", + "exporting": "Export {progress}%", + "cancelExport": "Cancel {progress}%" + }, + "redo": { + "title": "AI redo selection", + "description": "The selected range is extracted non-destructively and sent to one verified BeatAPI video-to-video path.", + "close": "Close AI redo", + "billingNotice": "Uses your configured BeatAPI key and may consume API credits.", + "placeholder": "Describe how this selected clip should be remade…", + "confirm": "Confirm and generate", + "running": "Generating…", + "takes": "Takes", + "original": "Original clip", + "rollback": "Restore original", + "preparing": "Preparing the selected range…", + "ready": "New Take is ready. Preview it before replacing the clip.", + "status": { + "validating": "Validating BeatAPI request…", + "uploading": "Uploading the derived clip…", + "submitting": "Submitting generation…", + "processing": "BeatAPI is generating a new Take…", + "succeeded": "Generation completed." + } + }, + "trim": { + "in": "In", + "out": "Out" + }, + "timeline": { + "tracks": "Tracks", + "title": "Timeline", + "visualTrack": "Visual 1", + "audioTrack": "Audio 1", + "start": "Start", + "dragHint": "Drag the clip to move · drag either edge to trim", + "trimStart": "Drag to trim the clip start", + "trimEnd": "Drag to trim the clip end", + "seek": "Move timeline playhead", + "seekHint": "Click to move the playhead", + "delete": "Delete", + "rippleDelete": "Ripple delete" + }, + "clipInspector": { + "sourceRange": "Source range", + "sourceFrom": "Source start", + "sourceTo": "Source end", + "stillDuration": "Still duration", + "clipDuration": "Clip duration", + "timelinePosition": "Timeline position", + "exactControls": "Exact", + "type": { + "image": "Still image", + "video": "Video clip", + "audio": "Audio clip" + } + }, + "audio": { + "volume": "Volume", + "fadeIn": "Fade in", + "fadeOut": "Fade out", + "mute": "Mute", + "unmute": "Unmute" + }, + "diagnostics": { + "timeline_gap": "Export is blocked because the visible video track contains a gap.", + "clip_overlap": "Export is blocked because clips overlap on the same track.", + "media_missing": "Export is blocked because a source file is missing.", + "take_duration_mismatch": "Export is blocked because the active AI Take is shorter than its timeline clip.", + "caption_overlap": "Export is blocked because captions overlap.", + "caption_out_of_video": "Export is blocked because a caption sits outside the video range.", + "tiny_clip": "A clip is too short to export reliably.", + "empty_track": "An empty track will be ignored." + }, + "errors": { + "videoOnly": "Choose a video file.", + "visualOnly": "Choose an image or video file.", + "audioOnly": "Choose an MP3, WAV, M4A, AAC, or OGG audio file.", + "noVideoTrack": "This file does not contain a readable video track.", + "noAudioTrack": "This file does not contain a readable audio track.", + "inspectFailed": "The browser could not read this media file.", + "exportFailed": "MP4 export failed.", + "exportCancelled": "MP4 export was cancelled.", + "loadFailed": "The saved timeline could not be loaded.", + "saveFailed": "The timeline could not be saved.", + "sourceUnavailable": "The source video is no longer available.", + "canvasNodeFailed": "The timeline could not be added to Canvas.", + "redoAssetMissing": "The generated Take was not saved into project assets.", + "redoFailed": "BeatAPI could not redo this selection.", + "clipOverlap": "This position overlaps another clip on the same track." + } + }, "header": { "backHome": "Back home", "untitledCanvas": "Untitled project", @@ -570,22 +695,40 @@ "studioDescription": "Focused generation and analysis", "canvas": "Canvas", "canvasDescription": "Visual multi-step workflow", + "editor": "Edit", "assets": "Assets", "projectAssets": { "triggerLabel": "Project assets", "title": "Project assets", - "description": "Studio and Canvas share this same collection. Every completed generation and project upload appears here.", + "description": "Studio, Canvas, and Edit share this collection. Uploads, generated media, tail frames, and timeline renders all appear here.", "canvasHint": "Click an asset to add it, or drag it anywhere onto the canvas.", "addToCanvas": "Add to canvas", + "addToTimeline": "Add to timeline", + "createTimeline": "Create timeline", + "continueFromTailFrame": "Continue from tail frame", + "extractingTailFrame": "Extracting the tail frame locally...", + "tailFrameName": "{name} tail frame", + "tailFrameInsertFailed": "The extracted tail frame could not be added to Canvas.", + "continuationInsertFailed": "The continuation generation node could not be created.", + "tailFrameFailed": "The tail frame could not be extracted.", + "continuationReady": "Tail frame saved. Describe the next shot to continue.", + "timelineCreated": "Timeline created from the selected clips", + "derivedClip": "Derived clip", + "tailFrame": "Tail frame", + "timelineRender": "Timeline render", + "timelineExtract": "Extracted clip", "openOriginal": "Open original", "addedToCanvas": "Added to canvas", + "addedToTimeline": "Added to timeline", "imageName": "Project image", "videoName": "Project video", + "audioName": "Project audio", "loading": "Loading project assets...", "loadFailed": "Project assets could not be loaded.", "empty": "Generate or upload an asset to start this project collection.", "images": "Images", - "videos": "Videos" + "videos": "Videos", + "audios": "Audio" }, "apiConfig": { "triggerLabel": "API configuration", diff --git a/messages/zh.json b/messages/zh.json index 56712d7..591bf24 100644 --- a/messages/zh.json +++ b/messages/zh.json @@ -138,8 +138,10 @@ "recentAssetsTitle": "最近素材", "recentImages": "最近图片", "recentVideos": "最近视频", + "recentAudios": "最近音频", "recentImageAlt": "最近图片", "recentVideoAlt": "最近视频", + "recentAudioAlt": "最近音频", "loading": "加载中...", "loadFailed": "加载素材失败", "emptyAssets": "还没有素材" @@ -558,6 +560,129 @@ "emptyDescription": "先创建第一个项目,再开始搭建基于画布的电商图片和视频工作流。", "updatedAt": "最近更新:{value}" }, + "editor": { + "media": { + "eyebrow": "本地素材", + "title": "项目源文件", + "drop": "拖入图片或视频", + "localOnly": "原文件只保留在当前设备", + "stillDuration": "静态画面 {duration} 秒" + }, + "empty": { + "title": "导入图片或视频,开始剪辑", + "description": "拖入本地图片、MP4 或选择文件,无需安装系统 FFmpeg,即可编排、裁剪、预览并导出。" + }, + "status": { + "ready": "就绪", + "reading": "正在读取素材…", + "saving": "保存中…", + "saved": "已保存到本地", + "canvasAdded": "时间线已加入 Canvas" + }, + "actions": { + "import": "导入图片或视频", + "importAudio": "导入音频", + "play": "播放选区", + "pause": "暂停", + "split": "分割", + "undo": "撤销剪辑", + "redoEdit": "重做剪辑", + "redo": "AI 重做", + "redoHint": "使用 BeatAPI 重做当前选区", + "extract": "截取为素材", + "addTimelineToCanvas": "时间线加入 Canvas", + "export": "导出 MP4", + "exporting": "导出 {progress}%", + "cancelExport": "取消 {progress}%" + }, + "redo": { + "title": "AI 重做选区", + "description": "系统会非破坏式截取当前选区,并通过一条已验证的 BeatAPI 视频转视频路径重做。", + "close": "关闭 AI 重做", + "billingNotice": "将使用你配置的 BeatAPI Key,可能消耗 API 点数。", + "placeholder": "描述你希望怎样重做这一段视频…", + "confirm": "确认并生成", + "running": "生成中…", + "takes": "版本 Takes", + "original": "原始片段", + "rollback": "恢复原片段", + "preparing": "正在准备选区…", + "ready": "新 Take 已完成,请预览后再替换当前片段。", + "status": { + "validating": "正在校验 BeatAPI 请求…", + "uploading": "正在上传派生片段…", + "submitting": "正在提交生成任务…", + "processing": "BeatAPI 正在生成新 Take…", + "succeeded": "生成完成。" + } + }, + "trim": { + "in": "入点", + "out": "出点" + }, + "timeline": { + "tracks": "轨道", + "title": "时间线", + "visualTrack": "画面 1", + "audioTrack": "声音 1", + "start": "起始位置", + "dragHint": "拖动片段调整位置 · 拖动两侧边缘裁剪", + "trimStart": "拖动以裁剪片段开头", + "trimEnd": "拖动以裁剪片段结尾", + "seek": "移动时间线播放头", + "seekHint": "点击以移动播放头", + "delete": "删除", + "rippleDelete": "波纹删除" + }, + "clipInspector": { + "sourceRange": "素材截取范围", + "sourceFrom": "素材起点", + "sourceTo": "素材终点", + "stillDuration": "画面持续时间", + "clipDuration": "片段时长", + "timelinePosition": "成片位置", + "exactControls": "精确参数", + "type": { + "image": "静态图片", + "video": "视频片段", + "audio": "音频片段" + } + }, + "audio": { + "volume": "音量", + "fadeIn": "淡入", + "fadeOut": "淡出", + "mute": "静音", + "unmute": "取消静音" + }, + "diagnostics": { + "timeline_gap": "可见视频轨存在空隙,已阻止导出,避免产生未解释的黑帧。", + "clip_overlap": "同一轨道存在片段重叠,已阻止导出。", + "media_missing": "有源素材文件缺失,已阻止导出。", + "take_duration_mismatch": "当前 AI Take 短于时间线片段,已阻止导出,避免尾帧冻结。", + "caption_overlap": "字幕存在重叠,已阻止导出。", + "caption_out_of_video": "有字幕超出视频范围,已阻止导出。", + "tiny_clip": "有片段过短,可能无法稳定导出。", + "empty_track": "空轨道将在导出时忽略。" + }, + "errors": { + "videoOnly": "请选择视频文件。", + "visualOnly": "请选择图片或视频文件。", + "audioOnly": "请选择 MP3、WAV、M4A、AAC 或 OGG 音频文件。", + "noVideoTrack": "这个文件里没有可读取的视频轨道。", + "noAudioTrack": "这个文件里没有可读取的音频轨道。", + "inspectFailed": "浏览器无法读取这个媒体文件。", + "exportFailed": "MP4 导出失败。", + "exportCancelled": "已取消 MP4 导出。", + "loadFailed": "无法加载已保存的时间线。", + "saveFailed": "时间线保存失败。", + "sourceUnavailable": "源视频已无法访问。", + "canvasNodeFailed": "无法将时间线加入 Canvas。", + "redoAssetMissing": "生成结果没有保存到项目素材。", + "redoFailed": "BeatAPI 无法重做这个选区。", + "clipOverlap": "这个位置会与同一轨道上的其他片段重叠。" + } + }, "header": { "backHome": "返回首页", "untitledCanvas": "未命名项目", @@ -571,22 +696,40 @@ "studioDescription": "专注生成与分析", "canvas": "Canvas", "canvasDescription": "可视化多步骤工作流", + "editor": "剪辑", "assets": "素材", "projectAssets": { "triggerLabel": "项目素材", "title": "项目素材", - "description": "Studio 与 Canvas 共用这组素材,项目内完成的生成结果和上传文件都会出现在这里。", + "description": "Studio、Canvas 和 Edit 共用这组素材。上传、生成结果、尾帧和时间线成片都会出现在这里。", "canvasHint": "点击素材即可加入,或直接拖到画布上的任意位置。", "addToCanvas": "加入画布", + "addToTimeline": "加入时间线", + "createTimeline": "创建时间线", + "continueFromTailFrame": "从尾帧续写", + "extractingTailFrame": "正在本地提取尾帧……", + "tailFrameName": "{name} 尾帧", + "tailFrameInsertFailed": "尾帧已提取,但无法加入画布。", + "continuationInsertFailed": "无法创建续写视频节点。", + "tailFrameFailed": "无法提取视频尾帧。", + "continuationReady": "尾帧已保存,请描述下一镜头后继续生成。", + "timelineCreated": "已用所选片段创建时间线", + "derivedClip": "派生片段", + "tailFrame": "尾帧", + "timelineRender": "时间线成片", + "timelineExtract": "截取片段", "openOriginal": "查看原文件", "addedToCanvas": "已加入画布", + "addedToTimeline": "已加入时间线", "imageName": "项目图片", "videoName": "项目视频", + "audioName": "项目音频", "loading": "正在加载项目素材...", "loadFailed": "暂时无法加载项目素材。", "empty": "生成或上传第一个素材,开始建立这个项目的素材集合。", "images": "图片", - "videos": "视频" + "videos": "视频", + "audios": "音频" }, "apiConfig": { "triggerLabel": "API 配置", diff --git a/package.json b/package.json index 578c4c7..92336b8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "beatdesign", - "version": "0.1.0", + "version": "0.2.0", "private": true, "description": "The open-source, local-first AI canvas for image and video creation and analysis", "license": "Apache-2.0", @@ -20,10 +20,11 @@ "type": "module", "scripts": { "dev": "tsx scripts/prepare-paraglide.ts && vite dev --host 127.0.0.1 --port 3020", - "test": "sh -c 'rg --files src -g \"*.test.ts\" -g \"*.test.tsx\" | sort | xargs node --import tsx --test'", + "test": "node scripts/run-tests.mjs", "typecheck": "tsc --noEmit", "i18n:check": "node scripts/check-i18n.mjs", "media:localize": "tsx scripts/with-env.ts pnpm exec tsx scripts/localize-project-media.ts", + "mcp": "tsx scripts/mcp-server.ts", "prebuild": "node scripts/db-setup.mjs", "build": "tsx scripts/prepare-paraglide.ts && vite build && node scripts/sanitize-output.mjs", "start": "tsx scripts/with-env.ts node .output/server/index.mjs", @@ -52,6 +53,7 @@ "@fontsource-variable/space-grotesk": "^5.3.0", "@libsql/client": "^0.14.0", "@mdx-js/react": "^3.1.1", + "@modelcontextprotocol/server": "^2.0.0", "@tanstack/react-query": "^5.101.0", "@tanstack/react-router": "1.170.11", "@tanstack/react-start": "1.168.19", @@ -62,6 +64,7 @@ "drizzle-orm": "^0.45.2", "jszip": "^3.10.1", "lucide-react": "^0.511.0", + "mediabunny": "1.25.3", "motion": "^12.38.0", "next-themes": "^0.4.6", "react": "^19.2.1", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 3dd39ab..4087f90 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -56,6 +56,9 @@ importers: '@mdx-js/react': specifier: ^3.1.1 version: 3.1.1(@types/react@19.2.14)(react@19.2.5) + '@modelcontextprotocol/server': + specifier: ^2.0.0 + version: 2.0.0 '@tanstack/react-query': specifier: ^5.101.0 version: 5.101.0(react@19.2.5) @@ -86,6 +89,9 @@ importers: lucide-react: specifier: ^0.511.0 version: 0.511.0(react@19.2.5) + mediabunny: + specifier: 1.25.3 + version: 1.25.3 motion: specifier: ^12.38.0 version: 12.38.0(react-dom@19.2.5(react@19.2.5))(react@19.2.5) @@ -1017,6 +1023,10 @@ packages: peerDependencies: rollup: '>=2' + '@modelcontextprotocol/core@2.0.0': + resolution: {integrity: sha512-pJCEwGG7Lfr/+PQp9ZTwKXNeO5wzbfKL7H3MYpCorM4oFBoQrdjnBgEoqG+RjhsvS1FKrDbKux+M1HhlnGWqcA==} + engines: {node: '>=20'} + '@modelcontextprotocol/sdk@1.29.0': resolution: {integrity: sha512-zo37mZA9hJWpULgkRpowewez1y6ML5GsXJPY8FI0tBBCd77HEvza4jDqRKOXgHNn867PVGCyTdzqpz0izu5ZjQ==} engines: {node: '>=18'} @@ -1027,6 +1037,10 @@ packages: '@cfworker/json-schema': optional: true + '@modelcontextprotocol/server@2.0.0': + resolution: {integrity: sha512-YhHWdHfpFMQfd0prsEnxKeS3Qz3ytIGmsS0sth4KDjnacIT7hxk6hXHkJ9KysxlkvTM+WZAtQbbcUhdoP4Hvtw==} + engines: {node: '>=20'} + '@napi-rs/lzma-linux-x64-gnu@1.5.1': resolution: {integrity: sha512-oTXEIha4SsuXdTA4Iyskj0kpdx2yVXdhd75c2v3xGrHFfVMsbhTPZU/nMPL4sWKo4pBHm3aucLaqGlF696dTyQ==} engines: {node: ^22.20 || ^24.12 || >=25} @@ -1842,6 +1856,12 @@ packages: '@types/debug@4.1.13': resolution: {integrity: sha512-KSVgmQmzMwPlmtljOomayoR89W4FynCAi3E8PPs7vmDVPe84hT+vGPKkJfThkmXs0x0jAaa9U8uW8bbfyS2fWw==} + '@types/dom-mediacapture-transform@0.1.12': + resolution: {integrity: sha512-d7/QsLRwF864A5mgIM/YrfiglHoYn7zgCcAoJgW404r+2DwnNr7EBbLnCWpmOMgH8y0te73L1AV6H1bmauaWFw==} + + '@types/dom-webcodecs@0.1.13': + resolution: {integrity: sha512-O5hkiFIcjjszPIYyUSyvScyvrBoV3NOEEZx/pMlsu44TKzWNkLVBBxnxJz42in5n3QIolYOcBYFCPZZ0h8SkwQ==} + '@types/estree-jsx@1.0.5': resolution: {integrity: sha512-52CcUVNFyfb1A2ALocQw/Dd1BQFNmSdkuC3BkZ6iqhdMfQz7JWOFRuJFloOzjk+6WijU56m9oKXFAXc7o3Towg==} @@ -3060,6 +3080,9 @@ packages: resolution: {integrity: sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw==} engines: {node: '>= 0.8'} + mediabunny@1.25.3: + resolution: {integrity: sha512-+LDXv/kybsElRQCmMlrdbKmPNvegeIyouGeOUzW5DJ8+M56H4G6wyEL2AJ1A45UcOB+U62qJMV7tuFO9S7yA7g==} + merge-descriptors@2.0.0: resolution: {integrity: sha512-Snk314V5ayFLhp3fkUREub6WtjBfPdCPY1Ln8/8munuLuiYhsABgBVWsozAG+MWMbVEvcdcpbi9R7ww22l9Q3g==} engines: {node: '>=18'} @@ -5123,6 +5146,10 @@ snapshots: transitivePeerDependencies: - supports-color + '@modelcontextprotocol/core@2.0.0': + dependencies: + zod: 4.4.3 + '@modelcontextprotocol/sdk@1.29.0(zod@3.25.76)': dependencies: '@hono/node-server': 1.19.13(hono@4.12.12) @@ -5145,6 +5172,11 @@ snapshots: transitivePeerDependencies: - supports-color + '@modelcontextprotocol/server@2.0.0': + dependencies: + '@modelcontextprotocol/core': 2.0.0 + zod: 4.4.3 + '@napi-rs/lzma-linux-x64-gnu@1.5.1': optional: true @@ -5866,6 +5898,12 @@ snapshots: dependencies: '@types/ms': 2.1.0 + '@types/dom-mediacapture-transform@0.1.12': + dependencies: + '@types/dom-webcodecs': 0.1.13 + + '@types/dom-webcodecs@0.1.13': {} + '@types/estree-jsx@1.0.5': dependencies: '@types/estree': 1.0.9 @@ -6985,6 +7023,11 @@ snapshots: media-typer@1.1.0: {} + mediabunny@1.25.3: + dependencies: + '@types/dom-mediacapture-transform': 0.1.12 + '@types/dom-webcodecs': 0.1.13 + merge-descriptors@2.0.0: {} merge-stream@2.0.0: {} diff --git a/scripts/localize-project-media.ts b/scripts/localize-project-media.ts index 53f5c9a..a4b8f2c 100644 --- a/scripts/localize-project-media.ts +++ b/scripts/localize-project-media.ts @@ -38,7 +38,9 @@ const localizeProject = async (projectId: string) => { if (!state) return { cards: 0, files: 0 }; const cardsWithRemoteMedia = state.snapshot.cards.filter( - (card) => Boolean(card.url && isOfficialBeatApiMediaUrl(card.url)) + (card): card is typeof card & { type: 'image' | 'video'; url: string } => + (card.type === 'image' || card.type === 'video') && + Boolean(card.url && isOfficialBeatApiMediaUrl(card.url)) ); if (cardsWithRemoteMedia.length === 0) return { cards: 0, files: 0 }; diff --git a/scripts/mcp-server.ts b/scripts/mcp-server.ts new file mode 100644 index 0000000..43f13b2 --- /dev/null +++ b/scripts/mcp-server.ts @@ -0,0 +1,37 @@ +#!/usr/bin/env node +import { existsSync, readFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + +function loadEnvFile(path: string) { + if (!existsSync(path)) return false; + for (const line of readFileSync(path, 'utf8').split('\n')) { + const trimmed = line.trim(); + if (!trimmed || trimmed.startsWith('#')) continue; + const separator = trimmed.indexOf('='); + if (separator < 1) continue; + const key = trimmed.slice(0, separator).trim(); + let value = trimmed.slice(separator + 1).trim(); + if ( + (value.startsWith('"') && value.endsWith('"')) || + (value.startsWith("'") && value.endsWith("'")) + ) { + value = value.slice(1, -1); + } + process.env[key] ??= value; + } + return true; +} + +const nodeEnv = process.env.NODE_ENV || 'development'; +for (const filename of [ + `.env.${nodeEnv}.local`, + `.env.${nodeEnv}`, + '.env.local', + '.env', +]) { + if (loadEnvFile(resolve(filename))) break; +} + +const { startBeatDesignMcpServer } = await import('../src/mcp/server'); +startBeatDesignMcpServer(); +console.error('BeatDesign MCP server is running on stdio.'); diff --git a/scripts/run-tests.mjs b/scripts/run-tests.mjs new file mode 100644 index 0000000..215a24a --- /dev/null +++ b/scripts/run-tests.mjs @@ -0,0 +1,28 @@ +import { spawnSync } from 'node:child_process'; +import { globSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const root = join(dirname(fileURLToPath(import.meta.url)), '..'); +const files = [ + ...globSync('src/**/*.test.ts', { cwd: root }), + ...globSync('src/**/*.test.tsx', { cwd: root }), +] + .sort() + .map((file) => join(root, file)); + +if (files.length === 0) { + console.error('No test files found.'); + process.exit(1); +} + +const result = spawnSync( + process.execPath, + ['--import', 'tsx', '--test', ...files], + { + cwd: root, + stdio: 'inherit', + } +); + +process.exit(result.status ?? 1); diff --git a/src/components/app/product-page-shell.test.ts b/src/components/app/product-page-shell.test.ts index 6d0d95c..5b8809a 100644 --- a/src/components/app/product-page-shell.test.ts +++ b/src/components/app/product-page-shell.test.ts @@ -21,11 +21,14 @@ test('workspace header keeps mode switching, shared assets, and API configuratio assert.match(shellSource, /\/assets\//); assert.match(shellSource, /header\.studio/); assert.match(shellSource, /header\.canvas/); + assert.match(shellSource, /header\.editor/); assert.match(shellSource, /header\.assets/); assert.match(shellSource, /href=\{`\/studio\/\$\{projectId\}`\}/); assert.match(shellSource, /href=\{`\/canvas\/\$\{projectId\}`\}/); + assert.match(shellSource, /href=\{`\/editor\/\$\{projectId\}`\}/); assert.match(shellSource, /aria-current=\{workspaceMode === 'studio'/); assert.match(shellSource, /aria-current=\{workspaceMode === 'canvas'/); + assert.match(shellSource, /aria-current=\{workspaceMode === 'editor'/); assert.match(shellSource, /aria-current=\{workspaceMode === 'assets'/); assert.doesNotMatch(shellSource, /DropdownMenu/); assert.doesNotMatch(shellSource, /ChevronDown/); diff --git a/src/components/app/product-page-shell.tsx b/src/components/app/product-page-shell.tsx index f18666c..620ab5a 100644 --- a/src/components/app/product-page-shell.tsx +++ b/src/components/app/product-page-shell.tsx @@ -2,7 +2,7 @@ import type { ReactNode } from 'react'; import { useCallback, useEffect, useRef, useState } from 'react'; -import { Images, LayoutPanelTop, Workflow } from 'lucide-react'; +import { Images, LayoutPanelTop, Scissors, Workflow } from 'lucide-react'; import { WorkspaceApiConfigDialog } from '@/components/app/workspace-api-config-dialog'; import { GitHubIcon } from '@/components/icons/github'; @@ -159,6 +159,15 @@ export function ProductPageShell({