Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Automatic extraction pipelines live under `src/Undefined/skills/pipelines/<name>
Consecutive messages from the same sender within `[message_batcher].window_seconds` are merged into a single AI invocation, so the AI sees the whole batch as `<message>` blocks and decides per-intent (independent request vs. correction/interruption). Pokes always bypass; an at-bot message arriving while a buffer already exists is processed individually so it is not blocked; a first at-bot message that opens the buffer routes the eventual batch through the mention lane. History writes remain unchanged. Configure under `[message_batcher]` (`enabled`, `window_seconds`, `strategy`, `max_window_seconds`, `max_messages_per_batch`, `group_enabled`, `private_enabled`, `pre_send_seconds`, `allow_cancel_after_send`); details in [docs/message-batching.md](docs/message-batching.md). Optional speculative pre-fire (`0 < pre_send_seconds < window_seconds`) dispatches the current batch to the LLM early once the user has been silent for `pre_send_seconds`; new messages can cancel the in-flight call as long as it has not yet sent any reply.

### User identification in prompts
The system prompt now includes a rule: **recognize and address users by their QQ ID (`sender_id`)** because nicknames can change. When needing to address a user, use the latest nickname obtained via `group.get_member_info(brief=true)`. Observations recorded in cognitive memory should always include the QQ ID, e.g., “QQ号12345678(昵称张三)做了某事”.
The system prompt now includes a rule: **recognize and address users by their QQ ID (`sender_id`)** because nicknames can change. When needing to address a user, use the latest nickname obtained via `group.get_member_info(brief=true)`. `end.observations` must be substantive facts worth future retrieval (prefer empty over noise); user-centered observations should always include the QQ ID, e.g., “QQ号12345678(昵称张三)做了某事”.

## Testing Guidelines
Write tests as `tests/test_<feature>.py`. Async tests use `pytest-asyncio`. Add or update coverage for behavior changes in APIs, config loading/hot reload, cognitive memory, meme or knowledge flows, and WebUI/runtime routes. If you touch `apps/undefined-console/` or `src/Undefined/webui/static/js/`, run `npm run check` in `apps/undefined-console/` in addition to the Python checks; if you touch `apps/undefined-chat/`, run `npm run check` in `apps/undefined-chat/` (it bundles Vitest unit/e2e suites, so cover changed behavior there). No fixed coverage threshold is configured, so cover touched paths well.
Expand Down
6 changes: 3 additions & 3 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,13 +158,13 @@ graph TB
EndSummaryStorage["EndSummaryStorage<br/>短期总结存储<br/>[end_summary_storage.py]"]
CognitiveService["CognitiveService<br/>认知记忆服务<br/>[cognitive/service/]<br/>• 事件检索 • 侧写读取<br/>• 入队 memory job"]
CognitiveJobQueue["JobQueue<br/>认知任务队列<br/>[cognitive/job_queue.py]<br/>• pending/processing/failed"]
CognitiveHistorian["HistorianWorker<br/>后台史官<br/>[cognitive/historian/]<br/>• 绝对化改写 • 闸门重试<br/>• 侧写合并(含历史事件注入)"]
CognitiveHistorian["HistorianWorker<br/>后台史官<br/>[cognitive/historian/]<br/>• 独立事实改写 • 闸门重试<br/>• 侧写合并(最新优先/去冗)"]
CognitiveVectorStore["CognitiveVectorStore<br/>向量存储<br/>[cognitive/vector_store.py]<br/>• events/profiles<br/>• 时间衰减加权排序<br/>• MMR 去重"]
CognitiveProfileStorage["ProfileStorage<br/>侧写存储<br/>[cognitive/profile_storage.py]<br/>• users/groups Markdown<br/>• 历史快照"]
MemeSystem["MemeSystem<br/>表情包存储<br/>[memes/]<br/>• worker.py (两阶段识别)<br/>• sqlite+chromadb<br/>• blob 持久化"]
FAQStorage["FAQStorage<br/>FAQ 存储<br/>[faq.py]<br/>• data/faq/{group_id}/"]
ScheduledTaskStorage["ScheduledTaskStorage<br/>定时任务存储<br/>[scheduled_task_storage.py]"]
TokenUsageStorage["TokenUsageStorage<br/>Token 使用统计<br/>[token_usage_storage.py]<br/>• 自动归档<br/>• gzip 压缩"]
TokenUsageStorage["TokenUsageStorage<br/>Token 使用统计<br/>[token_usage_storage.py]<br/>• 自动归档<br/>• gzip 压缩<br/>• 流式可选 TTFT/TPS"]
end

subgraph IOLayer["异步 IO 层 (src/Undefined/utils/)"]
Expand Down Expand Up @@ -573,7 +573,7 @@ graph LR
CognitiveProfile["ProfileStorage<br/>data/cognitive/profiles/"]
FAQ["FAQStorage<br/>data/faq/{group_id}/<br/>• ID: YYYYMMDD-NNN"]
Tasks["ScheduledTaskStorage<br/>data/scheduled_tasks.json<br/>• Cron 格式"]
TokenUsage["TokenUsageStorage<br/>data/token_usage.jsonl<br/>• 自动归档<br/>• gzip 压缩"]
TokenUsage["TokenUsageStorage<br/>data/token_usage.jsonl<br/>• 自动归档<br/>• gzip 压缩<br/>• 流式可选 TTFT/TPS"]
end

subgraph Persistence["持久化"]
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
## v3.11.1 史官记忆质量与流式调用指标

本版本围绕认知记忆质量与 LLM 可观测性做了针对性优化:史官侧写以最新事实为准并克制膨胀,事件改写提炼为带时间锚点的独立事实,`end.observations` 只保留值得日后检索的写实内容;流式模型调用额外记录首字延迟与生成吞吐。

- 优化史官侧写合并。合并时注入当前时刻与旧侧写 `updated_at`,稳定特征冲突时以当前输入批次为准覆盖过时内容;时间只用于判断取舍,侧写正文仍禁止写入时序描述。
- 引导侧写克制扩写、合并去冗。同维度复述会合并或覆盖,边缘或一次性特征可省略,过长条目会压缩重组;不设硬字数,由提示词按上下文灵活判断。
- 改进史官事件改写。在绝对化(消灭代词、相对时间、相对地点)基础上,尽量把“谁说了什么”提炼为带时间锚点的独立事实(如“张三在某时改用了 Rust”);言说行为本身才是要点、未兑现承诺、玩笑或无法核实时保留原意,并按上下文灵活判断而非套用固定模板。
- 收紧 `end.observations`。只写值得日后检索的写实实质事实,宁缺毋滥;禁止硬凑流程决策、否定清单、元评论或一次性闲聊碎碎念。无实质事实时用空数组,纯流水账写 `memo`。
- 为流式 LLM 调用记录 TTFT 与 TPS。`[API响应]` 日志与 `data/token_usage.jsonl` 在可计算时写入可选字段 `ttft_seconds` / `tokens_per_second`(吞吐按 `completion_tokens / (总耗时 − TTFT)`);非流式调用只记录总耗时,不计算、不落盘这两项。

---

## v3.11.0 主 Prompt 本地自定义与 Tool Call 兼容

本版本将部署者私有的身份、权限与人格补充从仓库主 Prompt 中解耦,新增可热更新的本地文件插槽,让不同部署可以在不修改受版本控制提示词的情况下完成定制;同时补充 `function` / `parameters` 文本 Tool Call 封包兼容。
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Management / Runtime 请求 → webui/app.py 或 api/app.py → routes/*
- `data/end_summaries.json` — 短期总结存储
- `data/scheduled_tasks.json` — 定时任务存储
- `data/faq/` — FAQ 存储
- `data/token_usage.jsonl` — Token 统计(自动 gzip 归档)
- `data/token_usage.jsonl` — Token 统计(自动 gzip 归档;流式调用可含可选 `ttft_seconds` / `tokens_per_second`
- `knowledge/` — 本地知识库数据目录(`texts/`、`intro.md`、`chroma/` 等)
- `res/prompts/` — 系统提示词模板

Expand All @@ -146,7 +146,7 @@ Management / Runtime 请求 → webui/app.py 或 api/app.py → routes/*
系统提示词(`res/prompts/undefined.xml`)包含用户识别规则:
- 以 QQ 号(`sender_id`)为用户唯一标识,昵称可能随时变动
- 称呼用户时使用当前最新昵称,不确定时可调用 `group.get_member_info(brief=true)` 查询
- 认知记忆(observations)必须包含 QQ 号,格式如:“QQ号12345678(昵称张三)做了某事”
- 认知记忆(`end.observations`)只写值得日后检索的写实实质事实(宁缺毋滥);必须含稳定实体标识,用户观察格式如:“QQ号12345678(昵称张三)做了某事”

## 配置系统

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Console 和 Chat 都需要连接到已经运行的 Undefined 服务。首次部
- **主 Prompt 本地自定义**:通过 `[prompt.file_includes]` 将身份、权限或人格补充文件插入 `p0`、`p1`、`p2`、`p3`、`summary` 五个稳定位置,无需修改仓库内的主提示词;配置路径和文件内容均支持热更新,推荐使用受 Git 与构建忽略规则保护的 `config/prompts/*.local.*` 文件。详见 [Prompt 本地文件插槽](docs/configuration.md#4112-promptfile_includes-主-prompt-本地文件插槽)。
- **三层分层记忆架构**:创新的分层记忆系统,模拟人类记忆机制——
- **短期记忆**(`end.memo`):每轮对话结束自动记录便签备忘,最近 N 条始终注入,保持短期连续性,零配置开箱即用
- **认知记忆**(`end.observations` + `cognitive.*`):核心层,AI 在每轮对话中主动观察并提取用户/群聊事实及有价值的自身行为,经后台史官异步改写后存入向量数据库;支持语义检索、时间衰减加权排序、MMR 多样性去重、跨群记忆联动与用户/群聊自动侧写(合并时注入历史事件防止特征丢失),前台零延迟
- **认知记忆**(`end.observations` + `cognitive.*`):核心层,AI 在每轮对话中主动提取写实新观察(用户/群聊实质事实及有价值的自身行为;宁缺毋滥),经后台史官异步改写为带时间锚点的独立事实后存入向量数据库;支持语义检索、时间衰减加权排序、MMR 多样性去重、跨群记忆联动与用户/群聊自动侧写(合并时注入历史事件与当前时刻、按克制扩写去冗),前台零延迟
- **置顶备忘录**(`memory.*`):AI 自身的置顶提醒(自我约束、待办事项),每轮固定注入,支持增删改查
详见 [认知记忆文档](docs/cognitive-memory.md)。
- **Management-first WebUI**:继续保留 `uv run Undefined-webui` 一键入口;即使 `config.toml` 缺失或未配完,也能先进入管理态补配置、看日志、校验并启动 Bot。
Expand Down
4 changes: 2 additions & 2 deletions apps/undefined-chat/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion apps/undefined-chat/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "undefined-chat",
"private": true,
"version": "3.11.0",
"version": "3.11.1",
"type": "module",
"scripts": {
"tauri": "tauri",
Expand Down
2 changes: 1 addition & 1 deletion apps/undefined-chat/src-tauri/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion apps/undefined-chat/src-tauri/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "undefined_chat"
version = "3.11.0"
version = "3.11.1"
description = "Undefined native chat client"
authors = ["Undefined contributors"]
license = "MIT"
Expand Down
2 changes: 1 addition & 1 deletion apps/undefined-chat/src-tauri/tauri.conf.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Undefined Chat",
"version": "3.11.0",
"version": "3.11.1",
"identifier": "com.undefined.chat",
"build": {
"beforeDevCommand": "npm run dev",
Expand Down
4 changes: 2 additions & 2 deletions apps/undefined-console/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion apps/undefined-console/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "undefined-console",
"private": true,
"version": "3.11.0",
"version": "3.11.1",
"type": "module",
"scripts": {
"tauri": "tauri",
Expand Down
2 changes: 1 addition & 1 deletion apps/undefined-console/src-tauri/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion apps/undefined-console/src-tauri/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "undefined_console"
version = "3.11.0"
version = "3.11.1"
description = "Undefined cross-platform management console"
authors = ["Undefined contributors"]
license = "MIT"
Expand Down
2 changes: 1 addition & 1 deletion apps/undefined-console/src-tauri/tauri.conf.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Undefined Console",
"version": "3.11.0",
"version": "3.11.1",
"identifier": "com.undefined.console",
"build": {
"beforeDevCommand": "npm run dev",
Expand Down
8 changes: 4 additions & 4 deletions docs/cognitive-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
认知记忆系统是 Undefined 的三层分层记忆架构,模拟人类记忆机制:

- **短期记忆**(`end.memo`):每轮对话结束自动记录便签备忘,最近 N 条始终注入,保持短期连续性,零配置开箱即用。若本轮由 MessageBatcher 合并多条消息,memo 应概括整个当前输入批次的处理结果。
- **认知记忆**(`end.observations` + `cognitive.*`):核心层,AI 在每轮对话中只观察当前输入批次,提取有价值的新观察(用户/群聊/第三方事实及有价值的自身行为)。`observations` 不要求与 bot 相关,也不要求长期稳定历史消息、认知记忆、侧写和最近消息参考只能用于消歧,不能作为新事实来源。后台史官会异步改写为绝对化事件并存入 ChromaDB 向量库,支持语义检索;当对话中出现可沉淀为稳定画像的新信息(偏好、身份、习惯等)时,史官自动合并更新 Markdown 侧写文件,下次对话时注入 prompt。
- **认知记忆**(`end.observations` + `cognitive.*`):核心层,AI 在每轮对话中只观察当前输入批次,提取**写实**新观察(用户/群聊/第三方实质事实及有价值的自身行为)。`observations` 不要求与 bot 相关,也不要求长期稳定,但必须值得日后检索;宁缺毋滥,无实质事实时用空数组,禁止硬凑流程决策、否定清单、元评论或闲聊碎碎念。用户中心观察须写成 `QQ号<sender_id>(昵称<name>)`,保留稳定数字标识。历史消息、认知记忆、侧写和最近消息参考只能用于消歧,不能作为新事实来源。后台史官会异步改写为绝对化事件并存入 ChromaDB 向量库,支持语义检索;当对话中出现可沉淀为稳定画像的新信息(偏好、身份、习惯等)时,史官自动合并更新 Markdown 侧写文件,下次对话时注入 prompt。
- **置顶备忘录**(`memory.*`):AI 自身的置顶提醒(自我约束、待办事项,如"用户要求以后用英文回复"),每轮固定注入,支持增删改查。注意:用户事实(偏好、身份、习惯等)不应写入此层,一律通过 `end.observations` 写入认知记忆。

三层记忆都只为当前请求提供背景、默认偏好和消歧信息,不能独立构成本轮可执行指令,也不能覆盖当前输入批次。任务目标、收件人、发送地址、工具参数和输出位置始终以当前输入及当前会话元数据为准;当前消息没有明确指定跨会话目标时,默认回复或发送到当前会话,不得从记忆、旧定时任务或历史工具调用中继承其他地址。只有当前输入明确要求沿用某项历史配置时,才可把对应记忆作为参数参考。
Expand Down Expand Up @@ -68,7 +68,7 @@ AI 调用 `end` 工具结束对话时,只做一次文件落盘(p95 < 5ms)
`end` 字段语义:

- `memo`:本轮便签纸,留给短期记忆看的简短备注(纯流水账动作写这里),可空。当前输入批次包含多条连续消息时,memo 应概括整批处理结果。
- `observations`:本轮从当前输入批次提取的有价值新观察列表(0..N 条),包括用户/群聊/第三方事实和有价值的自身行为(帮谁解决了什么)不要求与 bot 相关,也不要求长期稳定,严格一条一个要点;每条会独立改写与入库。当前输入批次包含多条连续消息时,必须覆盖整批消息中有价值的信息,不能只记录最后一条。历史消息、认知记忆、侧写和最近消息参考只能用于消歧,不能作为 observations 的新事实来源。
- `observations`:本轮从当前输入批次提取的写实新观察列表(0..N 条),包括用户/群聊/第三方实质事实和有价值的自身行为(帮谁解决了什么)不要求与 bot 相关,也不要求长期稳定,但必须值得日后检索;宁缺毋滥,无实质事实时用 `[]`。用户中心观察须写成 `QQ号<sender_id>(昵称<name>)`,保留稳定数字标识。严格一条一个要点;每条会独立改写与入库。当前输入批次包含多条连续消息且存在实质可记事实时,必须覆盖整批,不能只记录最后一条。禁止写入纯流水账动作(静默处理、闸门未通过、调了什么工具)、否定清单(“无新增任务/无隐私风险”等)、元评论或一次性闲聊/消费碎碎念——这些写 `memo` 或不写。历史消息、认知记忆、侧写和最近消息参考只能用于消歧,不能作为 observations 的新事实来源。
- 两字段都为空时,仅结束会话,不写认知队列。

### 后台史官流水线
Expand All @@ -79,7 +79,7 @@ pending/{job_id}.json
▼ dequeue(原子 os.replace)
processing/{job_id}.json
▼ LLM 绝对化改写(消灭代词/相对时间/相对地点;结合“当前输入批次原文 + 最近消息参考”做实体消歧)
▼ LLM 绝对化改写(消灭代词/相对时间/相对地点;尽量提炼为带时间锚点的独立事实;结合“当前输入批次原文 + 最近消息参考”做实体消歧)
▼ 正则闸门检查
│ 通过 → is_absolute=true
Expand Down Expand Up @@ -177,7 +177,7 @@ MMR_score = λ × relevance(doc, query) − (1 − λ) × max_similarity(doc, se

### 侧写合并:历史事件注入

史官合并侧写时,会在 merge LLM 调用前用当前 observations 作为 query 从 ChromaDB 检索该实体的 top-8 历史事件,注入 merge prompt。这让史官拥有更丰富的上下文来判断哪些特征应保留,避免因本轮未提及而误删长期稳定特征。
史官合并侧写时,会在 merge LLM 调用前用当前 observations 作为 query 从 ChromaDB 检索该实体的 top-8 历史事件,注入 merge prompt。这让史官拥有更丰富的上下文来判断哪些特征应保留,避免因本轮未提及而误删长期稳定特征。合并时还会注入**当前时刻**与旧侧写 `updated_at`:冲突时以当前输入批次为准覆盖过时特征;时间只用于判断取舍,侧写正文仍禁止写入时序描述。合并时按「克制扩写 / 合并去冗」压缩同维度复述,避免侧写无限膨胀(不设硬字数,由提示词灵活判断)。

### ChromaDB 前后台调度

Expand Down
2 changes: 2 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -883,6 +883,8 @@ summary = ""
- `merge` 兼容 `repack/lossless`
- `none` 兼容 `keep/off/disable`

每次生成模型调用会向 `data/token_usage.jsonl` 追加一条记录(含 `duration_seconds`、token 计数、`call_type` 等)。流式调用在可计算时额外写入可选字段 `ttft_seconds`(首字延迟)与 `tokens_per_second`(`completion_tokens / (duration_seconds − ttft_seconds)`);非流式或不具备首字采样点时省略这两项,不写 `null`。指标语义见[模型 API 与兼容层](model-compatibility.md#sdk-与-api-mode)。

---

### 4.19 `[mcp]`
Expand Down
Loading
Loading