Feat/harness wave0 - #12
Conversation
- Architecture audit (8.5/10): MessageContent design, agent loop, error handling - Security audit (8/10): path traversal, size limits, injection, session safety - Integration audit (8.5/10): backward compat, cross-crate contracts, GUI sync - M7-A7 formal logs: summary, verification, release, acceptance - Prephase logs: analysis plan, project management plan, GUI minimal experience Total: 3 independent claude-code audits (~64 turns, .44) + 3 doc writers (.36)
…h logs - Add agent-plan/ with phase-A, plan-mode, todolist runtime, and roadmap docs - Add Observability/phase-b thin trace/log/debug-bundle planning - Add research logs for planmode, observability, plan-mode-architecture, plan-pre-research, and session-research (May-June 2026) - Update docs/dev/README.md index to reference new docs
- Move docs/dev/archive/ -> docs/dev/archive(old-docs-dont-read-me)/ - Move docs/dev/gateway-to-gui/, multimodal/, upsp/ into old archive - Move docs/dev/nano-crates-io-checklist.md, nano-runtime-packaging-plan.md - These are legacy design notes, roadmaps, and research from earlier phases
Sync AGENTS.md commit policy to autocommit-without-push.\n\nValidation: not run (docs-only governance update).
- Circuit breaker with tool-call fingerprinting and loop wall-clock timeout - Subagent security: depth limit, concurrency control, least-privilege defaults - Context budget estimation, proactive compaction, overflow retry - Tool execution timeout at registry boundary (60s default) - Subagent policy-based tool rebuilding with credential minimization - Skill loading hardened with builtin dir resolution Refs: docs/logs/2026-06-agent-loop-safety/, docs/logs/2026-06-tool-timeout/
- SessionManager::save() uses temp file + backup promote for durability - SessionManager::load() explicit failure on read/parse errors (no silent reset) - Inbound user message persisted before LLM/tool execution - Raw turn durable save before consolidation - Manager runtime/state carry same session truth-source assumptions - CLI/manager wiring aligned with backend-first session contract Refs: docs/logs/2026-06-session-truth-source/
…OSE-03) - Credential scrubbing: Bearer, sk-*, ghp_*, xoxb-*, field-level redaction - Redacting writer for stdout/file tracing output - ErrorContext with actionable context, no secret leakage - Provider/http safety helpers with configurable timeouts - Filesystem tool path validation integration - Config schema/validate for context_budget and timeout settings Refs: docs/logs/2026-06-log-redaction/, docs/logs/2026-06-observability/
- Handler attachment/request boundary works without frontend page changes - Embedded gateway/server wiring remains valid on main - Config migration covers retained backend-only multimodal options - Runtime control alignment for multimodal paths
- TODOLIST references closeout plan and cards - Closeout plan defines what remains on main (backend/runtime/safety only) - Closeout cards mark product/UI files as moved-out - Survey and audit docs committed for traceability - Decisions.md updated with closeout context
Reclassify main TODOLIST to keep only stability work, archive research/product items, and sync closeout card statuses to the completed MAIN-CLOSE-01..05 sequence. Validation: not run (docs-only backlog and status update).
Update main backlog wording so sandbox remediation is treated as route-TBD work, validated on pro first instead of assumed to merge back into main. Validation: not run (docs-only backlog clarification).
Implement the Phase B minimum slice with typed trace types, append-only runtime JSONL logging, redaction/truncation before write, and agent runtime event emission. Also fix the docs/dev README nano packaging dead link and record iteration logs.\n\nValidation: cargo fmt --all; cargo check -p agent-diva-core -p agent-diva-agent -p agent-diva-cli -p agent-diva-manager; cargo test -p agent-diva-core trace --lib; cargo test -p agent-diva-core validate --lib; cargo test -p agent-diva-agent structured_runtime_logs --lib; cargo test -p agent-diva-agent process_inbound_stops_on_repeated_failed_tool_call --lib; just fmt-check; just check; just test; cargo run -p agent-diva-cli -- config path --json
Validation: cargo fmt --all; cargo check -p agent-diva-core -p agent-diva-agent -p agent-diva-manager -p agent-diva-cli; cargo test -p agent-diva-core debug --lib; cargo test -p agent-diva-manager debug_bundle --lib; cargo test -p agent-diva-cli gateway_debug_run --bin agent-diva; cargo run -p agent-diva-cli -- gateway run --help; cargo run -p agent-diva-cli -- gateway bundle --help; just fmt-check; just check; just test.
Validation: inspected staged diff and verified only docs/dev/20260611-general-audit-main Markdown files are included. Runtime tests were not run because this is a documentation-only audit writeup.
Validation: git diff --check and git diff --cached --check passed. Rust build/test not run because this change only adds audit documentation.
Correct 19 occurrences of harmnes/harmness to harness across the V1.2 PRD tree and its cross-references. - Rename directory: docs/prds/prd-harmnes-v1.2 -> docs/prds/prd-harness-v1.2 - Files updated (5): - docs/prds/prd-harness-v1.2/prd.md (2 path refs) - docs/prds/prd-harness-v1.2/prd-channel-manager.md (2 path refs) - docs/prds/prd-harness-v1.2/prd-gui-minimal.md (1 path ref) - docs/research/v1.2-ref-zeroclaw-todo-system.md (1 path ref) - docs/research/v1.2-ref-design-methodology-and-exemplars.md (1 path ref) - Note: TODOLIST.md H-8 verification rule was rewritten to a generic 'spelling errors' description (no literal target strings) so future grep audits of [Hh]armne(?:ss|s) return 0 matches project-wide. - Initial import of V1.2 PRD docs and two v1.2-ref research docs with the corrected naming. The other three v1.2-ref-* docs (codex/exemplar/openfang) and the remaining Wave 0 work are left in the working tree for separate, focused commits. Validation: grep -rn '[Hh]armne\\(?:ss\\|s\\)' docs/ TODOLIST.md returns 0 matches.
…hFilesTool E10-S8: Add ErrorKind enum (6 variants) and ToolError::error_kind() classification mapping E10-S9: Add ToolError::is_retryable() and retry loop in ToolRegistry::execute() (3 attempts, 100ms/200ms backoff) E10-S10: Add ToolError::error_code() with stable TE-000~TE-004 codes E7-S2: Add SearchFilesTool with ripgrep priority, built-in fallback, glob/regex, rate limiting Verification: 25/25 tests pass (17 tooling + 8 search_files)
- ProvidersConfig 13→3 slots (anthropic + openai_compatible + custom_providers) - ProviderError full classification (Auth/Transient/Permanent/ToolSchema/RateLimited) - AnthropicDriver: native Messages API with thinking/computer-use/caching (4 files, 208 tests) - RetryPolicy wired to client.rs HTTP calls (pre-existing, verified) - stream_options.include_usage on DTO + SSE parsing - Model ID security: raw model for native endpoints - Remove LiteLLM prefix logic (converge to raw model IDs) - Unified Usage type + Session token_usage + rollup API - Subagent TokenUsed bus emit + merge - StopSession cancel_subagent cascade - Tool CancellationToken support - Unified wall clock (single subagent_timeout_secs) - Usage fallback: warn + estimated flag - ProviderFallbackLayer: model/cross-driver fallback, max depth 3, loop detection
Add agent-diva-e2e crate to workspace with: - Cargo.toml with workspace dependencies (agent-diva-agent, core, providers, tooling, tools) - E2EScenario, E2EAssertion, E2EMessage, E2ESetup types (YAML deserialization) - E2EConfig loaded from env vars (DEEPSEEK_API_KEY, E2E_API_BASE, E2E_MODEL, E2E_PROVIDER_NAME, etc.) - Graceful skip when no API key set - 7+7 unit tests covering all types and config env operations
EventCollector: - Captures all 8 AgentEvent variants via mpsc::unbounded_channel - CRITICAL: drop(tx) fix prevents infinite blocking (Metis finding) - tokio::time::timeout guard for channel closure - Tool call tracking by call_id with orphan handling - 5 unit tests (all variants, drop, timeout, tool tracking, orphan) Assertion engine: - 6 assertion types: response_contains, response_matches, tool_called, file_exists, no_errors, judge - LLM-as-Judge with temperature=0, JSON fail-safe, markdown extraction - evaluate_assertions() and evaluate_assertions_with_judge() public API - 20 unit tests (PASS+FAIL for each type + integration + extract_json)
ScenarioRunner: - YAML discovery, deserialization, and sequential execution - Temp workspace creation with setup file provisioning - LiteLLMClient/AgentLoop construction with configurable provider (E2E_PROVIDER_NAME) - Multi-turn message processing with per-turn event channel lifecycle (drop(tx) fix) - Event merge across turns, assertion evaluation, trace writing - build_provider() with configurable provider_name (not hardcoded) - 8 unit tests (provider, runner, merge, discovery) 7 YAML scenarios: - smoke (AgentLoop connectivity) - basic_math (arithmetic verification) - echo (simple response validation) - tool_call (list_dir tool invocation) - multi_turn (context preservation across 2 turns) - error_handling (graceful tool failure) - long_response (regex + response_matches) 7 integration tests: - All gracefully skip when DEEPSEEK_API_KEY not set - 60-90s timeouts with cfg(feature=ci) gating - Smoke test as canonical integration pattern (env!(CARGO_MANIFEST_DIR))
E2ETracer:
- JSON traces written to {trace_dir}/{scenario}_{timestamp}.json
- Per-run metadata: scenario name, timestamp, duration, passed/failed, assertions, errors, event count
- Auto-creates trace directory
- 4 unit tests (file creation, content format, failed scenario, auto-dir)
Report generator:
- Aggregates all trace JSONs into E2EReport
- Flaky detection: ≥3 runs with <80% pass rate flagged as flaky
- Handles empty directory, corrupted JSON, non-JSON files gracefully
- 11 unit tests (empty, pass/fail, mixed, flaky above/below threshold, corrupted)
Justfile (5 recipes): - e2e: full suite (--test-threads=1, --nocapture) - e2e-basic: basic_math + echo + smoke - e2e-tools: tool_call - e2e-advanced: multi_turn + error_handling + long_response - e2e-report: full suite with tee output - NOT included in ci recipe (E2E is opt-in) CI workflow (.github/workflows/e2e.yml): - Triggers on push/PR to main and agent-diva-pro/** branches - Guard: job skips if DEEPSEEK_API_KEY secret not set (fork-safe) - Rust stable + just, serial execution, trace artifact upload (7-day retention)
E10-S10: add error_code() to SecurityError (SE-001~SE-010) and ProviderError (PE-001~PE-005) E7-S4: add 6 named toolset groups (CORE, FILE, SHELL, WEB, BROWSER, CODE) with find_toolset() lookup
E7-S3: ProcessTool with 8 operations (submit/list/poll/log/wait/kill/write/close) - tokio::process::Command for async process spawning - Arc<Mutex<HashMap>> for thread-safe process storage - Background stdout/stderr reader tasks - max_concurrent limit (default 5) - Windows GB18030 pipe encoding fallback - 8 unit tests (spawn, list, log, kill, stdin, concurrent limit) sprint-status: mark E7-S3, E7-S4, E10-S10 as done
…gateTool, MCP fixes - feat(config): add code_execution and delegate flags to BuiltInToolsConfig - feat(tools): implement ExecuteCodeTool with Python subprocess sandbox - feat(tools): enhance SpawnTool with Semaphore concurrency and JSON output - feat(agent): register ExecuteCodeTool in ToolAssembly - fix(mcp): zombie processes in to_dto(), Drop impl, race protection - fix(mcp): unify MAX_TOOL_RESULT_CHARS to 80K, fix .len()->.chars(), dead code - fix(mcp): upgrade stderr log level based on keyword detection - fix(mcp): MCP Error: prefix for is_error propagation - feat(mcp): auto-reconnect with exponential backoff via McpReconnectManager - feat(mcp): hot reload support for MCP config changes 10 implementation tasks + 4 verification reviews. 113 tests pass.
- fix(core): re-export Usage from session/mod.rs and lib.rs - fix(agent): fallback_report fallthrough, subagent Usage type - fix(neuron): NeuronResponse.usage to Option<Usage> - fix(migration): adapt ProvidersConfig to new 3-slot schema - restore: builtin.rs, schema.rs, spawn.rs, loop_guard.rs - restore: tool_assembly.rs, mcp_sdk.rs, hot_reload.rs - restore: reload_plan.rs, mcp_service.rs, runtime.rs, chat_commands.rs All changes verified: cargo check --workspace = 0 errors
Validation: - cargo test -p agent-diva-core heartbeat -- --nocapture - cargo test -p agent-diva-cli --test config_commands -- --nocapture - cargo check -p agent-diva-core -p agent-diva-manager - just fmt-check Deferred: - just check and just test remain red due pre-existing workspace issues captured in TODOLIST H-8.
Detects double-encoded (%252f, %255c) and triple-encoded (%25252f, %25255c) path separators. Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
There was a problem hiding this comment.
Code Review
This pull request introduces significant enhancements to the agent-diva runtime, including a new end-to-end testing framework (agent-diva-e2e), a four-state user presence state machine, structured runtime and audit logging, atomic file writes, configuration migration and hot-reloading, and a 3-layer prompt injection and PII redaction system. The review feedback highlights several critical issues: a cross-platform bug where atomic writes fail on Windows if the destination file exists; a security vulnerability in path traversal detection due to an incomplete nested URL-encoding blocklist; a regex bug in JSON secret redaction when handling escaped quotes; a startup crash in read-only environments during config migration; an inconsistent state vulnerability on hot-reload failures; and a redundant presence event storm on the message bus during inbound message processing.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
|
||
| temp_file.write_all(content)?; | ||
| temp_file.as_file().sync_all()?; | ||
| temp_file.persist(path).map_err(|error| error.error)?; |
There was a problem hiding this comment.
On Windows, tempfile::NamedTempFile::persist will fail if the destination file already exists. Since atomic_write is used to repeatedly save session and configuration files, this will cause subsequent write operations to fail on Windows. To ensure cross-platform compatibility, the existing file should be removed first on Windows before persisting.
#[cfg(windows)]\n if path.exists() {\n let _ = std::fs::remove_file(path);\n }\n temp_file.persist(path).map_err(|error| error.error)?;| pub fn contains_url_encoded_traversal(path: &str) -> bool { | ||
| let lower = path.to_lowercase(); | ||
| // Single encoding: ..%2f, %2f.., ..%5c, %5c.. | ||
| lower.contains("..%2f") | ||
| || lower.contains("%2f..") | ||
| || lower.contains("..%5c") | ||
| || lower.contains("%5c..") | ||
| // Double encoding: %252f (→ %2f), %255c (→ %5c) | ||
| || lower.contains("%252f") | ||
| || lower.contains("%255c") | ||
| // Triple encoding: %25252f (→ %252f → %2f), %25255c (→ %255c → %5c) | ||
| || lower.contains("%25252f") | ||
| || lower.contains("%25255c") | ||
| } |
There was a problem hiding this comment.
The contains_url_encoded_traversal function attempts to block nested URL-encoded path traversal by checking for specific hardcoded patterns (single, double, and triple encoding). This blocklist approach is incomplete and can be bypassed using higher levels of nested encoding (e.g., quadruple encoding %2525252f) or other encoding schemes like UTF-8 overlong encoding. A more robust security practice is to recursively decode the path until no percent-encodings remain, and then validate the fully decoded path.
| static JSON_FIELD_RE: Lazy<Regex> = Lazy::new(|| { | ||
| Regex::new( | ||
| r#"(?ix) | ||
| (?P<key>"(?:api_key|token|secret|password|authorization)") | ||
| \s*:\s* | ||
| (?P<value>"[^"]*"|null) | ||
| "#, | ||
| ) | ||
| .expect("valid json field regex") | ||
| }); |
There was a problem hiding this comment.
The JSON_FIELD_RE regex uses \"[^\"]*\" to match JSON string values. If a secret value contains an escaped quote (e.g., \"api_key\": \"some\\\"secret\"), this regex will prematurely stop matching at the escaped quote, leaving the rest of the secret unredacted in the logs and producing malformed JSON. To ensure complete redaction and maintain valid JSON structure, use a regex that correctly handles escaped characters.
static JSON_FIELD_RE: Lazy<Regex> = Lazy::new(|| {\n Regex::new(\n r#"(?ix)\n (?P<key>"(?:api_key|token|secret|password|authorization)")\n \s*:\s*\n (?P<value>"(?:[^"\\\\]|\\\\.)*"|null)\n "#,\n )\n .expect("valid json field regex")\n});| if let Some(base_value) = persist_migrated_base { | ||
| let persisted_config: Config = serde_json::from_value(base_value)?; | ||
| validate_config(&persisted_config)?; | ||
| self.save(&persisted_config)?; | ||
| } |
There was a problem hiding this comment.
If a configuration migration is triggered on startup, load() attempts to save the migrated configuration back to disk via self.save(&persisted_config). In read-only environments (such as Docker/Kubernetes with read-only ConfigMaps or restricted user permissions), this write operation will fail with a PermissionDenied or ReadOnlyFilesystem error, causing the entire application to crash on startup. Consider handling the save error gracefully (e.g., logging a warning) so that the application can still run using the successfully migrated in-memory configuration.
if let Some(base_value) = persist_migrated_base {\n let persisted_config: Config = serde_json::from_value(base_value)?;\n if let Err(e) = validate_config(&persisted_config) {\n tracing::error!("Migrated config failed validation: {}", e);\n } else if let Err(e) = self.save(&persisted_config) {\n tracing::warn!("Failed to persist migrated config to disk: {}", e);\n }\n }| async fn check_and_reload(&self) -> crate::Result<()> { | ||
| let current_mtime = match std::fs::metadata(&self.config_path) { | ||
| Ok(metadata) => metadata.modified().ok(), | ||
| Err(_) => return Ok(()), // File doesn't exist or can't be read | ||
| }; | ||
|
|
||
| let last_mtime = *self.last_modified.read().await; | ||
|
|
||
| // Check if file was modified | ||
| if current_mtime == last_mtime { | ||
| return Ok(()); | ||
| } | ||
|
|
||
| info!("Config file change detected, reloading..."); | ||
|
|
||
| // Read and parse the new config | ||
| let content = std::fs::read_to_string(&self.config_path)?; | ||
| let new_config: Config = match serde_json::from_str(&content) { | ||
| Ok(config) => config, | ||
| Err(e) => { | ||
| error!(error = %e, "Failed to parse config file, keeping current config"); | ||
| // Update mtime to avoid re-parsing the same bad file | ||
| *self.last_modified.write().await = current_mtime; |
There was a problem hiding this comment.
In the hot-reload loop, if any module fails to reload (returning an Err), all_succeeded is set to false, and the global current_config is not updated. However, the modules that were processed before the failure have already applied the new configuration, and there is no rollback mechanism to revert them. This leaves the system in an inconsistent state where some modules are running with the new configuration while others are running with the old one, but the global state still reports the old configuration. Consider implementing a two-phase commit or at least logging a critical warning about the inconsistent state.
| self.refresh_presence(); | ||
| if msg.sender_id != "cron" && !msg.metadata.contains_key("cron_job_id") { | ||
| self.note_user_activity(); | ||
| } |
There was a problem hiding this comment.
Calling refresh_presence() immediately followed by note_user_activity() when publishing an inbound message causes a redundant double-transition event storm. For example, if the user has been inactive, refresh_presence() will transition the state from Active to Distracted (emitting an event), and then note_user_activity() will immediately transition it back to Active (emitting another event). These two transitions happen within microseconds, creating unnecessary noise on the message bus. Consider checking the elapsed state silently before resetting, or combining these operations.
…1 double-encoding (verified) Changes: - B-06: ActionTracker refactored to per-session keyed HashMap<String, Vec<Instant>> - B-11: double/triple URL encoding already detected in feat/harness-wave0 (verified) - B-16: unbounded_channel() replaced with channel(256) + try_send overflow handling - B-10: deferred - exec policy in external sandbox crate
No description provided.