Summary
Add a workspace-level option (settable at Create workspace, editable later) that forces OCR re-processing for files whose OCR confidence falls below a threshold, rather than trusting a low-confidence text layer.
Motivation: a scanned PDF (or image) with a poor or partial embedded text layer can pass through detection with weak coverage, causing PII to be missed. When enabled, any file below the confidence threshold is (re-)OCR'd so downstream text recognizers match on reliable text.
Current state
- Workspaces already have a settings bag.
workspaces.settings JSONB (2B–8KB) — migrations/2025-05-21-222840_workspaces/up.sql. This option belongs there; no new column needed.
- DTOs exist.
CreateWorkspace / UpdateWorkspace in crates/nvisy-server/src/handler/request/workspaces.rs — the option would surface on both.
- Engine has OCR. The runtime exposes an OCR enricher (
OcrEnricherParams, image / PDF-raster modality) with a confidence concept — nvisy-schema plan/enricher.rs. Detection can already run OCR; this issue is about a workspace policy that mandates it below a confidence bar.
Proposal
- Setting — add a field to the workspace settings, e.g.:
A threshold (0.0–1.0) doubles as the on/off switch: absent = disabled.
- Create/Update surface — expose it on
CreateWorkspace and UpdateWorkspace, persisted into settings.
- Enforcement at detect — when a run's file reports OCR confidence below the workspace threshold, force the OCR enricher (re-OCR) before recognition instead of relying on the existing/low-confidence text layer.
Open questions
- Where does "OCR confidence" come from for a not-yet-OCR'd file? Need to confirm what the engine reports (per-page vs whole-doc, embedded-text-layer confidence vs OCR-output confidence) and at what point the threshold is evaluated. Depends on
OcrEnricherParams / engine capabilities — verify before implementing.
- Threshold granularity — single workspace-wide value, or per-file-type?
- Interaction with pipeline config — a pipeline may already toggle the OCR enricher. Precedence: does the workspace floor override/force it even when the pipeline left OCR off? (Likely yes — that's the point of a workspace-level mandate.)
- Cost/latency — forced re-OCR is expensive; surface it as opt-in (default off) and consider emitting an activity event when it triggers.
- Applies to image + PDF-raster modality only (OCR enricher scope); no-op for born-digital text.
Out of scope
- Changing the engine's OCR implementation; this is a workspace-level gating policy on top of existing OCR.
🤖 Generated with Claude Code
Summary
Add a workspace-level option (settable at Create workspace, editable later) that forces OCR re-processing for files whose OCR confidence falls below a threshold, rather than trusting a low-confidence text layer.
Motivation: a scanned PDF (or image) with a poor or partial embedded text layer can pass through detection with weak coverage, causing PII to be missed. When enabled, any file below the confidence threshold is (re-)OCR'd so downstream text recognizers match on reliable text.
Current state
workspaces.settings JSONB(2B–8KB) —migrations/2025-05-21-222840_workspaces/up.sql. This option belongs there; no new column needed.CreateWorkspace/UpdateWorkspaceincrates/nvisy-server/src/handler/request/workspaces.rs— the option would surface on both.OcrEnricherParams, image / PDF-raster modality) with a confidence concept —nvisy-schemaplan/enricher.rs. Detection can already run OCR; this issue is about a workspace policy that mandates it below a confidence bar.Proposal
{ "require_ocr_below_confidence": 0.80 } // null/absent = offCreateWorkspaceandUpdateWorkspace, persisted intosettings.Open questions
OcrEnricherParams/ engine capabilities — verify before implementing.Out of scope
🤖 Generated with Claude Code