Skip to content

perf(fs_watcher): fix _identify_actor lsof cache (~12-121x speedup) - #44

Merged
Raunplaymore merged 1 commit into
mainfrom
perf/lsof-cache-per-path-ttl
May 5, 2026
Merged

perf(fs_watcher): fix _identify_actor lsof cache (~12-121x speedup)#44
Raunplaymore merged 1 commit into
mainfrom
perf/lsof-cache-per-path-ttl

Conversation

@Raunplaymore

Copy link
Copy Markdown
Owner

Summary

v0.9 Track 1 deferred 항목 (_identify_bulk_actor lsof subprocess) 본격 조사. 측정 결과는 가설보다 훨씬 더 나쁨.

시나리오 Before After Speedup
repeat (1 path × 100) 33.9s 0.28s 121×
mixed (10 paths × 10) 36.0s 2.9s 12.5×
unique (100 unique) 33.6s 29.6s 1.13×

(v0.9 노트의 "1.4s/100 events"는 patch가 걸린 측정. 실제 lsof 부담은 macOS 환경에서 per-call 290~340ms.)

발견한 캐시 버그 2건

기존 _lsof_cache는 사실상 no-op였다.

  1. Negative result 미저장. if result.stdout.strip(): 가드 안에서만 캐시에 저장. 그런데 fs_events 대부분이 "이 파일 잡고 있는 프로세스 없음" 케이스라 빈 stdout → 저장 안 됨. cache_size=0 상태로 100-call burst 끝남.
  2. 단일 글로벌 TTL. _lsof_cache_time 하나로 모든 경로의 freshness 판정. 한 path의 subprocess 호출이 다른 path의 캐시 freshness까지 갱신하는 entanglement.

Fix

OrderedDict[str, tuple[int, str, float]]로 교체:

  • 엔트리별 timestamp (per-path TTL)
  • LRU bound _lsof_cache_max = 512
  • positive + negative 결과 모두 캐시 (가장 큰 이득)
  • TTL 2s → 30s — macOS lsof가 너무 느려서 10-path rotation이 2s 안에 한 바퀴를 못 돈다 (10 × 290ms ≈ 3s). 2s TTL은 cache가 영원히 cold. 30s는 "actor attribution은 약간 stale해도 OK" 허용범위 안.

Public API 변경 없음. Behavior 변경 없음 (캐시 hit 여부만 빨라짐).

Files

  • sentinel_mac/collectors/fs_watcher.py_identify_actor 재작성, 캐시 자료구조 변경 (~25줄 변경)
  • scripts/profile_lsof.py (신규) — 재현 가능한 측정 하네스. 기존 profile_workload.py는 lsof를 patch out 했었음
  • tests/test_lsof_cache.py (신규, 14 tests) — 구조 invariants / positive 캐싱 / negative 캐싱 / per-path TTL 독립성 / LRU 동작 / parse correctness
  • docs/perf/v0.11-lsof-2026-05-05.md (신규) — 베이스라인 / 분석 / 결과 / 향후 follow-up
  • CHANGELOG.md [Unreleased] Performance 항목

향후 작업 (별도 PR)

  • Phase 2 — directory-batch lsof. lsof +D <parent> -F pcn 한 번으로 디렉토리 내 모든 파일 커버. unique 시나리오 N×290ms → 290ms로. per-event actor identification 흐름을 미루는 변경이라 ADR 필요.
  • psutil alternative. subprocess 비용 없이 process_iter(['open_files']). macOS의 per-process syscall 비용 때문에 빨라질지는 측정 필요.

Verification

  • mypy clean (25 source files)
  • ruff clean (sentinel_mac + tests)
  • pytest 904 passed (890 + 14 신규)
  • python3 scripts/profile_lsof.py 본 환경 재현 결과는 위 표와 동일

Test plan

  • mypy / ruff / pytest local
  • 측정 스크립트 본 환경 재현
  • CI green 확인 후 머지

🤖 Generated with Claude Code

The prior cache was functionally a no-op for every realistic
workload. Two bugs:

1. Negative results were not cached — only `if result.stdout.strip()`
   stored entries. fs events mostly land on files with no holder
   (just-created tmp / write-then-close), so cache_size stayed 0
   after every burst and every call paid the macOS lsof cost
   (~290ms/call measured).
2. Single global TTL — `_lsof_cache_time` was bumped on every
   successful invocation and checked against that one timestamp for
   all paths, entangling cache freshness across unrelated entries.

Fix: per-entry (pid, name, timestamp) tuple in an OrderedDict, LRU
bound at 512, both positive and negative results cached under the
same TTL. TTL bumped from 2s to 30s — lsof on macOS is so slow that
even a 10-path rotation cannot complete a round in 2s
(10 * 290ms ≈ 3s), making any tighter TTL useless. 30s is well
within the "best-effort actor attribution" tolerance.

Measured speedup (scripts/profile_lsof.py):
- repeat (1 path × 100):   33.9s → 0.28s   (121x)
- mixed  (10 paths × 10):  36.0s → 2.9s    (12.5x)
- unique (100 unique):     33.6s → 29.6s   (1.13x)

`unique` is bound by lsof per-file cost; directory-batch is a Phase
2 follow-up that needs an ADR (changes the per-event contract).

Public API unchanged. 14 new unit tests cover positive/negative
caching, per-path TTL independence, LRU eviction, parse correctness.

mypy clean, ruff clean, 890 → 904 pytest pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Raunplaymore
Raunplaymore merged commit daf732e into main May 5, 2026
5 checks passed
@Raunplaymore Raunplaymore mentioned this pull request May 5, 2026
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant