fix(serve): resource governor for multi-instance system freezes by rajkumarsakthivel · Pull Request #142 · elara-labs/code-context-engine

rajkumarsakthivel · 2026-07-24T14:37:51Z

Summary

Fixes #139. Multiple cce serve processes (one per project per AI session) exhaust system memory and cause 10–20 minute desktop freezes on Linux. The reported case had 68 CCE processes with 2,179 threads consuming 5.4 GiB RSS, triggering kernel OOM and PSI pressure spikes above 85%.

This PR adds a resource governor (resource_governor.py) with four mechanisms:

ONNX Runtime thread caps: each cce serve process defaults to 2 intra-op threads instead of cpu_count. Override via CCE_ORT_THREADS env var or serve.max_ort_threads config key. Also sets TOKENIZERS_PARALLELISM=false to prevent the tokenizers library from spawning additional threads.
Per-project index lock: file-lock (fcntl.flock) so when multiple AI sessions spawn duplicate cce serve processes for the same project, only one indexes at a time. Others skip and retry on the next file-change event. No-op on Windows (advisory lock not available).
Linux PSI memory-pressure detection: reads /proc/pressure/memory and defers re-indexing when some avg10 exceeds 25%. This prevents CCE from contributing to the cascading page-cache thrashing described in the issue. No-op on non-Linux platforms.
Idle auto-shutdown: cce serve exits after 30 minutes of MCP inactivity (no call_tool invocations). This prevents zombie processes from accumulating when AI sessions disconnect or go idle. Configurable via CCE_IDLE_TIMEOUT_MINUTES env var, serve.idle_timeout_minutes config key, or set to 0 to disable.

New config keys

serve:
  idle_timeout_minutes: 30   # 0 = disabled
  max_ort_threads: 2         # 0 = use ONNX default

Impact on the reported scenario

With the defaults from this PR, the 68-process scenario from the issue would see:

Thread count reduced from ~2,179 to ~340 (68 × ~5 threads each vs. 68 × ~32)
Idle processes auto-shutdown after 30 minutes, reducing the steady-state count
Concurrent indexing serialized per-project, preventing I/O amplification
Memory-pressure detection pausing heavy work before the system enters OOM territory

…eezes (#139) Multiple cce serve processes (one per project per AI session) exhaust system memory and create 10-20 minute desktop freezes. Root cause: no thread caps, no idle shutdown, no cross-process coordination, no memory pressure awareness. Adds resource_governor.py with four mechanisms: - ONNX Runtime thread caps (default 2 per process, via CCE_ORT_THREADS) - Per-project file lock so duplicate instances don't index simultaneously - Linux PSI memory-pressure detection to pause indexing under pressure - Idle auto-shutdown after 30 minutes of MCP inactivity New config keys: serve.idle_timeout_minutes, serve.max_ort_threads New env vars: CCE_ORT_THREADS, CCE_IDLE_TIMEOUT_MINUTES

Copilot

Pull request overview

Adds a “resource governor” to cce serve to reduce multi-instance host overload (threads, indexing contention, and idle zombie processes), addressing issue #139’s reported Linux desktop freezes under heavy multi-process usage.

Changes:

Introduces resource_governor.py with ONNX/thread caps, per-project indexing lock, Linux PSI memory-pressure backoff, and idle auto-shutdown utilities.
Wires the governor into cce serve startup/reindex flow (thread caps, index lock, PSI backoff, and an idle watchdog).
Adds new serve.* config keys (idle_timeout_minutes, max_ort_threads) and maps them into Config.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
src/context_engine/resource_governor.py	New governor utilities (thread caps, lock, PSI check, idle tracking).
src/context_engine/integration/mcp_server.py	Tracks MCP tool activity to support idle shutdown.
src/context_engine/config.py	Adds and maps new `serve.*` configuration keys.
src/context_engine/cli.py	Applies governor behavior in `cce serve` (caps, lock/backoff, idle watchdog).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        try:
+            self._lock_path.parent.mkdir(parents=True, exist_ok=True)
+            fd = os.open(str(self._lock_path), os.O_CREAT | os.O_RDWR, 0o644)
+            fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+            self._fd = fd
+            return True
+        except (OSError, BlockingIOError):
+            return False


+    raw = os.environ.get("CCE_ORT_THREADS", "").strip()
+    if raw:
+        try:
+            n = int(raw)
+        except ValueError:
+            n = _DEFAULT_ORT_THREADS
+    elif max_threads is not None:
+        n = max_threads
+    else:
+        n = _DEFAULT_ORT_THREADS
+
+    if n <= 0:
+        return 0  # 0 = "let ONNX pick"
+
+    for var in (
+        "OMP_NUM_THREADS",
+        "MKL_NUM_THREADS",
+        "OPENBLAS_NUM_THREADS",
+        "ORT_NUM_THREADS",       # some ORT builds read this directly
+    ):
+        os.environ.setdefault(var, str(n))
+
+    # tokenizers library (used by fastembed) also spawns threads
+    os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")


+                # Back off under memory pressure (#139).
+                if is_memory_pressured():
+                    _log.info(
+                        "Memory pressure detected; deferring re-index of %s",
+                        rel,
+                    )
+                    _reindex_queue.task_done()
+                    await asyncio.sleep(30)
+                    continue


+    # Resource governor (#139) — caps per-process ONNX Runtime threads and
+    # auto-shuts down idle servers so zombie processes don't accumulate.
+    # 0 = disabled (no auto-shutdown / use ORT default threads).
+    serve_idle_timeout_minutes: int = 30
+    serve_max_ort_threads: int = 2


- Fix FD leak in ProjectIndexLock.try_acquire when flock fails - Force-set ORT thread env vars instead of setdefault - Re-queue reindex files under memory pressure instead of dropping - Fix docstring mechanism count (three → four) - Add config tests for serve.idle_timeout_minutes and serve.max_ort_threads

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

src/context_engine/resource_governor.py:81

When CCE_ORT_THREADS (or serve.max_ort_threads) is set to 0/negative to disable ORT caps, this returns early and never sets TOKENIZERS_PARALLELISM=false. That contradicts the PR’s stated goal of preventing tokenizers from spawning extra threads, and can reintroduce high thread counts even when ORT caps are disabled.

    if n <= 0:
        return 0  # 0 = "let ONNX pick"

src/context_engine/cli.py:3525

If the per-project index lock is held by another process, this worker drops the reindex request entirely. If the other process misses the same filesystem event (watcher queue overflow, transient watcher failure, etc.), the project index can remain stale until a later change happens to the same file. Consider re-queuing with a small backoff (similar to the PSI-pressure path) so every change is eventually indexed once the lock becomes available.

                if not index_lock.try_acquire():
                    _log.debug(
                        "Another process is indexing this project; "
                        "skipping %s", rel,
                    )

rajkumarsakthivel added 6 commits July 3, 2026 21:30

docs: add C# to AST-aware supported languages in README

8c00040

Merge branch 'main' of https://github.com/elara-labs/code-context-engine

a921ab5

Merge branch 'main' of https://github.com/elara-labs/code-context-engine

c8e4476

Merge branch 'main' of https://github.com/elara-labs/code-context-engine

f9c7ea1

Merge branch 'main' of https://github.com/elara-labs/code-context-engine

4b766e7

rajkumarsakthivel requested a review from fazleelahhee as a code owner July 24, 2026 14:37

rajkumarsakthivel mentioned this pull request Jul 24, 2026

Extreme (300x) load / system-wide freeze with multiple cce serve instances #139

Open

fazleelahhee previously approved these changes Jul 27, 2026

View reviewed changes

fazleelahhee requested a review from Copilot July 27, 2026 11:18

Copilot started reviewing on behalf of fazleelahhee July 27, 2026 11:18 View session

Copilot AI reviewed Jul 27, 2026

View reviewed changes

rajkumarsakthivel dismissed fazleelahhee’s stale review via e2be31a July 27, 2026 11:31

rajkumarsakthivel requested a review from fazleelahhee July 27, 2026 11:38

fazleelahhee requested a review from Copilot July 28, 2026 12:14

Copilot started reviewing on behalf of fazleelahhee July 28, 2026 12:14 View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(serve): resource governor for multi-instance system freezes - #142

fix(serve): resource governor for multi-instance system freezes#142
rajkumarsakthivel wants to merge 7 commits into
mainfrom
fix/resource-governor-139

rajkumarsakthivel commented Jul 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

rajkumarsakthivel commented Jul 24, 2026

Summary

New config keys

Impact on the reported scenario

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants