feat: detect and parse q27 (signalnine/q27 engine) LLM backend

CHANGELOG.md

@@ -10,6 +10,7 @@ Format: version sections are listed newest first.
 ## [Unreleased]
 
 ### Added
+- **q27 LLM backend** — detect and parse the signalnine/q27 engine (`/v1/models` `owned_by: "q27"` or Prometheus `q27_*` series) instead of mislabeling it as vLLM. Live tok/s from `q27_*_processed` counter diffs (real-time during generation; completion-based per-api totals as fallback for older binaries), exact computed-only prefill whose cached/uncached split doubles as the prefix-cache hit rate, KV usage, slots, TTFT/E2E/ITL p95 histograms, MTP accept, engine Active state, and constant-0 preemptions (FIFO admission — no wait queue, the Requests tile reads “N run”). Inference-health tiles render for q27 like vLLM. DecodeBench server-side tok/s reads the same q27 counters.
 - **DGX Spark CPU temperature** — remote Sparks collect CPU temp over SSH with the same hwmon allowlist as hosts (`acpitz` / `coretemp` / `k10temp` / `zenpower`; NVMe / CX7 filtered out). Overview shows a CPU bar and Spark pages show a CPU row on the GPU panel when the reading is above 0°C.
 
 ### Fixed

README.md

@@ -67,13 +67,13 @@ Full history: [CHANGELOG.md](./CHANGELOG.md)
 | **Non-Spark GPU hosts** | Linux boxes with a dedicated NVIDIA GPU are first-class units: same `nvidia-smi` collectors over SSH, detected hardware summary, and separate **RAM** / **VRAM** panels. Detail page: GPU (left) + **RAM → Network → Storage** (right column); Overview cards show RAM and VRAM bars |
 | **Live streaming** | WebSocket metrics with configurable poll intervals; central history store for sparklines across tab switches |
 | **Local + remote** | Host metrics via sysfs/proc/`nvidia-smi`; remotes over SSH (key or password) |
-| **LLM probe** | Auto-detects llama.cpp, vLLM, sglang, ds4-server, or EXL3; live decode/prefill tok/s; cached vs uncached prefill on ds4, llama.cpp, and SGLang; **daily peak** history on the LLM card |
+| **LLM probe** | Auto-detects llama.cpp, vLLM, sglang, ds4-server, EXL3, or q27; live decode/prefill tok/s; cached vs uncached prefill on ds4, llama.cpp, SGLang, and q27; **daily peak** history on the LLM card |
 | **ComfyUI** | Opt-in probe: queue/jobs, progress, cancel, Open link, inventory, overview chip |
 | **Hermes Agent** | Opt-in per unit: background update check (10 min), status badges, one-click or batch `hermes update` |
 | **Tailnet** | Opt-in probe: flags a unit that is healthy on the LAN but off its tailnet |
 | **Decode benchmark** | Multi-concurrency streaming decode tok/s; type picker (Structured / Prose / Code / JSON); lab protocol (temp 0, thinking off); persisted last run |
 | **Prompt Showcase** | Full-page multi-terminal LLM streaming demo (up to 32 prompts) with live tok/s and copy-out |
-| **vLLM health** | KV cache %, run/wait queue, TTFT/E2E/ITL p95, preemptions, prefix cache, MTP accept from Prometheus `/metrics` |
+| **LLM inference health** | KV cache %, run/wait queue, TTFT/E2E/ITL p95, preemptions, prefix cache, MTP accept from Prometheus `/metrics` (vLLM and q27; q27 FIFO-queues so the Requests tile reads “N run” without a wait gauge) |
 | **Multiple LLM ports** | Monitor several LLM servers on different ports simultaneously — each gets its own panel with independent backend detection and metrics |
 | **GPU processes** | See the top GPU processes by VRAM usage directly in the GPU panel, including process name and memory allocation |
 | **Spark uptime** | System uptime displayed inline on each Spark header for at-a-glance availability |
@@ -495,6 +495,8 @@ Each configured LLM port gets its own `LlmProbe` instance running in parallel. P
 
 - **llama.cpp** — `/slots` for live decode rates; model from `/props`
 - **ds4-server** (Entrpi/ds4-on-spark) — `/v1/models` (`owned_by: ds4.c`) + Prometheus `ds4_*` token counters for live tok/s
+- **EXL3** (ExLlamaV3 `tools/serve_openai.py`) — `/v1/models` (`owned_by: exl3`) or `/health` `{ok, busy}`; live tok/s from `/health` cumulative counters
+- **q27** (signalnine/q27 engine) — `/v1/models` (`owned_by: q27`) or Prometheus `q27_*` series; live tok/s from `q27_*_processed` counter diffs (completion-based totals as fallback), exact computed-only prefill with the cached/uncached split doubling as the prefix-cache hit rate, TTFT/E2E/ITL p95 histograms, and constant-0 preemptions (FIFO admission, no wait queue)
 - **vLLM / sglang** — `/v1/models`; sglang via `/server_info` (`last_gen_throughput` when metrics off; `/get_server_info` fallback), vLLM via Prometheus `/metrics` counters (scientific notation supported)
 
 Rates are derived from per-probe cumulative counter diffs (or SGLang sticky throughput while it moves). Multiple ports can be added or removed at runtime without restarting the monitor.

server/collectors/LlmProbe.js

@@ -66,7 +66,7 @@ export class LlmProbe {
     this.baseUrl = `http://${llmProbeHost(spark)}:${port}`;
 
     // State
-    this.backendType = null; // 'vllm' | 'llama.cpp' | 'sglang' | 'ds4' | 'exl3' | null
+    this.backendType = null; // 'vllm' | 'llama.cpp' | 'sglang' | 'ds4' | 'exl3' | 'q27' | null
     this.serverIsOpenAI = null; // true = OpenAI-compatible
     /** Whether /v1/models (or /slots) answered without credentials. null = unknown. */
     this.authOpen = null;
@@ -279,7 +279,8 @@ export class LlmProbe {
       this.backendType !== "vllm" &&
       this.backendType !== "sglang" &&
       this.backendType !== "ds4" &&
-      this.backendType !== "exl3"
+      this.backendType !== "exl3" &&
+      this.backendType !== "q27"
     ) {
       const slotUrl = `${this.baseUrl}/slots`;
       try {
@@ -326,19 +327,22 @@ export class LlmProbe {
   }
 
   /**
-   * Classify an OpenAI-compatible server: ds4, SGLang, EXL3, or vLLM (default).
+   * Classify an OpenAI-compatible server: ds4, SGLang, EXL3, q27, or vLLM (default).
    * @param {unknown} ownedBy
-   * @returns {Promise<"ds4" | "sglang" | "exl3" | "vllm">}
+   * @returns {Promise<"ds4" | "sglang" | "exl3" | "q27" | "vllm">}
    */
   async _classifyOpenAIBackend(ownedBy) {
     if (typeof ownedBy === "string") {
       if (/ds4/i.test(ownedBy)) return "ds4";
       if (/sglang/i.test(ownedBy)) return "sglang";
       if (/exl3/i.test(ownedBy)) return "exl3";
+      // q27's /v1/models reports owned_by: "q27" (signalnine/q27 engine).
+      if (/q27/i.test(ownedBy)) return "q27";
     }
     if (await this._probeIsDs4()) return "ds4";
     if (await this._probeIsSglang()) return "sglang";
     if (await this._probeIsExl3()) return "exl3";
+    if (await this._probeIsQ27()) return "q27";
     return "vllm";
   }
 
@@ -406,6 +410,25 @@ export class LlmProbe {
     return /(?:^|\n)ds4_tokens_decoded_total(?:\{|\s)/m.test(String(body || ""));
   }
 
+  /** True when Prometheus /metrics exposes q27-series (signalnine/q27 engine). */
+  async _probeIsQ27() {
+    try {
+      const res = await this._fetch(`${this.baseUrl}/metrics`);
+      if (!res.ok) return false;
+      const txt = await res.text();
+      return LlmProbe._metricsLookLikeQ27(txt);
+    } catch {
+      return false;
+    }
+  }
+
+  /** @param {string} body */
+  static _metricsLookLikeQ27(body) {
+    return /(?:^|\n)q27_(?:decode_tokens_(?:processed_)?total|requests_total)(?:\{|\s)/m.test(
+      String(body || "")
+    );
+  }
+
   // ─── OpenAI-compatible path (vLLM/sglang/ds4) ────────────
   async _probeOpenAICompatible() {
     const now = Date.now();
@@ -512,15 +535,31 @@ export class LlmProbe {
         ) {
           this.backendType = "ds4";
           this._applyDs4Metrics(txt, dtSec);
+        } else if (
+          this.backendType === "q27" ||
+          LlmProbe._metricsLookLikeQ27(txt)
+        ) {
+          this.backendType = "q27";
+          this._applyQ27Metrics(txt, dtSec);
         } else {
           this.backendType = "vllm";
           this._applyVllmMetrics(txt, dtSec);
         }
-      } else if (this.backendType !== "ds4" && this.backendType !== "exl3") {
+      } else if (
+        this.backendType !== "ds4" &&
+        this.backendType !== "exl3" &&
+        this.backendType !== "q27"
+      ) {
         this.backendType = "vllm";
       }
     } catch {
-      if (this.backendType !== "ds4" && this.backendType !== "exl3") this.backendType = "vllm";
+      if (
+        this.backendType !== "ds4" &&
+        this.backendType !== "exl3" &&
+        this.backendType !== "q27"
+      ) {
+        this.backendType = "vllm";
+      }
     }
 
     return this._getSnapshot();
@@ -613,6 +652,108 @@ export class LlmProbe {
     this.itlP95Seconds = null;
   }
 
+  /**
+   * Apply q27 (signalnine/q27 engine) Prometheus /metrics.
+   *
+   * The engine exposes the [req]-universe telemetry that was previously only
+   * in stderr logs, under q27_* series with an api= label (chat / completions /
+   * messages / responses). The probe sums across label sets, exactly like the
+   * ds4/vLLM paths: live tok/s from counter deltas so idle → 0.
+   *
+   * Semantics vs the vLLM path: prefill accounting is EXACT (per-request
+   * token counts, not vLLM's estimates) and the prefix split (computed/cached)
+   * doubles as the prefix-cache hit rate; the main prefill tile follows the
+   * ds4 convention and counts COMPUTED tokens only. The waiting tile stays
+   * null → the panel shows "—" (q27 FIFO-queues, no scheduler wait), while
+   * preemptions are exposed as a constant-0 counter so Preempts reads 0.
+   * @param {string} txt
+   * @param {number} dtSec
+   */
+  _applyQ27Metrics(txt, dtSec) {
+    // Live processed counters first (move during generation -> real-time
+    // tok/s); fall back to the completion-based per-api totals for older
+    // q27 binaries (step function: 0 during generation, jump at completion).
+    const decoded =
+      this._getPromMetric(txt, "q27_decode_tokens_processed_total") ??
+      this._getPromMetric(txt, "q27_decode_tokens_total");
+    // Exact prefill (not estimated like vLLM). Follow the ds4 convention:
+    // the main prefill tile counts COMPUTED tokens only -- cache-served
+    // tokens go to the cached/uncached split below, so a cache hit does not
+    // inflate the "real work" rate.
+    const computed =
+      this._getPromMetric(txt, "q27_prefill_computed_tokens_processed_total") ??
+      this._getPromMetric(txt, "q27_prefill_computed_tokens_total");
+    if (decoded != null) {
+      if (dtSec > 0 && dtSec < 10) {
+        const deltaOut = decoded - this.lastTokenCounts.output;
+        this.generationTps = Math.max(0, Math.round((deltaOut / dtSec) * 100) / 100);
+        if (computed != null) {
+          const deltaIn = computed - this.lastTokenCounts.input;
+          this.prefillTps = Math.max(0, Math.round((deltaIn / dtSec) * 100) / 100);
+        }
+      }
+      if (computed != null) this.lastTokenCounts.input = computed;
+      this.lastTokenCounts.output = decoded;
+      this.totalOutputTokens = decoded;
+    }
+
+    const inflight = this._getPromMetric(txt, "q27_requests_inflight");
+    this.requestsRunning = inflight;
+    if (inflight != null) this.slotsActive = Math.round(inflight);
+
+    const slotsTotal = this._getPromMetric(txt, "q27_slots_total");
+    if (slotsTotal != null) this.slotsTotal = Math.round(slotsTotal);
+
+    this.kvCacheUsage = this._getPromMetric(txt, "q27_kv_usage_perc");
+    this.requestsWaiting = null; // not exposed: q27 FIFO-queues, no wait gauge
+    // q27 never preempts (FIFO admission) — the server exposes a constant-0
+    // counter, so the Preempts tile reads 0 instead of "—".
+    this.preemptionsTotal = this._getPromMetric(txt, "q27_preemptions_total");
+    // Engine state: q27 keeps weights resident and is ready whenever the
+    // server is up (no sleep state / memory release), so report Active like
+    // the SGLang path does.
+    if (this.gpuMemoryUtilization == null) this.gpuMemoryUtilization = 1;
+
+    // Histograms (cumulative buckets, +Inf == _count by construction).
+    const ttftHist = this._parseHistogram(
+      txt,
+      "q27_ttft_seconds",
+      "q27_ttft_seconds_count"
+    );
+    const ttftP95 = this._histogramQuantile(ttftHist.buckets, ttftHist.total, 0.95);
+    this.ttftP95Seconds = ttftP95 == null ? null : Math.round(ttftP95 * 1000) / 1000;
+
+    const e2eHist = this._parseHistogram(
+      txt,
+      "q27_e2e_seconds",
+      "q27_e2e_seconds_count"
+    );
+    const e2eP95 = this._histogramQuantile(e2eHist.buckets, e2eHist.total, 0.95);
+    this.e2eP95Seconds = e2eP95 == null ? null : Math.round(e2eP95 * 1000) / 1000;
+
+    const itlHist = this._parseHistogram(
+      txt,
+      "q27_itl_seconds",
+      "q27_itl_seconds_count"
+    );
+    const itlP95 = this._histogramQuantile(itlHist.buckets, itlHist.total, 0.95);
+    this.itlP95Seconds = itlP95 == null ? null : Math.round(itlP95 * 1000) / 1000;
+
+    // Prefix-cache hit rate + live cached/uncached prefill split (live
+    // processed counters, with completion-based fallback).
+    const cachedSplit =
+      this._getPromMetric(txt, "q27_prefill_cached_tokens_processed_total") ??
+      this._getPromMetric(txt, "q27_prefill_cached_tokens_total");
+    const computedSplit =
+      this._getPromMetric(txt, "q27_prefill_computed_tokens_processed_total") ??
+      this._getPromMetric(txt, "q27_prefill_computed_tokens_total");
+    this._setPrefillSplitRates(cachedSplit, computedSplit, dtSec);
+
+    const specAccept = this._getPromMetric(txt, "q27_spec_accept_ratio");
+    this.mtpAcceptanceRate =
+      specAccept != null ? Math.round(specAccept * 10000) / 10000 : null;
+  }
+
   /**
    * Apply EXL3 tools/serve_openai.py GET /health.
    * Live tok/s from cumulative counter diffs so idle → 0.
@@ -1212,11 +1353,12 @@ export class LlmProbe {
   }
 
   /**
-   * Parse a vLLM Prometheus histogram from /metrics text.
-   * Returns { buckets: [{upper, count}], total } with cumulative counts per `le`,
-   * summed across label sets. `total` is the summed `_count` series (or null).
+   * Parse a Prometheus histogram from /metrics text.
+   * Returns { buckets: [{upper, count}], total } with cumulative counts per
+   * `le`, summed across label sets. `total` is the summed `_count` series
+   * (or null). `countMetricName` is the full metric name (with prefix).
    */
-  _parseVllmHistogram(body, metricPrefix) {
+  _parseHistogram(body, metricPrefix, countMetricName) {
     const esc = metricPrefix.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
     // Bucket lines: <metricPrefix>_bucket{...le="X"...} VALUE
     const bucketRe = new RegExp(
@@ -1235,7 +1377,7 @@ export class LlmProbe {
       if (upper === Infinity) infCount += count;
       byUpper.set(upper, (byUpper.get(upper) || 0) + count);
     }
-    const total = this._getVllmMetric(body, `${metricPrefix.replace(/^vllm:/, "")}_count`);
+    const total = this._getPromMetric(body, countMetricName);
     // Prometheus invariant: +Inf bucket count == _count. Mismatch → refuse quantile.
     if (total != null && infCount > 0 && Math.abs(infCount - total) > 1e-6) {
       return { buckets: [], total: null };
@@ -1245,6 +1387,14 @@ export class LlmProbe {
     return { buckets, total };
   }
 
+  /**
+   * Parse a vLLM Prometheus histogram from /metrics text (vllm: prefix).
+   */
+  _parseVllmHistogram(body, metricPrefix) {
+    const name = metricPrefix.replace(/^vllm:/, "");
+    return this._parseHistogram(body, metricPrefix, `vllm:${name}_count`);
+  }
+
   /**
    * Prometheus-style linear interpolation for a histogram quantile.
    * Returns null when empty / invalid or target is in the +Inf tail.

server/collectors/LlmStreaming.js

@@ -94,6 +94,16 @@ export async function readServerGenerationTokens(baseUrl, opts = {}) {
           /^sglang_generation_tokens_total(?:\{[^}]*\})?\s+([\d.eE+-]+)\s*$/gm
         );
       if (sglang != null) return sglang;
+      // q27 (signalnine/q27 engine) — live processed counter first, then the
+      // completion-based per-api total (same preference as LlmProbe).
+      const q27 =
+        fromSeries(
+          /^q27_decode_tokens_processed_total(?:\{[^}]*\})?\s+([\d.eE+-]+)\s*$/gm
+        ) ??
+        fromSeries(
+          /^q27_decode_tokens_total(?:\{[^}]*\})?\s+([\d.eE+-]+)\s*$/gm
+        );
+      if (q27 != null) return q27;
     }
   } catch {
     /* try next */

server/collectors/SystemCollector.js

@@ -561,7 +561,7 @@ export class SystemCollector {
     const lines = output.trim().split("\n").filter(Boolean);
     const disks = [];
     const disabledDevices = this.spark.disabledDevices || [];
-    const PSEUDO = new Set(["tmpfs", "devtmpfs", "proc", "sysfs", "efivarfs", "squashfs", "overlay", "devpts", "cgroup", "cgroup2"]);
+    const PSEUDO = new Set(["tmpfs", "devtmpfs", "proc", "sysfs", "efivarfs", "squashfs", "overlay", "devpts", "cgroup", "cgroup2", "swap"]);
 
     for (const line of lines) {
       const nameMatch = line.match(/NAME="([^"]*)"/);
@@ -1448,7 +1448,10 @@ export class SystemCollector {
         });
       }
     }
-    return this._readHostFile(`/proc/net/${relPath}`);
+    // No host bind mount: read straight from the local /proc/net.
+    // Do NOT re-dispatch through _readHostFile here — it routes /proc/net/*
+    // back into this method, causing infinite recursion.
+    return fs.readFileSync(`/proc/net/${relPath}`, "utf-8");
   }
 
   /** Lightweight liveness for local Sparks. */