feat(observability): add global logs, traces, errors and analytics APIs

[dviejokfs]

Instance-wide logs, traces, errors, and analytics can now be queried through global endpoints. Authorization, filtering, ordering, totals, and pagination are applied in the backend, so clients can load an instance view without assembling per-project responses.

Scope

• POST /logs/global/search: ordered application/database archive search with query-bound cursors, explicit scan limits, and access restrictions.
• GET /otel/global/trace-summaries and GET /otel/global/spans: global trace reads across local/Cloud storage with bounded streaming merges. Unscoped span statistics use batched project metadata reads.
• GET /error-groups: paginated issue groups with project identity and time-window event/user counts.
• GET /analytics/global and /analytics/global/projects: aggregate analytics and accessible project choices.
• Register the global log-chunk index and regenerate the web OpenAPI SDK.

This branch is based on main and contains no console redesign or design-system changes.

Load and limits

These are control-plane reads. Log search allows two active requests with no queue (429 at saturation), a 30-second timeout, at most 512 chunks / 64 MiB compressed bytes per request, 8 MiB per compressed chunk and 16 MiB per decompressed chunk. Results are capped at 100 rows; a budget-exhausted search explicitly reports an incomplete scan rather than returning a misleading partial page. Chunk timestamp bounds now cover out-of-order arrivals.

Trace merging retains the requested page and one lookahead row per storage source. Mixed-source deep offsets stream preceding rows, so memory and source-request count stay bounded while work still grows with offset. Metadata scope storage scales with project count. Analytics results are paginated (up to 100 rows), traffic series are bounded, and windows are capped at 90 days; aggregate query cost depends on matching events.

Evidence

• Built and started this main-based branch against a new, isolated TimescaleDB database.
• Authenticated requests to all six extracted routes returned 200 with valid empty-instance responses. The same six requests without authentication returned 401.
• The actual migrator recorded m20260909_000001_index_global_log_chunks, and PostgreSQL confirmed the index exists. The migration's exact create/drop SQL also passed against a temporary table; zero indexes remained after down.
• Regenerated the web SDK from that server with bun run openapi-ts; ./node_modules/.bin/tsc --noEmit passed afterward.

Focused global tests:

TEST_DATABASE_URL=<isolated PostgreSQL> cargo test -p temps-analytics -p temps-error-tracking -p temps-log-aggregator -p temps-otel --lib global -- --nocapture
analytics: 3 passed; errors: 3 passed; logs: 2 passed; traces: 6 passed

The PostgreSQL tests exercised aggregate sorting/filtering, hidden projects, deep pagination, service ownership, query-bound log cursors, and scan budgets. The trace merge tests exercised mixed sources, exact totals, source failures, and a 105-project catalog with exactly two metadata queries.

Additional compiled log tests passed: standalone/deployment-token access (2), out-of-order chunk bounds (1), and filesystem range reads (1).

cargo check --lib -p temps-analytics -p temps-error-tracking -p temps-log-aggregator -p temps-otel passed. Source attribution passed for 3,772 source files, and the staged diff passed whitespace checks.

Trace storage integration:

cargo test -p temps-otel --test global_traces_postgres_test --test clickhouse_trace_summaries_test global -- --nocapture
3 passed, 8 filtered out (2 suites)

This covers PostgreSQL summary/raw-span scope consistency, ClickHouse global sorting/pagination, and Cloud-schema pagination after aggregation.

cargo clippy --all-targets --all-features -- -D warnings passed. The commit hooks passed, including formatting, Clippy, whitespace and typos; the commit includes a DCO sign-off.

Local setup note: main's web/bun.lock currently has a duplicate console-kit entry that prevents frozen installation. Dependencies were installed using a temporary local repair; the original lockfile was restored and is not part of this PR.

Review fixes

• Both global trace responses now expose per-project storage source, effective start/end, and window_clamped_at. This preserves the existing ADR-040/041 source-selection contract while making shortened windows explicit, including empty pages. Regression tests cover both cutover directions and both response shapes.
• Global Speed applies the same hosting-provider exclusion as project performance before counting and calculating percentiles. A PostgreSQL regression mixes ordinary, hosting-provider, crawler, unknown-IP and missing-IP samples, with project/environment scope checks.
• Rebuilt and restarted the isolated server, verified all six endpoints return 200, regenerated the web SDK, and passed web TypeScript, all 14 focused global tests, full all-target/all-feature Clippy, and source attribution checks.

[github-actions]

📓 Changelog preview

This is what your commits will add to the generated CHANGELOG.md at release time (via git-cliff). Do not edit CHANGELOG.md by hand — it is generated from your Conventional Commit messages.

## [Unreleased]

### Added

- **observability:** Add global logs traces errors and analytics endpoints

### Fixed

- **observability:** Disclose trace cutovers and filter hosting-provider speed samples

[greptile-apps]

Greptile Summary

The follow-up adds explicit per-project source and effective-window metadata to both global trace response shapes and aligns global Speed filtering with project-level performance behavior.

• Global trace summaries and spans now disclose clamped windows even when result pages are empty.
• Global Speed excludes crawler and hosting-provider samples before calculating counts and percentiles while retaining unknown or missing geolocation samples.
• Focused regression tests cover both trace cutover directions, both response shapes, filtering categories, and project/environment scope.

Confidence Score: 5/5

The PR appears safe to merge because both previously reported blocking failures are resolved.

No blocking failure remains; trace truncation is explicitly disclosed through per-project effective-window metadata in both response shapes, and global Speed now applies the same hosting-provider exclusion as project performance.

Important Files Changed

Filename	Overview
crates/temps-otel/src/services/telemetry_write_mode.rs	Resolves each project's effective trace window and records the selected source and cutover clamp metadata in both transition directions.
crates/temps-otel/src/handlers/global_traces.rs	Exposes the resolved source, effective bounds, and clamp timestamp in both global trace response types, including empty pages.
crates/temps-analytics/src/global.rs	Applies the established hosting-provider exclusion before global Speed counts and percentile aggregation, with focused regression coverage.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Q[Global trace request] --> S[Resolve project storage scopes]
  S --> L[Local effective window]
  S --> C[Cloud effective window]
  L --> M[Merge bounded results]
  C --> M
  M --> R[Response rows and per-project window metadata]

Loading

_{Reviews (2): Last reviewed commit: "fix(observability): disclose trace cutov..." | Re-trigger Greptile}

[dviejokfs]

@greptile Please re-review commit 96e5b35. Both P1 findings are addressed:

• Global trace summaries and spans expose per-project source, effective time range, and window_clamped_at, including empty pages. This follows the existing ADR-040/041 source-selection contract. Tests cover both cutover directions and response serialization.
• Global Speed excludes hosting-provider samples before counts and percentiles, matching project performance. A PostgreSQL regression covers ordinary, hosting-provider, crawler, unknown and missing-IP samples, plus project/environment scope.

All 14 focused global tests, full all-target/all-feature Clippy, web TypeScript, source attribution, and commit hooks passed. The rebuilt server passed authenticated endpoint smoke checks and supplied the regenerated web SDK.