Skip to content

Repository files navigation

ULog

Coverage

ULog is an initiative to standardize and make actionable the telemetry generated by AI systems. By introducing versioned JSON logging contracts, a normalization and enrichment pipeline, and a deterministic classification engine, ULog ensures that heterogeneous logs from APIs, LLMs, agents, and CV pipelines become reliable, auditable signals.

What lives where

  • schemas/ – JSON Schemas (wrapper + versioned pattern)
    • Entry points (wrappers): schemas/<domain>.schema.json (stable refs)
    • Versioned files: schemas/<domain>/vN/<domain>.schema.json
    • Shared enums: schemas/_common.jsonvocab/controlled_vocabulary.json
  • rules/ – rule definitions (JSON) - See the Rules Engine README for details on ordering and how to add new rules.
  • data/ – sample or synthetic datasets
  • docs/ – project documentation
  • tests/ – unit tests (CI runs pytest)
  • local_pipeline/ – docker-compose demo with a placeholder classifier service
    • in/ and out/ are mounted as volumes at /in and /out inside the container
  • scripts/ – build and packaging scripts
  • dist/ – build artifacts (wheel, CLI bundle, Lambda ZIP, checksums)

Privacy & Data-Handling (must read)

We work locally, with synthetic-only data. Do not paste real logs or connect any cloud services here.
See docs/PRIVACY.md for redaction rules, unsafe→safe examples, and the reviewer checklist.

Quickstart (local)

Batch Processing

  1. Copy env: cp .env.example .env
  2. Generate a sample input: make generate
  3. Run the pipeline: make classify
  4. Check outputs in local_pipeline/out/

HTTP Service

  1. Copy env: cp .env.example .env (set PORT if you want a custom port)
  2. Start the service: make http.up
  3. Health check: curl http://localhost:${PORT:-8080}/health
  4. Run smoke tests: make http.test
  5. Update screenshots (JSON + PNG): make http.screens
  6. Stop the service: make http.down

Endpoints (served by the FastAPI app in src/ulog/classifier/http.py):

  • GET /health — liveness check
  • POST /parsenormalize only (no rule classification)
    Requires each item to have @timestamp and @message.
    Content-Type:
    • application/json with a JSON array of objects, e.g.:
      [
        {"@timestamp":"2025-01-01T12:34:56.789Z","@message":"INFO: Service started"}
      ]
    • application/x-ndjson where each line is a JSON object with those fields.
  • POST /classifyfull pipeline (parse → validate → classify → annotate)
    Content-Type:
    • application/json with a JSON array of objects
    • application/x-ndjson (objects per line) Optional query params:
    • input_format=auto|raw|json (default auto)
    • schema=core_api|llm|agentic|cv (optional hint)

Responses: both endpoints return a JSON array of items.

Outputs: smoke results and screenshots are written to local_pipeline/out/ and docs/screenshots/.

Build & Packaging

Build reproducible distribution artifacts locally:

./scripts/build.sh

Outputs wheel, CLI bundle, Lambda ZIP, and checksums under dist/.

📦 See Build & Packaging for details.

Schema harness

Validate schemas and examples locally:

  • Text summary: make test.schemas
  • JSON: make test.schemas.json

Wrappers only are collected (files matching schemas/*.schema.json).

CI

GitHub Actions runs on PRs and main:

  • Lint: ruff check .
  • Tests: pytest
  • Schemas: make test.schemas (produces JUnit/JSON reports in tests/reports/)
  • Coverage: writes coverage.xml and docs/coverage.svg

No secrets required.

Environment Variables

Variable Default What it controls
LOG_LEVEL INFO Local verbosity for CLI/services (DEBUG, INFO, WARN, ERROR).
PORT 8080 HTTP service port exposed by docker-compose.
IN_DIR /in Input mount for the local Docker Compose pipeline.
OUT_DIR /out Output mount for the local Docker Compose pipeline.
CLASSIFIER_IMAGE ulog-classifier:local Image tag used by the classifier service in local_pipeline/docker-compose.yml.
PYTHON_VERSION 3.12 Python version used by CI and local dev tools.
RULES_PATH / ULOG_RULES_PATH /app/rules/rules.json Path to rules/rules.json.
SCHEMAS_DIR / ULOG_SCHEMAS_DIR /app/schemas Root folder for schemas.
VOCAB_PATH / ULOG_VOCAB_PATH /app/vocab/controlled_vocabulary.json Controlled vocabulary file.
CLASSIFIER_NO_VALIDATION 0 "1" to bypass JSON Schema validation (useful for quick parser inspection).

Common Make targets

  • make help – list common commands
  • make lint – run ruff locally
  • make test – run pytest
  • make yamllint – lint YAML with 4-space indentation
  • make test.schemas – run the two-phase parse→validate harness (writes JUnit/JSON)
  • make coverage – run tests with coverage and write docs/coverage.svg
  • make rules.validate – JSON-lint & schema-validate rules/rules.json (if present)
  • make data.generate / make data.generate.raw – synthetic dataset + raw mirror
  • make generate – create a sample input file for the local pipeline
  • make classify / make down – run/stop the Docker Compose (v2) pipeline
  • make build - build all distribution artifacts (wheel, CLI, Lambda ZIP)
  • make build.verify - verify build reproducibility (builds twice, compares checksums)
  • make clean - remove local dist/ directory

Rule evaluation (first-match-wins)

Rules are evaluated in the order they appear in rules/rules.json. The first matching rule wins; subsequent rules are not applied.

Reproducible requirements

CI exports a pinned requirements.txt artifact for Docker/prod installs:

poetry export -f requirements.txt --output requirements.txt --without-hashes

Use this when building images that do not run Poetry at build time.

About

No description, website, or topics provided.

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages