Skip to content

Centralize sensitive log redaction across services - #308

Open
quarj0 wants to merge 40 commits into
mainfrom
feat/ic-066-structured-log-redaction
Open

Centralize sensitive log redaction across services#308
quarj0 wants to merge 40 commits into
mainfrom
feat/ic-066-structured-log-redaction

Conversation

@quarj0

@quarj0 quarj0 commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Closes #150

Summary

  • adds one shared backend redaction processor for Django, Celery, storage/provider logging paths, and managed AI
  • installs the boundary before Django/Celery/AI application logging can emit records
  • sanitizes nested structured context, positional args, exception text/tracebacks, credentials, common PII shapes, evidence locations, biometric fields, and binary values
  • adds frontend safeLog and rejects direct production console.* calls in lint
  • adds adversarial cross-service tests and operational logging guidance
  • copies the shared backend logging package into both production images

Verification

  • DJANGO_SETTINGS_MODULE=config.settings.testing uv run --project backend python backend/django/manage.py test common
  • cd backend/ai-service && uv run --project .. pytest tests/test_safe_logging.py
  • cd frontend && pnpm --filter @identitycore/api-client lint
  • full CI and Security workflows

Security notes

  • safe operational identifiers/status/reason codes remain available
  • credentials, subject PII, document/OCR content, evidence storage keys, selfies/embeddings/liveness media, and byte payloads are redacted
  • exception strings are sanitized at the Python LogRecord creation boundary so callers cannot bypass redaction by using logger.exception(...) or passing an exception as a formatting argument

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@quarj0

quarj0 commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e31f024100

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread backend/shared/logging_redaction.py Outdated
Comment thread frontend/packages/api-client/src/safe-logging.ts Outdated
Comment thread backend/shared/logging_redaction.py
Comment thread backend/shared/logging_redaction.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80733db288

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread backend/shared/logging_redaction.py
Comment thread backend/shared/logging_redaction.py
Comment thread backend/shared/logging_redaction.py Outdated
Comment thread backend/shared/logging_redaction.py Outdated
Comment thread backend/shared/logging_redaction.py
Comment thread backend/shared/logging_redaction.py
@quarj0

quarj0 commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@quarj0

quarj0 commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 89c3f7ff1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread backend/shared/logging_redaction.py
@quarj0

quarj0 commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b2dfd71615

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread frontend/scripts/check-safe-logging.mjs Outdated
Comment thread frontend/packages/api-client/src/safe-logging.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e372580d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread backend/shared/logging_redaction.py Outdated
Comment thread frontend/scripts/check-safe-logging.mjs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aacd38b78c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

record.stack_info = redact_text(record.stack_info)
if record.exc_info:
record.exc_text = _redact_exception(record.exc_info)
record.exc_info = None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve exception data for Django's email handler

When DEBUG=False and Django's mail_admins handler is enabled, django.utils.log.AdminEmailHandler.emit() inspects record.exc_info to construct its ExceptionReporter. Clearing the tuple here causes every 500-error email to fall back to (None, record.getMessage(), None), so the sanitized exc_text may still reach ordinary formatters but the admin report loses the actual exception traceback. Preserve a sanitized exception representation in a form this handler can consume rather than discarding its structured exception data.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[IC-066] Create centralized structured-log redaction for secrets, tokens, PII, and biometric payloads.

1 participant