Skip to content

test(backend): cover parser sandbox UTF-8 output size boundaries (#2163) - #2482

Open
Rakshak05 wants to merge 2 commits into
utksh1:mainfrom
Rakshak05:issue-#2163
Open

test(backend): cover parser sandbox UTF-8 output size boundaries (#2163)#2482
Rakshak05 wants to merge 2 commits into
utksh1:mainfrom
Rakshak05:issue-#2163

Conversation

@Rakshak05

Copy link
Copy Markdown
Contributor

Description

This PR adds byte-boundary test coverage for multibyte UTF-8 stdout and stderr handling in the parser sandbox (parser_sandbox.py). It verifies that when parser output or error logs hit or cross size limits on partial byte boundaries, truncation and failure messages remain safe without raising UnicodeDecodeError or corrupting internal diagnostic logs.

Key additions in testing/backend/unit/test_parser_sandbox.py:

  • Added TestMultibyteUtf8OutputBoundaries test class covering:
    • Valid multibyte UTF-8 JSON output (2-byte, 3-byte, and 4-byte characters like ñ, , 🚀, 日本語) decoding correctly within byte limits.
    • Streaming multibyte UTF-8 output straddling max_output_bytes boundaries, asserting ParserSandboxError is raised safely with output limit reasons.
    • Oversized multibyte UTF-8 stderr exceeding the 64 KB limit, ensuring replacement characters (errors="replace") handle partial UTF-8 byte sequences cleanly.
    • Non-JSON stdout containing partial multibyte UTF-8 sequences.
  • Updated _write_parser test helper to use explicit encoding="utf-8".

Related Issues

Closes #2163

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

How Has This Been Tested?

Executed backend unit tests locally using pytest:

python -m pytest testing/backend/unit/test_parser_sandbox.py
python -m pytest testing/backend/unit/test_parser_sandbox_helpers.py testing/backend/unit/test_parser_sandbox_timeout_cleanup.py

All 31 tests in test_parser_sandbox.py passed cleanly.

Checklist

  • My code follows the code style of this project.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(backend): cover parser sandbox UTF-8 output size boundaries

1 participant