feat: zero-copy buffer-protocol subject support (mmap, bytearray, array)

[Qubitium]

Summary

Allow Pattern.match(), search(), fullmatch(), finditer(), findall(), split(), and sub() to operate directly on any object that implements the Python buffer protocol (e.g. mmap.mmap, bytearray, array.array) without first copying it into a bytes object. The change is safe under GIL=0 because the memoryview that exposes the buffer pins the exporter's storage for the lifetime of the match/finditer iterator.

Upstream credit: The buffer-protocol subject support is based on Jack Wong's work in iforapsy/PyPcre (commit e08f2ab). This port rebases it onto main, adds UTF-8 validation for the zero-copy buffer path, and includes expanded tests.

What changed

• pcre_ext/util.c adds buffer_view_from_object(obj, &data, &len), which uses PyMemoryView_FromObject to borrow a contiguous, single-byte C pointer into the exporter's storage. Callers keep the returned memoryview alive as utf8_owner, which both Pattern_execute() and Pattern_create_finditer() now do.
• pcre_ext/pcre2.c adds a PyObject_CheckBuffer(subject_obj) branch in Pattern_execute() and Pattern_create_finditer(). It uses the helper above, treats the subject as raw byte data (byte offsets, bytes group values), and calls ensure_valid_utf8_for_bytes_subject() before the PCRE2_NO_UTF_CHECK fast-path so UTF-8 patterns cannot receive invalid bytes.
• pcre/re_compat.py broadens is_bytes_like() to accept other buffer-protocol objects by attempting memoryview(value), while still returning False for str.
• tests/test_mmap.py adds coverage for mmap.mmap and bytearray subjects across pcre module-level functions and pattern methods, including findall/finditer, split/sub, pos/endpos byte offsets, non-contiguous buffer rejection, and the lifetime contract that prevents mmap.close() while a match object is alive.

GIL=0 / safety notes

• Object lifetime: PyMemoryView_FromObject increments the exporter's buffer export count and holds the exporter alive. The C MatchObject / FinditerObject stores the memoryview as utf8_owner, so the buffer stays pinned until the match/finditer is freed.
• Mutability: a memoryview only prevents resize/close, not concurrent element writes from other threads/processes. That is a caller contract; the README already warns not to mutate writable buffers during a match.
• UTF-8 safety: the new buffer branch validates UTF-8 for PCRE2_UTF patterns, matching the existing bytes behavior, so PCRE2_NO_UTF_CHECK is safe to use.

Example

import mmap, pcre

with open("data.txt", "rb") as f, mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
    match = pcre.search(rb"needle-(\d+)", mm)
    print(match.group(1))  # b"42"

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Devin Review found 1 potential issue.

[devin-ai-integration]

🟡 Test claims to cover mmap splitting but actually tests plain bytes

The split test passes an already-sliced value (self.mm[:20] at tests/test_mmap.py:81) which is a plain bytes object rather than the memory-mapped file, so the new zero-copy path is never exercised for splitting.

Impact: The feature area appears covered but has no real test, so a future regression in splitting directly on a buffer object would go undetected.

Why the sliced value is bytes, not mmap

Slicing an mmap.mmap (self.mm[:20]) returns a bytes object, not another mmap or memoryview. Therefore pattern.split(self.mm[:20]) splits over a bytes subject and never enters the buffer-protocol branch added in pcre_ext/pcre2.c. To actually exercise the new path the test should split over self.mm directly (e.g. pattern.split(self.mm)), which routes through Pattern.split -> finditer(self.mm) -> the new buffer branch.

Suggested change

	parts = pattern.split(self.mm[:20])
	parts = pattern.split(self.mm)

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

Good catch — self.mm[:20] returns plain bytes, so the zero-copy buffer path was not exercised. Fixed in 9710449 by passing self.mm directly to pattern.split(). Test still passes and now routes through Pattern.split -> finditer(self.mm) -> the new buffer branch.