Skip to content

Expose the matching engine to Python via pybind11 - #2

Merged
rrohit2901 merged 1 commit into
mainfrom
feature/python-bindings
Aug 9, 2026
Merged

Expose the matching engine to Python via pybind11#2
rrohit2901 merged 1 commit into
mainfrom
feature/python-bindings

Conversation

@rrohit2901

Copy link
Copy Markdown
Owner

The engine could only be driven from C++. This adds an importable matching_engine module built with pybind11 and packaged with scikit-build-core, so pip install . yields a working extension.

MatchingEngine cannot be bound directly. Its constructor takes an event queue that nothing drains unless a Logger is running on its own thread, so a caller handed the raw class could build an engine whose unbounded queue grows until it exhausts memory. Teardown is equally load-bearing: ~MatchingEngine publishes SESSION_CLOSE and Logger::stop() publishes the sentinel that ends the drain loop, so the engine must die before stop() or the last record never reaches the file. PyMatchingEngine owns the queue, logger, thread and engine, and makes that wiring non-optional.

Only MatchingEngine is bound -- the class is templated on a template template parameter with a requires-clause, and that is the only instantiation used anywhere.

Book views are returned as flat PriceLevel values rather than bound BookLevel objects. BookLevel is a live handle that re-reads the book on every accessor, so a Python object wrapping one would report the book as it is when read rather than when it was fetched.

close() holds the GIL across engine_.reset() and releases it only around the thread join: another thread may be parked inside a bound method dereferencing engine_, and it can only stay parked while we hold the GIL.

num_levels is validated because OrderBook's view takes an int but forwards to an unsigned parameter, so a negative count wrapped to about four billion and quietly returned the whole book.

Build changes are option-guarded so every default reproduces the previous build byte for byte. Without the guards pip install . would clone googletest and Google Benchmark to build neither. me_core gains POSITION_INDEPENDENT_CODE because a static library cannot otherwise link into a shared module; measured A/B against a non-PIC build, the hot path P50s are unchanged.

The engine could only be driven from C++. This adds an importable
`matching_engine` module built with pybind11 and packaged with
scikit-build-core, so `pip install .` yields a working extension.

MatchingEngine cannot be bound directly. Its constructor takes an event
queue that nothing drains unless a Logger is running on its own thread,
so a caller handed the raw class could build an engine whose unbounded
queue grows until it exhausts memory. Teardown is equally load-bearing:
~MatchingEngine publishes SESSION_CLOSE and Logger::stop() publishes the
sentinel that ends the drain loop, so the engine must die before stop()
or the last record never reaches the file. PyMatchingEngine owns the
queue, logger, thread and engine, and makes that wiring non-optional.

Only MatchingEngine<LockQueue> is bound -- the class is templated on a
template template parameter with a requires-clause, and that is the only
instantiation used anywhere.

Book views are returned as flat PriceLevel values rather than bound
BookLevel objects. BookLevel is a live handle that re-reads the book on
every accessor, so a Python object wrapping one would report the book as
it is when read rather than when it was fetched.

close() holds the GIL across engine_.reset() and releases it only around
the thread join: another thread may be parked inside a bound method
dereferencing engine_, and it can only stay parked while we hold the GIL.

num_levels is validated because OrderBook's view takes an int but
forwards to an unsigned parameter, so a negative count wrapped to about
four billion and quietly returned the whole book.

Build changes are option-guarded so every default reproduces the previous
build byte for byte. Without the guards `pip install .` would clone
googletest and Google Benchmark to build neither. me_core gains
POSITION_INDEPENDENT_CODE because a static library cannot otherwise link
into a shared module; measured A/B against a non-PIC build, the hot path
P50s are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rrohit2901
rrohit2901 merged commit a4878f6 into main Aug 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant