Expose the matching engine to Python via pybind11 - #2
Merged
Conversation
The engine could only be driven from C++. This adds an importable `matching_engine` module built with pybind11 and packaged with scikit-build-core, so `pip install .` yields a working extension. MatchingEngine cannot be bound directly. Its constructor takes an event queue that nothing drains unless a Logger is running on its own thread, so a caller handed the raw class could build an engine whose unbounded queue grows until it exhausts memory. Teardown is equally load-bearing: ~MatchingEngine publishes SESSION_CLOSE and Logger::stop() publishes the sentinel that ends the drain loop, so the engine must die before stop() or the last record never reaches the file. PyMatchingEngine owns the queue, logger, thread and engine, and makes that wiring non-optional. Only MatchingEngine<LockQueue> is bound -- the class is templated on a template template parameter with a requires-clause, and that is the only instantiation used anywhere. Book views are returned as flat PriceLevel values rather than bound BookLevel objects. BookLevel is a live handle that re-reads the book on every accessor, so a Python object wrapping one would report the book as it is when read rather than when it was fetched. close() holds the GIL across engine_.reset() and releases it only around the thread join: another thread may be parked inside a bound method dereferencing engine_, and it can only stay parked while we hold the GIL. num_levels is validated because OrderBook's view takes an int but forwards to an unsigned parameter, so a negative count wrapped to about four billion and quietly returned the whole book. Build changes are option-guarded so every default reproduces the previous build byte for byte. Without the guards `pip install .` would clone googletest and Google Benchmark to build neither. me_core gains POSITION_INDEPENDENT_CODE because a static library cannot otherwise link into a shared module; measured A/B against a non-PIC build, the hot path P50s are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The engine could only be driven from C++. This adds an importable
matching_enginemodule built with pybind11 and packaged with scikit-build-core, sopip install .yields a working extension.MatchingEngine cannot be bound directly. Its constructor takes an event queue that nothing drains unless a Logger is running on its own thread, so a caller handed the raw class could build an engine whose unbounded queue grows until it exhausts memory. Teardown is equally load-bearing: ~MatchingEngine publishes SESSION_CLOSE and Logger::stop() publishes the sentinel that ends the drain loop, so the engine must die before stop() or the last record never reaches the file. PyMatchingEngine owns the queue, logger, thread and engine, and makes that wiring non-optional.
Only MatchingEngine is bound -- the class is templated on a template template parameter with a requires-clause, and that is the only instantiation used anywhere.
Book views are returned as flat PriceLevel values rather than bound BookLevel objects. BookLevel is a live handle that re-reads the book on every accessor, so a Python object wrapping one would report the book as it is when read rather than when it was fetched.
close() holds the GIL across engine_.reset() and releases it only around the thread join: another thread may be parked inside a bound method dereferencing engine_, and it can only stay parked while we hold the GIL.
num_levels is validated because OrderBook's view takes an int but forwards to an unsigned parameter, so a negative count wrapped to about four billion and quietly returned the whole book.
Build changes are option-guarded so every default reproduces the previous build byte for byte. Without the guards
pip install .would clone googletest and Google Benchmark to build neither. me_core gains POSITION_INDEPENDENT_CODE because a static library cannot otherwise link into a shared module; measured A/B against a non-PIC build, the hot path P50s are unchanged.