Skip to content

Repository files navigation

Nexus

A C++23 order-matching engine with price-time priority, lock-free SPSC pipelines, zero-allocation hot paths, and io_uring-based asynchronous UDP networking.


Overview

Nexus implements the path from a framed UDP order through decoding, price-time-priority matching, and fill encoding. The matching path uses preallocated pools, cache-aware containers, and intrusive lists to avoid heap allocation after initialization.

Two dedicated threads communicate through lock-free SPSC queues. The network thread uses kernel-managed UDP sockets through io_uring on Linux, with a non-blocking POSIX fallback; the matching thread spin-polls inbound messages and dispatches them to per-symbol order books. CPU affinity and real-time scheduling hooks are available, but the committed measurements below were collected in an ARM64 Linux VM rather than a tuned bare-metal deployment.

Architecture

                        +-----------------+
                        |   UDP Client    |
                        +--------+--------+
                                 | sendto / recvfrom
                                 v
+-----------------------------------------------------------------------+
|                         Nexus Server                                  |
|                                                                       |
|   +-------------------+      SPSC Queue      +-------------------+    |
|   |  Network Thread   | ---(InboundMessage)-->|  Matching Thread  |    |
|   |                   |                       |                   |    |
|   |  io_uring / POSIX |                       |  MatchingEngine   |    |
|   |  Frame decode     | <-(OutboundMessage)---|  OrderBook[]      |    |
|   |  Frame encode     |      SPSC Queue       |  PoolAllocator    |    |
|   +-------------------+                       +-------------------+    |
|                                                                       |
+-----------------------------------------------------------------------+

OrderBook (one per symbol):
  +------------------+
  |  Bids (sorted    |  FlatPriceMap<greater> -- contiguous sorted array
  |   desc by price) |  Each level: intrusive doubly-linked list of Orders
  +------------------+
  |  Asks (sorted    |  FlatPriceMap<less> -- binary search O(log n) lookup
  |   asc by price)  |  Price-time priority within each level
  +------------------+
  |  FlatHashMap     |  Robin Hood hashing, O(1) cancel/modify
  |  PoolAllocator   |  O(1) alloc/dealloc, zero system calls
  +------------------+

Measured Performance

These results were recorded on a 12-core ARM64 system reported at 2 GHz in an OrbStack Linux VM, using a release build with -O3 and LTO. They are local microbenchmark observations, not production latency claims; raw benchmark output is not committed. The benchmark programs under benchmarks/ reproduce the measurement paths.

Core Matching Engine

Operation Latency Throughput
Insert (no match) 27 ns 37M orders/sec
Full match (cross) 43 ns 23M orders/sec
Cancel 25 ns 40M orders/sec
Modify (qty down) 1.4 ns 710M ops/sec
Iceberg replenish 22 ns 46M ops/sec
Stop trigger + fill 228 ns 4.4M ops/sec
Depth snapshot (10 lvl) 34 ns 30M ops/sec

Full Pipeline (wire decode → match → wire encode)

Operation Latency
No match 38 ns
Full match 59 ns
Cancel 27 ns

Wire-to-Wire Latency (UDP loopback)

Percentile ARM64 VM
p50 17.5 μs
p90 45.9 μs
p99 2.2 ms

The full round trip is client sendto() → kernel UDP stack → io_uring receive → decode → SPSC → match → fill → SPSC → encode → io_uring send → kernel UDP stack → client recvfrom(). The VM result includes scheduler and hypervisor jitter; no cause is inferred from the numbers alone.

Thread-to-Thread (SPSC hop, no kernel I/O)

This benchmark removes socket and kernel-networking work and measures the SPSC and matching-thread path directly. It is not a DPDK benchmark; Nexus does not implement DPDK or another kernel-bypass data path.

Percentile ARM64 VM
p50 875 ns
p90 917 ns
p99 1.0 μs
p99.9 14.0 μs

Infrastructure

Component ARM64 VM
SPSC push+pop (single thread) 420M ops/sec
SPSC throughput (cross-thread) 370M+ ops/sec

Projected Results (Not Measured)

The following ranges were retained from the original design notes for transparency. They are estimates for hardware/configurations that have not been benchmarked in this repository and must not be read as observed results.

Metric x86_64 bare metal (estimated) x86_64 + isolcpus (estimated)
UDP round-trip p50 8–12 μs 5–8 μs
UDP round-trip p90 15–25 μs 10–15 μs
UDP round-trip p99 50–100 μs 20–40 μs
Thread-to-thread p50 350–500 ns
Thread-to-thread p90 400–550 ns
Thread-to-thread p99 500–700 ns
Thread-to-thread p99.9 2–5 μs
SPSC push+pop 700M–1B ops/sec
SPSC cross-thread throughput 500–800M ops/sec

Features

  • Price-time priority limit order book with intrusive doubly-linked lists
  • Order types: Limit, Market, IOC, FOK, Iceberg (hidden quantity), Stop-Limit
  • io_uring-based asynchronous UDP networking (Linux) with a non-blocking POSIX fallback (macOS/Linux)
  • Lock-free SPSC queues for inter-thread communication
  • Custom data structures: FlatPriceMap (sorted contiguous array), Robin Hood hash map
  • Pool allocators with O(1) alloc/dealloc and zero hot-path heap allocation
  • CPU core pinning and real-time scheduling support
  • Market data depth snapshots with visible-only quantities (iceberg-aware)
  • Binary wire protocol with 16-byte frame headers and sequence numbering
  • Latency histograms with percentile reporting
  • Profile-Guided Optimization (PGO) build infrastructure

Design Decisions

Why fixed-point pricing?

Floating-point arithmetic is non-deterministic across platforms and compiler settings (-ffast-math, FMA contraction). Real exchanges use fixed-point integer pricing for exact reproducibility. We use int64_t with 6 decimal places ($150.25 = 150'250'000).

Why FlatPriceMap over std::map?

std::map is a node-based tree, while FlatPriceMap stores price levels in a contiguous sorted array. That makes lookup O(log n) and insertion O(n) due to shifting, in exchange for sequential storage when the matching loop walks levels from the best price. The included matching benchmark exercises multi-level walks; it does not claim a committed std::map comparison.

Why Robin Hood hashing over std::unordered_map?

std::unordered_map uses separate chaining — each bucket is a heap-allocated linked list, causing cache misses on every lookup. Our FlatHashMap uses open addressing with Robin Hood probe distance balancing: entries with longer probe chains "steal" slots from entries with shorter chains, reducing worst-case variance. Backward-shift deletion avoids tombstone accumulation, which is critical for cancel-heavy workloads where erasures are as frequent as insertions. Fibonacci hashing distributes sequential order IDs uniformly across the table.

Why pool allocation?

malloc/new are unbounded in latency — a single allocation can trigger mmap, brk, or page faults. Our pool pre-allocates a contiguous block of Order-sized slots and manages a singly-linked free list threaded through unused slots. Allocate pops the head; deallocate pushes to the head. O(1), zero system calls, deterministic.

Why intrusive linked lists?

Non-intrusive containers (like std::list) allocate a separate node for each element. Intrusive lists embed the prev/next pointers directly in the Order struct. Since orders come from the pool, they are already linked-list nodes — zero extra allocation, zero indirection. O(1) insertion (append to tail), O(1) removal (unlink by pointer without search).

Why io_uring over epoll?

io_uring exposes memory-mapped submission and completion rings and can batch work submitted to the kernel. Nexus uses those rings around ordinary kernel-managed UDP sockets to reduce control-path overhead and keep multiple operations in flight. This is asynchronous networking, not kernel bypass; packets still traverse the Linux networking stack.

Why two threads (not one, not many)?

The network thread and matching thread have fundamentally different performance profiles. The network thread is I/O-bound; the matching thread is compute-bound. Separating them allows:

  • CPU pinning each to an isolated core (no scheduler interference)
  • The matching thread never touches a syscall (pure compute)
  • Natural backpressure via the SPSC queue (no locks, no mutexes)

More threads would add synchronization complexity without benefit — the order book is inherently sequential (price-time priority requires total ordering).

Why SPSC queues (not MPMC, not mutexes)?

With exactly one producer and one consumer, SPSC queues need only atomic load/store with acquire/release ordering — no CAS loops, no mutexes, no contention. The head and tail counters are placed on separate cache lines to eliminate false sharing between cores. Power-of-2 capacity enables branchless index wrapping via bitmask. On x86, the TSO memory model means acquire/release compiles to plain loads/stores — zero fence overhead.

Limitations

  • The networking backends are io_uring and non-blocking POSIX UDP. DPDK, AF_XDP, RDMA, and other kernel-bypass transports are not implemented.
  • The committed performance table is from one ARM64 virtualized environment; bare-metal x86_64 figures are explicitly projections.
  • Raw output for the recorded measurements is not committed, so results should be reproduced with the included benchmark binaries before comparison.
  • The server is an exchange-engine prototype, not a production venue: it does not provide persistence, replication, risk checks, authentication, or a production market-data protocol.

Building

# Prerequisites: CMake 3.22+, C++23 compiler (GCC 13+ or Clang 17+)
# Linux: liburing-dev for io_uring support

# Debug build (with sanitizers)
cmake -B build -DCMAKE_BUILD_TYPE=Debug -DENABLE_ASAN=ON -DENABLE_UBSAN=ON
cmake --build build --parallel

# Release build (with LTO)
cmake -B build-release -DCMAKE_BUILD_TYPE=Release
cmake --build build-release --parallel

# Profile-Guided Optimization build
./scripts/pgo_build.sh          # Full PGO pipeline
./scripts/pgo_build.sh --compare  # With before/after comparison

# Run tests (18 suites, 216 declared test cases)
cd build-release && ctest --output-on-failure

# Run benchmarks
./build-release/benchmarks/bench_matching
./build-release/benchmarks/bench_pipeline
./build-release/benchmarks/bench_wire_to_wire

# Start the server
./build-release/nexus_server --port 9000 --symbols 1,2,3

# Run the latency test client
./build-release/test_client --host 127.0.0.1 --port 9000 --orders 10000

Testing

18 test suites with 216 declared test cases covering:

  • Core: Order book matching (limit, market, IOC, FOK, iceberg, stop-limit), cancel, modify, depth snapshots, market data callbacks
  • Data structures: FlatPriceMap (sorted array), FlatHashMap (Robin Hood), PoolAllocator, SPSC queue (including multi-threaded stress tests)
  • Protocol: Binary codec round-trips, frame encode/decode, message dispatch pipeline, sequence tracking
  • Network: io_uring source, POSIX source, UDP loopback
  • Integration: End-to-end UDP round-trip (order → fill), burst tests (100 concurrent pairs), graceful shutdown, invalid frame handling

Project Structure

nexus/
├── src/
│   ├── core/                     # Matching engine + order book
│   │   ├── types.h                # Order, Fill, Price, CancelRequest, DepthSnapshot
│   │   ├── price_level.h          # Intrusive doubly-linked list FIFO queue
│   │   ├── flat_price_map.h       # Sorted contiguous array (replaces std::map)
│   │   ├── order_book.h/cpp       # Single-symbol LOB with price-time priority
│   │   ├── matching_engine.h/cpp  # Multi-symbol order router
│   │   └── matching_thread.h/cpp  # Dedicated matching thread with spin-poll loop
│   ├── memory/                   # Zero-allocation infrastructure
│   │   ├── pool_allocator.h       # Fixed-size object pool (free-list based)
│   │   ├── flat_hash_map.h        # Robin Hood hash map with backward-shift deletion
│   │   └── spsc_queue.h           # Lock-free Lamport queue, cache-line separated
│   ├── protocol/                 # Wire format + framing
│   │   ├── messages.h             # Cache-line aligned wire structs (64B orders, 32B cancel)
│   │   ├── messages_internal.h    # Compact SPSC transport types (no alignment padding)
│   │   ├── binary_codec.h         # Zero-copy encode/decode (memcpy-based)
│   │   ├── framing.h              # 16-byte UDP frame header (magic, version, seq)
│   │   ├── message_dispatcher.h   # Type-safe wire-to-engine dispatch
│   │   └── sequencer.h            # Inbound gap/duplicate detection, outbound numbering
│   ├── network/                  # Packet I/O
│   │   ├── io_uring_source.h/cpp  # Async UDP via io_uring (Linux)
│   │   ├── posix_source.h/cpp     # Non-blocking UDP fallback (macOS/Linux)
│   │   └── network_thread.h/cpp   # Dedicated I/O event loop
│   ├── server/                   # Orchestration
│   │   ├── nexus_server.h/cpp     # Thread lifecycle, ordered startup/shutdown
│   │   └── main.cpp               # CLI args, POSIX signal handling
│   ├── platform/
│   │   └── thread_utils.h/cpp     # CPU pinning, RT scheduling, thread naming
│   └── telemetry/
│       ├── latency_tracker.h      # Fixed-bucket histogram (100ns resolution)
│       └── stats.h                # Throughput counters
├── tests/                        # 18 Google Test suites (216 declared cases)
├── benchmarks/                   # 5 Google Benchmark suites
├── tools/
│   ├── order_generator.cpp        # Realistic order flow generator (1M orders)
│   └── test_client.cpp            # UDP latency measurement client
├── cmake/
│   ├── CompilerWarnings.cmake     # -Wall -Wextra -Wpedantic -Wshadow ...
│   ├── Sanitizers.cmake           # ASan, UBSan, TSan
│   ├── IoUring.cmake              # Conditional io_uring detection
│   └── PGO.cmake                  # Profile-Guided Optimization support
└── scripts/
    └── pgo_build.sh               # Automated PGO build pipeline

Platform Support

Platform Status Notes
Linux x86_64 Primary Full support including io_uring
Linux ARM64 Supported Full support; 128-byte cache lines
macOS ARM64 Dev only Matching engine + tests work; no io_uring

License

MIT

About

C++23 order-matching engine with price-time priority, lock-free SPSC pipelines, zero-allocation hot paths, and io_uring asynchronous UDP networking.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages