Add nonunix platform support by ryanofsky · Pull Request #274 · bitcoin-core/libmultiprocess

ryanofsky · 2026-04-22T21:53:45Z

This PR implements API changes and fixes needed to allow libmultiprocess to work on nonunix platforms.

These changes were originally part of #231, which adds windows support, but were split out to allow windows and nonwindows changes to be reviewed separately.

DrahtBot · 2026-04-22T21:54:07Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline and AI policy for information on the review process.

Type	Reviewers
ACK	Sjors, ViniciusCestarii
Stale ACK	enirox001

If your review is incorrectly listed, please copy-paste  into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#298 (Fix error handling when creating clients (mp::ConnectStream) by xyzconstant)
#296 (ci: Bump channel to nixos-26.05 by maflcko)
#209 (cmake: Increase cmake policy version by ryanofsky)
#175 (Set cmake_minimum_required(VERSION 3.22) by maflcko)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

ryanofsky

Thanks for the review! Will update with suggestions.

Sjors · 2026-06-17T14:23:06Z

I opened stratum-mining/sv2-tp#110 to test the changes in the Template Provider.

Sjors

Studied two first three commits...

Sjors

Reviewed the rest as well, except the CMake changes in c9aa806 and 7cb83a5.

Concept ACK

Sjors · 2026-06-18T17:29:46Z

 //! errors in python unit tests.
 std::string LogEscape(const kj::StringTree& string, size_t max_size);

+using Stream = kj::Own<kj::AsyncIoStream>;


In 091f5e1 proxy, refactor: Change ConnectStream and ServeStream to accept stream objects: would it make sense to introduce this earlier, in 3c81cf2?

re: #274 (comment)

In 091f5e1 proxy, refactor: Change ConnectStream and ServeStream to accept stream objects: would it make sense to introduce this earlier, in 3c81cf2?

It could make sense but I feel it's a little clearer if the EventLoop code is using kj::Own<kj::AsyncIoStream> and kj::Own<kj::OutputStream> types directly and not tied to the mp::Stream type for external callers and meant to be more opaque.

But I did extend this commit to use Stream stream type in ConnectStream and ServeStream functions since these are external functions.

In 174604a "proxy, refactor: Change ConnectStream and ServeStream to accept stream objects"

Had a concern that the name Stream is fairly broad, so using it too widely could hide useful information at call sites. For example, inside EventLoop it seems helpful to see the concrete KJ type details directly, otherwise future code might start using mp::Stream everywhere just because the alias exists.

I agree with keeping it limited to external facing methods like ConnectStream and ServeStream, where the abstraction is useful, and avoiding it in internal code where the underlying type is more informative.

re: #274 (comment)

Agreed, glad the reasoning made sense!

ryanofsky

Thanks for the reviews!

Rebased 7cb83a5 -> 68ed129 (pr/wins.1 -> pr/wins.2, compare) implementing review suggestions and fixing conflict with #279

Updated 68ed129 -> a43e5a8 (pr/wins.2 -> pr/wins.3, compare) to fix environ undeclared on macOS/BSD: add explicit declaration before posix_spawn() https://github.com/bitcoin-core/libmultiprocess/actions/runs/27974097183/job/82787313836

Updated a43e5a8 -> 6cc729f (pr/wins.3 -> pr/wins.4, compare) to fix clang-tidy readability-redundant-declaration on environ declaration https://github.com/bitcoin-core/libmultiprocess/actions/runs/27978804911/job/82803323635

ryanofsky · 2026-06-22T16:28:08Z

 //! errors in python unit tests.
 std::string LogEscape(const kj::StringTree& string, size_t max_size);

+using Stream = kj::Own<kj::AsyncIoStream>;


re: #274 (comment)

In 091f5e1 proxy, refactor: Change ConnectStream and ServeStream to accept stream objects: would it make sense to introduce this earlier, in 3c81cf2?

It could make sense but I feel it's a little clearer if the EventLoop code is using kj::Own<kj::AsyncIoStream> and kj::Own<kj::OutputStream> types directly and not tied to the mp::Stream type for external callers and meant to be more opaque.

But I did extend this commit to use Stream stream type in ConnectStream and ServeStream functions since these are external functions.

ryanofsky · 2026-07-16T10:19:41Z

@Sjors @enirox001 Are you able to Re-ACK this? I'd like to merge it to unblock windows support. This PR and bitcoin/bitcoin#35084 contain most of the code changes needed to support windows, with the other changes just being non-invasive #ifdef WIN32 additions.

There are not many changes since your last reviews. They are:

Rebasing
Dropping cmake commits
Catching std::stio exception for more descriptive spawn error messages
Catching another exception (in ~Connection and declaring it noexcept(false). These changes just precautionary, the new exception and other exceptions in general are not expected here.
In EventLoopRef::reset adding comment and tweaking post_writer call to avoid potential TSAN error (not a real race).
Tweaking F_SETFD call to preserve previous flags
Changing posix_spawn to posix_spawnp to preserve previous PATH search behavior.

Sjors · 2026-07-16T12:49:20Z

Everything else in 254c3e3 looks good to me, just two questions above.

enirox001

ACK 254c3e3 Changes look fine to me. Left a few questions, but they are non-blocking nits.

ryanofsky

Updated 254c3e3 -> 6e29e22 (pr/wins.7 -> pr/wins.8, compare) fixing version documentation

ryanofsky

Updated 6e29e22 -> 9953981 (pr/wins.8 -> pr/wins.9, compare) changing MakeStream signature to avoid potential thread safety issues in the future. Note that this requires an update to bitcoin/bitcoin#35084 so CI will fail until that is pushed (should happen shortly)

Squashed 9953981 -> 1b0f605 (pr/wins.9 -> pr/wins.10, compare) just updating commit message and commit order as suggested

Sjors · 2026-07-16T17:28:34Z

ACK 1b0f605

ViniciusCestarii · 2026-07-16T18:55:30Z

ACK 1b0f605 I like that these changes make the code more readable too, require clauses and KJ Stream abstractions are nice

ryanofsky

Thanks for the reviews! I plan to merge this soon (as-is) but rereviewed this myself and left some notes for followups

ryanofsky · 2026-07-17T10:33:57Z

        } else if (done()) {
            // Intentionally do not break if m_post_fn was set, even if done()
-            // would return true, to ensure that the EventLoopRef write(post_fd)
+            // would return true, to ensure that the post() m_post_writer->write()


In commit "proxy, refactor: Replace EventLoop wakeup fd integers with KJ stream objects" (e96d5d7)

Note for followup. Unclear why I changed this comment to reference post() instead of EventLoopRef to. I think this is just a mistake and this should reference EventLoopRef::reset() specifically. Both of these methods write but only reset() sets the done() state and needs this logic.

I was also looking into why newTwoWayPipe is used here instead of newOneWayPipe, and it doesn't seem like there is a good reason. And there seems to be a newer kj::Executor mechanism https://github.com/capnproto/capnproto/blob/v2/kjdoc/tour.md#threads that would avoid the need for these pipes entirely.

I think I want to focus on getting windows support fully implemented in #231 and tested in CI but after that it seems like there is room for simplification here with kj::Executor

ryanofsky · 2026-07-17T10:46:26Z


+//! Information about parent process passed to child process as a command-line
+//! argument. On unix this is the child socket fd number formatted as a string.
+using SpawnConnectInfo = std::string;


In commit "util, refactor: Add SpawnConnectInfo type alias and use it" (1389cf3)

Note for followup: Looking at this code I think these type SpawnConnectInfo aliases actually hurt readability more than they help. I think originally I was thinking SpawnConnectInfo might have to be different types on different platforms to deal with the windows implementation #231 which sends a serialized socketed handle over a pipe to the child process instead of passing a file descriptor. But either way the application only sees an opaque string so std::string is a clearer way to represent this than SpawnConnectInfo

3f221b5bfd Merge bitcoin-core/libmultiprocess#274: Add nonunix platform support 1b0f605606 doc: Remove trailing whitespace d8f8ca3119 ipc: Wrap mpgen main() in try-catch to print errors fbe5a14ad4 ci: Check out bitcoin/bitcoin PR #35084 instead of master 39d3690d83 types: Replace SFINAE with requires clauses to avoid MSVC C2039 error ba68520203 proxy, refactor: Fix C4305 truncation warning in Accessor on MSVC 1d81d47811 util, refactor: Fix PtrOrValue constructor for move-only types on MSVC b883fe1e52 proxy: Fix shutdownWrite() exception handling on macOS with dynamic libraries 0012411ccc proxy: Call shutdownWrite() in Connection destructor 38312ad191 proxy, refactor: Change ConnectStream and ServeStream to accept stream objects e96d5d742a proxy, refactor: Replace EventLoop wakeup fd integers with KJ stream objects db4f9a3d73 cmake: Bump minimum required Cap'n Proto version to 0.9 652934fb79 util, refactor: Add SocketPair() and use it in SpawnProcess 1c6ef7a26c util, refactor: Do not fork() and exec() separately 1389cf3132 util, refactor: Add SpawnConnectInfo type alias and use it c7ca1f00b6 util, refactor: Add SocketId type alias and use it be46a35203 util, refactor: Add ProcessId type alias and use it 91a78db780 doc: Bump version 13 > 14 git-subtree-dir: src/ipc/libmultiprocess git-subtree-split: 3f221b5bfd7ee0e7972e3c5ed4bb7ee86e457f6d

Since bitcoin-core#274, connection setup has been split into two calls: `MakeStream` followed by `ConnectStream`. Between these calls, the event loop's reference count can temporarily drop to zero, allowing `EventLoop::loop()` to exit before `ConnectStream` posts its work, crashing the example on startup. Keep an `EventLoopRef` alive in main() so the loop stays running while it is in use.

d3d74e7 ipc, refactor: Update mp::g_thread_context references (Ryan Ofsky) 2d3f72f ipc, refactor: Update mp::SpawnProcess call (Ryan Ofsky) e9f1981 ipc, refactor: Add Stream type alias and use it (Ryan Ofsky) 3859805 ipc, refactor: Add SocketId type alias and use it (Ryan Ofsky) 2ee9b69 ipc, refactor: Add ProcessId type alias and use it (Ryan Ofsky) 3449797 ipc: Avoid 'unistd.h' error with MSVC (Ryan Ofsky) dbcc192 ipc, refactor: fix include order (Ryan Ofsky) 7c86d48 ipc, refactor: use native path separators in test (Ryan Ofsky) 00287b9 ipc, refactor: Change Protocol class field order (Ryan Ofsky) 33d37f3 ipc, refactor: Drop connect/listen/serve exe_name parameters (Ryan Ofsky) 7949404 ipc, moveonly: combine ipc_test.cpp and ipc_tests.cpp (Ryan Ofsky) Pull request description: This PR makes Bitcoin Core changes needed to be compatible with bitcoin-core/libmultiprocess#274, which changes the libmultiprocess API to stop using unix-specific types so it is compatible with windows. (Windows support is added in followups: bitcoin-core/libmultiprocess#231 and #32387.) The PR uses some [compatibility shims](https://github.com/ryanofsky/bitcoin/blob/pr/ipc-wins/src/ipc/util.h) so it can be reviewed and merged without needing to merge bitcoin-core/libmultiprocess#274 first and bump the libmultiprocess subtree. These can be deleted when the subtree is updated. --- Review note: All the changes here are refactoring, and you don't really need to know anything about IPC or Windows to review this code. It is also a mostly move-only change (131 lines added, 96 removed, 215 moved) ACKs for top commit: xyzconstant: tACK d3d74e7 enirox001: ACK d3d74e7 Sjors: ACK d3d74e7 ViniciusCestarii: re-ACK d3d74e7 tested locally on Linux Tree-SHA512: cd48708f9fd086ac8127dc75cfaf4bd8f8da81e07d11b2c9e65fd9061ffa33478bffc6fd6fa4b3505e86c6437752578fe6e5bd590c683c3bc9969093103a5608

Since bitcoin-core#274, connection setup has been split into two calls: `MakeStream` followed by `ConnectStream`. Between these calls, the event loop's reference count can temporarily drop to zero, allowing `EventLoop::loop()` to exit before `ConnectStream` posts its work, crashing the example on startup. Keep an `EventLoopRef` alive in main() so the loop stays running while it is in use.

…e fork 1e0c7ff util: Clear FD_CLOEXEC in child instead of parent before fork (Sjors Provoost) 8550ee6 util, refactor: Add ChildFail helper for post-fork child errors (Sjors Provoost) Pull request description: Commit 652934f in #274 _sets_ `FD_CLOEXEC` in order "to ensure sockets are not leaked if processes are spawned". However it's _cleared_ too early, before forking. This PR clears it _after_ forking, but still before exec. The first commit adds a helper for signal-safely emitting an error message, since the second commit also needs this. The second commit and a code comment explain why it's unsafe to clear `FD_CLOEXEC` before fork. ryanofsky also described it: > As I understand it, this race has always existed, and was not introduced in [652934f](652934f). What [652934f](652934f) did was start using `FD_CLOEXEC` which narrowed the race window, without completely closing it. This followup PR fixes the race more completely, but there is still a small race between creating the socketpair and applying the cause the CLOEXEC flags. > > The race happens when a `SpawnProcess` call happens at the same time as a separate `fork` call in unrelated thread not using `SpawnProcess`. Because `SpawnProcess` creates a socket pair, if a separate `fork` happens in another thread, it could inherit the socket pair file descriptors and keep them open too long if it is not looping over them and closing them like `SpawnProcess` is. As I understand it this could result in `socketpair` connections staying open even after the `SpawnProcess` parent or child have closed them, so `onDisconnect` events might not be triggered, and resources might not be freed. A test in Sjors@1e1ff03 demonstrates the issue, but is not included in the PR to keep things simple. This should not be an actual problem in the way Bitcoin Core uses libmultiprocess today, but it may be with Windows and/or multiple connection support. ACKs for top commit: ViniciusCestarii: ACK 1e0c7ff verified that without this change the test Sjors@1e1ff03 fails and that this really tightens the race window to just between socketpair and fcntl syscalls. ryanofsky: Code review ACK 1e0c7ff. Thanks for the fix! I think it's be good to add bugfix: to the title and make PR description describe bug more practically. xyzconstant: tACK 1e0c7ff Tree-SHA512: eaf127de19ad579dc2adc1276f7e160e9b6478257cadccfa7b4dd3faf348cb11b58394d1e7987e81a6fc0001410c73f923c0d7ce63c235f43d0c599226bffc13

36c6c63 doc: Document reference-counted EventLoop lifetime (xyzconstant) 3a997e1 Fix startup race in mpexample (xyzconstant) Pull request description: Since #274, connection setup has been split into two calls: `MakeStream` followed by `ConnectStream`. Between these calls, the event loop's reference count can temporarily drop to zero, allowing `EventLoop::loop()` to exit (and the EventLoop to be destroyed) before `ConnectStream` posts its work. This crashes mpexample on startup. Fix this by keeping an `EventLoopRef` alive in main() so the loop stays running while it is in use. This same pattern is already used in Bitcoin Core (`m_loop_ref` member of `CapnpProtocol`). A second commit documents the reference-counted EventLoop lifetime in the `EventLoopRef` class comment and in `doc/usage.md`. NOTE: On macOS, mpexample was crashing intermittently on startup (sometimes failing the m_post_fn == nullptr assert in the EventLoop destructor, sometimes with a mutex lock failure), depending on timing. With this change, the crashes are gone. ACKs for top commit: ryanofsky: Code review ACK 36c6c63. Thanks for the fix, documentation, and simplification! Tree-SHA512: 69ed48484bb0c4889116fe98ed002cce2aeba65f6bfb1fd3d250aa34f7205669caa10990d6fac71cb4733a3b29a178e65be34fcd1dfb03b11de5a9fb956377da

6101a2e ipc, refactor: Update mp::g_thread_context references (Ryan Ofsky) 8f9f52c ipc, refactor: Update mp::SpawnProcess call (Ryan Ofsky) ff56e7a ipc, refactor: Add Stream type alias and use it (Ryan Ofsky) c0a7490 ipc, refactor: Add SocketId type alias and use it (Ryan Ofsky) 56c0011 ipc, refactor: Add ProcessId type alias and use it (Ryan Ofsky) c4c23f6 ipc: Avoid 'unistd.h' error with MSVC (Ryan Ofsky) 8a9cd6e ipc, refactor: fix include order (Ryan Ofsky) a75df3e ipc, refactor: use native path separators in test (Ryan Ofsky) 83817f3 ipc, refactor: Change Protocol class field order (Ryan Ofsky) e93ffef ipc, refactor: Drop connect/listen/serve exe_name parameters (Ryan Ofsky) 4b41ceb ipc, moveonly: combine ipc_test.cpp and ipc_tests.cpp (Ryan Ofsky) Pull request description: This PR makes Bitcoin Core changes needed to be compatible with bitcoin-core/libmultiprocess#274, which changes the libmultiprocess API to stop using unix-specific types so it is compatible with windows. (Windows support is added in followups: bitcoin-core/libmultiprocess#231 and bitcoin/bitcoin#32387.) The PR uses some [compatibility shims](https://github.com/ryanofsky/bitcoin/blob/pr/ipc-wins/src/ipc/util.h) so it can be reviewed and merged without needing to merge bitcoin-core/libmultiprocess#274 first and bump the libmultiprocess subtree. These can be deleted when the subtree is updated. --- Review note: All the changes here are refactoring, and you don't really need to know anything about IPC or Windows to review this code. It is also a mostly move-only change (131 lines added, 96 removed, 215 moved) ACKs for top commit: xyzconstant: tACK 6101a2e enirox001: ACK 6101a2e Sjors: ACK 6101a2e ViniciusCestarii: re-ACK 6101a2e tested locally on Linux Tree-SHA512: cd48708f9fd086ac8127dc75cfaf4bd8f8da81e07d11b2c9e65fd9061ffa33478bffc6fd6fa4b3505e86c6437752578fe6e5bd590c683c3bc9969093103a5608

ryanofsky mentioned this pull request Apr 23, 2026

Add windows support #231

Open

This was referenced Apr 23, 2026

ci: add Windows job #272

Draft

proxy: add local connection limit to ListenConnections #269

Merged

cmake: Increase cmake policy version #209

Open

Set cmake_minimum_required(VERSION 3.22) #175

Open

ViniciusCestarii reviewed May 8, 2026

View reviewed changes

Comment thread include/mp/util.h Outdated

ViniciusCestarii reviewed May 8, 2026

View reviewed changes

Comment thread src/mp/util.cpp Outdated

ViniciusCestarii reviewed May 8, 2026

View reviewed changes

Comment thread include/mp/util.h Outdated

DrahtBot mentioned this pull request May 12, 2026

build: prepare for subtree split #276

Closed

Sjors mentioned this pull request May 28, 2026

Mining interface tracking issue bitcoin/bitcoin#33777

Open

13 tasks

ryanofsky commented Jun 2, 2026

View reviewed changes

Comment thread include/mp/util.h Outdated

Comment thread src/mp/util.cpp Outdated

Comment thread include/mp/util.h Outdated

DrahtBot added the Needs rebase label Jun 9, 2026

Sjors mentioned this pull request Jun 17, 2026

Add nonunix platform support (libmultiprocess v14) stratum-mining/sv2-tp#110

Draft

Sjors reviewed Jun 17, 2026

View reviewed changes

Comment thread include/mp/util.h

Comment thread include/mp/util.h

Sjors reviewed Jun 17, 2026

View reviewed changes

Comment thread include/mp/util.h

Sjors reviewed Jun 18, 2026

View reviewed changes

Comment thread include/mp/util.h

Comment thread include/mp/util.h

Comment thread src/mp/util.cpp

Comment thread src/mp/util.cpp Outdated

Sjors reviewed Jun 18, 2026

View reviewed changes

Sjors mentioned this pull request Jun 18, 2026

ipc: Add nonunix platform support bitcoin/bitcoin#35084

Merged

ryanofsky force-pushed the pr/wins branch from 7cb83a5 to 68ed129 Compare June 22, 2026 18:13

ryanofsky commented Jun 22, 2026

View reviewed changes

DrahtBot removed the Needs rebase label Jun 22, 2026

ryanofsky force-pushed the pr/wins branch 2 times, most recently from a43e5a8 to 6cc729f Compare June 22, 2026 21:32

DrahtBot mentioned this pull request Jun 23, 2026

ci: Bump channel to nixos-26.05 #296

Merged

DrahtBot added the Needs rebase label Jul 14, 2026

ryanofsky force-pushed the pr/wins branch from 491f1e8 to 254c3e3 Compare July 14, 2026 01:28

DrahtBot removed the Needs rebase label Jul 14, 2026

DrahtBot mentioned this pull request Jul 16, 2026

Fix error handling when creating clients (mp::ConnectStream) #298

Open

Sjors reviewed Jul 16, 2026

View reviewed changes

Comment thread doc/versions.md Outdated

Sjors reviewed Jul 16, 2026

View reviewed changes

Comment thread include/mp/proxy-io.h Outdated

ryanofsky force-pushed the pr/wins branch from 254c3e3 to 6e29e22 Compare July 16, 2026 12:53

enirox001 reviewed Jul 16, 2026

View reviewed changes

Comment thread src/mp/util.cpp

Comment thread ci/configs/olddeps.bash

ryanofsky commented Jul 16, 2026

View reviewed changes

Comment thread doc/versions.md Outdated

Comment thread include/mp/proxy-io.h Outdated

ryanofsky force-pushed the pr/wins branch from 6e29e22 to 9953981 Compare July 16, 2026 13:49

ryanofsky commented Jul 16, 2026

View reviewed changes

Comment thread include/mp/proxy-io.h Outdated

ryanofsky force-pushed the pr/wins branch from 9953981 to 1b0f605 Compare July 16, 2026 14:19

DrahtBot requested a review from enirox001 July 16, 2026 17:28

ryanofsky commented Jul 17, 2026

View reviewed changes

ryanofsky merged commit 3f221b5 into bitcoin-core:master Jul 17, 2026
13 checks passed

Sjors mentioned this pull request Jul 20, 2026

bugfix: clear FD_CLOEXEC in child instead of parent before fork #311

Merged

xyzconstant mentioned this pull request Jul 22, 2026

Fix startup race in example #315

Merged

ryanofsky mentioned this pull request Aug 3, 2026

Remove code for Cap'n Proto versions before 0.9 #331

Merged

Uh oh!

Conversation

ryanofsky commented Apr 22, 2026

Uh oh!

DrahtBot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews

Conflicts

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryanofsky left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sjors commented Jun 17, 2026

Uh oh!

Sjors left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sjors left a comment

Choose a reason for hiding this comment

Uh oh!

Sjors Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

ryanofsky Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

enirox001 Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

ryanofsky Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryanofsky left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryanofsky Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryanofsky commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sjors commented Jul 16, 2026

Uh oh!

enirox001 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ryanofsky left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DrahtBot commented Apr 22, 2026 •

edited

Loading

ryanofsky left a comment •

edited

Loading

ryanofsky commented Jul 16, 2026 •

edited

Loading

ryanofsky left a comment •

edited

Loading