test: cover immediate client disconnects for ListenConnections

[xyzconstant]

Following on testing the reversed direction ryanofsky suggested in #298, this PR adds a test to cover immediate client disconnects on the server side.

Additionally, this test adds DefaultLogHandler and a log_handler parameter to ListenSetup, allowing individual tests to observe logs by passing a custom log handler. The new test takes advantage of this by catching and skipping Uncaught exception in daemonized task. logs.

NOTE: an issue surfaced on the macOS job, the accept() call in ListenConnections fails for the closed connection, this is a Cap'n Proto bug as reported by ViniciusCestarii in #310 (review) and its fix is available in the v2 branch.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline and AI policy for information on the review process.

Type	Reviewers
ACK	ryanofsky
Stale ACK	ViniciusCestarii

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

[xyzconstant]

the macos job failed with:

[ TEST ] listen_tests.cpp:292: ListenConnections survives a client that disconnects before being accepted
mp/proxy.cpp:48: error: Uncaught exception in daemonized task.; exception = kj/async-io-unix.c++:1365: failed: setsocketopt(IPPROTO_TCP, TCP_NODELAY): Invalid argument
stack: 1020dc7cb 1020dcaff 1020dcc60 10079556c
/Users/runner/work/libmultiprocess/libmultiprocess/test/mp/test/listen_tests.cpp:189: failed: expected matched
stack: 10078b28b 1007ac7bf

I came across the same issue on my machine (MacBook Pro M2) too. I'll push a fix in a moment.

[ViniciusCestarii]

ACK 38391eb Nice finding! This is nicer than the old code that would just stop listening because of a transient error that KJ didn't catch but also with this change now a non-transient error will spin forever retrying an unrecoverable error which I believe could be fixed in a follow up.

Also I have dug in and I found that this is a KJ bug because it should treat transient errors like this and it was already fixed for capnp v2 which isn't released yet.

Given that, I still think this PR is correct and the defensive retry is the right call regardless. The spin forever on permanent failure case remains a valid follow-up on top of this.

[xyzconstant]

Thanks for the review @ViniciusCestarii!

Also I have dug in and I found that this is a KJ bug because it should treat transient errors like this and it was already fixed for capnp v2 which isn't released yet.

Nice catch! Now it's clear why this wasn't failing on other platforms.

Given that, I still think this PR is correct and the defensive retry is the right call regardless. The spin forever on permanent failure case remains a valid follow-up on top of this.

You're right on this, I was considering this too. I'm yet to figure out which exceptions we want to stop the accept loop for and which we want to retry, so a follow-up is the right call to me. If other reviewers would rather it be solved in this PR, I'm happy to tackle it here though.

[ryanofsky]

Code review 38391eb

Nice catch and elegant test! However, I think the suggested fix is too dangerous because retrying whenever accept fails could lead to an infinite loop if the accept failed for reasons other than this macos/capnproto bug, like file descriptors being closed, or resources being exhausted.

I think it'd be good to add the test but not the fix here, and make the test allow the setsocketopt(IPPROTO_TCP, TCP_NODELAY): Invalid argument error (https://github.com/bitcoin-core/libmultiprocess/actions/runs/29545518652/job/87776862017#step:4:126) that results on macos when the capnproto version is less than whatever capnproto version fixes this to document the bug. Separately we should make sure this is fixed in the capnproto version we are using in Bitcoin Core and consider adding a patch to the Bitcoin Core depends build if needed temporarily.

[ViniciusCestarii]

Thinking back here, I agree with @ryanofsky. In my review I understimated the spin forever on permanent failure case and the fix truly belongs to KJ and not here because if we were to try to reliably distinguish a transient error from a non transient error here we would basically be doing what KJ was supposed to do

[xyzconstant]

Thanks for the reviews, addressed feedback and force-pushed: d8e5395 -> b5b357f (compare)

re: #310 (review)

I think it'd be good to add the test but not the fix here, and make the test allow the setsocketopt(IPPROTO_TCP, TCP_NODELAY): Invalid argument error (https://github.com/bitcoin-core/libmultiprocess/actions/runs/29545518652/job/87776862017#step:4:126) that results on macos when the capnproto version is less than whatever capnproto version fixes this to document the bug. Separately we should make sure this is fixed in the capnproto version we are using in Bitcoin Core and consider adding a patch to the Bitcoin Core depends build if needed temporarily.

Fair concern and I think you're right. The fix was designed to pass the test, assuming this behavior would be tolerated. So it's better we drop it, even though it means the test has a workaround for this macOS bug, which I think we could be okay with.

[xyzconstant]

Just opened #319 to keep track of the macOS accept crash.

[ryanofsky]

Code review ACK b5b357f. Thanks for the test! Left suggestions below but think this could also be merged as is.

[xyzconstant]

Thanks for the review @ryanofsky!

I force-pushed changes based on your suggestion to tighten the KJ_EXPECT(...) check, plus the KJ_REQUIRE() check on accept_error_seen() for non-macOS builds.

b5b357f -> 496fb84 (compare)

[ryanofsky]

Code review ACK 496fb84. Since last review just tightened the checks to only allow the error on macos

[ryanofsky]

In commit "test: cover immediate client disconnects for ListenConnections" (496fb84)

Now that the fix is merged would be nice to simplify the comment to something like This a Cap'n Proto bug, fixed by https://github.com/capnproto/capnproto/pull/2748