Skip to content

net: non-blocking host netdev I/O with an epoll poller - #77

Open
0pcom wants to merge 2 commits into
tinygo-org:mainfrom
0magnet:epoll-poller
Open

net: non-blocking host netdev I/O with an epoll poller#77
0pcom wants to merge 2 commits into
tinygo-org:mainfrom
0magnet:epoll-poller

Conversation

@0pcom

@0pcom 0pcom commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

The host netdev now uses non-blocking sockets and a small epoll-based poller (netpoll_native.go): when a syscall would block (EAGAIN), the goroutine parks on a channel until the fd is ready, its deadline passes, or the fd is closed. The last case is the important one — closing an fd now wakes every goroutine parked on it (real cancellation), so a blocked Accept/Recv/Send returns promptly on shutdown instead of pinning a thread and hanging teardown.

This replaces the previous blocking-syscall + SO_*TIMEO approach so that:

  • a blocked read/accept parks the goroutine instead of pinning an OS thread;
  • closing an fd unblocks every goroutine parked on it, which is what lets a graceful shutdown actually complete.

Follow-on to the host netdev that landed via #60.

Verified against tinygo dev (0.42.0): blocked Accept unblocks on Close; 4 concurrent connections doing echo round-trips; 20 sequential request/response round trips on one connection; and a full software-TLS handshake + HTTPS GET served through it (with the crypto/tls host change in the tinygo repo). Also validated by a real skycoin daemon serving its API and syncing blocks.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C3X3gvq8ZggMRvzVzWm66i

The host netdev now uses non-blocking sockets and a small epoll-based poller
(netpoll_native.go): when a syscall would block (EAGAIN), the goroutine parks on
a channel until the fd is ready, its deadline passes, or the fd is CLOSED. The
last case is the important one — closing an fd now wakes every goroutine parked
on it (real cancellation), so a blocked Accept/Recv/Send returns promptly on
shutdown instead of pinning a thread and hanging teardown.

Verified: unit test (blocked Accept unblocks on Close; concurrent connect/
accept/read/write) and a real skycoin daemon (serves the API and syncs blocks).

Note: this fixes I/O cancellation, but the skycoin daemon's Ctrl+C issue is a
separate, pre-existing signal-delivery problem (SIGINT doesn't reach skycoin's
handler under TinyGo, though it does in isolation) — still to be root-caused.
A deadline was captured when Recv/Send entered the poller wait, so a
concurrent SetReadDeadline/SetWriteDeadline had no effect until the
next wakeup. The visible symptom: net/http's abortPendingRead — which
interrupts the server's background read by setting a past read deadline
— blocked until the client happened to send data or close, so an
HTTP/1.1 response with "Connection: close" did not close the connection
until the client gave up.

Deadline setters now call the netdev's PollInterrupt (through an
optional interface, so netdevs without a poller are unaffected), which
wakes waiters parked in that direction with an internal sentinel error;
Read/Write retry with the fresh deadline, and a Write that was mid-way
through a buffer continues from where it left off. An interrupt that
arrives between a waiter capturing its deadline and parking is
remembered on the pollDesc and consumed by the next wait, so the change
cannot be missed.
@0pcom

0pcom commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Pushed a second commit: deadline changes now interrupt I/O that is already parked in the poller.

The first version captured the deadline when Recv/Send entered the wait, so a concurrent SetReadDeadline had no effect until the next wakeup. The visible symptom was in net/http: abortPendingRead interrupts the server's background read by setting a past read deadline, and with that inert, an HTTP/1.1 response with Connection: close did not actually close the connection until the client gave up.

Mechanism: the conn deadline setters call the netdev's PollInterrupt (optional interface — netdevs without a poller are unaffected), which wakes parked waiters with an internal sentinel; Read/Write retry under the fresh deadline, a partially-sent Write continues from where it left off, and an interrupt that lands between deadline-capture and parking is remembered on the pollDesc so it cannot be missed.

Verified: an HTTP/1.1 Connection: close request now closes promptly (plain and TLS), and the earlier tests (accept-unblock-on-close, concurrent echo, 20 sequential round trips, TLS handshake + HTTPS GET) still pass.

@yohimik

yohimik commented Sep 5, 2026

Copy link
Copy Markdown

I audited the local Crier shutdown patch. Repeated Close on a TCP listener, TCP connection, or UDP connection can close a new socket that has reused the descriptor. A test now closes the old socket, installs a replacement at the same descriptor, calls Close again, and checks the replacement. All three cases failed before the guard. A separate test makes 32 concurrent Close calls and checks that only one reaches the netdev.

The guard and tests are available in a0c3ff5. The combined fork passes these tests and blocked Accept, Read, and Write tests on Linux arm64 and amd64 under emulation, using released TinyGo 0.43.0-net.1 with an isolated source root.

This is a limited Close guard, not an equivalent to this poller. It does not solve stale descriptor use by other operations or changes to deadlines during blocked I/O. The shutdown call stays a fork stopgap. I opened no duplicate shutdown PR. Please keep cancellation and deadline work here. The guard and regression tests are available to combine with your lifecycle work. #82 only implements ListenConfig and is independent of #78 and #80.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants