interactive threadpool question

[ericphanson]

HTTP.jl v1 used interactive threadpool by default, so we made sure to configure our julias to have one (eg --threads=2,1). With HTTP.jl v2, Reseau makes its own foreign thread? Would the migration from v1 to v2 be: drop the interactive threadpool (eg --threads=2), but still set cpu=$nthreads+1 in eg k8s manifests so that the Reseau thread has room? Appreciate any thoughts.

[quinnj]

Well shoot, this is a great question that caught a regression: #1344.

But also to clarify: Reseau.jl starts a foreign thread on its own (via direct OS ccalls and then julia "adopts" it); Julia puts the thread in the :foreign pool because it's not sure where it came from or whether its safe to run tasks on. Reseau.jl itself doesn't run any tasks on this thread, but exclusively uses it to check IO readiness notifications for any-threadpool tasks who need a readiness notification. i.e. any task can eagerly do network IO work itself via Reseau and if it hits a case where it needs to wait, registers the socket fd w/ Reseau internal list, and parks itself (wait()). When foreign thread gets an OS notification for the fd, it schedules the task w/ a notification that the IO operation can now be retried and should be ready.

So, from HTTP's perspective, we actually still want to run the server task (and a few children tasks that get spawned) on the :interactive threadpool, since it was built for exaclty this use-case: separate threadpool from default so certian tasks can try to remain more responsive than default pool where hot compute kernels who don't yield may prevent other tasks from working. The PR above should fix that, but yeah, I missed this in the HTTP 2.x internals rewrite.

So, long-story short: should keep the same configuration you had before! I wouldn't worry about doing cpu=$nthreads+1 though just for Reseau's foreign thread. It's a very low-traffic/overhead thread generally and is meant to easily "share" a cpu core w/ interactive/default threads doing the bulk of work.