net/http: follow redirects in the client again

[yohimik]

net/http: follow redirects in the client again

Repository tinygo-org/net. Branch upstream-pr/http-redirects, base main.

What this does

The client keeps CheckRedirect and ErrUseLastResponse, but Client.do lost
its redirect loop, so a 3xx went straight back to the caller with an unread
body. A request for a GitHub release asset returned 302 and zero bytes where Go
returns 200 and the file.

This ports the loop of Go back into http/client.go. It has

• up to 10 hops through checkRedirect and defaultCheckRedirect,
• redirectBehavior with the 301, 302 and 303 change to GET and the 307 and 308
  replay with GetBody,
• Location resolved against the URL of the current request,
• the drop of sensitive headers when a hop leaves the initial domain,
• the read and close of an intermediate body,
• ErrUseLastResponse honoured,
• refererForURL with the https to http rule.

The code follows the upstream Go source, with its comments, so a later update
from upstream is a straight comparison.

Two changes from upstream

• shouldCopyHeaderOnRedirect compares the hostnames as they are and not
  through idnaASCII, which needs golang.org/x/net/idna and its tables. The
  Unicode and the punycode spelling of one IDN host thus count as two hosts and
  lose their sensitive headers. That is the safe direction. The rule for a
  subdomain is the upstream one.
• roundTrip uses Request.URL.Host when Request.Host is empty, because only
  NewRequest fills Request.Host and a redirect hop builds its request
  directly.

Evidence

There is no CI in this repository. Checked by hand on macOS 26.6 arm64 with a
TinyGo build that carries the matching toolchain change.

• Before, a GET of a GitHub release asset returned 302 and 0 bytes.
• After, the same GET returns 200 and the file.
• A program that does http.Get("https://example.com/") builds and completes.

A downstream product ships binaries built with these changes in a production
release. dispat v1.4.0 is published and is not a prerelease. It carries
dispat-tiny-linux-amd64 and dispat-tiny-linux-arm64, built by the fork
release v0.42.0-net.4 from sha256-pinned tarballs and smoke-executed under
binfmt before upload, beside six binaries from the gc toolchain.
https://github.com/yohimik/dispat/releases/tag/services%2Fdispat%2Fv1.4.0

The acceptance record of that repository is committed at
packages/docs/docs/internals/tinygo.md. It reports the net.2 to net.4
acceptance history, an integration suite of 694 rows that passes with 0 failures
and 1 documented skip on darwin, and a size table of 0.58x to 0.63x against the
gc equivalents with TinyGo -opt=z -no-debug against go build -trimpath -ldflags "-s -w". Those figures come from that document. They are not a
measurement of this branch.

The self-update path of that program downloads a release asset over HTTPS, which
is the redirect case above.

Scope

• One file, http/client.go. No API change.
• The change is independent of the darwin work in this series and applies to
  linux as it stands.

Known gaps

• http.Client.Timeout is still inert in this port. The loop does not add a
  deadline of its own. That is a separate piece of work.
• There is no Transport implementation, so a caller-supplied Transport is
  still driven directly and does not take part in the redirect handling.

Related pull requests

This change is part of one body of work. Together the changes make programs that use the network and child processes work on hosted linux and macOS. A full CLI was tested end to end with all of them and ships binaries built this way, see dispat v1.4.0 in the evidence section.

In tinygo-org/tinygo

• sync: hand the RWMutex over instead of a re-test of the reader count tinygo#5630 sync, RWMutex hand-over. Bug fix, standalone.
• runtime: deliver os/signal notifications under the threads scheduler tinygo#5631 runtime, os/signal delivery with the threads scheduler. Bug fix, standalone.
• compiler,runtime: call fcntl through a C wrapper on darwin tinygo#5632 compiler and runtime, fcntl on darwin. Bug fix, standalone.
• runtime: implement weak.runtime_makeStrongFromWeak tinygo#5633 runtime, weak.runtime_makeStrongFromWeak. Needed by real crypto/tls.
• os: start processes with posix_spawn tinygo#5634 os, start processes with posix_spawn.
• loader: use the real crypto/tls on hosted linux and darwin tinygo#5635 loader, real crypto/tls on hosted linux and darwin.
• builder: declare the BSD socket API in the darwin libSystem stub tinygo#5636 builder, BSD socket symbols in the darwin libSystem stub.

In this repository

• net/http: follow redirects in the client again #72 net/http, follow redirects in the client again.
• net: report a name that does not exist as a not-found lookup error #73 net, report a name that does not exist as a not-found error.
• net: extend the host netdev to darwin #74 net, extend the host netdev to darwin.
• net/http: give the HTTPS client trust roots on darwin #75 net/http, give the HTTPS client trust roots on darwin.

A merge order that works. The three bug fixes are independent. tinygo-org/tinygo#5633 goes before tinygo-org/tinygo#5635. HTTPS on linux needs only tinygo-org/tinygo#5633 and tinygo-org/tinygo#5635. Full darwin support also needs tinygo-org/tinygo#5636, the net changes and a new src/net submodule pin.

[yohimik]

tinygo-org/net main has not moved since this branch was opened. It is still
70037cf, so the branch needs no rebase and it is current. v0.42.0 of the
toolchain pins src/net at that same commit, so the change applies to the
released toolchain.