Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ A modern HTTP(S) client for the command line, implemented in Go.
- **Response formatting** - Automatic formatting and syntax highlighting for JSON, XML, YAML, HTML, CSS, CSV, Markdown, MessagePack, Protocol Buffers, and more
- **Image rendering** - Display images directly in your terminal
- **WebSocket support** - Bidirectional WebSocket connections with automatic JSON formatting
- **WebTransport support** - HTTP/3 stream and datagram sessions over direct UDP
- **gRPC support** - Make gRPC calls with automatic reflection, discovery, and JSON-to-protobuf conversion
- **Authentication** - Built-in support for Basic Auth, Bearer Token, AWS Signature V4, and mTLS
- **Compression** - Select automatic, Brotli, gzip, zstd, or disabled response decoding
Expand Down Expand Up @@ -58,6 +59,7 @@ fetch picsum.photos/1024/1024
- **[Output Formatting](docs/output-formatting.md)** - Supported content types and formatting options
- **[Image Rendering](docs/image-rendering.md)** - Terminal image protocols and formats
- **[WebSocket](docs/websocket.md)** - Bidirectional WebSocket connections
- **[WebTransport](docs/webtransport.md)** - HTTP/3 stream and datagram sessions
- **[gRPC](docs/grpc.md)** - Making gRPC requests with Protocol Buffers
- **[Advanced Features](docs/advanced-features.md)** - DNS, proxies, TLS, HTTP versions, and more
- **[Encrypted ClientHello](docs/ech.md)** - ECH modes, discovery, and downgrade safety
Expand Down
12 changes: 12 additions & 0 deletions docs/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,18 @@ valid only with a `ws://` or `wss://` URL. Text lines, interactive entries, and
incoming messages are bounded to 16 MiB; binary stdin is streamed in bounded
chunks. See [WebSocket](websocket.md).

### WebTransport options

`--webtransport URL` opens an HTTPS WebTransport session over HTTP/3 and direct
UDP. The default `--wt-mode stream` uses one reliable bidirectional stream.
`--wt-mode datagram` uses unreliable datagrams. Use `--wt-datagram-mode
lines|binary` to split piped input, and repeat `--wt-protocol PROTOCOL` to offer
application protocols. Received datagrams are compact JSON Lines records with
base64 data. WebTransport does not support proxies, redirects, retries,
formatting, HAR, output files, Unix sockets, or Digest authentication. EOF on
datagram input does not close the session; use Ctrl+C when the peer remains
open. `--dry-run` does not access the network or consume stdin.

## Agent Skill Options

### `--skill`
Expand Down
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ reference by task.
- [DNS, proxy, HTTP versions, and TLS/ECH](advanced-features.md)
- [Encrypted ClientHello](ech.md)
- [WebSockets](websocket.md)
- [WebTransport](webtransport.md)
- [gRPC](grpc.md)
- [Image rendering](image-rendering.md)
- [Self-update and installation](updates.md)
Expand Down
25 changes: 25 additions & 0 deletions docs/webtransport.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# WebTransport

Use `fetch --webtransport https://host/path` to open a WebTransport session.
WebTransport uses HTTP/3 over a direct UDP connection. It does not support
proxies, Unix sockets, redirects, retries, output files, formatting, or
Digest authentication.

The default mode is one reliable bidirectional stream. `-d` and `-j` are sent
after the session handshake, followed by piped standard input. The stream is
closed for writing at EOF, but fetch continues to read until the peer closes.
Stream output is raw bytes when stdout is redirected and is escaped when it is
a terminal.

Use `--wt-mode datagram` for unreliable datagrams. `--wt-datagram-mode lines`
sends one datagram per line; `binary` sends 1 KiB chunks. Received datagrams
are JSON Lines records with `sequence`, `length`, and base64 `data` fields.
Datagram input ending does not close the session. Use Ctrl+C when the peer does
not close it.

Repeat `--wt-protocol` to advertise application protocols. Protocols are
validated and sent in offer order. `--dry-run` prints the CONNECT metadata and
does not resolve DNS, open UDP, or consume standard input.

This implementation uses WebTransport draft-16. Use HTTPS and TLS 1.3-capable
HTTP/3 servers.
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@ go 1.27.0
require (
github.com/andybalholm/brotli v1.2.2
github.com/coder/websocket v1.8.15
github.com/dunglas/httpsfv v1.1.0
github.com/goccy/go-yaml v1.19.2
github.com/klauspost/compress v1.19.2
github.com/mattn/go-runewidth v0.0.28
github.com/quic-go/quic-go v0.61.0
github.com/quic-go/webtransport-go v0.12.0
github.com/ryanfowler/readability v0.1.1
github.com/tinylib/msgp v1.6.4
github.com/yuin/goldmark v1.8.5
Expand Down
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ github.com/coder/websocket v1.8.15 h1:6B2JPeOGlpff2Uz6vOEH1Vzpi0iUz20A+lPVhPHtNU
github.com/coder/websocket v1.8.15/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dunglas/httpsfv v1.1.0 h1:Jw76nAyKWKZKFrpMMcL76y35tOpYHqQPzHQiwDvpe54=
github.com/dunglas/httpsfv v1.1.0/go.mod h1:zID2mqw9mFsnt7YC3vYQ9/cjq30q41W+1AnDwH8TiMg=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
Expand All @@ -24,6 +26,8 @@ github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.61.0 h1:ui88A53s8MSVYLC56en0KQ17HARk+9986Dn0SBfKNvA=
github.com/quic-go/quic-go v0.61.0/go.mod h1:9So2anK4Tp22URSQq00k+Vo2PNkle96ycDPDHL4s9vs=
github.com/quic-go/webtransport-go v0.12.0 h1:CpnKNwZvdV0LD73xoHO8QaR0NI3llqpWRwnazdZS0sE=
github.com/quic-go/webtransport-go v0.12.0/go.mod h1:GHne8aRFJ24h73pAMrcywXtuaz/ShBXCLXLvG/NPFdU=
github.com/ryanfowler/readability v0.1.1 h1:MpvDWXpeawWSvvUj6YTXL8vqWhz06FuKRvaap2NVrZc=
github.com/ryanfowler/readability v0.1.1/go.mod h1:rNsnkYiYbZXCpEyIZyXbg26Dd0Z+6+11QIm6PBeeS1A=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
Expand Down
83 changes: 79 additions & 4 deletions internal/cli/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"os"
"strings"

"github.com/dunglas/httpsfv"
"github.com/ryanfowler/fetch/internal/aws"
"github.com/ryanfowler/fetch/internal/config"
"github.com/ryanfowler/fetch/internal/core"
Expand Down Expand Up @@ -52,7 +53,12 @@ type App struct {
Force bool
SortHeaders bool
WS bool // set when URL scheme is ws:// or wss://
WebTransport bool
WTMode core.WTMode
WTDgramMode core.WTDatagramMode
WTProtocols []string
Method string
MethodExplicit bool
Multipart []core.KeyVal[string]
Output string
ProtoDesc string
Expand All @@ -78,6 +84,9 @@ type App struct {
pagerSet bool
noPagerSet bool
wsMessageModeSet bool
wsInteractiveSet bool
wtDgramModeSet bool
wtModeSet bool

provenance map[string]OptionProvenance
}
Expand Down Expand Up @@ -571,12 +580,13 @@ func (a *App) CLI() *CLI {
},

boolFlag(&a.Version, "version", "V", "Print version"),
boolFlag(&a.WebTransport, "webtransport", "", "Use WebTransport over HTTP/3"),

Flag{
Long: "ws-interactive",
Args: "MODE",
Description: "WebSocket prompt mode",
IsSet: func() bool { return a.WSInteractive != core.WSInteractiveAuto },
IsSet: func() bool { return a.wsInteractiveSet },
Fn: a.parseWSInteractiveFlag,
}.WithValues([]core.KeyVal[string]{
{Key: "auto", Val: "Use interactive prompt when attached to a terminal"},
Expand All @@ -599,6 +609,14 @@ func (a *App) CLI() *CLI {
{Key: "binary", Val: "Send binary messages"},
}),

Flag{Long: "wt-datagram-mode", Args: "MODE", Description: "WT datagram stdin mode", IsSet: func() bool { return a.wtDgramModeSet }, Fn: a.parseWTDgramModeFlag}.WithValues([]core.KeyVal[string]{
{Key: "lines", Val: "One datagram per line"}, {Key: "binary", Val: "One datagram per 1 KiB"},
}),
Flag{Long: "wt-mode", Args: "MODE", Description: "WT data mode", IsSet: func() bool { return a.wtModeSet }, Fn: a.parseWTModeFlag}.WithValues([]core.KeyVal[string]{
{Key: "stream", Val: "Reliable bidirectional stream"}, {Key: "datagram", Val: "Unreliable datagrams"},
}),
{Long: "wt-protocol", Args: "PROTOCOL", Description: "WT application protocol (repeatable)", IsSet: func() bool { return len(a.WTProtocols) > 0 }, Fn: a.parseWTProtocolFlag},

// Custom: XML body
{
Short: "x",
Expand Down Expand Up @@ -676,9 +694,15 @@ func (a *App) parseDataFlag(value string) error {
if err != nil {
return err
}
a.Data, a.ContentType, err = core.DetectContentType(r, path)
if err != nil {
return err
// Stdin is one-shot. Do not sniff it while parsing: WebTransport defers
// application input until after its handshake, and dry-run must not read it.
if value == "@-" && a.WebTransport {
a.Data, a.ContentType = r, "application/octet-stream"
} else {
a.Data, a.ContentType, err = core.DetectContentType(r, path)
if err != nil {
return err
}
}
a.dataSet = true
return nil
Expand Down Expand Up @@ -811,7 +835,58 @@ func (a *App) parseXMLFlag(value string) error {
return nil
}

func (a *App) parseWTModeFlag(value string) error {
switch value {
case "stream":
a.WTMode = core.WTStream
case "datagram":
a.WTMode = core.WTDatagram
default:
return core.NewValueError("wt-mode", value, "must be one of [stream, datagram]", false)
}
a.wtModeSet = true
return nil
}

func validateWTProtocol(value string) error {
_, err := httpsfv.Marshal(httpsfv.NewItem(value))
return err
}

func (a *App) parseWTProtocolFlag(value string) error {
if strings.TrimSpace(value) == "" {
return core.NewValueError("wt-protocol", value, "must not be empty", false)
}
for _, protocol := range a.WTProtocols {
if protocol == value {
return core.NewValueError("wt-protocol", value, "duplicate protocol", false)
}
}
// Validate with the same Structured Fields encoder used by webtransport-go.
// Keep this dependency out of normal parsing by validating the item syntax
// through the small shared helper.
if err := validateWTProtocol(value); err != nil {
return core.NewValueError("wt-protocol", value, err.Error(), false)
}
a.WTProtocols = append(a.WTProtocols, value)
return nil
}

func (a *App) parseWTDgramModeFlag(value string) error {
switch value {
case "lines":
a.WTDgramMode = core.WTDatagramLines
case "binary":
a.WTDgramMode = core.WTDatagramBinary
default:
return core.NewValueError("wt-datagram-mode", value, "must be one of [lines, binary]", false)
}
a.wtDgramModeSet = true
return nil
}

func (a *App) parseWSInteractiveFlag(value string) error {
a.wsInteractiveSet = true
switch value {
case "auto":
a.WSInteractive = core.WSInteractiveAuto
Expand Down
89 changes: 89 additions & 0 deletions internal/cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ package cli

import (
"bytes"
"crypto/tls"
"errors"
"fmt"
"io"
"net/url"
Expand All @@ -12,6 +14,7 @@ import (
"strings"
"time"

"github.com/ryanfowler/fetch/internal/client"
"github.com/ryanfowler/fetch/internal/core"
"github.com/ryanfowler/fetch/internal/curl"
)
Expand Down Expand Up @@ -241,6 +244,14 @@ func isFlagVisibleOnOS(flagOS []string) bool {

func Parse(args []string) (*App, error) {
var app App
// Mark this before parsing so a one-shot stdin body can skip MIME sniffing
// even when --webtransport appears after -d @-.
for _, arg := range args {
if arg == "--webtransport" {
app.WebTransport = true
break
}
}

cli := app.CLI()
long, err := parseWithFlags(cli, args)
Expand Down Expand Up @@ -273,6 +284,9 @@ func Parse(args []string) (*App, error) {
if app.wsMessageModeSet && !app.WS {
return &app, fmt.Errorf("'--ws-message-mode' requires a ws:// or wss:// URL")
}
if err := ValidateWebTransport(&app); err != nil {
return &app, err
}

if err := validateSchemeExclusives(&app, cli, long); err != nil {
return &app, err
Expand All @@ -290,6 +304,81 @@ func Parse(args []string) (*App, error) {
return &app, nil
}

// ValidateWebTransport validates mode-specific options after CLI and config
// values have been merged. It performs no I/O and is safe to call preflight.
func ValidateWebTransport(app *App) error {
if app == nil || !app.WebTransport {
if app != nil && (app.wtModeSet || app.wtDgramModeSet || len(app.WTProtocols) > 0) {
return errors.New("WebTransport options require --webtransport")
}
return nil
}
if app.WS {
return errors.New("WebSocket and WebTransport cannot be used together")
}
if app.InspectDNS || app.InspectTLS || app.Update || app.CheckUpdate || app.Skill || app.InstallSkill != "" || app.UninstallSkill != "" {
return errors.New("WebTransport cannot be combined with an inspection, update, or skill command")
}
if app.wtDgramModeSet && app.WTMode != core.WTDatagram {
return errors.New("--wt-datagram-mode requires --wt-mode datagram")
}
if name, ok := app.CLI().Options().Unsupported(ModeWebTransport); ok {
return fmt.Errorf("--%s cannot be used with WebTransport", name)
}
if app.URL == nil {
return nil
}
if !strings.EqualFold(app.URL.Scheme, "https") {
return errors.New("WebTransport requires an https:// URL")
}
if app.Cfg.HTTP == core.HTTP1 || app.Cfg.HTTP == core.HTTP2 {
return fmt.Errorf("WebTransport requires HTTP/3; cannot use %s", app.Cfg.HTTP.String())
}
if app.Cfg.Format != core.FormatUnknown {
return errors.New("--format cannot be used with WebTransport")
}
if app.Cfg.TLSMax != nil && *app.Cfg.TLSMax < tls.VersionTLS13 {
return errors.New("WebTransport requires max-tls 1.3 or higher")
}
// These values can come from a merged config file, so registry IsSet
// checks alone are not sufficient here.
for _, unsupported := range []struct {
name string
set bool
}{
{"compress", app.Cfg.Compress != core.CompressionUnknown},
{"copy", app.Cfg.Copy != nil}, {"ignore-status", app.Cfg.IgnoreStatus != nil},
{"no-encode", app.Cfg.NoEncode != nil}, {"redirects", app.Cfg.Redirects != nil},
{"retry", app.Cfg.Retry != nil}, {"retry-delay", app.Cfg.RetryDelay != nil},
{"retry-unsafe", app.Cfg.RetryUnsafe != nil},
} {
if unsupported.set {
return fmt.Errorf("--%s cannot be used with WebTransport", unsupported.name)
}
}
if app.UnixSocket != "" {
return errors.New("WebTransport cannot be used with a unix socket")
}
if app.URL != nil {
decision, err := client.SelectProxy(app.Cfg.Proxy, app.URL)
if err != nil {
return err
}
if decision.URL != nil {
return errors.New("WebTransport cannot be used with a proxy")
}
}
for _, h := range app.Cfg.Headers {
if strings.EqualFold(h.Key, "Host") {
return errors.New("host header cannot be used with WebTransport")
}
if strings.EqualFold(h.Key, "WT-Available-Protocols") || strings.EqualFold(h.Key, "WT-Protocol") {
return fmt.Errorf("header %q cannot be supplied with WebTransport", h.Key)
}
}
return nil
}

func validateEquivalentAliases(app *App) error {
if app.compressSet && app.noEncodeSet && app.explicitCompress != core.CompressionOff {
return newExclusiveFlagsError("compress", "no-encode")
Expand Down
Loading