Summary
For HTTP responses that carry neither Content-Length nor Transfer-Encoding: chunked — i.e. bodies delimited by connection close (HTTP/1.0-style responses, CGI gateways, ICY/Shoutcast radio streams) — s->filesize stays at its "unknown" sentinel UINT64_MAX, so target_end in http_buf_read() is UINT64_MAX. The premature-end check then treats every normal connection close as a premature termination: it logs Stream ends prematurely at N, should be 18446744073709551615 and returns AVERROR(EIO) instead of AVERROR_EOF. The original upstream commit 3668701 guarded this check with s->filesize >= 0 (signed, -1 = length unknown); the signed→unsigned conversion in 2a05c8f dropped the equivalent target_end >= 0 term, silently changing the semantics so the check fires whenever the length is unknown.
Location
http_buf_read() — premature-end check:
|
uint64_t file_end = s->end_off ? s->end_off : s->filesize; |
|
uint64_t target_end = s->range_end ? s->range_end : file_end; |
|
if ((!s->willclose || s->chunksize == UINT64_MAX) && s->off >= file_end) |
|
return AVERROR_EOF; |
|
if (s->off == target_end && target_end < file_end) |
|
return AVERROR(EAGAIN); /* reached end of content range */ |
|
len = ffurl_read(s->hd, buf, size); |
|
if ((!len || len == AVERROR_EOF) && |
|
(!s->willclose || s->chunksize == UINT64_MAX) && s->off < target_end) { |
|
av_log(h, AV_LOG_ERROR, |
|
"Stream ends prematurely at %"PRIu64", should be %"PRIu64"\n", |
|
s->off, target_end |
|
); |
|
return AVERROR(EIO); |
|
} |
http_read_stream() — is_premature computed against filesize == UINT64_MAX:
|
retry: |
|
read_ret = http_buf_read(h, buf, size); |
|
while (read_ret < 0) { |
|
uint64_t target = h->is_streamed ? 0 : s->off; |
|
bool is_premature = s->filesize > 0 && s->off < s->filesize; |
|
|
|
if (read_ret == AVERROR_EXIT) |
|
break; |
|
else if (read_ret == AVERROR(EAGAIN)) { |
|
/* send new request for more data on existing connection */ |
|
AVDictionary *options = NULL; |
|
if (s->willclose) |
|
ffurl_closep(&s->hd); |
|
s->initial_requests = 0; /* continue streaming uninterrupted from now on */ |
|
read_ret = http_open_cnx(h, &options); |
|
av_dict_free(&options); |
|
if (read_ret == 0) |
|
goto retry; |
|
} |
|
|
|
if (h->is_streamed && !s->reconnect_streamed) |
|
break; |
|
|
|
if (!(s->reconnect && is_premature) && |
|
!(s->reconnect_at_eof && read_ret == AVERROR_EOF)) { |
|
if (is_premature) |
|
return AVERROR(EIO); |
|
else |
|
break; |
|
} |
Local checkout: /Users/ronagy/GitHub/nxtedition/ffmpeg/libavformat/http.c
Details
} else {
uint64_t file_end = s->end_off ? s->end_off : s->filesize;
uint64_t target_end = s->range_end ? s->range_end : file_end;
if ((!s->willclose || s->chunksize == UINT64_MAX) && s->off >= file_end)
return AVERROR_EOF;
if (s->off == target_end && target_end < file_end)
return AVERROR(EAGAIN); /* reached end of content range */
len = ffurl_read(s->hd, buf, size);
if ((!len || len == AVERROR_EOF) &&
(!s->willclose || s->chunksize == UINT64_MAX) && s->off < target_end) {
av_log(h, AV_LOG_ERROR,
"Stream ends prematurely at %"PRIu64", should be %"PRIu64"\n",
s->off, target_end
);
return AVERROR(EIO);
}
}(libavformat/http.c lines 1824–1838)
Step-by-step trace:
- Server responds e.g.
HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n<body> — no Content-Length, no Transfer-Encoding. In process_line(), Connection: close sets s->willclose = 1 (lines 1331–1333). s->filesize keeps the UINT64_MAX it was initialized to in http_connect() (line 1725), and s->chunksize keeps the not-chunked sentinel UINT64_MAX (line 1483).
- In the
http_buf_read() else-branch (lines 1824–1825): file_end = s->end_off ? s->end_off : s->filesize = UINT64_MAX; target_end = s->range_end ? s->range_end : file_end = UINT64_MAX (no Content-Range, so range_end is 0).
- When the server finishes sending the body and closes the connection,
tcp_read() returns AVERROR_EOF; retry_transfer_wrapper() (libavformat/avio.c lines 541–542) returns AVERROR_EOF for a zero-length read, so len = ffurl_read(...) at line 1830 is AVERROR_EOF.
- The condition at lines 1831–1832 evaluates fully true:
(!len || len == AVERROR_EOF) → true; (!s->willclose || s->chunksize == UINT64_MAX) → true even though willclose == 1, because chunksize == UINT64_MAX (not chunked); s->off < target_end → off < UINT64_MAX → always true when the length is unknown.
- Lines 1833–1837 log
Stream ends prematurely at N, should be 18446744073709551615 and return AVERROR(EIO).
- In
http_read_stream(), is_premature = s->filesize > 0 && s->off < s->filesize (line 1912) is also true (UINT64_MAX > 0), so the clean end of stream either returns AVERROR(EIO) to the caller (line 1934) or — with -reconnect_streamed/-reconnect on a seekable context — enters a pointless reconnect/backoff loop until the retry limits are exhausted (lines 1939–1958).
Git archaeology: the check was introduced by upstream 3668701 ("avformat/http: Return an error in case of prematurely ending data", 2015) as !len && (!s->willclose || s->chunksize < 0) && s->filesize >= 0 && s->off < s->filesize — the s->filesize >= 0 term meant "expected length is known" (signed, -1 = unknown). The signed→unsigned conversion 2a05c8f ("http: make length/offset-related variables unsigned.", 2016) rewrote target_end >= 0 && s->off < target_end as just s->off < target_end, dropping the "length known" precondition. With unsigned target_end == UINT64_MAX, s->off < target_end is effectively always true.
Impact
Severity: medium.
- Every close-delimited HTTP resource (HTTP/1.0 servers, CGI/streaming gateways without
Content-Length, ICY/Shoutcast audio streams) ends with a spurious AVERROR(EIO) error and a misleading Stream ends prematurely ... should be 18446744073709551615 log line instead of a clean EOF. Downstream demuxer/player code sees an I/O error at what is actually a normal end of stream.
- With reconnect options enabled, the false "premature" classification triggers needless reconnect attempts with exponential backoff against a stream that has legitimately ended, delaying shutdown and hammering the origin.
Suggested fix
Restore the "expected length is known" precondition that 2a05c8f dropped — an unknown length means a connection close is a normal EOF:
--- a/libavformat/http.c
+++ b/libavformat/http.c
@@ static int http_buf_read(URLContext *h, uint8_t *buf, int size)
len = ffurl_read(s->hd, buf, size);
- if ((!len || len == AVERROR_EOF) &&
- (!s->willclose || s->chunksize == UINT64_MAX) && s->off < target_end) {
+ if ((!len || len == AVERROR_EOF) && target_end != UINT64_MAX &&
+ (!s->willclose || s->chunksize == UINT64_MAX) && s->off < target_end) {
av_log(h, AV_LOG_ERROR,
"Stream ends prematurely at %"PRIu64", should be %"PRIu64"\n",
s->off, target_end
);
return AVERROR(EIO);
}Complementary fix in http_read_stream() so the unknown-length case is not classified as premature there either (otherwise line 1934 still converts the propagated AVERROR_EOF into AVERROR(EIO) on non-streamed contexts, and the reconnect loop still triggers):
@@ static int http_read_stream(URLContext *h, uint8_t *buf, int size)
- bool is_premature = s->filesize > 0 && s->off < s->filesize;
+ bool is_premature = s->filesize > 0 && s->filesize != UINT64_MAX &&
+ s->off < s->filesize;
This matches the pre-2a05c8f813 semantics (unknown length ⇒ connection close is a normal EOF).
Upstream status
The identical condition exists in upstream FFmpeg master (git show upstream/master:libavformat/http.c, lines 1766–1774 — same (!len || len == AVERROR_EOF) && (!s->willclose || s->chunksize == UINT64_MAX) && s->off < target_end check, and the same s->filesize > 0 && s->off < s->filesize premature test in its reconnect logic). This is not a fork regression; the http_buf_read() part of the fix is worth reporting/submitting upstream as well.
Consolidates duplicate findings from the review: avformat/http: premature-end check turns clean EOF of read-until-close responses (no Content-Length) into AVERROR(EIO).
Summary
For HTTP responses that carry neither
Content-LengthnorTransfer-Encoding: chunked— i.e. bodies delimited by connection close (HTTP/1.0-style responses, CGI gateways, ICY/Shoutcast radio streams) —s->filesizestays at its "unknown" sentinelUINT64_MAX, sotarget_endinhttp_buf_read()isUINT64_MAX. The premature-end check then treats every normal connection close as a premature termination: it logsStream ends prematurely at N, should be 18446744073709551615and returnsAVERROR(EIO)instead ofAVERROR_EOF. The original upstream commit 3668701 guarded this check withs->filesize >= 0(signed,-1= length unknown); the signed→unsigned conversion in 2a05c8f dropped the equivalenttarget_end >= 0term, silently changing the semantics so the check fires whenever the length is unknown.Location
http_buf_read()— premature-end check:FFmpeg/libavformat/http.c
Lines 1824 to 1838 in 9a83bff
http_read_stream()—is_prematurecomputed againstfilesize == UINT64_MAX:FFmpeg/libavformat/http.c
Lines 1908 to 1937 in 9a83bff
Local checkout:
/Users/ronagy/GitHub/nxtedition/ffmpeg/libavformat/http.cDetails
(
libavformat/http.clines 1824–1838)Step-by-step trace:
HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n<body>— noContent-Length, noTransfer-Encoding. Inprocess_line(),Connection: closesetss->willclose = 1(lines 1331–1333).s->filesizekeeps theUINT64_MAXit was initialized to inhttp_connect()(line 1725), ands->chunksizekeeps the not-chunked sentinelUINT64_MAX(line 1483).http_buf_read()else-branch (lines 1824–1825):file_end = s->end_off ? s->end_off : s->filesize=UINT64_MAX;target_end = s->range_end ? s->range_end : file_end=UINT64_MAX(noContent-Range, sorange_endis 0).tcp_read()returnsAVERROR_EOF;retry_transfer_wrapper()(libavformat/avio.clines 541–542) returnsAVERROR_EOFfor a zero-length read, solen = ffurl_read(...)at line 1830 isAVERROR_EOF.(!len || len == AVERROR_EOF)→ true;(!s->willclose || s->chunksize == UINT64_MAX)→ true even thoughwillclose == 1, becausechunksize == UINT64_MAX(not chunked);s->off < target_end→off < UINT64_MAX→ always true when the length is unknown.Stream ends prematurely at N, should be 18446744073709551615and returnAVERROR(EIO).http_read_stream(),is_premature = s->filesize > 0 && s->off < s->filesize(line 1912) is also true (UINT64_MAX > 0), so the clean end of stream either returnsAVERROR(EIO)to the caller (line 1934) or — with-reconnect_streamed/-reconnecton a seekable context — enters a pointless reconnect/backoff loop until the retry limits are exhausted (lines 1939–1958).Git archaeology: the check was introduced by upstream 3668701 ("avformat/http: Return an error in case of prematurely ending data", 2015) as
!len && (!s->willclose || s->chunksize < 0) && s->filesize >= 0 && s->off < s->filesize— thes->filesize >= 0term meant "expected length is known" (signed,-1= unknown). The signed→unsigned conversion 2a05c8f ("http: make length/offset-related variables unsigned.", 2016) rewrotetarget_end >= 0 && s->off < target_endas justs->off < target_end, dropping the "length known" precondition. With unsignedtarget_end == UINT64_MAX,s->off < target_endis effectively always true.Impact
Severity: medium.
Content-Length, ICY/Shoutcast audio streams) ends with a spuriousAVERROR(EIO)error and a misleadingStream ends prematurely ... should be 18446744073709551615log line instead of a clean EOF. Downstream demuxer/player code sees an I/O error at what is actually a normal end of stream.Suggested fix
Restore the "expected length is known" precondition that 2a05c8f dropped — an unknown length means a connection close is a normal EOF:
Complementary fix in
http_read_stream()so the unknown-length case is not classified as premature there either (otherwise line 1934 still converts the propagatedAVERROR_EOFintoAVERROR(EIO)on non-streamed contexts, and the reconnect loop still triggers):This matches the pre-2a05c8f813 semantics (unknown length ⇒ connection close is a normal EOF).
Upstream status
The identical condition exists in upstream FFmpeg master (
git show upstream/master:libavformat/http.c, lines 1766–1774 — same(!len || len == AVERROR_EOF) && (!s->willclose || s->chunksize == UINT64_MAX) && s->off < target_endcheck, and the sames->filesize > 0 && s->off < s->filesizepremature test in its reconnect logic). This is not a fork regression; thehttp_buf_read()part of the fix is worth reporting/submitting upstream as well.Consolidates duplicate findings from the review: avformat/http: premature-end check turns clean EOF of read-until-close responses (no Content-Length) into AVERROR(EIO).