Hysteria2 went silent after an uploadThe 37-line fix

At a glance

Hysteria2 is the UDP-based transport our apps prefer on many networks. In an on-device test it carried traffic for a minute during a speed test and then stopped for good, while the QUIC connection itself stayed alive. The cause was in the Hysteria server: a destination that stops reading its upload could pin the whole connection’s flow-control window, so the client could not send anything new on any stream. We reproduced it in a lab, patched the server with about 37 lines, and rolled the patch out to the servers.

What this means for you

• If you were on Hysteria2 and ran a big upload or an upload speed test, the connection could stall. Colitu noticed within seconds and moved your traffic to the TCP spare (VLESS Reality). You stayed online, but you lost Hysteria2 until the app returned to it.
• The fix is on the server side. You do not need to update the app: Android, iOS, Windows and Linux all benefit.
• The patch is on almost all servers, rolled out the same night, one server at a time. Each server’s VPN service restarted for about 2 seconds.

How we found it

We were testing port hopping for Hysteria2 on an Android phone on a home Wi-Fi network. Hopping worked: the client moved to a new UDP port about every 30 seconds (we saw five hops), and the server’s replies arrived on every new port.

The problem showed up under a speed test. Hysteria2 carried traffic for about 50 to 60 seconds and then stopped. The app’s health check (a request every 5 seconds, 4 seconds to answer, three misses allowed) declared the transport dead and moved traffic to the TCP spare. Users stayed online; Hysteria2 was lost.

We repeated the test on a server without port hopping and got the same result, so hopping was not the cause.

What the packets showed

We captured traffic on the phone. After the stall the QUIC connection was alive: acknowledgements and small control packets went both ways, and DNS and the apps’ own QUIC traffic (UDP datagrams) still flowed. What was missing: the client sent no new TCP stream data at all. The server logged nothing unusual.

A connection that is alive but will not send stream data points at flow control, not at the network.

What we ruled out

Each of these was tested, not assumed.

Reproducing it on one PC

We built the official Hysteria server v2.11.0 from source and ran it on loopback with an Xray 26.10.10 client. A harness opened parallel uploads, aborted them with a reset, and then checked whether new connections worked. The only thing that varied was how the upload’s target behaved:

A larger connection window (64 MB, with 4 MB per stream; configuration only) did not fix it. It only delayed the stall: with 6 streams it failed in round 3, with 20 streams immediately.

An upload speed test behaves exactly like stall: the test server stops reading once it has measured enough, while the browser’s request is still open.

The mechanism

1. 1A client uploads through a Hysteria2 stream. The server opens a TCP connection to the target and copies the stream into it.
2. 2The target stops reading but does not close. Its receive buffer fills, so the server’s write into the target blocks, and the server stops reading from the stream.
3. 3The stream’s unread upload data keeps occupying the connection-level receive window. A few such streams use up the whole window. From then on the client cannot send on any stream of that connection, including brand-new ones.
4. 4The client (Xray) never gives the stream up, because its own writer is blocked on the same window. On an instrumented server the “stream given up” signal arrived only when we killed the client process.
5. 5UDP datagrams are not covered by stream flow control, so DNS and app QUIC kept flowing. That is why the connection looked healthy in the capture.

The server cannot tell a stalled target from a slow one by looking at the stream. It can tell by progress: a write that makes no progress for a long time.

The fix

About 37 lines in core/server/server.go, on top of the official v2.11.0 source (commit bc766a2). Two parts:

1. every write into a TCP target:
     set write deadline = now + 10 s   (renewed before each write)
     no progress for 10 s -> the write fails -> close the stream,
     which releases its share of the connection window

2. when the client gives a stream up (stream context ends):
     close the target immediately

A slow target that keeps moving is never cut off, because every write gets a fresh 10 seconds. Only a target that has stopped accepting data is closed. We build it as v2.11.0-colitu1 from the official source tag, and the node image now builds Hysteria from source with this patch.

Results in the lab

Hysteria’s own server unit tests pass with the patch.

Result on the phone

On the patched Berlin server we ran four speed tests in a row over 160 seconds. Hysteria2 was never declared dead and traffic stayed on Hysteria2. Before the patch it died at about 60 seconds every time, on the Berlin server and on a server in Vilnius.

Rollout

The patched build went out the same night, server by server. Each server’s VPN service restarted for about 2 seconds. Almost all servers have it now; two more are scheduled. No app release was needed, so the fix reaches the Android, iOS, Windows and Linux apps at once.

Reported upstream

The bug sits in the open-source Hysteria server, and its client-side half in Xray, so every service that runs them is affected, not only ours. On 16 October we reported both and offered our fix:

• Hysteria issue #1702: the bug, how to reproduce it and what we ruled out.
• Hysteria pull request #1703: the fix ported to the current development branch, with a configurable stall timeout (10 seconds by default) and an integration test that fails without the fix and passes with it.
• Xray-core issue #7169: the client side. A client that cancels a stream it no longer needs would free the window even on servers without the fix.

Until the projects decide, our servers run the patched build described above. We will update this report when the fix is merged or changed.

Limitations

• The lab runs on loopback, with no packet loss and no added latency. It shows the mechanism and the effect of the patch, not real-world timings.
• The device result is one phone on one network: four speed tests, 160 seconds. It is encouraging, not a long-term measurement.
• During the up to 10 seconds before the deadline fires, TCP over that connection can still pause. The patch shortens the stall; it does not make it zero.
• A client-side fix would complement this: the client cancelling a stream it no longer needs would free the window without waiting for the server’s deadline.
• We do not name servers or networks beyond the city of the server, and we publish no addresses.