PacketRelayDocs

How a connection works#

PacketRelay has no HTTP framework. Every TCP connection is accepted raw, its request head is read and hand-parsed, and it is classified into one of four routes before anything WebSocket-shaped happens — src/main.rs:454-486. A connection that survives classification is upgraded, announces a role in its first application frame, and is joined to a session.

This page traces that path in order, from the accept call to the cleanup, and names the three points at which a connection dies without ever being told why.

The path#

  TCP accept                                          src/main.rs:358-360
      │
      ▼
  limiter.try_connect(peer_ip)                        src/main.rs:365
      │   Err → warn! → drop(stream)      ← no response at all
      ▼
  tokio::spawn per-connection task                    src/main.rs:372-376
      │
      ▼
  read_request_head                                   src/main.rs:389, 431-452
      │   ≤ 16 KiB, ≤ --hello-timeout
      │   breach / EOF / timeout → drop  ← no response at all
      ▼
  classify_request                                    src/main.rs:393, 454-486
      │
      ├── Health  → 200 "ok\n"            → shutdown
      ├── Ready   → 200 "ready\n"         → shutdown
      ├── Reject  → 404 "not found\n"     → shutdown
      │
      ▼ Legacy
  route_capacity.try_acquire                          src/main.rs:409-412
      │   None → 503 "route capacity reached\n" → shutdown
      ▼
  ReplayStream → accept_async_with_config             src/main.rs:416-423
      │   handshake failure → warn! → close
      ▼
  read_hello (--hello-timeout)                        src/connection.rs:491-546
      │   timeout / binary / bad JSON → drop, no error frame
      ▼
  role dispatch                                       src/connection.rs:68-163
      │   Desktop │ Mobile │ Host │ Spectator │ Member
      ▼
  register → session_ready or room_ready              src/connection.rs:256-266
      │
      ▼
  forwarding loop (tokio::select!)                    src/connection.rs:333-409
      │
      ▼
  ownership-aware cleanup                             src/connection.rs:411-465

Admission runs before the first byte is read#

The accept loop consults the rate limiter on the peer IP the instant a socket arrives, before the socket is handed to a task and before a single byte is read — src/main.rs:360-369. try_connect checks a sliding per-IP connection-rate window, then the per-IP concurrency cap, then the global cap, and increments both counters on success — src/rate_limit.rs:92-125.

Health probes therefore consume connection budget, because at the moment the limiter runs nothing has been read and the relay cannot know the request is a probe. A monitoring system polling /healthz once a second from one address spends 60 slots per minute against --max-connect-rate-per-ip, whose default is 240.

Note

A rejected attempt still pushes its timestamp onto the IP's history — src/rate_limit.rs:111-112. An IP that has tripped the rate limit keeps extending its own lockout for as long as it keeps trying, on purpose: the alternative rewards a client that retries hardest.

A rejection is silent#

On breach the stream is dropped. There is no HTTP status, no WebSocket close frame, no TCP reset the relay generates deliberately — only a warn! log line carrying the peer address and the reason — src/main.rs:365-369. The client observes a connection that opened and then closed with nothing on it, which is indistinguishable at the socket layer from a network fault.

The same silence covers every failure inside read_request_head: a head exceeding MAX_HTTP_HEAD_BYTES (16 KiB), a peer that closes before sending \r\n\r\n, a read error, or the --hello-timeout deadline elapsing all return None, and the caller returns without writing anything — src/main.rs:389-392, src/main.rs:431-452.

Head parsing happens in the connection's own task#

read_request_head is called inside the spawned task, not in the accept loop — src/main.rs:372-376, src/main.rs:389. A peer that opens a socket and sends nothing for the full --hello-timeout window stalls only itself; the accept loop is already back at listener.accept(). The comment at src/main.rs:361-363 states the intent, and the test slow_request_head_does_not_block_a_second_connection pins it — src/main.rs:636-667.

That slow peer still holds a limiter slot for the duration, so the defence is against head-of-line blocking, not against connection exhaustion. Exhaustion is the rate limiter's job.

Only the request line is parsed#

classify_request looks at the bytes before the first \r\n\r\n, takes the first line, and splits it on ASCII whitespace — src/main.rs:454-464. Nothing else in the head is read. No header is inspected, no Host is required, and Origin is never checked anywhere in the codebase.

The line must be exactly three fields: the method GET, a target beginning /, and the version HTTP/1.1src/main.rs:465-473. Anything else classifies as Reject. The full rule list and the exact responses are in HTTP endpoints.

Classification is by exact target string, with a prefix guard that turns /health?detail=1 and its neighbours into a 404 rather than into a WebSocket upgrade — src/main.rs:474-486. Every other target, including /, falls to Legacy.

Legacy is the only route that takes a permit#

RouteCapacity holds a single Semaphore sized --max-connections, and try_acquire returns a permit only for RequestRoute::Legacy — every other route matches the _ => None arm — src/main.rs:52-70. The health, ready and reject routes have already returned by the time the permit is requested, so the None they would receive never matters.

A Legacy connection that cannot get a permit is answered with 503 Service Unavailable and the body route capacity reached\nsrc/main.rs:409-412. This is the one capacity failure that produces an HTTP response, because by this point a valid request line has been parsed and the relay knows it is talking to something that speaks HTTP.

The permit is bound to _route_permit and released when the connection task ends. The semaphore and the rate limiter are separate mechanisms sized from the same flag: the limiter counts every TCP connection including probes, the semaphore counts only upgrades in flight.

The handshake re-reads bytes the relay already consumed#

By the time the route is known, the request head has been drained from the socket. ReplayStream wraps the TcpStream with those bytes as a prefix and serves them from the prefix before falling through to the socket — src/main.rs:81-134. Tungstenite is then handed the ReplayStream and performs its own complete handshake validation, including Sec-WebSocket-Key and Sec-WebSocket-Versionsrc/main.rs:416-423.

Classification is therefore independent of TCP packet boundaries without any weakening of handshake checking, which is the failure mode the design exists to avoid: a hand-rolled path sniffer that also accepts the handshake would be a second, weaker WebSocket implementation. The packet-boundary half of that is pinned by fragmented_request_head_is_classified_only_after_the_exact_target_arrivessrc/main.rs:586-605.

The socket configuration applied at the same call is fixed except for one flag: 16 KiB read and write buffers, a 128 KiB maximum write buffer, and max_message_size and max_frame_size both set to --max-message-bytessrc/main.rs:417-422.

Route selection is by hello, not by path#

The upgraded connection is passed to handle_connection, which reads exactly one frame under the --hello-timeout deadline — src/connection.rs:491-546. That frame must be text; a binary first frame is refused. It must parse as one of five tagged hello envelopes, and the tag alone selects the role: Desktop, Mobile, Host, Spectator or Membersrc/protocol.rs:76-110.

Every non-reserved path reaches the same handler, so the path carries no protocol meaning. /, /ws, and /anything/at/all are identical to the relay. See Core concepts for what each role then does.

A hello that times out, arrives binary, fails to parse, or arrives after the peer has already closed produces a log line and a dropped connection — no error frame is sent. Failures after a successful parse do send one: an invalid session_id yields {"type":"error","message":"invalid session ID"} and then a close — src/connection.rs:41-50.

Important

No path in the relay ever sends a WebSocket Close frame with a status code. Error envelopes are sent as ordinary text frames and the sink is then dropped. A client that distinguishes close codes will see none of them.

The forwarding loop#

The loop is a tokio::select! over two arms: this connection's bounded outbound mpsc receiver, and its WebSocket stream — src/connection.rs:333-409.

Every inbound text frame is charged to the per-connection MessageBudget with text.len() before any role dispatch — src/connection.rs:346. A breach sends an error envelope and breaks the loop. Because the charge happens first, keepalive {"type":"ping"} frames are charged even though they are then dropped; because only the Message::Text arm charges, binary frames, pongs and raw frames cost nothing and are silently discarded — src/connection.rs:405.

Dispatch is by session kind and role. A bridge frame is forwarded verbatim to the peer, a room frame is wrapped in a member_frame envelope carrying the relay-authenticated sender, and a host frame is fanned out verbatim to every spectator — src/connection.rs:357-394. There is no spectator arm, so spectator-to-host frames are charged to the budget and then discarded.

Fan-out awaits each recipient's send sequentially with no timeout and no drop policy — src/connection.rs:383-394. One recipient whose --outbound-queue is full stalls the sender's entire fan-out.

Cleanup is ownership-aware#

When the loop breaks, the handler logs the disconnect, drops its receiver, and unregisters — src/connection.rs:411-465. Every unregister path requires that the stored channel is the same channel this connection registered, tested with same_channelsrc/session.rs:299-331, src/session.rs:378-451.

That check exists because a desktop or host reconnect silently replaces the stored sender — src/session.rs:169-175, src/session.rs:202-206. Without it, the displaced connection's later cleanup would evict its own successor and emit a peer_disconnected that is untrue. With it, the stale cleanup is a no-op and no event is sent at all; src/session.rs:617-647 pins the bridge case and src/session.rs:586-615 the room case.

Only a cleanup that actually removed a slot notifies anyone. A departing bridge peer produces peer_disconnected to the survivor, a departing host produces host_disconnected to every spectator followed by a fresh viewer_count, and a departing member produces member_left then member_count to the rest of the room. Emptying a session deletes it, and emptying a room additionally drops that room's trust-on-first-use pins — src/connection.rs:447-449.

What the path does not contain#

These absences are deliberate and documented rather than pending.

Stage Status
TLS termination Not in process. The relay speaks plain HTTP and WebSocket; the hosting edge terminates TLS, so clients use wss://
Origin header check Not present anywhere in src/. Any origin may upgrade
Post-hello idle timeout None exists. --hello-timeout covers the request head and the first hello only — src/connection.rs:333-409
Server-initiated keepalive None. The relay never sends a WebSocket Ping; it echoes Pong to a client Ping — src/connection.rs:397-399
Graceful shutdown None. The accept loop is unconditional and no signal is handled — src/main.rs:357-381
Cross-process state None. Sessions, pins and limiter state are process-local; a restart loses all of it
Warning

With no post-hello idle timeout and no server-initiated ping, a connection that completes its hello and then goes silent holds a session slot, a limiter slot and a semaphore permit until the peer or the network closes it. The reaper only removes sessions, and it measures age from creation rather than from activity — src/session.rs:137-147.

  • Core concepts — sessions, roles and the three session kinds the hello selects between.
  • HTTP endpoints — the exact route table, every validation rule and the literal response bytes.
  • Message reference — every hello a client may send and every envelope the relay may return.
  • Limits & capacity — the caps, budgets and backpressure this path enforces, with defaults.
  • Security model — what admission proves, and the four roles it proves nothing about.
  • Troubleshooting — the client-visible symptoms these silent drops produce.