Hardening the WebSocket Server in sgcWebSockets

· Components

A WebSocket server that faces the public internet is a tempting target. The protocol has no built-in message-size limit, so a single connection can try to drag the whole process down with one crafted frame, and the upgrade handshake is an easy place to probe for sloppy validation. The latest sgcWebSockets ships a set of protections that close these doors by default, so your server stays up even under hostile traffic.

Everything below applies to all three WebSocket servers: TsgcWebSocketServer, TsgcWebSocketHTTPServer and the http.sys based TsgcWebSocketServer_HTTPAPI. The .NET edition inherits the same protections, because it runs on the same native library.

One limit that stops three memory attacks

The headline addition is a single property, MaxMessageSize, that bounds how large an inbound message may be. It defaults to 64 MB, which is generous for almost every application, and you can raise or lower it, or set it to 0 to disable the limit.

That one property defends against three different memory-exhaustion techniques that all end the same way, with the server out of RAM:

In every case the connection is closed cleanly with WebSocket close code 1009 (Message Too Big), and the server's memory never moves past the cap.

oServer := TsgcWebSocketServer.Create(nil);
oServer.Port := 80;
// accept messages up to 16 MB, reject anything larger with close 1009
oServer.MaxMessageSize := 16 * 1024 * 1024;
oServer.Active := True;

Safe frame-length parsing

WebSocket frames can carry a 64-bit length field. Per RFC 6455 the most significant bit of that field must be zero. sgcWebSockets now rejects a frame whose 64-bit length has the high bit set instead of trusting it, so the size limit above cannot be slipped past with an integer that wraps around. This check is always on and does not depend on MaxMessageSize.

Stricter handshakes

Two new options under SecurityOptions harden the WebSocket upgrade itself, and both are enabled by default.

EnforceWebSocketVersion answers any handshake that asks for a version other than 13 with 426 Upgrade Required and a Sec-WebSocket-Version: 13 header, instead of completing the upgrade. ValidateWebSocketKey rejects, with 400 Bad Request, any handshake whose Sec-WebSocket-Key is missing or is not a valid 16-byte base64 nonce. Both checks apply only to the RFC 6455 path, so older clients on legacy specifications are unaffected.

oServer := TsgcWebSocketServer.Create(nil);
oServer.SecurityOptions.EnforceWebSocketVersion := True;  // 426 on a wrong version
oServer.SecurityOptions.ValidateWebSocketKey := True;     // 400 on a malformed key
// lock the server to your own site while you are at it
oServer.SecurityOptions.OriginsAllowed := 'https://app.example.com';
oServer.Active := True;

How it fits with the firewall

These protections live at the protocol layer, inside the frame parser, which is exactly where the memory attacks happen. That is the part the Firewall and RateLimiter components cannot reach, because they see a message only after it has been decoded. The two layers complement each other: keep using the firewall and rate limiter for IP filtering, connection and message rate limits and origin policy, and let the new built-in limits guard the parser itself. For a public server we recommend setting MaxMessageSize to your real maximum, locking OriginsAllowed to your front end, and capping MaxConnections.

Upgrading

The new protections are active as soon as you update, with safe defaults, so most servers gain the memory and handshake hardening without a single code change. If your application legitimately exchanges messages larger than 64 MB, raise MaxMessageSize accordingly. Existing client code is unaffected.

Update from the sgcWebSockets download page, or grab it through GetIt or your registered account.

Questions, feedback or migration help? Get in touch, you will get a reply from the people who wrote the code.