Backpressure: Send-Queue Limits for Slow WebSocket Clients | eSeGeCe Blog

Backpressure: Send-Queue Limits for Slow WebSocket Clients

· Components
Backpressure: Send-Queue Limits for Slow WebSocket Clients

A server that broadcasts a fast feed produces messages at its own pace, not at the pace of the slowest client. Market data, telemetry, a live log tail: the server pushes, and every connected client is expected to keep up. Most of them do. One of them, sooner or later, does not.

When a client stops draining its socket, the messages you send it do not disappear. They pile up in that connection's send queue, and the queue is memory. A phone that went into a train tunnel, a browser tab that got throttled, a client process that stopped calling read, all of them quietly grow your server's memory until something falls over. sgcWebSockets 2026.7 lets you bound that queue and decide what happens when it is full.

The slow consumer

The problem is not the network being slow, it is the mismatch. Your feed publishes 500 messages a second. A client on a bad link drains 50. The other 450 have to go somewhere, and "somewhere" is a per connection list in the server process. After a minute that one connection is holding tens of thousands of messages. After ten minutes it is holding your RAM.

The damage is not limited to the slow client either. The memory it eats is memory the healthy clients do not get, and once the process starts swapping or the allocator starts struggling, everybody's latency goes with it. One phone in a tunnel becomes an outage for everyone.

Until now the send queue in sgcWebSockets was unbounded, which is fine while every client keeps up and unpleasant the first time one does not. The fix is to put a ceiling on it.

Bounding the queue

The queue lives under the component's QueueOptions, next to the existing Text, Binary and Ping level sub-options. Two new properties bound it: MaxQueueSize caps how many pending messages a single connection may hold, and OverflowPolicy says what to do with the next message once that cap is reached.

uses
  sgcWebSocket, sgcWebSocket_Classes, sgcWebSocket_Types;

begin
  // messages are queued per connection and flushed by the queue thread
  oServer.QueueOptions.Text.Level   := qmLevel1;
  oServer.QueueOptions.Binary.Level := qmLevel1;

  // at most 1000 pending messages per connection
  oServer.QueueOptions.MaxQueueSize := 1000;

  // when the queue is full, throw away the oldest pending message
  oServer.QueueOptions.OverflowPolicy := qopDropOldest;
end;

QueueOptions is published on TsgcWebSocketServer, TsgcWebSocketHTTPServer and TsgcWebSocketClient, and the component copies it onto every connection as it is accepted, so the limit is per connection, not per server. A thousand connections with MaxQueueSize = 1000 gives you a worst case of a thousand pending messages each, and that worst case is now a number you can multiply out on paper before you deploy.

Choosing an overflow policy

There is no universally right answer here, which is why there are three. The policy you want depends on what your messages mean.

qopDropOldest is right for a live feed where only the newest value matters. A price tick, a sensor reading, a dashboard gauge: the message from four seconds ago is worthless the moment the next one arrives. Dropping from the front of the queue means the client that fell behind and then recovered gets the current picture, not a backlog of stale ones it has to chew through before it catches up. This is the default.

qopDropNewest is right when the earlier messages are the ones that matter and you would rather shed the new load. If a client is being walked through a sequence and the head of that sequence is the meaningful part, keep what is already queued and refuse the newcomer.

qopDisconnect is right when a client that cannot keep up is a client you do not want. It is the honest option: instead of silently feeding that connection a gap-ridden, corrupted view of the stream, you close it. The client's WatchDog reconnects, it resubscribes, and it starts again from a known state. A client that knows it was disconnected can recover. A client that does not know it lost messages cannot.

// A market data feed: only the newest tick is worth anything.
oServer.QueueOptions.MaxQueueSize   := 500;
oServer.QueueOptions.OverflowPolicy := qopDropOldest;

// A sequence where the head matters: shed the new load instead.
oServer.QueueOptions.OverflowPolicy := qopDropNewest;

// A client that falls this far behind is a client we do not want.
oServer.QueueOptions.OverflowPolicy := qopDisconnect;

Whatever you pick, pick it deliberately. Dropping messages is a trade-off, and the point of these three values is that you get to choose which trade-off you are making instead of discovering it in production.

Knowing before you drop: OnSendBufferFull

A queue that silently drops messages is a queue that lies to you. OnSendBufferFull is the escape hatch. It fires before anything is discarded, it hands you the connection and the current queue size, and the var DropMessage parameter lets your code veto the drop.

procedure TForm1.WSServerSendBufferFull(Connection: TsgcWSConnection;
  const QueueSize: Integer; var DropMessage: Boolean);
begin
  // fires BEFORE anything is dropped: log it, count it, alert on it
  DoLog('slow consumer ' + Connection.IP + ' [' + Connection.Guid + '] queue=' +
    IntToStr(QueueSize));
  Inc(FSlowConsumerHits);

  // this connection is a paying feed we must not truncate: keep the message
  // and let the queue grow past the cap for this one send
  if IsPriorityConnection(Connection) then
    DropMessage := False;
end;

Set DropMessage := False and the overflow policy is skipped for that message: it is queued anyway, above the cap. Use that sparingly, because a veto you always apply is the same thing as no limit at all. The everyday use of the event is the boring one, and the valuable one: knowing that it happened, which connection it happened to, and how far behind it had fallen. That number is your early warning long before the process runs out of memory.

OnSendBufferFull is published on TsgcWebSocketClient, TsgcWebSocketServer and TsgcWebSocketHTTPServer.

What to do when you cannot afford to drop anything

Sometimes the honest answer is that no message may ever be lost. If that is your case, none of the three policies above is what you want, and neither is a bigger queue. A bigger queue does not fix the problem, it only postpones it, and it postpones it to a worse moment: further into a burst, deeper into memory pressure, and further from the point where you could still have done something about it.

The answer is a different design. Do not hold undelivered messages in a socket queue, persist them, and let the client fetch what it missed when it is able to. That is exactly what message history and reconnect recovery does: the server keeps a bounded per channel history with offsets, and a client that fell behind or dropped its connection replays the messages it missed on resubscribe. Pair that with qopDisconnect and you get the strongest combination available: a client that cannot keep up is cut loose fast, and when it comes back it recovers everything it missed rather than pretending it did not miss it.

Unlimited is still the default

MaxQueueSize defaults to 0, which means unlimited, which is exactly the behaviour sgcWebSockets had before. Upgrading changes nothing. The overflow path is not even entered unless you set a positive cap, so existing servers keep queueing the way they always have until you decide otherwise.

That said, if you broadcast to clients you do not control, and on the public internet you never control them, an unbounded per connection queue is an open-ended memory commitment made on behalf of the worst client that will ever connect to you. Setting a cap costs one line.

Availability

Send-queue limits, the three overflow policies and OnSendBufferFull ship in sgcWebSockets 2026.7 for Delphi 7 through 13 and C++Builder, on Win32/Win64, Linux64, macOS, Android and iOS.

Customers with an active subscription can download the new build from the customer area. Trial users can grab the updated installer at esegece.com/products/websockets/download.

Questions, feedback or help picking the right policy for your feed? Get in touch, and you will get a reply from the people who wrote the code.