The question we get asked the most
Whenever we publish a new HTTP/2 feature in sgcWebSockets — server push, HPACK dynamic table tuning, SETTINGS frame negotiation — someone politely asks the same question on the forum: “Why didn't you just extend Indy's TIdHTTP? It already does TCP, TLS and HTTP/1.1, surely HTTP/2 is just another framing format on top.”
It is a reasonable question. The short answer is that HTTP/2 is not a layer you can graft onto an HTTP/1.1 client — it is a fundamentally different transport that breaks almost every assumption Indy makes. This post walks through the specific reasons we built our HTTP/2 implementation from scratch, what that took, and how it influenced the rest of sgcWebSockets.
HTTP/1.1 is text, HTTP/2 is binary frames
Indy's HTTP implementation, like every classical HTTP/1.1 client, reads a CRLF-terminated request line, then headers, then either a content-length body or a chunked body. It is line-oriented text. The transport is one request, one response, then either close or pipeline (and nobody actually pipelines in practice).
HTTP/2 is a binary framing protocol. Every byte after the connection preface belongs to a frame: DATA, HEADERS, PRIORITY, RST_STREAM, SETTINGS, PUSH_PROMISE, PING, GOAWAY, WINDOW_UPDATE, CONTINUATION. Frames are not aligned with requests — a single request's headers may span several HEADERS / CONTINUATION frames, the body may arrive as many interleaved DATA frames, and frames from different requests are intermixed on the same socket. The reader is not a line parser, it is a state machine over fixed-length frame headers and variable-length payloads.
Retrofitting that on top of TIdHTTP.ReadHeaderFromStream would mean rewriting the read loop completely. At that point you are not extending Indy — you are writing a parallel implementation that happens to share a socket class.
Multiplexing breaks the one-request-per-connection assumption
Indy's HTTP client model is built around a single in-flight request. You call Get, the method blocks, the response arrives, you process it. The TCP socket belongs to that call for the duration. HTTP/2 inverts this: a single connection multiplexes many concurrent streams, each identified by a stream id. A typical Apple Push Notification connection sends thousands of independent POST requests across the same TCP socket, with responses arriving out of order.
To support that with Indy's blocking model you would need either (a) one Indy client per HTTP/2 stream — defeating the whole point of HTTP/2 — or (b) a separate dispatcher thread that owns the socket, reads frames, dispatches them to per-stream queues, and exposes a futures/promises API to callers. Option (b) is essentially writing a new HTTP/2 client and parking Indy underneath as a thin socket wrapper. We chose to do that explicitly with the most appropriate transport for the platform instead.
HPACK is a state machine, not a string
HTTP/1.1 headers are name-value text. HTTP/2 headers are compressed with HPACK (RFC 7541), which combines a static table of common headers, a dynamic table that grows as the connection proceeds, and Huffman-coded literals. Encoding a header set is not WriteLn(Stream, 'Content-Type: application/json') — it is a lookup against two tables, a choice between indexed/literal-with-indexing/literal-without-indexing/literal-never-indexed representations, optional Huffman compression, and an update to the dynamic table that both endpoints must keep in sync.
Get the table size wrong and the connection dies with a COMPRESSION_ERROR. Forget to apply the SETTINGS update for max header table size and a previously working server starts rejecting your requests. There is no shortcut — you either implement HPACK in full or you do not have HTTP/2. We implemented the static table, the dynamic table with eviction, Huffman encode/decode, and the four representation forms in roughly 1,200 lines of Pascal. That code does not exist anywhere in Indy and would have had no natural home there.
Flow control is per-stream and per-connection
HTTP/2 has two layered windows: a connection-level window and one per stream. Both start at 65,535 bytes and grow only when the peer sends WINDOW_UPDATE frames. A naive implementation that writes DATA without checking the window will hang the connection after the first 64 KB. A correct implementation needs an event loop that pauses writers when the window is exhausted, splits large bodies into window-sized chunks, and resumes when updates arrive.
None of that exists in HTTP/1.1 and therefore none of it exists in Indy. It is a real piece of work that has to live somewhere — we put it in the same place as the frame reader and the HPACK codec.
What our writer loop looks like
Here is a small slice of the frame writer, simplified for readability. Note the explicit flow control check and the loop that splits the payload into window-bounded chunks:
procedure TsgcHTTP2Stream.WriteData(const aData: TBytes; aEndStream: Boolean);
var
vOffset, vChunk, vMax: Integer;
vFlags: Byte;
begin
vOffset := 0;
while vOffset < Length(aData) do
begin
// Respect both connection and stream flow-control windows.
vMax := Min(FConnection.SendWindow, Self.SendWindow);
vMax := Min(vMax, FConnection.PeerSettings.MaxFrameSize);
if vMax <= 0 then
begin
// Block until a WINDOW_UPDATE wakes us.
FWindowEvent.WaitFor(INFINITE);
Continue;
end;
vChunk := Min(Length(aData) - vOffset, vMax);
vFlags := 0;
if aEndStream and ((vOffset + vChunk) = Length(aData)) then
vFlags := FLAG_END_STREAM;
FConnection.SendFrame(FRAME_DATA, vFlags, Self.StreamId,
Copy(aData, vOffset, vChunk));
Dec(FConnection.SendWindow, vChunk);
Dec(Self.SendWindow, vChunk);
Inc(vOffset, vChunk);
end;
end;
This is a few dozen lines but it captures the heart of why HTTP/2 cannot be a thin add-on. There is no equivalent code path in any HTTP/1.1 client because the concept simply does not exist. Bolt it on top of TIdHTTP.Get and you immediately discover that the whole calling convention — one blocking call per request — is the wrong shape.
ALPN, settings negotiation and connection lifecycle
HTTP/2 over TLS is selected via ALPN (Application-Layer Protocol Negotiation), which has to be configured on the TLS context before the handshake. Indy's OpenSSL IO handler did not expose ALPN for years — it was eventually added — but ALPN is only the first step. After the TLS handshake the client must send the 24-byte connection preface, then a SETTINGS frame, then wait for the server's SETTINGS and acknowledge it. Only then is the connection ready for requests. SChannel on Windows requires a different ALPN setup path than OpenSSL. msquic has yet another. Abstracting these neatly was easier in a clean module than threaded through Indy's existing TLS handshake code.
Performance: the numbers
The pay-off for doing all of this work is measurable. On a representative workload — a Delphi client sending one million Apple Push Notifications to APNs (apns2 in HTTP/2 mode) on a modern Windows server:
- HTTP/1.1 with
TIdHTTP, one TCP connection per request: ~110 requests/sec, limited by TLS handshake cost. - HTTP/1.1 with
TIdHTTP, connection reuse: ~850 requests/sec, limited by the one-in-flight-per-connection rule. - HTTP/2 with sgcWebSockets, single connection, multiplexed: ~9,500 requests/sec, limited by APNs server-side throttling rather than the client.
Roughly a 10x improvement on the same hardware and the same network, without changing the application logic. That is the value HTTP/2 multiplexing delivers when you build the stack natively rather than trying to fake it with parallel HTTP/1.1 connections.
What we re-used from Indy
Not everything was rewritten. We use Indy's OpenSSL TIdSSLIOHandlerSocketOpenSSL as one of the underlying TLS providers (with our own ALPN patch), Indy's TIdTCPClient for the raw socket layer in the default backend, and Indy's TIdHTTPServer stays in the picture for HTTP/1.1 fallback. The HTTP/2 layer sits on top, owning the read loop, the frame parser, HPACK, flow control and stream lifecycle. The split lets us share TLS plumbing while keeping the HTTP/2 semantics independent of Indy's blocking call model.
Lessons for other protocols
The same pattern repeats throughout sgcWebSockets. WebSocket is a binary framing protocol bolted onto an HTTP upgrade — we wrote the framer from scratch. MQTT 5 has its own variable-length integer encoding, properties table and flow control — written from scratch. AMQP 1.0 has a full type system and link credit flow — written from scratch. HTTP/3 is over QUIC and not TCP at all — we wrap msquic / ngtcp2. The common thread is that every modern protocol has its own state machine and pretending it is “just another mode of HTTP/1.1” leads to fragile, slow code.
Closing thoughts
Extending Indy is the obvious first instinct — it is the library every Delphi developer already trusts. But HTTP/2 is not an extension of HTTP/1.1, it is a redesign. Frame multiplexing, HPACK, flow control and out-of-order responses each require infrastructure that simply does not exist in a blocking HTTP/1.1 client. Building it natively cost real engineering effort but it is the only way to deliver the order-of-magnitude performance HTTP/2 promises — and it set the template for every other modern protocol sgcWebSockets has shipped since.