Delphi WebRTC: Peer to Peer Audio, Video and Data

Q: Does Delphi support WebRTC?

Not in the RTL, and not through the VCL. TsgcRTCPeerConnection is a native Object Pascal implementation of the W3C peer connection surface: CreateOffer, CreateAnswer, SetLocalDescription, SetRemoteDescription, AddIceCandidate, CreateDataChannel and AddTrack, with ICE, DTLS, SCTP and SRTP underneath. There is no embedded Chromium, no TWebBrowser and no JavaScript bridge in the process.

Q: Which sgcWebSockets edition do I need?

Two steps. Enterprise defines SGC_ICE, SGC_DTLS, SGC_TURN and SGC_RTCPEERCONNECTION, which is what puts TsgcRTCPeerConnection, TsgcICEClient, TsgcTURNClient and TsgcTURNServer on the palette. The offer and answer API, data channels and media tracks are gated behind SGC_SDP, SGC_DATACHANNEL and SGC_RTP, which only SGC_PACK_WEBRTC defines, and that is the sgcWebRTC add-on, also included in All-Access. Below that, a STUN client is in Standard, and a STUN server and the WebSocket server component are in Professional.

Q: Which thread do the events fire on?

Worker threads, always. OnLocalDescription, OnIceCandidate, OnConnectionStateChange, OnTrack and OnError arrive on the ICE, network or timer thread. Data channel events arrive on the thread driving the SCTP association. Track audio and video events arrive on the network or tick thread. None of them is marshalled to the main thread for you, so wrap anything that touches a control in TThread.Queue, which is what the shipped demos do.

Q: Do I need a certificate for the DTLS handshake?

No. Leave RTCOptions.DTLSOptions.CertFile empty and a self-signed certificate and key are generated in memory, once per component, and reused for every peer. Its fingerprint is published as the a=fingerprint attribute of the local description, which is what authenticates you to the other side. That is the WebRTC trust model: the chain is never verified, the fingerprint carried over the signalling channel is the anchor. You can still point CertFile and KeyFile at your own PEM files when you want a stable identity.

OVERVIEW

What actually has to happen

WebRTC is four separate problems wearing one name. Only one of them is about media, and it is the easy one. These are the four, in the order you have to solve them.

1. Describe the session

One side builds an offer, a text document (SDP) that says which media it wants to send, which codecs it speaks, the fingerprint of the certificate it will present and the ICE credentials it will use. The other side answers with the subset it accepts. CreateOffer and CreateAnswer produce those documents, SetRemoteDescription consumes them.

2. Carry it to the other side

WebRTC deliberately does not say how the offer reaches the other peer. That channel is called signalling and it is your job. It is a few hundred bytes of text each way, so a WebSocket connection to a small relay is enough, and sgcWebSockets already gives you both halves of one.

3. Find a path through the NATs

Neither peer knows its public address, and both are usually behind a router. ICE gathers every address a peer might be reachable on, sends them across the signalling channel as they appear, and probes each pairing until one works. STUN finds the public address, TURN provides a relay when nothing direct works.

4. Move the bytes

Once one candidate pair is nominated, a DTLS handshake runs over it and everything after that is encrypted. A data channel is SCTP over that DTLS transport, an audio or video track is SRTP over it. Both share the one connection and the one open port.

THE SERVER QUESTION

Is there a WebRTC server?

Not in the media path, which is the whole point. Once the two applications have found each other, the audio, the video and the data travel directly between them. Nothing you host sees the payload, and nothing you host has to scale with the number of minutes your users spend on a call.

There are still two servers in the picture, and it helps to be exact about what each one does, because they are often confused.

The signalling server is yours. It relays a handful of text messages between two peers before the call starts, then goes quiet. It never sees media. In this walkthrough it is fifteen lines of Delphi built on TsgcWebSocketServer.

The STUN and TURN server exists because of NAT, not because of WebRTC. A STUN server answers one question, "what public address did this packet arrive from", and that is all. A TURN server relays packets for the pairs that cannot reach each other any other way, so it is the only piece that carries media, and only for the calls that need it. Public STUN servers are free and plentiful, TURN you host yourself, and sgcWebSockets Enterprise ships both a STUN server and a TURN server component if you would rather not run a separate daemon.

who-talks-to-whom.txt

  App A                                App B
    |                                    |
    |---- offer / answer / candidate --->|   your signalling
    |<-------- (WebSocket relay) --------|   server, text only
    |                                    |
    |---> "what is my public address?"   |   STUN, once per
    |     (STUN binding request)         |   candidate
    |                                    |
    |====== audio, video and data ======>|   direct, encrypted,
    |<===================================|   no server involved
    |                                    |
    |== only when nothing direct works ==|   TURN relay,
    |    (relayed candidate pair)        |   your server

BEFORE YOU START

Editions, units and platforms

The peer connection and the media engine are two different licence steps. It is worth getting this straight before you write any code, because the compiler will simply not see half the API otherwise.

What you want to do	What it needs	Where it comes from
STUN client, to discover a public address	`TsgcSTUNClient`	sgcWebSockets Standard and above
Run your own STUN server	`TsgcSTUNServer`	sgcWebSockets Professional and above
Relay a WebSocket signalling channel	`TsgcWebSocketServer`	sgcWebSockets Professional and above for the server half. The client half, `TsgcWebSocketClient`, arrives with Standard.
ICE, TURN client and server, and the peer connection component itself	`TsgcICEClient`, `TsgcTURNClient`, `TsgcTURNServer`, `TsgcRTCPeerConnection`	sgcWebSockets Enterprise
Offer and answer, data channels, audio and video tracks	`CreateOffer`, `CreateAnswer`, `SetRemoteDescription`, `AddIceCandidate`, `CreateDataChannel`, `AddTrack`	The sgcWebRTC pack, on top of Enterprise. Also included in All-Access.

Why the split, in the compiler's own words

The Enterprise block of sgcVer.inc defines SGC_ICE, SGC_DTLS, SGC_RTCPEERCONNECTION and SGC_TURN. That is what puts TsgcRTCPeerConnection on the palette and gives it an ICE and TURN transport.

Everything this page is actually about sits one level further in. SGC_PACK_WEBRTC is what defines SGC_SDP, SGC_SCTP, SGC_DATACHANNEL, SGC_RTP and SGC_SRTP, and it checks first that ICE, DTLS and the peer connection are already there. The manual signalling methods, the data channel API and the media API sit inside {$IFDEF SGC_SDP}, {$IFDEF SGC_DATACHANNEL} and {$IFDEF SGC_RTP} respectively, so on Enterprise alone they do not compile.

If CreateOffer does not resolve, that is what happened. The data channel demo that ships in the box says so out loud rather than failing silently.

sgcVer.inc

{$IFDEF SGC_PACK_WEBRTC} { PACK WEBRTC }
  {$IFDEF SGC_INDY_LIB}
    {$IFDEF SGC_ICE} { requires ICE + DTLS + RTCPeerConnection }
      {$IFDEF SGC_DTLS}
        {$IFDEF SGC_RTCPEERCONNECTION}
          {$DEFINE SGC_SDP}
          {$DEFINE SGC_SCTP}
          {$DEFINE SGC_DATACHANNEL}
          {$DEFINE SGC_RTP}
          {$DEFINE SGC_SRTP}
          {$DEFINE SGC_CODEC_OPUS}
          {$DEFINE SGC_CODEC_VP8}
          {$DEFINE SGC_CODEC_H264}
        {$ENDIF}
      {$ENDIF}
    {$ENDIF}
  {$ENDIF}
{$ENDIF}

The units both applications need

sgcP2P is the barrel unit that publishes TsgcRTCPeerConnection and re-exports the handler types. It is enough to declare the component, but enumerated constants come from the units that declare their types, so pull those in as well when you name rtctkAudio or cctAudioOpus.

The two applications in this walkthrough are the same program with a different button pressed. Everything below goes in both.

uPeer.pas

uses
  Classes, SysUtils,
  // sgc
  sgcWebSocket,             // signalling carrier
  sgcWebSocket_Classes,     // TsgcWSConnection
  sgcJSON,                  // wraps the SDP and the candidates
  sgcP2P,                   // TsgcRTCPeerConnection
  sgcP2P_RTCPeerConnection, // TsgcRTCConnectionState
  sgcP2P_DataChannel,       // TsgcRTCDataChannel
  sgcP2P_RTC_Media,         // TsgcRTCTrack, rtctkAudio
  sgcP2P_Codec_Types,       // cctAudioOpus, TsgcVideoFrame
  sgcP2P_Media_Factory,     // sgcCreateAudioCapture
  sgcP2P_MediaCapture,      // TsgcMediaCaptureSource
  sgcP2P_MediaRenderer;     // TsgcMediaRenderer

STEP 1

The signalling channel

Three message kinds, one dumb relay. This is the part every WebRTC tutorial waves at, and the part you actually have to write.

A relay, not a broker

The signalling server does not need to understand a single byte of what it forwards. It takes the text one peer sent and hands it to the other. Broadcast already has an Exclude parameter that takes a connection Guid, so "send to everyone except the sender" is one line.

Keep it that dumb. The moment the relay starts parsing SDP it becomes a component you have to update every time a codec changes, and it stops being able to relay a call to a browser.

In production you would key the relay on a room identifier so two calls cannot collide, and put it behind TLS. TsgcWebSocketServer carries the same TLSOptions, Authentication and WatchDog surface as the rest of the library.

uSignallingServer.pas

procedure TFormServer.Start;
begin
  FServer := TsgcWebSocketServer.Create(nil);
  FServer.Port := 5000;
  FServer.OnMessage := OnSignallingMessage;
  FServer.Active := True;
end;

procedure TFormServer.OnSignallingMessage(
  Connection: TsgcWSConnection; const Text: string);
begin
  // relay verbatim to the other peer. The server never
  // parses the SDP, so it never learns about codecs.
  FServer.Broadcast(Text, '', '', Connection.Guid);
end;

The peer side of the same channel

Each application opens a TsgcWebSocketClient to that relay and speaks a three word vocabulary: offer, answer and candidate. TsgcJSON from the same library serialises them, so there is no extra dependency.

Note where the branches go. An incoming offer is set as the remote description and immediately answered. An incoming answer is only set. An incoming candidate is fed to AddIceCandidate, and it can arrive before or after the description, which is the whole point of trickle ICE.

uPeer.pas

procedure TFormPeer.OnSignallingMessage(
  Connection: TsgcWSConnection; const Text: string);
var
  oJSON: TsgcJSON;
  vKind: string;
begin
  oJSON := TsgcJSON.Create(nil);
  try
    oJSON.Read(Text);
    vKind := oJSON.Node['kind'].Value;

    if vKind = 'offer' then
    begin
      FPeer.SetRemoteDescription('offer',
        oJSON.Node['sdp'].Value);
      FPeer.CreateAnswer;  // fires OnLocalDescription
    end
    else if vKind = 'answer' then
      FPeer.SetRemoteDescription('answer',
        oJSON.Node['sdp'].Value)
    else if vKind = 'candidate' then
      FPeer.AddIceCandidate(oJSON.Node['candidate'].Value,
        oJSON.Node['sdpMid'].Value,
        oJSON.Node['sdpMLineIndex'].Value);
  finally
    FreeAndNil(oJSON);
  end;
end;

STEP 2

Build the peer connection

Identical in both applications. The only asymmetry in the whole exchange is which one presses Call.

Configuration, and the events that matter

RTCOptions.ICEServers is the W3C iceServers list. AddURL takes a stun: or turn: URL and fills in the type, host, port and TLS flag from the scheme, with an optional username and credential for TURN.

RTCOptions.DTLS defaults to False. Leave it that way and there is no encryption and no SRTP keying material, so media will not work. CreateDataChannel turns it on for you, because a data channel is SCTP over DTLS and there is no valid configuration with DTLS off. AddTrack does not, so set it yourself when you add media.

You do not need a certificate file. Leave RTCOptions.DTLSOptions.CertFile empty and an in-memory self-signed certificate is generated once per component, which is exactly the WebRTC model: identity is anchored on the a=fingerprint line in the SDP, not on a chain. A remote description that carries no fingerprint is refused rather than allowed to handshake against anything.

Every event below fires on a worker thread, the ICE, network or timer thread, never on the main thread. Marshal with TThread.Queue before you touch a control.

uPeer.pas

procedure TFormPeer.CreatePeer;
begin
  FPeer := TsgcRTCPeerConnection.Create(nil);

  FPeer.RTCOptions.ICEServers.AddURL(
    'stun:stun.l.google.com:19302');
  FPeer.RTCOptions.ICE.STUN := True;
  FPeer.RTCOptions.ICE.TURN := False;  // no TURN server yet
  FPeer.RTCOptions.DTLS     := True;   // default is False

  FPeer.OnLocalDescription      := OnLocalDescription;
  FPeer.OnIceCandidate          := OnIceCandidate;
  FPeer.OnConnectionStateChange := OnConnectionStateChange;
  FPeer.OnDataChannel           := OnDataChannel;
  FPeer.OnTrack                 := OnTrack;
  FPeer.OnError                 := OnError;

  FSignalling := TsgcWebSocketClient.Create(nil);
  FSignalling.Host := 'signalling.example.com';
  FSignalling.Port := 5000;
  FSignalling.OnMessage := OnSignallingMessage;
  FSignalling.Active := True;
end;

Publishing the local description

OnLocalDescription hands you the type, the string 'offer' or 'answer', and the SDP itself. Both peers use the same handler, and it does one thing: put it on the signalling channel.

The SDP arrives already complete. CreateOffer gathers ICE candidates first and waits up to RTCOptions.GatheringTimeout milliseconds, 3000 by default, ending early after GatheringIdleTimeout milliseconds with no new candidate, 500 by default. That is the non-trickle path.

Set TrickleICE to True and the description goes out immediately with candidates following behind it. You rarely have to. RTCOptions.TrickleICEAuto is True by default, so when the remote description says a=ice-options:trickle, which every browser does, the component switches itself over and stops burning the gathering timeout.

uPeer.pas

procedure TFormPeer.OnLocalDescription(Sender: TObject;
  const aType, aSDP: string);
var
  oJSON: TsgcJSON;
begin
  oJSON := TsgcJSON.Create(nil);
  try
    oJSON.AddPair('kind', aType);  // 'offer' or 'answer'
    oJSON.AddPair('sdp', aSDP);
    FSignalling.WriteData(oJSON.Text);
  finally
    FreeAndNil(oJSON);
  end;
end;

// App A only. App B answers from OnSignallingMessage.
procedure TFormPeer.btnCallClick(Sender: TObject);
begin
  FPeer.CreateDataChannel('chat');  // forces DTLS on
  FPeer.CreateOffer;
end;

STEP 3

ICE candidates, STUN and TURN

This is where peer to peer connections fail, and where the failures are hardest to read. Three kinds of candidate, three reasons they exist.

host

An address the machine can see on itself, one per network interface. Free, instant, and enough when both applications are on the same LAN or the same VPN. If your two Delphi applications only ever run inside one office, host candidates are all you need and you can skip STUN entirely.

srflx, server reflexive

The public address a STUN server saw the peer's packet arrive from. This is what gets two peers behind ordinary home routers talking directly, and it covers the large majority of real connections. It costs one round trip to a STUN server that carries no traffic afterwards.

relay

An address on a TURN server that forwards to the peer. Needed for symmetric NAT, restrictive corporate firewalls and some mobile carriers. Every byte of the call crosses your TURN server, so it is the expensive path and the one you only fall back to.

Trickling them across

OnIceCandidate fires once per candidate, as it is discovered, with the candidate line, its sdpMid and its sdpMLineIndex. Those three fields are exactly what the browser API expects, so the same JSON works whether the far end is Delphi or Chrome.

Send each one immediately. Do not wait, do not batch. A candidate that arrives before the remote description is held and applied when the description lands, so ordering is not your problem.

When the pair is finally nominated, SelectedLocalCandidate and SelectedRemoteCandidate tell you which two addresses won. That one log line answers "why is this call going through my TURN server" faster than anything else.

uPeer.pas

procedure TFormPeer.OnIceCandidate(Sender: TObject;
  const aCandidate, aSdpMid: string;
  aSdpMLineIndex: Integer);
var
  oJSON: TsgcJSON;
begin
  oJSON := TsgcJSON.Create(nil);
  try
    oJSON.AddPair('kind', 'candidate');
    oJSON.AddPair('candidate', aCandidate);
    oJSON.AddPair('sdpMid', aSdpMid);
    oJSON.AddPair('sdpMLineIndex', aSdpMLineIndex);
    FSignalling.WriteData(oJSON.Text);
  finally
    FreeAndNil(oJSON);
  end;
end;

procedure TFormPeer.OnConnectionStateChange(Sender: TObject;
  aState: TsgcRTCConnectionState);
begin
  // rtccsNew, rtccsGathering, rtccsConnecting, rtccsConnected,
  // rtccsDisconnected, rtccsFailed, rtccsClosed
  if aState = rtccsConnected then
    Log(FPeer.SelectedLocalCandidate + ' -> ' +
        FPeer.SelectedRemoteCandidate);
end;

Adding TURN, and the switch you must not forget

A turn: entry in ICEServers carries its own host, port, username and credential, and that is what the allocation uses. Adding it is one more AddURL.

The trap is the other direction. RTCOptions.ICE.TURN defaults to True, and when the server list holds no TURN entry at all the gathering falls back to the single server in RTCOptions.ICE, whose host defaults to 127.0.0.1 and port to 3478. So a peer configured with nothing but a STUN URL still tries a TURN allocation against localhost, fails, and reports it. It is noise rather than a fault, but it looks alarming in a log and it sends you hunting in the wrong place. Set RTCOptions.ICE.TURN := False until you actually have a TURN server.

RTCOptions.ICE.STUN behaves the same way and also defaults to True.

uPeer.pas

// STUN for the public address, TURN for the fallback relay
FPeer.RTCOptions.ICEServers.AddURL(
  'stun:stun.example.com:3478');
FPeer.RTCOptions.ICEServers.AddURL(
  'turn:turn.example.com:3478', 'user', 'secret');

FPeer.RTCOptions.ICE.STUN := True;
FPeer.RTCOptions.ICE.TURN := True;

// a stalled call is usually a candidate problem. Lower the
// gathering waits on a LAN, where there is nothing to gather.
FPeer.RTCOptions.GatheringTimeout     := 1000;
FPeer.RTCOptions.GatheringIdleTimeout := 200;

ICE client reference Run your own TURN server

STEP 4

The data channel

Text and binary between the two applications, with the reliability you choose per channel. This is usually the first thing you get working, and it proves the whole transport.

Opening one, and receiving the other side's

The peer that calls CreateDataChannel gets the object back straight away. The peer that did not gets the same channel through OnDataChannel. Attach the handlers in both places, because either side can open a channel at any point in the session.

The channel is not usable the instant you create it. Its Id stays unassigned until the SCTP association is up and the DTLS role is resolved, and Send returns False while it is not open. Wait for OnOpen.

Reliability is decided at creation. The defaults are ordered and fully reliable, a TCP-like channel. Pass aOrdered = False for unordered delivery, or an aMaxRetransmits or aMaxPacketLifeTime for partial reliability, which is what you want for positional updates or anything where a late packet is worse than a lost one.

Do not free a channel, it is owned by the peer connection. Close starts the shutdown and OnClose fires when the far end agrees.

uPeer.pas

// caller: ordered and reliable, the default
FChannel := FPeer.CreateDataChannel('chat');
AttachChannel(FChannel);

// unordered, give up after 3 retransmits
FState := FPeer.CreateDataChannel('state', False, 3);

// callee: the same channel arrives here
procedure TFormPeer.OnDataChannel(Sender: TObject;
  aChannel: TsgcRTCDataChannel);
begin
  AttachChannel(aChannel);
end;

procedure TFormPeer.AttachChannel(
  aChannel: TsgcRTCDataChannel);
begin
  FChannel := aChannel;
  FChannel.OnOpen          := OnChannelOpen;
  FChannel.OnMessage       := OnChannelText;
  FChannel.OnMessageBinary := OnChannelBinary;
  FChannel.OnClose         := OnChannelClose;
  FChannel.OnError         := OnChannelError;
end;

procedure TFormPeer.OnChannelText(Sender: TObject;
  const aText: string);
begin
  // fires on the SCTP thread, queue before touching a control
  TThread.Queue(nil,
    procedure
    begin
      memoChat.Lines.Add(aText);
    end);
end;

Sending, and not flooding

Send takes a string and SendBytes takes a TBytes. Both return False rather than raising when the channel is not open, so a send during teardown is a returned False, not an exception on a worker thread.

MaxMessageSize is the largest message the peer said it accepts, read from the a=max-message-size attribute of its description. A message over that limit is refused locally instead of being put on the wire, where it would abort the whole association and take every other channel with it. Zero means no description ever named a limit and the check is off.

BufferedAmount is how many bytes are queued in SCTP for this stream and not yet acknowledged. Watch it when you are streaming a file: send until it passes a threshold, then wait for it to drain rather than queueing gigabytes into memory.

uPeer.pas

procedure TFormPeer.btnSendClick(Sender: TObject);
begin
  if not Assigned(FChannel) then
    Exit;

  if not FChannel.Send(txtMessage.Text) then
    Log('channel not open');
end;

procedure TFormPeer.SendChunk(const aBytes: TBytes);
begin
  if (FChannel.MaxMessageSize > 0) and
     (Cardinal(Length(aBytes)) > FChannel.MaxMessageSize) then
  begin
    Log('too big for this peer, split it');
    Exit;
  end;

  if FChannel.BufferedAmount < 262144 then
    FChannel.SendBytes(aBytes);
end;

STEP 5

Audio and video tracks

A track is not a data channel with pictures in it. Different transport, different failure modes, different code. This is the distinction most people get wrong first.

	Data channel	Media track
Carried by	SCTP over DTLS (RFC 8831)	SRTP over the same DTLS transport (RFC 3711)
Delivery	Your choice, from fully reliable and ordered to fire and forget	Always lossy by design. Late is worse than lost, so nothing is retransmitted forever
Unit of work	A message. You decide when to send one	A clock. Audio is fed in 20 ms frames, video at a frame rate
Opened with	`CreateDataChannel`, at any time in the session	`AddTrack`, which needs a new offer to publish it
Received through	`OnDataChannel`, then the channel's own `OnMessage`	`OnTrack`, then the track's `OnAudio` or `OnVideoFrame`
Needs DTLS on	Yes, and `CreateDataChannel` sets it for you	Yes, and `AddTrack` does not. Set `RTCOptions.DTLS` yourself
Use it for	Chat, file transfer, remote control, game state, telemetry	Microphone, camera, screen share, anything with a timeline

Sending the microphone

AddTrack takes a kind and a codec and returns a TsgcRTCTrack. The audio codecs are cctAudioOpus, cctAudioPCMU and cctAudioPCMA, the video codecs are cctVideoVP8, cctVideoVP9, cctVideoH264 and cctVideoJPEG.

Capture is a separate object, because you may not want the platform microphone at all. sgcCreateAudioCapture builds the right implementation for the platform the code was compiled for, waveIn on Windows, ALSA on Linux, AudioRecord on Android, a VoiceProcessingIO Audio Unit on iOS and macOS, so nothing in your code names a platform class. It returns nil on a target with no implementation, so test the result.

SendPCM wants 16-bit signed interleaved PCM at the encoder's rate and channel count, which for Opus is 48000 Hz and for G.711 is 8000 Hz. The capture source publishes what it actually delivers through AudioSampleRate, AudioChannels and AudioFrameDurationMs, so you can check rather than assume.

Adding a track after the session is already up sets the negotiation-needed flag and fires OnNegotiationNeeded. Call CreateOffer again to publish it, and the re-offer is built without touching the transport.

uPeer.pas

procedure TFormPeer.StartCall;
begin
  FPeer.RTCOptions.DTLS := True;  // SRTP keys come from DTLS

  FAudioTrack := FPeer.AddTrack(rtctkAudio, cctAudioOpus);
  FVideoTrack := FPeer.AddTrack(rtctkVideo, cctVideoVP8);

  FCapture := sgcCreateAudioCapture;
  if Assigned(FCapture) then
  begin
    FCapture.OnAudioCapture := OnAudioCaptured;
    FCapture.Start;
    if not FCapture.Active then
      Log(FCapture.LastError);
  end;

  FRenderer := sgcCreateAudioRenderer;
  if Assigned(FRenderer) then
    FRenderer.Start;

  FPeer.CreateOffer;
end;

procedure TFormPeer.OnAudioCaptured(Sender: TObject;
  const aPCM: TBytes;
  aSampleRate, aChannels, aSamplesPerChannel: Integer);
begin
  if Assigned(FAudioTrack) then
    FAudioTrack.SendPCM(aPCM, aSamplesPerChannel);
end;

Playing what the other side sent

OnTrack fires once per remote media line, when the remote description brings one in. The track it hands you is owned by the peer connection, so wire its events and never free it.

Audio arrives as decoded PCM through OnAudio, with the rate and channel count the decoder produced. Hand it straight to the TsgcMediaRenderer that sgcCreateAudioRenderer built, which converts the format when the device could not be opened to match.

Video arrives as a decoded TsgcVideoFrame through OnVideoFrame: raw pixels in Data, with Width, Height, Format and Stride. The formats are vffI420, vffNV12, vffRGB24, vffRGBA32, vffBGR24 and vffBGRA32, and the two BGR ones follow the Windows GDI byte order, so blitting a vffBGR24 frame into a bitmap is a memory copy and not a conversion.

If a frame arrives broken after packet loss, RequestKeyFrame asks the sender for a fresh one.

uPeer.pas

procedure TFormPeer.OnTrack(Sender: TObject;
  aTrack: TsgcRTCTrack);
begin
  if aTrack.Kind = rtctkAudio then
    aTrack.OnAudio := OnRemoteAudio
  else
  begin
    FRemoteVideo := aTrack;
    aTrack.OnVideoFrame := OnRemoteVideoFrame;
  end;
  aTrack.OnEnded := OnRemoteTrackEnded;
end;

procedure TFormPeer.OnRemoteAudio(Sender: TObject;
  const aPCM: TBytes;
  aSampleRate, aChannels, aSamplesPerChannel: Integer);
begin
  if Assigned(FRenderer) then
    FRenderer.RenderAudio(aPCM, aSampleRate, aChannels,
      aSamplesPerChannel);
end;

procedure TFormPeer.OnRemoteVideoFrame(Sender: TObject;
  const aFrame: TsgcVideoFrame);
begin
  // aFrame.Data holds Width x Height pixels in aFrame.Format
  if aFrame.Format = vffBGR24 then
    BlitToBitmap(aFrame);
end;

The camera, on Windows

Audio capture is abstracted behind the factory because every supported platform has an implementation. Video capture is not, so you name the platform class. On Windows that is TsgcVideoCapture_Win from sgcP2P_MediaCapture_Win, which drives Video for Windows and delivers frames through the same OnVideoCapture event the base class declares.

The neighbouring unit sgcP2P_ScreenCapture_Win gives you TsgcScreenCapture_Win and TsgcWindowCapture_Win, both of them TsgcMediaCaptureSource descendants, so screen sharing is the same three lines with a different constructor.

uPeer.pas

uses
  sgcP2P_MediaCapture_Win;   // MSWINDOWS only

procedure TFormPeer.StartCamera;
begin
  FVideoCapture := TsgcVideoCapture_Win.Create(640,
    480, 30);
  FVideoCapture.DeviceIndex := 0;
  FVideoCapture.OnVideoCapture := OnVideoCaptured;
  FVideoCapture.Start;
end;

procedure TFormPeer.OnVideoCaptured(Sender: TObject;
  const aFrame: TsgcVideoFrame);
begin
  if Assigned(FVideoTrack) then
    FVideoTrack.SendVideoFrame(aFrame);
end;

WHAT GOES WRONG

The failures worth knowing in advance

Peer to peer connections fail in ways that produce no error at all, which is what makes them hard. These are the ones that come up most.

Media is silent and nothing errors

RTCOptions.DTLS is False. That is the default, CreateDataChannel turns it on but AddTrack does not, so a session that only carries media never runs a DTLS handshake and therefore never gets SRTP keys. Set it explicitly.

A TURN error you did not ask for

RTCOptions.ICE.TURN defaults to True and falls back to 127.0.0.1:3478 when the server list has no TURN entry. Set it to False until you really have a TURN server, or the log fills with allocation failures that have nothing to do with your problem.

The remote description is refused

A description with no a=fingerprint is rejected outright and reported through OnError. In the WebRTC trust model that line is the only thing authenticating the peer, so accepting a description without one would let the handshake complete against any certificate at all.

An access violation in an event handler

Every peer connection, data channel and track event fires on a worker thread, the ICE, network, SCTP or tick thread, never on the main thread. Touching a VCL or FMX control directly from one is undefined. Wrap it in TThread.Queue.

Both sides re-offer at once

That is called glare, and it is resolved with the W3C perfect negotiation rule. The impolite peer, Polite = False, keeps its own offer and reports the incoming one through OnError. The polite peer rolls its own back and answers. The two peers of a session must not both be polite.

One large message kills every channel

A message over the peer's a=max-message-size would abort the whole SCTP association, taking every other data channel with it. Send and SendBytes check MaxMessageSize and refuse locally instead. Split large payloads yourself.

Connection takes three seconds to start

That is RTCOptions.GatheringTimeout, the non-trickle wait. GatheringIdleTimeout ends it early once candidates stop arriving, and TrickleICEAuto switches to trickle mode when the remote description advertises it. Lower both timeouts on a LAN.

The audio is fast, slow or garbled

A sample rate or channel count mismatch between the capture device and the encoder. Opus is negotiated at 48000 Hz and G.711 at 8000 Hz. Read AudioSampleRate and AudioChannels back off the capture source rather than assuming the device honoured what you asked for.

FAQ

Delphi WebRTC, frequently asked

What developers ask before wiring two applications together peer to peer.

Not in the RTL, and not through the VCL. TsgcRTCPeerConnection is a native Object Pascal implementation of the W3C peer connection surface: CreateOffer, CreateAnswer, SetLocalDescription, SetRemoteDescription, AddIceCandidate, CreateDataChannel and AddTrack, with ICE, DTLS, SCTP and SRTP underneath. There is no embedded Chromium, no TWebBrowser and no JavaScript bridge in the process.

You need something that can carry a few hundred bytes of text between the two peers before the call starts, because neither one knows how to reach the other yet. That is signalling, and it can be a WebSocket relay, an existing message queue, a REST endpoint, even a copy and paste for a demo. Once the offer, the answer and the ICE candidates have crossed, the media and the data go directly between the two applications and the signalling channel can close. No server sits in the media path unless a TURN relay turned out to be the only route that worked.

A WebSocket, almost always, because it is bidirectional and the server can push an incoming offer without the peer polling for it. This page builds one from TsgcWebSocketServer and TsgcWebSocketClient in about fifteen lines, relaying every message to the other peer with Broadcast and the sender's Connection.Guid as the exclude. sgcWebSockets also ships a ready-made signalling protocol component, TsgcWSPServer_RTCPeerConnection, which drives the exchange for you through RTCOptions.WebSocket and GatherCandidates when you would rather not write the relay at all.

If both applications are on the same LAN or the same VPN, neither. Host candidates already describe reachable addresses. If they are on different networks behind ordinary routers, you need STUN, which tells each peer the public address its packets appear to come from, and that covers most real connections. You need TURN when no direct pairing works at all: symmetric NAT, restrictive corporate firewalls and some mobile carriers. TURN relays every byte of the call, so it is the expensive fallback rather than the default. Add both to RTCOptions.ICEServers and ICE picks the cheapest pair that actually connects.

A data channel is SCTP over DTLS and moves messages, with the reliability you choose: ordered and fully reliable like TCP, or unordered with a retransmit or lifetime limit for anything where a late packet is useless. A media track is SRTP over the same DTLS transport and moves a timeline: audio in 20 ms frames, video at a frame rate, always lossy by design. Use a data channel for chat, file transfer, remote control and game state. Use a track for a microphone, a camera or a screen. They share one connection and one open port.

Two steps. Enterprise defines SGC_ICE, SGC_DTLS, SGC_TURN and SGC_RTCPEERCONNECTION, which is what puts TsgcRTCPeerConnection, TsgcICEClient, TsgcTURNClient and TsgcTURNServer on the palette. The offer and answer API, data channels and media tracks are gated behind SGC_SDP, SGC_DATACHANNEL and SGC_RTP, which only SGC_PACK_WEBRTC defines, and that is the sgcWebRTC add-on, also included in All-Access. Below that, a STUN client is in Standard, and a STUN server and the WebSocket server component are in Professional.

Yes, and nothing in this page changes. The SDP is standard, the candidate lines carry the same sdpMid and sdpMLineIndex the browser API expects, and the offer and answer state machine follows RFC 8829, so the JSON your relay forwards works unmodified in both directions. Browsers trickle candidates from the first millisecond, which is exactly what RTCOptions.TrickleICEAuto detects: it sees a=ice-options:trickle in the remote description and answers immediately instead of waiting out the gathering timeout.

Worker threads, always. OnLocalDescription, OnIceCandidate, OnConnectionStateChange, OnTrack and OnError arrive on the ICE, network or timer thread. Data channel events arrive on the thread driving the SCTP association. Track audio and video events arrive on the network or tick thread. None of them is marshalled to the main thread for you, so wrap anything that touches a control in TThread.Queue, which is what the shipped demos do.

No. Leave RTCOptions.DTLSOptions.CertFile empty and a self-signed certificate and key are generated in memory, once per component, and reused for every peer. Its fingerprint is published as the a=fingerprint attribute of the local description, which is what authenticates you to the other side. That is the WebRTC trust model: the chain is never verified, the fingerprint carried over the signalling channel is the anchor. You can still point CertFile and KeyFile at your own PEM files when you want a stable identity.

A DTLS handshake runs over the nominated candidate pair and derives the SRTP keys, so every RTP, RTCP and SCTP packet on the connection is encrypted. For data channels there is no way to switch it off: CreateDataChannel sets RTCOptions.DTLS to True unconditionally, because an RTCDataChannel is SCTP over DTLS by definition and there is no valid configuration without it.

The P2P and WebRTC units ship in every runtime package from Delphi 7 through RAD Studio 13, and in the matching C++ Builder packages. Audio capture and playback have platform implementations for Windows, Linux, Android, iOS and macOS, so the factory functions return a working object on all five. Video capture is the exception: it is named per platform rather than built by a factory, and on Windows that is TsgcVideoCapture_Win, alongside TsgcScreenCapture_Win and TsgcWindowCapture_Win for screen sharing.

Yes, that is renegotiation. AddTrack on an established session marks it as needing negotiation and fires OnNegotiationNeeded. Call CreateOffer again and a re-offer is built synchronously, with no new ICE gathering and no new DTLS or SCTP handshake, so the transport is never interrupted. Existing media lines keep their position and their mid, the new one is appended at the end. RemoveTrack works the same way in reverse: the media line stays and is republished as recvonly or inactive.

GO DEEPER

Component references and technical documents

Every piece this page uses has its own reference page, and most have a standalone technical PDF with the full property, method and event list.

TsgcRTCPeerConnection

The component this whole page is about. Offer and answer, ICE, DTLS, SCTP data channels and RTP media on one class.

Component page →

sgcWebRTC

The media engine pack: SDP, SCTP, RTP, SRTP, Opus and G.711 audio, VP8 and H.264 video, bandwidth estimation.

Product page →

Feature breakdown

Codec by codec and platform by platform, what the media engine does and where each encoder comes from.

See the features →

ICE client

Candidate gathering, the check list, nomination and the ICE server collection, the layer under the peer connection.

Component page →

STUN client and server

Binding requests, retransmission options, and the server component when you would rather host your own.

STUN client →

TURN client and server

Allocations, permissions, channel binds, and a TURN server component for the calls that need a relay.

TURN server →

All P2P components

UDP, STUN, TURN, ICE and RTCPeerConnection, the whole peer to peer family in one index.

Browse P2P →

Delphi WebRTC overview

The library-level view of WebRTC in sgcWebSockets, with the signalling protocol components and the demo list.

Which edition do I need?

The full edition matrix, feature by feature, when WebRTC is not the only thing you are weighing up.

Compare editions →

Other use cases

This page is one of the Delphi use cases, each taking a single job end to end. The others so far are calling an LLM from Delphi and signing a user in with OAuth2 and PKCE.

All use cases →

	RTCPeerConnection technical document (PDF) Properties, methods, events and code samples for the peer connection component alone.	Open
	ICE client technical document (PDF) Candidate gathering, the check list and the ICE server collection in detail.	Open
	TURN client technical document (PDF) Allocations, permissions and channel binds, the client ICE drives for a relayed candidate.	Open
	STUN client technical document (PDF) Binding requests and retransmission, the cheapest way to learn your own public address.	Open
	Demo projects Demos\35.P2P\05.RTCPeerConnection and Demos\35.P2P\06.DataChannel ship inside the package.	Open

STANDARDS

The specifications behind each step

Primary sources, when you would rather read what the component implements than take our word for it.

RFC 8829, JSEP

The offer and answer state machine behind CreateOffer, CreateAnswer and SetRemoteDescription, including renegotiation and rollback.

Read the RFC →

RFC 8445, ICE

Candidate gathering, priority, the check list and nomination. The reason a connection sometimes takes a second and sometimes fails.

Read the RFC →

RFC 8489 and RFC 8656

STUN and TURN. What a binding request asks, and what an allocation costs you.

Read the RFC →

RFC 8831 and RFC 8832

WebRTC data channels over SCTP, and the DCEP open handshake that assigns stream ids by DTLS role.

Read the RFC →

RFC 8122, SDP fingerprints

Why a=fingerprint is the identity of a peer connection, and why a description without one is refused.

Read the RFC →

WebRTC 1.0 (W3C)

The API this component mirrors, including perfect negotiation, which is where Polite comes from.

Read the spec →

Delphi WebRTC: Audio, Video and Data Between Two Applications

What actually has to happen

1. Describe the session

2. Carry it to the other side

3. Find a path through the NATs

4. Move the bytes

Is there a WebRTC server?

Editions, units and platforms

Why the split, in the compiler's own words

The units both applications need

The signalling channel

A relay, not a broker

The peer side of the same channel

Build the peer connection

Configuration, and the events that matter

Publishing the local description

ICE candidates, STUN and TURN

host

srflx, server reflexive

relay

Trickling them across

Adding TURN, and the switch you must not forget

The data channel

Opening one, and receiving the other side's

Sending, and not flooding

Audio and video tracks

Sending the microphone

Playing what the other side sent

The camera, on Windows

The failures worth knowing in advance

Media is silent and nothing errors

A TURN error you did not ask for

The remote description is refused

An access violation in an event handler

Both sides re-offer at once

One large message kills every channel

Connection takes three seconds to start

The audio is fast, slow or garbled

Delphi WebRTC, frequently asked

Component references and technical documents

TsgcRTCPeerConnection

sgcWebRTC

Feature breakdown

ICE client

STUN client and server

TURN client and server

All P2P components

Delphi WebRTC overview

Which edition do I need?

Other use cases

The specifications behind each step

RFC 8829, JSEP

RFC 8445, ICE

RFC 8489 and RFC 8656

RFC 8831 and RFC 8832

RFC 8122, SDP fingerprints

WebRTC 1.0 (W3C)

Put two of your applications on a call