OpenAI Assistants Streaming

Streaming Events

Instead of waiting the full response from the assistant, you can stream the response using Server-Sent Events. Just pass the param Stream = True when using the CreateRun function and the response will be using streaming.

The following events are used to handle the streaming responses

• OnStreamRun: the event is called when there is an update in the run object.
• OnStreamMessage: the event is called when there is an update in the message object: created, in-progess, completed...
• OnStreamMessageDelta: Occurs when parts of a Message are being streamed.
• OnStreamDone: Occurs when an error occurs. This can happen due to an internal server error or a timeout.
• OnStreamError: Occurs when a stream ends.

Step 1: Create an Assistant

An Assistant represents an entity that can be configured to respond to a user's messages using several parameters like model, instructions, and tools. 

1
2
3
4
5
6
7
8
9
10
11
// ... create a new assistant
oAssistant := TsgcAIOpenAIAssistant.Create(nil);
// ... set your api key
oAssistant.OpenAIOptions.ApiKey := txtAPIKey.Text;
// ... assistant options
oAssistant.AssistantOptions.Name := 'Math Tutor';
oAssistant.AssistantOptions.Instructions.Text := 'You are a personal math tutor. Write and run code to answer math questions.';
oAssistant.AssistantOptions.Model := 'gpt-4o';
// ... create the assistant
oAssistant.CreateAssistant(); 
X

Step 2: Create a Thread

A Thread represents a conversation between a user and one or many Assistants. You can create a Thread when a user (or your AI application) starts a conversation with your Assistant. 

1
2
oThread := oAssistant.CreateThread; 
X

Step 3: Add a Message to the Thread and Run using Streaming

Enter your text here ...

1
2
3
4
5
6
7
8
9
10
11
12
13
procedure SendMessage(const oAssistant: TsgcAIOpenAIAssistant; const oThread: TsgcAIClass_Thread; const aMessage: string);
var
  i: Integer;
  oMessage: TsgcOpenAIClass_Message;
  oMessages: TsgcOpenAIClass_Response_List_Messages;
  oRun: TsgcOpenAIClass_Run;
begin
  DoLog('[user]: ' + aMessage);
  oMessage := oAssistant.CreateMessageText(oThread.Id, aMessage);
  if Assigned(oMessage) then
    oRun := oAssistant.CreateRun(oThread.Id, True);
end; 
X

Step 4: Handle the Response

Use the event OnStreamMessageDelta to read the server-sent stream message. 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
procedure OnStreamMessageDelta(Sender: TObject; const aMessageDelta: TsgcOpenAIClass_MessageDelta; const aRaw: string);
var
  i: Integer;
  vResponse: string;
  vType: string;
begin
  for i := Low(aMessageDelta.Content) to High(aMessageDelta.Content) do
  begin
    vType := aMessageDelta.Content[i]._Type;
    if vType = 'text' then
    begin
      vResponse := TsgcOpenAIClass_MessageDeltaContent_Text
        (aMessageDelta.Content[i]).Value;
    end;
  end;
end; 
X

OpenAI Delphi Sample

The following compressed file contains the source code of the Assistants demo built for windows using the sgcWebSockets library.