Ollama

Ollama is an open-source tool for running large language models locally. It supports a wide range of models including Llama, Mistral, Gemma, Phi, and many others, enabling local AI inference without requiring cloud API access.

The sgcWebSockets library provides a Delphi component TsgcHTTP_API_Ollama to interact with the Ollama API.

Ollama API

The Ollama API provides access to locally running models for AI-powered applications. The API supports text generation, streaming, model management, and embeddings. No API key is required by default since models run locally. The host and port are configurable.

Features

• Messages
  • Ollama Messages Examples

• Models
  • Ollama Models Examples

• Embeddings
  • Ollama Embeddings Examples

Configuration

Ollama runs locally and by default listens on http://localhost:11434. Configure the host in the OllamaOptions.Host property. An API key is optional and only needed if you have configured authentication on your Ollama instance.

Ollama := TsgcHTTP_API_Ollama.Create(nil);
Ollama.OllamaOptions.Host := 'http://localhost:11434';

Messages

Send a structured list of input messages with text content, and the model will generate the next message in the conversation.

• _CreateMessage: Creates a message with the specified model and user prompt.
  • Model: The model to use (e.g. llama3).
  • Message: The user message content.
• _CreateMessageWithSystem: Creates a message with a system prompt.
  • System: System prompt that sets the behavior of the assistant.
• _CreateMessageStream: Creates a message with streaming (SSE) enabled. Events are delivered through the OnHTTPAPISSE event handler.

Models

Manage and query locally available models.

• _GetModels: Lists all available models.
• _GetTags: Lists all locally available model tags.
• _ShowModel: Retrieves detailed information about a specific model.
  • Model: The name of the model to query.
• _PullModel: Downloads a model from the Ollama model library.
  • Model: The name of the model to pull.
• _DeleteModel: Deletes a locally available model.
  • Model: The name of the model to delete.

Embeddings

Generate vector embeddings from text input using locally running models.

• _CreateEmbeddings: Generates embeddings for the given text.
  • Model: The model to use for embeddings (e.g. llama3).
  • Input: The text to generate embeddings for.