Once a client creates a session, it then sends JSON-formatted events containing text and audio chunks. The server will respond in kind with audio containing voice output, a text transcript of that voice output, and function calls (if functions are provided by the client).

A realtime Session represents the overall client-server interaction, and contains default configuration.

It has a set of default values which can be updated at any time (via session.update) or on a per-response level (via response.create).

SemanticVad

ServerVad

TracingConfig

TranscriptionConfig

Configuration for input audio transcription, defaults to off and can be set to null to turn off once on. Input audio transcription is not native to the model, since the model consumes audio directly. Transcription runs asynchronously through the /audio/transcriptions endpoint and should be treated as guidance of input audio content rather than precisely what the model heard. The client can optionally set the language and prompt for transcription, these offer additional guidance to the transcription service.

TurnDetection

Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.

Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.

Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability.
For example, if user audio trails off with "uhhm", the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.

Namespace Glitch9.AIDevKit.OpenAI.Realtime

Classes

Structs

Enums