OpenAI Realtime Transcription

Realtime transcription receives continuous audio over a persistent WebSocket connection and returns incremental text and a final transcript for each audio item. It is suitable for live captions, meeting notes, call transcription, and voice input. To upload a complete recording in one request, use the audio transcription API.

gpt-live-transcribe is designed for low-latency incremental transcription. gpt-transcribe is designed for higher-accuracy transcription after a complete speech segment has been submitted and can return detected languages in the final event. Use the same model in the connection URL and session configuration.

Authentication and connection

Server applications should use a Bearer API key in the WebSocket handshake:

The WebSocket API in ordinary browsers cannot customize the Authorization header. Do not place a long-lived API key in frontend HTML, JavaScript, or client application bundles. Browser products should connect through a trusted backend or use a controlled short-lived credential mechanism.

Audio input requirements

The examples on this page use headerless, mono PCM16 little-endian audio at 24 kHz. The audio value in input_audio_buffer.append is a Base64 string containing raw PCM bytes, not a WAV file or another audio container.

Send small, consistently sized audio chunks in realtime. A single event must not exceed 15 MiB. Incorrect audio format, sample rate, or container handling can cause transcription to fail.

Session configuration

For session.update, set session.type to transcription. Transcription settings are placed under session.audio.input.transcription:

Use noise_reduction.type=near_field for headsets and far_field for meeting rooms or laptop microphones. Set it to null to disable noise reduction.

Manual commit and server VAD

• turn_detection: null: The client calls input_audio_buffer.commit after sending audio. Use this when the application explicitly controls segmentation.
• Server VAD: Configure the threshold, speech prefix, and silence duration so the server automatically detects speech start and end.

Do not commit an empty buffer in manual mode. With VAD enabled, use speech_started and speech_stopped to update the recording UI, but treat the completed event as the final transcript.

Connection examples

The following examples assume that audio.pcm has already been converted to raw, mono PCM16 audio at 24 kHz. A realtime application should continuously send small audio chunks rather than read an entire recording at once.

import fs from 'node:fs';
import WebSocket from 'ws';

const ws = new WebSocket('wss://api.agtcloud.cn/v1/realtime?model=MODEL_ID', {
  headers: {
    Authorization: 'Bearer your-api-key',
  },
});

ws.on('open', async () => {
  ws.send(JSON.stringify({
  "type": "session.update",
  "session": {
    "type": "transcription",
    "audio": {
      "input": {
        "format": { "type": "audio/pcm", "rate": 24000 },
        "noise_reduction": { "type": "near_field" },
        "transcription": {
          "model": "MODEL_ID",
          "prompt": "Chinese-language customer support call",
          "keywords": ["AGTCloud", "API gateway"],
          "languages": ["zh-cn", "en"]
        },
        "turn_detection": null
      }
    }
  }
}));

  const audio = fs.createReadStream('audio.pcm', { highWaterMark: 4800 });
  for await (const chunk of audio) {
    ws.send(JSON.stringify({
      type: 'input_audio_buffer.append',
      audio: chunk.toString('base64'),
    }));
    await new Promise((resolve) => setTimeout(resolve, 100));
  }
  ws.send(JSON.stringify({ type: 'input_audio_buffer.commit' }));
});

ws.on('message', (message) => {
  const event = JSON.parse(message.toString());
  if (event.type.endsWith('.delta')) process.stdout.write(event.delta);
  if (event.type.endsWith('.completed')) console.log('\n', event.transcript);
});

Event reference

The client uses session and audio-buffer events to control input. The server associates incremental text, final text, and errors with Item IDs. Different audio items can complete out of order, so aggregate text by item_id.

{
  "event_id": "evt_session_001",
  "type": "session.update",
  "session": {
    "type": "transcription",
    "audio": {
      "input": {
        "format": {
          "type": "audio/pcm",
          "rate": 24000
        },
        "noise_reduction": {
          "type": "near_field"
        },
        "transcription": {
          "model": "MODEL_ID",
          "prompt": "Chinese-language customer support call",
          "keywords": [
            "AGTCloud",
            "API gateway"
          ],
          "languages": [
            "zh-cn",
            "en"
          ],
          "delay": "low"
        },
        "turn_detection": null
      }
    }
  }
}

Error handling and recovery

Incremental text can be revised by later results. Do not write a delta value to an immutable record. Only the transcript value in completed should be treated as the final text for the current audio item.