Realtime voice agents with GPT-Realtime-2.1 and Parallel | Parallel

Introducing Parallel Search Turbo: the fastest and lowest-cost search for AI. Learn more.Learn more.

HumanMachine

July 12, 2026

\# Building a realtime voice agent with GPT-Realtime-2.1 and Parallel Search Turbo

A concise tutorial for building a realtime voice agent that uses GPT-Realtime-2.1 for conversation and Parallel Search Turbo when a question needs current web context.

Tags:Developers

Reading time: 4 min

Voice agents are most useful when they can answer questions about what is happening now. The hard part is adding web search without turning every turn into a slow, opaque research job.

This demo uses GPT-Realtime-2.1 for conversation and Parallel Search Turbo for current web context. The model decides whether a turn needs research. If it does, the app runs one Turbo search, shows the returned sources, and asks Realtime to answer from that evidence. Greetings and clarification questions can skip search entirely.

Try asking about tomorrow’s weather, the latest box office results, or planning a summer trip.

Parallel Voice Search Demo

Ask the web.

Tap the mic and ask a question.

Try asking

“What will the weather be like in San Francisco tomorrow?”
“How do index funds differ from mutual funds?”
“What should I consider before visiting Kyoto in the summer?”

### GPT-Realtime-2.1 voice research demo

Interactive GPT-Realtime-2.1 voice research demo. In non-interactive or machine-readable contexts, try it at https://turbo-voice-research.vercel.app/embed.

Open interactive demo

\## How the handoff works

Realtime handles the conversation, Turbo searches the web, and the app controls the handoff between them.

The user speaks. Semantic voice activity detection decides when the turn is complete.
Realtime makes a routing decision: search the web, respond directly, or wait for clearer input.
For a substantive factual question, search_web creates a self-contained question and a few targeted search queries. The server sends one request to Parallel Search Turbo.
The app shows the returned sources and passes what Turbo found back into the same Realtime session.
The app turns tools off and requests one short audio answer grounded in that evidence.

The two-pass design keeps the first decision silent, prevents the model from speaking before research finishes, and avoids unnecessary searches for conversational turns.

Reference docs:

\## 1. Create a short-lived Realtime session

Create the Realtime client secret on your server. The standard OpenAI API key should never reach the browser.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

export async function POST(request: Request) {
  const response = await fetch(
    "https://api.openai.com/v1/realtime/client_secrets",
    {
      method: "POST",
      signal: AbortSignal.timeout(10_000),
      headers: {
        Authorization: "Bearer " + process.env.OPENAI_API_KEY,
        "Content-Type": "application/json",
        "OpenAI-Safety-Identifier": hashUser(request),
      },
      body: JSON.stringify({
        session: {
          type: "realtime",
          model: "gpt-realtime-2.1",
          instructions:
            "Wait for the client session configuration before responding.",
          reasoning: { effort: "low" },
          tools: [],
          tool_choice: "auto",
          audio: {
            input: {
              noise_reduction: { type: "far_field" },
              transcription: {
                language: "en",
                model: "gpt-realtime-whisper",
              },
              turn_detection: {
                type: "semantic_vad",
                eagerness: "medium",
                interrupt_response: true,
                create_response: true,
              },
            },
            output: { voice: "marin" },
          },
        },
      }),
    },
  );

const data = await response.json();
  return Response.json({ value: data.value }, { status: response.status });
}``` export async function POST(request: Request) {
  const response = await fetch(
    "https://api.openai.com/v1/realtime/client_secrets",
    {
      method: "POST",
      signal: AbortSignal.timeout(10_000),
      headers: {
        Authorization: "Bearer " + process.env.OPENAI_API_KEY,
        "Content-Type": "application/json",
        "OpenAI-Safety-Identifier": hashUser(request),
      },
      body: JSON.stringify({
        session: {
          type: "realtime",
          model: "gpt-realtime-2.1",
          instructions:
            "Wait for the client session configuration before responding.",
          reasoning: { effort: "low" },
          tools: [],
          tool_choice: "auto",
          audio: {
            input: {
              noise_reduction: { type: "far_field" },
              transcription: {
                language: "en",
                model: "gpt-realtime-whisper",
              },
              turn_detection: {
                type: "semantic_vad",
                eagerness: "medium",
                interrupt_response: true,
                create_response: true,
              },
            },
            output: { voice: "marin" },
          },
        },
      }),
    },
  );

const data = await response.json();
  return Response.json({ value: data.value }, { status: response.status });
}
```

The browser gives the agent and tool config after it receives this credential. The first automatic response is configured as a required, text-only routing pass, so Realtime can choose what to do without speaking too early.

\## 2. Connect with WebRTC

In the browser, request microphone access, create the Agents SDK WebRTC transport, and connect with the short-lived token.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

import {
OpenAIRealtimeWebRTC,
RealtimeAgent,
RealtimeSession,
} from "@openai/agents/realtime";

async function connectRealtime() {
const media = await navigator.mediaDevices.getUserMedia({ audio: true });
const token = await fetch("/api/realtime-token", { method: "POST" })
    .then((response) => response.json());

const audio = document.createElement("audio");
audio.autoplay = true;

const transport = new OpenAIRealtimeWebRTC({
    mediaStream: media,
    audioElement: audio,
});

const agent = new RealtimeAgent({
    name: "Parallel voice assistant",
    instructions: voiceAgentInstructions(),
    tools: [searchWebTool, respondDirectlyTool, waitForUserTool],
});

const session = new RealtimeSession(agent, {
    model: "gpt-realtime-2.1",
    transport,
    config: {
      toolChoice: "required",
      parallelToolCalls: false,
      outputModalities: ["text"],
      reasoning: { effort: "low" },
    },
});

await session.connect({
    apiKey: token.value,
    model: "gpt-realtime-2.1",
});

return session;
}``` import {
OpenAIRealtimeWebRTC,
RealtimeAgent,
RealtimeSession,
} from "@openai/agents/realtime";

const audio = document.createElement("audio");
audio.autoplay = true;

const transport = new OpenAIRealtimeWebRTC({
    mediaStream: media,
    audioElement: audio,
});

await session.connect({
    apiKey: token.value,
    model: "gpt-realtime-2.1",
});

return session;
}
```

Only search_web starts web research. The other routes handle simple conversational responses or ignore audio that was not directed at the assistant.

\## 3. Search with Parallel Turbo

The search route turns the user’s request into a self-contained question and one to three searches. The server includes the resolved question in the final query list, adds the current date and time zone to the objective, and calls the Search API in turbo mode.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

async function searchParallelTurbo(
  question: string,
  generatedQueries: string[],
) {
  const searchQueries = unique([\
    question,\
    ...generatedQueries,\
  ]).slice(0, 3);

const response = await fetch("https://api.parallel.ai/v1/search", {
    method: "POST",
    signal: AbortSignal.timeout(8_000),
    headers: {
      "Content-Type": "application/json",
      "x-api-key": process.env.PARALLEL_API_KEY!,
    },
    body: JSON.stringify({
      objective: buildSearchObjective(question),
      search_queries: searchQueries,
      mode: "turbo",
      max_chars_total: 10_000,
      advanced_settings: {
        max_results: 20,
        excerpt_settings: { max_chars_per_result: 1_000 },
      },
    }),
  });

if (!response.ok) throw new Error("Parallel Search failed.");
  return response.json();
}``` async function searchParallelTurbo(
  question: string,
  generatedQueries: string[],
) {
  const searchQueries = unique([\
    question,\
    ...generatedQueries,\
  ]).slice(0, 3);

if (!response.ok) throw new Error("Parallel Search failed.");
  return response.json();
}
```

The demo retrieves broadly, then passes a smaller, bounded evidence packet to Realtime. That gives the model enough context to answer without stuffing the full search response into the latency-sensitive audio turn.

Conversation history helps create better follow-ups. If someone asks about San Francisco weather and then says “what about tomorrow?”, Realtime carries the location forward and resolves the date before searching.

\## 4. Speak the grounded answer

When the search tool finishes, the app explicitly requests an audio response with tools disabled. The immediately preceding Turbo result is the evidence for factual claims in that answer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

session.on(
"agent_tool_end",
(_context, _agent, toolDefinition) => {
    if (toolDefinition.name !== "search_web") return;

session.transport.requestResponse({
      tools: [],
      tool_choice: "none",
      output_modalities: ["audio"],
      max_output_tokens: 800,
      reasoning: { effort: "low" },
      instructions: [\
        "Answer directly from the preceding Parallel Search output.",\
        "Use one or two complete spoken sentences under 60 words.",\
        "Do not mention tools, snippets, JSON, citations, or URLs.",\
        "Start with the answer.",\
      ].join("\n"),
    });
},
);``` session.on(
"agent_tool_end",
(_context, _agent, toolDefinition) => {
    if (toolDefinition.name !== "search_web") return;

session.transport.requestResponse({
      tools: [],
      tool_choice: "none",
      output_modalities: ["audio"],
      max_output_tokens: 800,
      reasoning: { effort: "low" },
      instructions: [\
        "Answer directly from the preceding Parallel Search output.",\
        "Use one or two complete spoken sentences under 60 words.",\
        "Do not mention tools, snippets, JSON, citations, or URLs.",\
        "Start with the answer.",\
      ].join("\n"),
    });
},
);
```

The sources appear in the UI as soon as search completes. Realtime then speaks the answer; the demo does not render a separate written answer transcript.

\## Latency notes

We don’t label the whole voice interaction as search latency. The full turn also includes voice activity detection, routing, answer generation, and audio playback.

- **Search latency: ** research request → Parallel response
- **Voice latency: ** speech stopped → first audio

The app records more detailed timings for debugging, but keeping these two numbers separate is enough to explain where users are waiting.

\## When to use this stack

This tech works well when a voice experience benefits from fast, sourced web context: current events, explainers, weather, travel planning, product research, and conversational follow-ups.

Turbo is a fast grounding step, not a full research pipeline. Use a specialized feed for authoritative live prices or scores, and a deeper research mode when the question needs broad retrieval or multi-hop reasoning.

\## Before you ship

- Keep OpenAI and Parallel API keys on the server.
- Use short-lived Realtime client secrets in the browser.
- Resolve follow-up references and relative dates before search.
- Measure search latency separately from time to first audio.

The pattern is simple: Realtime decides how to handle the turn, Turbo grounds factual answers in current web sources, and the app controls the boundary between search and speech.

\## Ready to get started?

Try Parallel Try Parallel Contact sales Contact sales

Are you an agent? Read this to onboard Parallel Are you an agent? Read this to onboard Parallel

By George Pickett

July 12, 2026

\## Related Posts80

Jul 30, 2026\ - Building an always-on background agent to proactively support customers

Tags: Developers

Author: By Khushi Shelat