Eleven v4 Turbo WebSocket Error: Fix + Latency Test

Because ElevenLabs runs two WebSocket products, and the one every voice-agent tutorial uses, the per-voice /v1/text-to-speech/{voice_id}/stream-input socket, only serves the older model line: Flash, Turbo v2.5, Multilingual v2. Point it at eleven_v4_turbo and the upgrade request never becomes a socket:

GET wss://api.elevenlabs.io/v1/text-to-speech/{voice_id}/stream-input?model_id=eleven_v4_turbo

HTTP/1.1 400 Bad Request
{"detail": {
  "type": "validation_error",
  "code": "unsupported_model",
  "message": "Model 'eleven_v4_turbo' is not supported on the text-to-speech
              websocket endpoint. Use the text-to-dialogue websocket endpoint instead.",
  "param": "model_id"
}}

Most WebSocket clients hide the body. Python's websockets library gave me InvalidStatus: server rejected WebSocket connection: HTTP 400 and nothing else, so I had to replay the upgrade with curl to read the message. eleven_v4 and eleven_v3_conversational fail the same way. The reverse mistake is worse: Flash v2.5 on the dialogue socket came back as a bare 403 with an empty body.

The model ID swap is the trap, the same way it was for Claude 5.5. ElevenLabs shipped v4 Turbo on September 28 as the model for "conversational agents," so the obvious move is changing one string in a config that already streams Flash. That config is pointed at the wrong endpoint. The TTS vs Text to Dialogue WebSockets guide spells it out: no eleven_v3 or eleven_v4 on the TTS socket, and model_id on the dialogue socket must start with one of them.

Four protocol changes, all small:

• URL: wss://api.elevenlabs.io/v1/text-to-dialogue/stream-input?model_id=eleven_v4_turbo. No voice ID in the path.
• First message registers voices: {"voices": ["<voice_id>"]}. The docs say v4 Turbo allows one registered voice per connection, Eleven v4 up to ten.
• Text goes in an inputs array: each item is {text, voice_id, new_turn} instead of a bare text string.
• A flushed turn ends with a different flag: is_final_audio_for_turn: true. The TTS socket's isFinaldoesn't exist here, and the snake_case is_final only arrives after you send close_socket. My first version of the client below waited for is_final and hung on the first reply.

This is the client I tested against the live API, trimmed to the parts that matter:

import asyncio, base64, json, os
import websockets

KEY = os.environ["ELEVENLABS_API_KEY"]
VOICE = "EXAVITQu4vr4xnSDxMaL"
URI = ("wss://api.elevenlabs.io/v1/text-to-dialogue/stream-input"
       "?model_id=eleven_v4_turbo&output_format=pcm_16000")

async def keep_alive(ws):
    while True:
        await asyncio.sleep(12)            # server closes at 20s of silence
        await ws.send(json.dumps({"keep_alive": True}))

async def speak(ws, text, on_audio):
    # flush=True: short lines never start without it
    await ws.send(json.dumps({
        "inputs": [{"text": text + " ", "voice_id": VOICE}],
        "flush": True,
    }))
    async for raw in ws:
        msg = json.loads(raw)
        if msg.get("error"):
            raise RuntimeError(msg)
        if msg.get("audio"):
            on_audio(base64.b64decode(msg["audio"]))
        if msg.get("is_final_audio_for_turn"):   # not is_final
            break

async def call_session():
    async with websockets.connect(URI, additional_headers={"xi-api-key": KEY}) as ws:
        await ws.send(json.dumps({"voices": [VOICE]}))   # replaces voice_id in the URL
        pinger = asyncio.create_task(keep_alive(ws))
        try:
            ...  # per agent turn: await speak(ws, llm_reply, play)
        finally:
            pinger.cancel()

The two lines doing the real work are "flush": True and the keep-alive task. Both exist because of how this socket behaves on a phone call, which is the next section.

The socket dies while the caller talks

The dialogue socket has a fixed 20-second inactivity limit, counted from your last message. I opened one, registered a voice and waited 25 seconds. The next send failed with close code 1008 and the reason No message received within 20s.A caller explaining a leaking water heater easily talks for 20 seconds, so without keep-alives your agent's next reply hits a dead socket. Sending {"keep_alive": true} every 5 seconds over the same 25-second wait kept it open, and the next line streamed normally.

The old TTS socket let you set inactivity_timeoutup to 180 seconds. This one doesn't, so the keep-alive (or a reconnect per turn) is mandatory, not a tuning knob. Reconnecting is cheap: the handshake took a median 66 ms from here.

Short replies never play

The server waits for about 40 characters and 8 words before it generates anything. I sent "Sure, one moment." without a flush and got no audio for 8 seconds, at which point my test gave up. The same line with flush: truestarted in 109 ms. Agent replies like "Got it.", "Yes." and "One sec." are exactly the short ones, and that silence is what makes callers say "hello?" and trigger interruption loops. Flush at the end of every LLM turn, always.

Concurrency now counts open sockets

This is the one that changes your capacity math.On the TTS socket only active generation counted toward concurrency, so a Pro plan's limit of 20 could carry a lot of calls. Each open dialogue socket holds one session for its whole life, from a separate pool: 14 on Free, 21 on Starter, 35 on Creator, 70 on Pro, 105 on Scale and Business, per the models and concurrency docs. One socket per call means 70 simultaneous calls on Pro, and call 71 gets too_many_concurrent_requests. For an outbound dialer that's a hard ceiling to plan around, not a soft queue.

ElevenLabs says no: about 100 ms median model latency for v4 Turbo against about 75 ms for Flash v2.5, network excluded. I measured what a caller actually waits for, time from sending the text to the first audio byte, network included, with the same 110-character agent line in English and Spanish. Milliseconds, median with range, 8 runs each (5 for Eleven v4):

On its own socket, v4 Turbo started speaking about 24 ms sooner than Flash. Both numbers include the same network hop, so the vendor's 25 ms gap in Flash's favor didn't show up from where I sit. One English v4 Turbo run spiked to 528 ms, but the median was 114 in both languages and every Spanish run landed between 105 and 120. That matters if your agents switch to Spanish mid-call.

Flash still finishes the line about five times faster. Over HTTP streaming, Flash generated the whole 5.5-second line in 260 ms. v4 Turbo needed about 1.4 seconds for 6.3 seconds of audio. That's still well ahead of playback, so the caller never hears a gap, but it means each turn ties up generation capacity roughly five times longer.

Don't use plain Eleven v4 on a call.Half a second to first audio on either transport, and that sits on top of your STT and LLM time. It's a voiceover model. Eleven v3 Conversational, the previous expressive option for agents, did 223 ms in my single HTTP check.

Not in credits. Every ElevenLabs TTS response carries a character-cost header. For the same 110-character line, Flash v2.5 charged 55, v4 Turbo charged 55, and Eleven v4 charged 110. The Models API agrees: eleven_v4_turbo and eleven_flash_v2_5 both list a character_cost_multiplier of 0.5, eleven_v4lists 1.0. The models page still has a line saying v4 Turbo costs more per character than Flash, so check your own plan's pricing before you budget off my header.

The headers also expose the concurrency split. On the same free account, Flash requests reported maximum-concurrent-requests: 4 and every v4-family request reported 2. They sit in different concurrency groups, so moving HTTP traffic from Flash to v4 Turbo halves your headroom before you touch the dialogue socket at all.

If you run your own pipeline, the kind where you own the Twilio media stream, the STT and the LLM call, yes for anything where tone sells: booking calls, warm outbound follow-ups, anything in Spanish. Speed isn't the reason to stay on Flash anymore. Stay on Flash if you need more than your plan's dialogue sessions at once, or if your agent reads back a lot of long confirmations and you want the generation capacity back.

If you use a hosted platform, this isn't your socket to manage. The platform owns the connection, the keep-alives and the session pool, and you pick the voice model from a dropdown. Retell has already deprecated eleven_turbo_v2_5in favor of Flash and migrates agents automatically, which is a good reminder that the platform decides what's available. I compared the two big hosted options in ElevenLabs Agents vs Retell AI and Retell vs Vapi, and the honest limits of all of them are in what actually works in voice AI in 2026.

How I tested this: October 21, 2026, from Allen, Texas to api.elevenlabs.io (responses reported x-region: us-central1), free plan, premade voice Sarah, pcm_16000 output, Python requests and websockets 17.3. WebSocket times start when the text frame is sent on an already open socket. HTTP times start at the request on a warm keep-alive connection, after one warm-up call per model. 8 runs per cell, 5 for Eleven v4. The free plan has the lowest queue priority, so paid plans may be a little faster across the board. Credit costs come from the character-cost response header. About 5,500 credits in total.