Run your Retell AI agent on an open model: the custom LLM bridge

Retell AI prices the brain of your voice agent by the minute: the LLM line on its pricing page runs from $0.045 to $0.16 per minute for frontier models, on top of the voice engine. Retell also has a custom LLM slot that skips that line entirely: it opens a WebSocket to your server during the call, streams the live transcript, and speaks whatever text you stream back. That slot is a clean fit for an open model on EcoHash. A small bridge server, about 90 lines, receives Retell's events and answers with streaming completions from llama-3.1-8b-instruct, which costs a fraction of a cent per call minute in tokens. This post walks through the protocol, the bridge, a local test that does not need a phone call, and the honest math.

Same agent, cheaper brain. Retell's custom LLM slot plus a 90-line bridge puts an open model behind your voice agent for well under a cent per minute in tokens.

What you are wiring together

• Retell slot: Custom LLM, WebSocket; Retell connects to your server per call

• Protocol: Retell sends transcript events; you stream back response events

• Bridge: integrations/retell in ecohash-examples, about 90 lines

• Default model: llama-3.1-8b-instruct, $0.10 in / $0.10 out per 1M tokens

• Measured on EcoHash: 205 ms TTFT (p95), ~143 tokens/sec single stream

• What stays on Retell: Voice engine, STT, TTS, telephony (their built-in providers)

• API base URL: https://api.ecohash.com/v1

How the pieces fit

Retell does not call a chat completions URL directly. When a call starts, Retell opens a WebSocket to your server and the two sides exchange JSON events:

1. Your server sends a config event and a first sentence to speak.

2. When the caller finishes a turn, Retell sends response_required with the live transcript and a response_id.

3. The bridge forwards the transcript to EcoHash /v1/chat/completions with stream: true.

4. Each token delta goes back to Retell as a response event; a final content_complete closes the turn. If the caller interrupts, a newer response_id arrives and the stale reply is dropped.

The bridge

The full server is in integrations/retell/server.py. The core of it:

@app.websocket("/llm-websocket/{call_id}")
async def llm_websocket(ws: WebSocket, call_id: str):
    await ws.accept()
    await ws.send_json({"response_type": "config",
                        "config": {"auto_reconnect": True, "call_details": False}})
    await ws.send_json({"response_type": "response", "response_id": 0,
                        "content": BEGIN_SENTENCE, "content_complete": True})

    async for event in ws.iter_json():
        if event.get("interaction_type") in ("response_required", "reminder_required"):
            stream = await client.chat.completions.create(
                model="llama-3.1-8b-instruct",
                messages=to_messages(event["transcript"]),
                stream=True,
            )
            async for chunk in stream:
                delta = chunk.choices[0].delta.content or ""
                if delta:
                    await ws.send_json({"response_type": "response",
                                        "response_id": event["response_id"],
                                        "content": delta, "content_complete": False})
            await ws.send_json({"response_type": "response",
                                "response_id": event["response_id"],
                                "content": "", "content_complete": True})

Run it and expose it publicly (ngrok is fine for a first test):

pip install fastapi uvicorn openai
export ECOHASH_API_KEY=eco_...   # create one at console.ecohash.com
uvicorn server:app --host 0.0.0.0 --port 8000

The key comes from the EcoHash console: create a free account, open API Keys, create one, and copy the eco_ value into ECOHASH_API_KEY.

Point the agent at it

In the Retell dashboard, click Create an Agent, open Other options, and pick Custom LLM:

Then set the Custom LLM URL to your bridge:

wss://your-host.example.com/llm-websocket

Retell appends the call id and connects when a call starts. Note the cost estimate in the agent header: it now covers voice and telephony only, with no LLM line in it, which is the point of this post. Everything else about the agent, voice, transcriber, phone number, stays exactly as it was. Retell's own custom LLM setup guide covers the dashboard walkthrough in detail; this bridge is the part their guide leaves as an exercise, wired to an open model.

Test it without making a call

The bridge speaks a documented protocol, so you can fake Retell from your terminal with a WebSocket client:

import asyncio, json, websockets

async def main():
    async with websockets.connect("ws://localhost:8000/llm-websocket/test-1") as ws:
        print(json.loads(await ws.recv()))          # config
        print(json.loads(await ws.recv())["content"])  # first sentence
        await ws.send(json.dumps({
            "interaction_type": "response_required", "response_id": 1,
            "transcript": [{"role": "user", "content": "Do you deliver on weekends?"}]}))
        reply = ""
        while True:
            ev = json.loads(await ws.recv())
            reply += ev.get("content", "")
            if ev.get("content_complete"):
                break
        print(reply)

asyncio.run(main())

We ran this exact loop while writing the post: the bridge answered the config handshake, spoke its first sentence, and streamed a coherent 18-chunk reply from llama-3.1-8b-instruct, with ping and interrupt handling working as documented.

What it costs

With Retell's built-in models, the LLM is billed per minute. With a custom LLM, that line drops off Retell's bill and you pay your own provider per token instead. List prices, checked 2026-07-23:

• GPT 5.5 via Retell: $0.16

• GPT 5.4 or Claude 4.6 Sonnet via Retell: $0.08

• GPT 4.1 via Retell: $0.045

• llama-3.1-8b-instruct on EcoHash via bridge: ~$0.0003 in tokens

The EcoHash figure assumes a typical support call burns about 2,000 input and 250 output tokens per minute at $0.10 per 1M each; even a context-heavy agent at ten times that usage stays under a cent. Retell's other lines, the voice engine at $0.055 per minute plus TTS and telephony, are unchanged, so the saving applies to the LLM line only. You also take on hosting a small server, which is the real trade.

The trade-offs

• The custom LLM slot covers the brain only. Retell's STT and TTS stay on its built-in providers, so you cannot plug EcoHash's Kokoro in here. For that pattern, Vapi has an open TTS slot: see Give your Vapi voice agent a cheaper voice.

• You are hosting a WebSocket server on the call path. Keep it close to Retell's region and return the first token fast; llama-3.1-8b-instruct measures 205 ms TTFT (p95) on EcoHash, and your hosting adds on top. Measure before promising numbers.

• Voice replies should be short. Keep the system prompt strict about one-sentence answers or the agent will monologue.

• The cost table uses list prices as of the date above and stated token assumptions. Recheck both before building a business case.

FAQ

Does Retell support an OpenAI-compatible base URL directly? No. The custom LLM slot is a WebSocket protocol, not a base URL field, which is why you run a small bridge. The bridge in this post is about 90 lines and the protocol is documented.

Which models can the bridge use? Any chat model on EcoHash. llama-3.1-8b-instruct is a good default for voice; gpt-oss-20b adds capability at $0.05 in / $0.18 out per 1M tokens. See /models.

What happens when the caller interrupts? Retell sends a new response_required with a higher response_id. The bridge tracks the latest id and drops any reply still streaming for an older one.

Can I add tool calls or RAG? Yes, that is the point of owning the bridge: run retrieval, call tools, or apply guardrails between the transcript and the model. The same key also serves /v1/embeddings and a reranker for RAG.

Does this change Retell's voice quality or latency budget? Voice stays on Retell's engine. Your contribution to latency is bridge hosting plus model TTFT, so host near Retell and pick a fast model.

How do I transcribe or analyze calls afterwards? Use EcoHash whisper-large-v3-turbo on the recordings and a chat model for summaries, on the same API key. See Build a voice agent with Whisper, Kokoro, and an OpenAI-compatible API.

Next steps

• Create a free EcoHash API key: console.ecohash.com

• Bridge source: integrations/retell

• Browse chat models and prices: /models

• The Vapi version of this pattern, for the voice side: /blog/vapi-custom-tts-kokoro

• LLM benchmarks: ecohash-benchmarks