A Voice AI Agent on a 16 GB Laptop: Local Speech, Small Model, No Cloud
Voice assistants used to mean a stack of paid cloud services, each billed per minute or per word. We ran a complete voice agent, listening, thinking and speaking, on one office laptop with open models. That means a voice agent can sit on one server with one fixed monthly price.
The short answer: a voice AI agent has three parts: speech-to-text, a language model, and text-to-speech. All three now have open models small enough to run together on an everyday machine. Our Agent Engineering team ran IBM’s Granite 4.2 8B model with OpenWhispr for speech-to-text and Kokoro-82M for text-to-speech, on a 16 GB Windows laptop with no dedicated graphics card. This is a feasibility result, not a production launch, and it points to one clear benefit: a voice agent with a cost you can budget.
Three words explained
- Speech-to-text (STT): software that listens and writes down what was said.
- Text-to-speech (TTS): software that reads text aloud in a natural voice.
- Voice agent: a system that listens, understands, looks things up or takes an action, and replies by voice.
Why are voice AI costs hard to predict?
A typical cloud voice agent pays three meters at once: per minute of audio heard, per token the AI model reads and writes, and per character spoken back. A long call, a talkative customer or a busy month moves all three. For a founder planning a budget, that is three numbers nobody can promise in advance.
Running all three parts on your own server replaces three meters with one fixed price.
Longer calls and busier months move every meter on the left; the single server on the right costs the same either way.
Can a full voice agent run on a 16 GB laptop?
Yes. We ran speech-to-text, a language model and text-to-speech together on one everyday laptop:
| Part | What we used |
|---|---|
| Laptop | 11th-gen Intel Core i5 (i5-1145G7), 16 GB RAM, Intel Iris Xe integrated graphics, no dedicated GPU |
| Listening (STT) | OpenWhispr, an open-source tool that runs Whisper-family speech models on the device |
| Thinking (language model) | Granite 4.2 8B, open-weight, Apache 2.0 |
| Speaking (TTS) | Kokoro-82M, a small open-weight text-to-speech model, Apache 2.0 |
For voice we used the larger 8B model, the size recommended for voice agents, and our own testing confirmed it. Each part of this stack can be swapped for another open model that suits your languages and hardware. All three parts ran together within the laptop’s 16 GB of memory.
How does the voice loop work?
A local voice agent works in four steps, all on the same machine:
- Listen. The microphone captures the question; speech-to-text turns it into text, on the device.
- Understand and answer. The language model reads the question, searches your documents if needed, and writes a short answer.
- Speak. Text-to-speech reads the answer aloud.
- Log. The question, the answer and its source are saved, so quality can be checked.
A spoken question passes through three open models on the same laptop and comes back as speech; no audio leaves the machine.
Where does a local voice agent fit?
- Reception or kiosk: visitors ask about services, directions or opening hours.
- Internal voice lookup: staff ask about stock, procedures or policies hands-free, on a shop floor or in a warehouse.
- Voice intake: the agent collects a caller’s details and request so a person can follow up.
- Privacy-sensitive settings: audio never leaves your own machine.
An always-on phone line serving many callers at once is a bigger build. It needs telephony integration, more server capacity and careful testing.
What does a voice agent cost in the cloud?
The same stack moves to one cloud server. On AWS, a server with an NVIDIA T4 graphics card (g4dn.xlarge: 4 vCPU, 16 GB, 1 GPU) costs about $264 a month on a one-year commitment, or about $384 pay-as-you-go, in the US East region (public price list, November 2026; storage and data transfer extra). A GPU is worth having for voice, because all three models share it.
That price does not change with the number of calls or minutes. You know your voice AI bill on day one of the year.
What should you get right before you build?
- Test with real voices. Accents, background noise and phone-quality audio all affect speech-to-text. Record 30 to 50 real questions and test with those.
- Keep answers short. People listen more slowly than they read. Instruct the model to answer in two or three sentences.
- Plan the hand-off. The agent should know when to pass the caller to a person, and do it smoothly.
- Check every licence. Each part of the stack, STT, model and TTS, has its own licence. See Open-Source Voice AI Models Worth Testing for Your Voice Agent.
Common mistakes
- Testing only with clean, studio-quality audio.
- Letting the model give long, written-style answers that sound robotic when spoken.
- Forgetting that a voice clone of a real person needs that person’s consent.
- Treating a laptop demo as a phone system.
How SHVX can help
Want a voice agent with a fixed monthly cost? Our Agent Engineering team tests your use case on open voice models, sizes the server and gives you the full cost picture before you build.
Learn more about Agent Engineering.
FAQ
Can a voice AI agent run without the cloud?
Yes. We ran speech-to-text, a language model and text-to-speech together on one 16 GB laptop with open models.
Which model size do I need for voice?
We used an 8B model, the size recommended for voice, and our testing confirmed it works well. Test it on your own recorded questions before you commit.
Does my audio leave the building?
Not when every part runs on your own machine or private server.
What does it cost to run in the cloud?
One AWS server with a T4 GPU costs about $264 a month on a one-year commitment in the US East region, plus storage, whatever the number of calls.
Server prices checked against public AWS price lists in November 2026. Cloud prices change often; re-check before you budget.