In Part 1 I described the painful reality of building voice-native systems: every team ends up rewriting the same brittle runtime plumbing around interruptions, audio transport, context injection, and backend switching.
That first release established the core promise: stable runtime contracts should let builders swap providers and deployment models without rewriting the product around them. Vona v0.2.0 delivers concrete progress on one of the most requested directions for that promise: high-quality local voice inference, with strong emphasis on Apple Silicon and simpler local LLM integration via Ollama.
Native MLX Voice on Apple Silicon
The biggest addition in this release is first-class native support for running speech models locally using Apple’s MLX framework.
v0.2.0 introduces:
vona-mlx-whisper- native MLX loader for Whisper-based speech recognitionvona-mlx-qwen3-tts- native MLX loader for Qwen3 TTS- Shared utilities for safetensors discovery, metadata parsing, and tensor materialisation
- Model provisioning manifests so applications can explicitly download and cache the required checkpoints (Distil-Whisper Large V3 and bf16 Qwen3 TTS)
This is a significant step. Instead of routing every audio frame through cloud APIs, you can now run both transcription and synthesis locally on Apple Silicon with a clean, modular integration into the Vona runtime.
A new mlx_ollama_voice_bench example and updated benchmark documentation show how to validate the full local voice pipeline end-to-end.
Ollama Streaming Adapter
Many developers want to use strong local (or self-hosted) LLMs without managing complex inference servers. v0.2.0 adds vona-ollama, a streaming text adapter for Ollama’s /api/generate endpoint.
This brings first-class support for using Ollama as the text generation backend inside Vona sessions, with proper token streaming. It’s exposed through a simple feature flag and integrates cleanly with the core runtime contracts.
Better Modularity & Contracts
v0.2.0 also includes important structural improvements:
- Core generation contracts were expanded to support text token streaming alongside decoupled audio transcription and synthesis traits.
- The MLX backend was split into more granular crates and feature gates. You can now enable just the MLX runtime (
native-mlx) or opt into the heavier model loading stack (mlx-models-loader) independently. - Several dependency and build cleanups were made to keep local-only builds lean.
These changes make it easier to compose exactly the voice stack you need without pulling in unnecessary weight.
What This Unlocks
With v0.2.0, Vona becomes significantly more practical for teams that want to build voice-native products with a local or hybrid footprint:
- Run high-quality STT + TTS entirely on-device (Apple Silicon)
- Combine local speech models with Ollama (or other) text generation
- Keep the same interruption handling, context injection, and deterministic testing guarantees regardless of backend
- Maintain a clean separation between your product logic and the voice runtime
This aligns directly with Vona’s original goal: giving developers a stable, testable foundation so they can focus on product differentiation instead of rebuilding runtime infrastructure.
What’s Next
The modular foundation is now in place for more local and hybrid voice backends. Future work will likely expand adapter coverage, improve the test harness, and refine the experience of mixing local and cloud components within the same session.
I’m especially interested in feedback from people running real voice workloads on Apple Silicon or using Ollama in production voice flows.
Try Vona v0.2.0
- GitHub Release: deliberium/vona v0.2.0
- Add it to your project with the features you need (e.g.
ollama,mlx-whisper-native)
If you’re building voice-native applications and want a runtime layer that doesn’t force you into a single provider or deployment model, Vona is ready for you to try and contribute to.
What kind of voice backends or local inference setups are you most interested in seeing supported next?