On-Device Push-to-Talk Dictation without Cloud Latency
Published: August 15, 2026 · Read Time: 4 min read · Category: DSP & Audio
Author: Abhishek Shivakumar (Systems & Audio Engineering)
Designing a zero-latency audio pipeline for Whisper models on macOS using local ring buffers, voice activity detection, and Core ML.
Local Audio Pipeline
Cloud dictation services transmit audio payloads over network connections, introducing variable latency and external data dependencies. WhisperGo processes speech locally on Apple Silicon devices.
The audio capture layer runs a high-priority CoreAudio input callback writing into a lock-free circular ring buffer. When the push-to-talk shortcut activates, the recording window opens without blocking the UI thread.
Voice Activity Detection
Energy estimation and spectral flatness metrics determine speech boundaries. Background room noise is estimated continuously while the key is depressed, allowing the speech segment to be windowed with precise start and end margins.
{
"capture": {
"sample_rate_hz": 16000,
"ring_buffer_capacity_sec": 30.0,
"vad_threshold_db": -42.0
},
"model": {
"engine": "coreml_ane",
"checkpoint": "whisper_base_en_q8.mlmodelc",
"compute_units": "all"
}
}Core ML Transcription
Audio frames are converted to log-Mel spectrograms and dispatched to a quantized Whisper model compiled for the Apple Neural Engine and GPU. Decoding begins within 12 ms of key release.
Generated tokens are piped through text post-processing, resolving punctuation, numbers, and system voice commands before inserting characters into the active frontmost application.
Execution ProfileTranscription of a five-second speech snippet completes in approximately 95 ms on an M3 Pro processor.
Casing and Vocabulary Cache
A local database tracks custom programming terminology, acronyms, and proper nouns. Frequently spoken phrases receive boosted priority during greedy token selection, preventing repeated correction cycles.