Fix silent live preview on stereo and multi-input audio interfaces

AVAudioConverter keeps only channel 1 when downmixing stereo and outputs
pure silence for layout-less multi-channel inputs (e.g. a 4-input USB
interface such as the UA Volt 476P), so the dictation model heard nothing
and the HUD never showed text. Pick the channel that carries the voice
(loudest on smoothed energy, switching only on a clear 6 dB margin) and
convert it as mono. Also reserve the dictation locale, which the Speech
framework warns will become required, and log the mic format.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.gitignore

@@ -3,6 +3,7 @@ macos/build/
 macos/WhisperDictate.app/Contents/MacOS/
 macos/WhisperDictate.app/Contents/Frameworks/
 macos/WhisperDictate.app/Contents/_CodeSignature/
+macos/WhisperDictate.app/Contents/CodeResources
 WhisperDictate
 *.o
 *.dSYM/

README.md

@@ -12,21 +12,42 @@ A simple menu bar app for voice dictation using OpenAI Whisper (local, offline).
 
 ## macOS
 
+### What's new in 1.4.0
+
+- 💬 **Live preview while recording** — the recording overlay shows what you're saying as you speak, using macOS's on-device dictation model (works in Hungarian too). It's a preview: the pasted text still comes from Whisper, so it can differ slightly. Toggle it in Settings.
+
+### What's new in 1.3.1
+
+- 🪶 **Quantized models** — *Large v3 Turbo q5_0* keeps the transcription server at ~0.7 GB RAM instead of ~1.75 GB, at the same speed (also Small q5_1, Medium q5_0, Turbo q8_0 and Large v3 q5_0)
+- 📥 **Download progress window** — progress bar, size, speed, time remaining, Cancel/Retry; progress also shows in the menu bar and Settings
+- 🔥 **Model warm-up at recording start** — if the model was paged out after a pause, it starts loading as soon as you start talking, not after you stop; the HUD tells you when a transcription is slower because the model is still loading
+
+### What's new in 1.3.0
+
+- ⚡ **Warm transcription server** — the Whisper model stays loaded in a background `whisper-server`, so each dictation is a fast local call (~0.5s) instead of reloading the 1.5–1.6 GB model on every utterance
+- 📊 **Live mic-level HUD** while recording, which warns ("No audio input?") if the microphone stays silent
+- 📖 **Custom dictionary** — an editable *Heard → Replacement* table that fixes names/jargon in the transcript and also biases recognition
+
 ### Features
 
 - 🎤 Global hotkey (⌃⌥D) to start/stop recording
+- ⚡ Fast transcription via a warm whisper-server (model stays loaded — no per-dictation reload), with automatic fallback to `whisper-cli`
+- 📊 Live mic-level meter while recording, with a "no audio input" warning
+- 💬 Live transcript preview in the recording overlay (on-device Apple dictation model; Whisper still produces the pasted text)
+- 📖 Custom dictionary (heard → replacement) that also biases recognition
 - 🔒 Fully offline - uses local Whisper model
-- ⚡ Automatic paste into any focused app
+- ⌨️ Automatic paste into any focused app
 - 📋 Clipboard preservation - your copied content is restored after paste
 - ⚙️ Settings window with model selection dropdown
-- 📥 Built-in model downloader with progress indicator
+- 📥 Built-in model downloader with a progress window (speed, time remaining, cancel/retry)
+- 🪶 Quantized model options for much lower memory use
 - 🚀 Launch at login support
 - 🔊 Sound feedback (optional)
-- 📦 Self-contained - whisper-cli bundled in app
+- 📦 Self-contained - whisper-cli and whisper-server bundled in the app
 
 ### Requirements
 
-- macOS 13.0+
+- macOS 26 (Tahoe) or later
 - Apple Silicon (M1/M2/M3) or Intel Mac
 
 ### Quick Install (Download)
@@ -74,8 +95,10 @@ make install
 
 Click the menu bar icon → Settings to configure:
 - **Language**: Auto-detect or 31 supported languages
-- **Model**: Select from installed models or download new ones
+- **Model**: Select from installed models or download new ones (✓ marks models already downloaded)
+- **Dictionary**: Add *Heard → Replacement* pairs to fix names/jargon (use `\n` for a newline); the replacement terms also improve recognition
 - **Sound feedback**: Toggle audio feedback on/off
+- **Live preview while recording**: Show a rough live transcript in the recording overlay (on by default)
 - **Launch at login**: Start automatically when you log in
 
 ### Whisper Models
@@ -91,8 +114,22 @@ Download models directly from the app or manually:
 | Large v3 Turbo | 1.6 GB | ~4 sec | Best | **Recommended** - fast like Medium, accurate like Large |
 | Large v3 | 3.1 GB | ~8 sec | Maximum | Difficult audio, max accuracy |
 
+**Quantized variants** (q5/q8) have much smaller weights, so the model uses far less memory, with a small accuracy trade-off:
+
+| Model | Size | Full-size equivalent |
+|-------|------|----------------------|
+| Small q5_1 | 181 MB | Small (466 MB) |
+| Medium q5_0 | 514 MB | Medium (1.5 GB) |
+| Large v3 Turbo q5_0 | 547 MB | Large v3 Turbo (1.6 GB) — measured ~0.7 GB RAM vs ~1.75 GB, same speed |
+| Large v3 Turbo q8_0 | 833 MB | Large v3 Turbo (1.6 GB) — near-identical accuracy |
+| Large v3 q5_0 | 1.0 GB | Large v3 (3.1 GB) |
+
 Models are stored in `~/.whisper-models/`
 
+> With the warm server (1.3.0+), the selected model loads once at startup (a few seconds); after that each dictation skips the reload, so the speeds above are roughly the per-dictation transcription time.
+>
+> The server keeps the whole model in memory while it runs. If macOS pages it out after a long pause, 1.3.1+ starts loading it back as soon as you start recording, and the HUD shows *"Loading model into memory…"* if that takes a moment.
+
 ### Audio Feedback
 
 - 🔔 **Tink** - Recording started
@@ -114,6 +151,8 @@ Grant these in System Settings → Privacy & Security:
 ## Security
 
 - All processing is done locally - no data leaves your device
+- The live preview uses macOS's on-device dictation model (SpeechAnalyzer); it needs no extra permission beyond the microphone
+- The bundled `whisper-server` is bound to `127.0.0.1` (loopback only) and is never exposed on the network
 - Audio files are stored in private temp directory and deleted after transcription
 - Clipboard is cleared after paste (transcript doesn't remain accessible)
 - Original clipboard content is preserved and restored after paste

macos/Makefile

@@ -2,7 +2,7 @@
 
 APP_NAME = WhisperDictate
 APP_BUNDLE = $(APP_NAME).app
-VERSION = 1.2.0
+VERSION = 1.4.0
 
 # Directories
 SRC_DIR = src
@@ -11,7 +11,7 @@ BUNDLE_DIR = $(APP_BUNDLE)/Contents
 
 # Compiler settings
 SWIFT = swiftc
-SWIFT_FLAGS = -O -framework Cocoa -framework AVFoundation -framework Carbon -framework CoreGraphics -framework ServiceManagement
+SWIFT_FLAGS = -O -framework Cocoa -framework AVFoundation -framework Speech -framework Carbon -framework CoreGraphics -framework ServiceManagement
 
 .PHONY: all clean build bundle install run dmg
 

macos/WhisperDictate.app/Contents/Info.plist

@@ -2,35 +2,40 @@
 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 <plist version="1.0">
 <dict>
-    <key>CFBundleDevelopmentRegion</key>
+	<key>CFBundleDevelopmentRegion</key>
-    <string>en</string>
+	<string>en</string>
-    <key>CFBundleExecutable</key>
+	<key>CFBundleDisplayName</key>
-    <string>WhisperDictate</string>
+	<string>WhisperDictate</string>
-    <key>CFBundleIconFile</key>
+	<key>CFBundleExecutable</key>
-    <string>AppIcon</string>
+	<string>WhisperDictate</string>
-    <key>CFBundleIdentifier</key>
+	<key>CFBundleIconFile</key>
-    <string>com.whisperdictate.app</string>
+	<string>AppIcon</string>
-    <key>CFBundleInfoDictionaryVersion</key>
+	<key>CFBundleIdentifier</key>
-    <string>6.0</string>
+	<string>com.whisperdictate.app</string>
-    <key>CFBundleName</key>
+	<key>CFBundleInfoDictionaryVersion</key>
-    <string>WhisperDictate</string>
+	<string>6.0</string>
-    <key>CFBundleDisplayName</key>
+	<key>CFBundleName</key>
-    <string>WhisperDictate</string>
+	<string>WhisperDictate</string>
-    <key>CFBundlePackageType</key>
+	<key>CFBundlePackageType</key>
-    <string>APPL</string>
+	<string>APPL</string>
-    <key>CFBundleShortVersionString</key>
+	<key>CFBundleShortVersionString</key>
-    <string>1.0.0</string>
+	<string>1.4.0</string>
-    <key>CFBundleVersion</key>
+	<key>CFBundleVersion</key>
-    <string>1</string>
+	<string>5</string>
-    <key>LSMinimumSystemVersion</key>
+	<key>LSMinimumSystemVersion</key>
-    <string>13.0</string>
+	<string>13.0</string>
-    <key>LSUIElement</key>
+	<key>LSUIElement</key>
-    <true/>
+	<true/>
-    <key>NSHighResolutionCapable</key>
+	<key>NSAppTransportSecurity</key>
-    <true/>
+	<dict>
-    <key>NSMicrophoneUsageDescription</key>
+		<key>NSAllowsLocalNetworking</key>
-    <string>WhisperDictate needs microphone access to record your voice for transcription.</string>
+		<true/>
-    <key>NSAppleEventsUsageDescription</key>
+	</dict>
-    <string>WhisperDictate needs accessibility access to paste transcribed text.</string>
+	<key>NSAppleEventsUsageDescription</key>
+	<string>WhisperDictate needs accessibility access to paste transcribed text.</string>
+	<key>NSHighResolutionCapable</key>
+	<true/>
+	<key>NSMicrophoneUsageDescription</key>
+	<string>WhisperDictate needs microphone access to record your voice for transcription.</string>
 </dict>
 </plist>

macos/WhisperDictate.entitlements

@@ -0,0 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<!-- Not sandboxed: WhisperDictate uses global Accessibility / synthetic
+	     keystrokes and spawns a bundled CLI, which the App Sandbox forbids.
+	     Distributed via Developer ID, not the Mac App Store. -->
+	<key>com.apple.security.app-sandbox</key>
+	<false/>
+	<!-- The app bundles whisper-cli + ggml/whisper dylibs in the app. Disabling
+	     library validation lets the process tree load them without tripping the
+	     hardened runtime's same-Team-ID check. -->
+	<key>com.apple.security.cs.disable-library-validation</key>
+	<true/>
+	<!-- Hardened-runtime resource-access entitlement. REQUIRED for microphone
+	     access under the hardened runtime: without it tccd refuses to even
+	     prompt and denies kTCCServiceMicrophone outright, so the app records
+	     silence (and whisper hallucinates on the empty audio). Must be present
+	     on any Developer ID / notarized build. -->
+	<key>com.apple.security.device.audio-input</key>
+	<true/>
+	<!-- whisper.cpp's Metal/ggml backend generates code at runtime; the
+	     hardened runtime needs allow-jit so the bundled whisper-cli/dylibs
+	     don't get killed for JIT. -->
+	<key>com.apple.security.cs.allow-jit</key>
+	<true/>
+</dict>
+</plist>

macos/bundle-whisper.sh

@@ -25,6 +25,17 @@ mkdir -p "$FRAMEWORKS_DIR"
 cp "$WHISPER_CLI_REAL" "$MACOS_DIR/whisper-cli"
 chmod +x "$MACOS_DIR/whisper-cli"
 
+# Copy whisper-server (warm, preloaded-model HTTP backend) from the same bin dir
+WHISPER_BIN_DIR=$(dirname "$WHISPER_CLI_REAL")
+WHISPER_SERVER_REAL="$WHISPER_BIN_DIR/whisper-server"
+if [ -f "$WHISPER_SERVER_REAL" ]; then
+    cp "$WHISPER_SERVER_REAL" "$MACOS_DIR/whisper-server"
+    chmod +x "$MACOS_DIR/whisper-server"
+    echo "Copied: whisper-server"
+else
+    echo "Warning: whisper-server not found at $WHISPER_SERVER_REAL"
+fi
+
 # List of dylibs to copy
 DYLIBS=(
     "libwhisper.1.dylib"
@@ -49,9 +60,12 @@ for dylib in "${DYLIBS[@]}"; do
     fi
 done
 
-# Fix dylib paths in whisper-cli
+# Fix dylib paths in the bundled executables (whisper-cli + whisper-server)
-for dylib in "${DYLIBS[@]}"; do
+for exe in whisper-cli whisper-server; do
-    install_name_tool -change "@rpath/$dylib" "@executable_path/../Frameworks/$dylib" "$MACOS_DIR/whisper-cli" 2>/dev/null || true
+    [ -f "$MACOS_DIR/$exe" ] || continue
+    for dylib in "${DYLIBS[@]}"; do
+        install_name_tool -change "@rpath/$dylib" "@executable_path/../Frameworks/$dylib" "$MACOS_DIR/$exe" 2>/dev/null || true
+    done
 done
 
 # Fix dylib paths in each dylib (they reference each other)
@@ -67,8 +81,10 @@ for dylib in "${DYLIBS[@]}"; do
     fi
 done
 
-# Sign everything
+# Sign everything (ad-hoc; the release flow re-signs with Developer ID afterward)
-codesign --force --sign - "$MACOS_DIR/whisper-cli" 2>/dev/null || true
+for exe in whisper-cli whisper-server; do
+    [ -f "$MACOS_DIR/$exe" ] && codesign --force --sign - "$MACOS_DIR/$exe" 2>/dev/null || true
+done
 for dylib in "${DYLIBS[@]}"; do
     codesign --force --sign - "$FRAMEWORKS_DIR/$dylib" 2>/dev/null || true
 done