Add warm whisper-server backend (no per-dictation model reload)

Each dictation previously spawned whisper-cli with -m, reloading the full
1.5-1.6GB model (+ ~10s Metal init) every single time. This adds a
WhisperServer manager that keeps a whisper-server child process alive with
the model preloaded, so each utterance is a fast local HTTP POST to
/inference (~0.5s) instead of a multi-second cold start.

- Bundles whisper-server alongside whisper-cli (bundle-whisper.sh).
- Server (re)starts on launch and on model change; stops on terminate;
  picks a free loopback port; -nt (plain text) + -sns (fewer hallucinations).
- transcribe() prefers the warm server and falls back to a per-call
  whisper-cli spawn if the server is unavailable.
- Warm-up now starts BEFORE the blocking accessibility prompt so the model
  load overlaps with the user granting permissions.
- Info.plist: NSAllowsLocalNetworking for the loopback HTTP call.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
hariel1985
2026-06-27 08:56:45 +02:00
szülő 865b194eeb
commit 21be80ddb4
3 fájl változott, egészen pontosan 274 új sor hozzáadva és 44 régi sor törölve

Fájl megtekintése

@@ -25,6 +25,17 @@ mkdir -p "$FRAMEWORKS_DIR"
cp "$WHISPER_CLI_REAL" "$MACOS_DIR/whisper-cli"
chmod +x "$MACOS_DIR/whisper-cli"
# Copy whisper-server (warm, preloaded-model HTTP backend) from the same bin dir
WHISPER_BIN_DIR=$(dirname "$WHISPER_CLI_REAL")
WHISPER_SERVER_REAL="$WHISPER_BIN_DIR/whisper-server"
if [ -f "$WHISPER_SERVER_REAL" ]; then
cp "$WHISPER_SERVER_REAL" "$MACOS_DIR/whisper-server"
chmod +x "$MACOS_DIR/whisper-server"
echo "Copied: whisper-server"
else
echo "Warning: whisper-server not found at $WHISPER_SERVER_REAL"
fi
# List of dylibs to copy
DYLIBS=(
"libwhisper.1.dylib"
@@ -49,9 +60,12 @@ for dylib in "${DYLIBS[@]}"; do
fi
done
# Fix dylib paths in whisper-cli
for dylib in "${DYLIBS[@]}"; do
install_name_tool -change "@rpath/$dylib" "@executable_path/../Frameworks/$dylib" "$MACOS_DIR/whisper-cli" 2>/dev/null || true
# Fix dylib paths in the bundled executables (whisper-cli + whisper-server)
for exe in whisper-cli whisper-server; do
[ -f "$MACOS_DIR/$exe" ] || continue
for dylib in "${DYLIBS[@]}"; do
install_name_tool -change "@rpath/$dylib" "@executable_path/../Frameworks/$dylib" "$MACOS_DIR/$exe" 2>/dev/null || true
done
done
# Fix dylib paths in each dylib (they reference each other)
@@ -67,8 +81,10 @@ for dylib in "${DYLIBS[@]}"; do
fi
done
# Sign everything
codesign --force --sign - "$MACOS_DIR/whisper-cli" 2>/dev/null || true
# Sign everything (ad-hoc; the release flow re-signs with Developer ID afterward)
for exe in whisper-cli whisper-server; do
[ -f "$MACOS_DIR/$exe" ] && codesign --force --sign - "$MACOS_DIR/$exe" 2>/dev/null || true
done
for dylib in "${DYLIBS[@]}"; do
codesign --force --sign - "$FRAMEWORKS_DIR/$dylib" 2>/dev/null || true
done