4 Commit-ok

Szerző SHA1 Üzenet Dátum
hariel1985
78caac8d04 Fix silent live preview on stereo and multi-input audio interfaces
AVAudioConverter keeps only channel 1 when downmixing stereo and outputs
pure silence for layout-less multi-channel inputs (e.g. a 4-input USB
interface such as the UA Volt 476P), so the dictation model heard nothing
and the HUD never showed text. Pick the channel that carries the voice
(loudest on smoothed energy, switching only on a clear 6 dB margin) and
convert it as mono. Also reserve the dictation locale, which the Speech
framework warns will become required, and log the mic format.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:25:58 +02:00
hariel1985
fef0cd4ade Update README for 1.4.0
Also correct the requirement to macOS 26: the app is built against the
macOS 26 SDK with a macOS 26 deployment target.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 11:29:30 +02:00
hariel1985
3890cd8ab4 Bump version to 1.4.0
Minor release: live transcript preview in the recording HUD using the
on-device Apple dictation model, with Whisper still producing the final
pasted text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 11:29:30 +02:00
hariel1985
0983ba0fde Add live transcript preview in the recording HUD
While recording, the HUD now shows a rough live transcript using macOS's
on-device dictation model (SpeechAnalyzer + DictationTranscriber), which
also supports Hungarian. It is only a preview: Whisper still produces the
pasted text.

The preview captures the microphone with its own AVAudioEngine, leaving
the AVAudioRecorder path for Whisper untouched, so a preview failure just
means no preview. Audio is buffered from the moment recording starts so
the first words aren't lost while the model spins up; dictionary terms
are passed as contextual strings and replacements are applied to the
preview text. The dictation model for the selected language is
registered/downloaded at launch and on language change, and awaited
inline if needed.

The HUD is wider and shows the last three lines of the transcript; after
recording it stays up as "Transcribing…" until the final text is pasted.
A "Live preview while recording" setting (on by default) turns it off.
No extra permission is required beyond the microphone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 11:29:30 +02:00
4 fájl változott, egészen pontosan 326 új sor hozzáadva és 21 régi sor törölve

Fájl megtekintése

@@ -12,6 +12,10 @@ A simple menu bar app for voice dictation using OpenAI Whisper (local, offline).
## macOS ## macOS
### What's new in 1.4.0
- 💬 **Live preview while recording** — the recording overlay shows what you're saying as you speak, using macOS's on-device dictation model (works in Hungarian too). It's a preview: the pasted text still comes from Whisper, so it can differ slightly. Toggle it in Settings.
### What's new in 1.3.1 ### What's new in 1.3.1
- 🪶 **Quantized models** — *Large v3 Turbo q5_0* keeps the transcription server at ~0.7 GB RAM instead of ~1.75 GB, at the same speed (also Small q5_1, Medium q5_0, Turbo q8_0 and Large v3 q5_0) - 🪶 **Quantized models** — *Large v3 Turbo q5_0* keeps the transcription server at ~0.7 GB RAM instead of ~1.75 GB, at the same speed (also Small q5_1, Medium q5_0, Turbo q8_0 and Large v3 q5_0)
@@ -29,6 +33,7 @@ A simple menu bar app for voice dictation using OpenAI Whisper (local, offline).
- 🎤 Global hotkey (⌃⌥D) to start/stop recording - 🎤 Global hotkey (⌃⌥D) to start/stop recording
- ⚡ Fast transcription via a warm whisper-server (model stays loaded — no per-dictation reload), with automatic fallback to `whisper-cli` - ⚡ Fast transcription via a warm whisper-server (model stays loaded — no per-dictation reload), with automatic fallback to `whisper-cli`
- 📊 Live mic-level meter while recording, with a "no audio input" warning - 📊 Live mic-level meter while recording, with a "no audio input" warning
- 💬 Live transcript preview in the recording overlay (on-device Apple dictation model; Whisper still produces the pasted text)
- 📖 Custom dictionary (heard → replacement) that also biases recognition - 📖 Custom dictionary (heard → replacement) that also biases recognition
- 🔒 Fully offline - uses local Whisper model - 🔒 Fully offline - uses local Whisper model
- ⌨️ Automatic paste into any focused app - ⌨️ Automatic paste into any focused app
@@ -42,7 +47,7 @@ A simple menu bar app for voice dictation using OpenAI Whisper (local, offline).
### Requirements ### Requirements
- macOS 13.0+ - macOS 26 (Tahoe) or later
- Apple Silicon (M1/M2/M3) or Intel Mac - Apple Silicon (M1/M2/M3) or Intel Mac
### Quick Install (Download) ### Quick Install (Download)
@@ -93,6 +98,7 @@ Click the menu bar icon → Settings to configure:
- **Model**: Select from installed models or download new ones (✓ marks models already downloaded) - **Model**: Select from installed models or download new ones (✓ marks models already downloaded)
- **Dictionary**: Add *Heard → Replacement* pairs to fix names/jargon (use `\n` for a newline); the replacement terms also improve recognition - **Dictionary**: Add *Heard → Replacement* pairs to fix names/jargon (use `\n` for a newline); the replacement terms also improve recognition
- **Sound feedback**: Toggle audio feedback on/off - **Sound feedback**: Toggle audio feedback on/off
- **Live preview while recording**: Show a rough live transcript in the recording overlay (on by default)
- **Launch at login**: Start automatically when you log in - **Launch at login**: Start automatically when you log in
### Whisper Models ### Whisper Models
@@ -145,6 +151,7 @@ Grant these in System Settings → Privacy & Security:
## Security ## Security
- All processing is done locally - no data leaves your device - All processing is done locally - no data leaves your device
- The live preview uses macOS's on-device dictation model (SpeechAnalyzer); it needs no extra permission beyond the microphone
- The bundled `whisper-server` is bound to `127.0.0.1` (loopback only) and is never exposed on the network - The bundled `whisper-server` is bound to `127.0.0.1` (loopback only) and is never exposed on the network
- Audio files are stored in private temp directory and deleted after transcription - Audio files are stored in private temp directory and deleted after transcription
- Clipboard is cleared after paste (transcript doesn't remain accessible) - Clipboard is cleared after paste (transcript doesn't remain accessible)

Fájl megtekintése

@@ -2,7 +2,7 @@
APP_NAME = WhisperDictate APP_NAME = WhisperDictate
APP_BUNDLE = $(APP_NAME).app APP_BUNDLE = $(APP_NAME).app
VERSION = 1.3.1 VERSION = 1.4.0
# Directories # Directories
SRC_DIR = src SRC_DIR = src
@@ -11,7 +11,7 @@ BUNDLE_DIR = $(APP_BUNDLE)/Contents
# Compiler settings # Compiler settings
SWIFT = swiftc SWIFT = swiftc
SWIFT_FLAGS = -O -framework Cocoa -framework AVFoundation -framework Carbon -framework CoreGraphics -framework ServiceManagement SWIFT_FLAGS = -O -framework Cocoa -framework AVFoundation -framework Speech -framework Carbon -framework CoreGraphics -framework ServiceManagement
.PHONY: all clean build bundle install run dmg .PHONY: all clean build bundle install run dmg

Fájl megtekintése

@@ -19,9 +19,9 @@
<key>CFBundlePackageType</key> <key>CFBundlePackageType</key>
<string>APPL</string> <string>APPL</string>
<key>CFBundleShortVersionString</key> <key>CFBundleShortVersionString</key>
<string>1.3.1</string> <string>1.4.0</string>
<key>CFBundleVersion</key> <key>CFBundleVersion</key>
<string>4</string> <string>5</string>
<key>LSMinimumSystemVersion</key> <key>LSMinimumSystemVersion</key>
<string>13.0</string> <string>13.0</string>
<key>LSUIElement</key> <key>LSUIElement</key>

Fájl megtekintése

@@ -1,5 +1,6 @@
import Cocoa import Cocoa
import AVFoundation @preconcurrency import AVFoundation
@preconcurrency import Speech
import Carbon.HIToolbox import Carbon.HIToolbox
import ServiceManagement import ServiceManagement
import ApplicationServices import ApplicationServices
@@ -11,6 +12,7 @@ struct Defaults {
static let modelPath = "whisperModelPath" static let modelPath = "whisperModelPath"
static let playSounds = "playSounds" static let playSounds = "playSounds"
static let replacements = "customReplacements" static let replacements = "customReplacements"
static let livePreview = "livePreview"
} }
// MARK: - Whisper Models // MARK: - Whisper Models
@@ -411,6 +413,208 @@ final class WhisperServer {
// A small floating, non-interactive panel shown while recording. Doubles as a // A small floating, non-interactive panel shown while recording. Doubles as a
// diagnostic: if the input stays silent it flags "no input" — exactly the // diagnostic: if the input stays silent it flags "no input" — exactly the
// failure that silently produced empty recordings + whisper hallucinations. // failure that silently produced empty recordings + whisper hallucinations.
// MARK: - Live Preview (on-device Apple dictation model)
// Streams a rough live transcript into the recording HUD using the system's
// on-device dictation model (SpeechAnalyzer + DictationTranscriber, macOS 26).
// It is only a preview: the pasted text still comes from Whisper. It captures
// the microphone with its own AVAudioEngine so the AVAudioRecorder path used
// for Whisper stays untouched; if anything here fails there is simply no preview.
final class LivePreview {
// Called on the main thread with the transcript so far
var onText: ((String) -> Void)?
private var session = 0
private var engine: AVAudioEngine?
private var analyzer: SpeechAnalyzer?
private var input: AsyncStream<AnalyzerInput>.Continuation?
private var resultsTask: Task<Void, Never>?
// App language code -> supported dictation locale ("hu" -> hu-HU); nil if unsupported
static func locale(for languageCode: String) async -> Locale? {
let requested = languageCode == SupportedLanguages.autoDetect ? Locale.current : Locale(identifier: languageCode)
return await DictationTranscriber.supportedLocale(equivalentTo: requested)
}
private static func makeTranscriber(locale: Locale) -> DictationTranscriber {
DictationTranscriber(locale: locale, contentHints: [], transcriptionOptions: [.punctuation],
reportingOptions: [.volatileResults, .frequentFinalization], attributeOptions: [])
}
// Download the on-device model for this language in the background if it's missing
static func installModelIfNeeded(languageCode: String) {
Task {
guard let locale = await locale(for: languageCode) else { return }
do {
if let request = try await AssetInventory.assetInstallationRequest(supporting: [makeTranscriber(locale: locale)]) {
NSLog("Live preview: installing dictation model for \(locale.identifier)")
try await request.downloadAndInstall()
NSLog("Live preview: dictation model installed for \(locale.identifier)")
}
} catch {
NSLog("Live preview: model install failed: \(error)")
}
}
}
func start(languageCode: String, contextualStrings: [String]) {
stop()
let current = session
Task { @MainActor in
do {
guard let locale = await LivePreview.locale(for: languageCode) else {
NSLog("Live preview: no on-device dictation model for '\(languageCode)'")
return
}
let transcriber = LivePreview.makeTranscriber(locale: locale)
// Even when the model is already on disk the app may first have to register it;
// that takes a fraction of a second. A real download just means no preview this time.
_ = try? await AssetInventory.reserve(locale: locale)
if await AssetInventory.status(forModules: [transcriber]) != .installed,
let request = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) {
NSLog("Live preview: preparing dictation model for \(locale.identifier)")
try await request.downloadAndInstall()
}
guard current == self.session,
let analyzerFormat = await SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith: [transcriber]) else { return }
// Start the microphone first; the stream buffers audio until the analyzer is running,
// so the first words aren't lost while the model spins up.
let (stream, continuation) = AsyncStream<AnalyzerInput>.makeStream()
let engine = AVAudioEngine()
let micFormat = engine.inputNode.outputFormat(forBus: 0)
guard micFormat.sampleRate > 0, current == self.session,
let monoFormat = AVAudioFormat(standardFormatWithSampleRate: micFormat.sampleRate, channels: 1),
let converter = AVAudioConverter(from: monoFormat, to: analyzerFormat) else { return }
NSLog("Live preview: mic %d ch @ %.0f Hz -> %@", micFormat.channelCount, micFormat.sampleRate, analyzerFormat.description)
engine.inputNode.installTap(onBus: 0, bufferSize: 4096, format: micFormat,
block: LivePreview.tapBlock(converter: converter, format: analyzerFormat, continuation: continuation))
try engine.start()
self.engine = engine
self.input = continuation
let analyzer = SpeechAnalyzer(modules: [transcriber])
self.analyzer = analyzer
if !contextualStrings.isEmpty {
let context = AnalysisContext()
context.contextualStrings[.general] = contextualStrings
try await analyzer.setContext(context)
}
self.resultsTask = Task { @MainActor [weak self] in
// Finalized segments accumulate; the volatile tail is replaced on every update
var finalized = ""
do {
for try await result in transcriber.results {
let text = String(result.text.characters)
let shown = LivePreview.join(finalized, text)
if result.isFinal { finalized = shown }
guard let self = self, current == self.session else { return }
self.onText?(shown)
}
} catch {
NSLog("Live preview: results ended: \(error)")
}
}
guard current == self.session else { return } // stop() already tore everything down
try await analyzer.start(inputSequence: stream)
} catch {
NSLog("Live preview failed: \(error)")
if current == self.session { self.stop() }
}
}
}
func stop() {
session += 1
if let engine = engine {
engine.inputNode.removeTap(onBus: 0)
engine.stop()
}
engine = nil
input?.finish()
input = nil
resultsTask?.cancel()
resultsTask = nil
if let analyzer = analyzer {
Task { await analyzer.cancelAndFinishNow() }
}
analyzer = nil
}
// Segments don't always carry a separating space
static func join(_ head: String, _ tail: String) -> String {
guard let last = head.last, let first = tail.first, !last.isWhitespace, !first.isWhitespace else { return head + tail }
return head + " " + tail
}
// Runs on the audio thread: take the voice channel as mono, convert it to the
// analyzer's format. Built in a static context so it carries no actor assumptions.
private static func tapBlock(converter: AVAudioConverter, format: AVAudioFormat,
continuation: AsyncStream<AnalyzerInput>.Continuation) -> AVAudioNodeTapBlock {
let downmixer = VoiceChannelPicker()
return { micBuffer, _ in
guard let buffer = downmixer.mono(from: micBuffer) else { return }
let ratio = format.sampleRate / buffer.format.sampleRate
let capacity = AVAudioFrameCount(Double(buffer.frameLength) * ratio) + 64
guard let out = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: capacity) else { return }
var consumed = false
var error: NSError?
converter.convert(to: out, error: &error) { _, status in
if consumed {
status.pointee = .noDataNow
return nil
}
consumed = true
status.pointee = .haveData
return buffer
}
if error == nil, out.frameLength > 0 {
continuation.yield(AnalyzerInput(buffer: out))
}
}
}
}
// Picks the input channel that carries the voice and returns it as a mono buffer.
// AVAudioConverter keeps only channel 1 when downmixing stereo, and outputs pure
// silence for layout-less multi-channel inputs (e.g. a 4-input USB interface),
// so the live preview model would hear nothing. The loudest channel is chosen on
// smoothed energy, and only replaced by one that is clearly (6 dB) louder.
final class VoiceChannelPicker {
private var energy: [Float] = []
private var active = 0
private var monoFormat: AVAudioFormat?
func mono(from buffer: AVAudioPCMBuffer) -> AVAudioPCMBuffer? {
let channels = Int(buffer.format.channelCount)
let frames = Int(buffer.frameLength)
guard frames > 0, let data = buffer.floatChannelData else { return nil }
if channels == 1 { return buffer }
if energy.count != channels {
energy = Array(repeating: 0, count: channels)
active = 0
}
for c in 0..<channels {
var sum: Float = 0
for i in 0..<frames { sum += data[c][i] * data[c][i] }
energy[c] = energy[c] * 0.9 + sum / Float(frames) * 0.1
}
if let loudest = energy.indices.max(by: { energy[$0] < energy[$1] }), energy[loudest] > energy[active] * 4 {
active = loudest
}
if monoFormat?.sampleRate != buffer.format.sampleRate {
monoFormat = AVAudioFormat(standardFormatWithSampleRate: buffer.format.sampleRate, channels: 1)
}
guard let format = monoFormat,
let out = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(frames)),
let dest = out.floatChannelData else { return nil }
dest[0].update(from: data[active], count: frames)
out.frameLength = AVAudioFrameCount(frames)
return out
}
}
final class MeterView: NSView { final class MeterView: NSView {
enum Mode { case recording, transcribing } enum Mode { case recording, transcribing }
enum Notice { case loadingModel, modelLoaded } enum Notice { case loadingModel, modelLoaded }
@@ -419,15 +623,55 @@ final class MeterView: NSView {
private var silent = false private var silent = false
var mode = Mode.recording { didSet { needsDisplay = true } } var mode = Mode.recording { didSet { needsDisplay = true } }
var notice: Notice? { didSet { needsDisplay = true } } var notice: Notice? { didSet { needsDisplay = true } }
var previewText: String? { didSet { if previewText != oldValue { cachedPreview = nil; needsDisplay = true } } }
var hasPreview: Bool { previewBlock() != nil }
// Height for the current mode + notice private static let maxPreviewLines = 3
private static let previewFont = NSFont.systemFont(ofSize: 13)
private var cachedPreview: (width: CGFloat, text: NSAttributedString, height: CGFloat)?
private var noticeHeight: CGFloat {
switch notice {
case .loadingModel?: return 35
case .modelLoaded?: return 19
case nil: return 0
}
}
// Height for the current mode, notice and live preview text
var preferredHeight: CGFloat { var preferredHeight: CGFloat {
let base: CGFloat = mode == .recording ? 72 : 44 let base: CGFloat = mode == .recording ? 72 : 44
switch notice { let preview = previewBlock().map { 6 + $0.height } ?? 0
case .loadingModel?: return base + 34 return base + noticeHeight + preview
case .modelLoaded?: return base + 20 }
case nil: return base
// The tail of the live transcript that fits in a few lines; older words scroll off the top
private func previewBlock() -> (text: NSAttributedString, height: CGFloat)? {
guard let raw = previewText, !raw.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty else { return nil }
let width = bounds.width - 32
if let cached = cachedPreview, cached.width == width { return (cached.text, cached.height) }
let attrs: [NSAttributedString.Key: Any] = [
.font: MeterView.previewFont,
.foregroundColor: NSColor(calibratedWhite: 1, alpha: 0.92)
]
func measure(_ string: NSAttributedString) -> CGFloat {
ceil(string.boundingRect(with: NSSize(width: width, height: 10_000), options: [.usesLineFragmentOrigin, .usesFontLeading]).height)
} }
let lineHeight = measure(NSAttributedString(string: "Ag", attributes: attrs))
let maxHeight = lineHeight * CGFloat(MeterView.maxPreviewLines) + 1
let words = raw.split(whereSeparator: { $0.isWhitespace })
var dropped = 0
var text = NSAttributedString(string: words.joined(separator: " "), attributes: attrs)
var height = measure(text)
while height > maxHeight && dropped < words.count - 1 {
dropped += 1
text = NSAttributedString(string: "…" + words[dropped...].joined(separator: " "), attributes: attrs)
height = measure(text)
}
cachedPreview = (width, text, height)
return (text, height)
} }
func set(level: CGFloat, silent: Bool) { func set(level: CGFloat, silent: Bool) {
@@ -435,7 +679,7 @@ final class MeterView: NSView {
self.silent = silent self.silent = silent
needsDisplay = true needsDisplay = true
} }
func reset() { level = 0; silent = false; mode = .recording; notice = nil; needsDisplay = true } func reset() { level = 0; silent = false; mode = .recording; notice = nil; previewText = nil; needsDisplay = true }
override func draw(_ dirtyRect: NSRect) { override func draw(_ dirtyRect: NSRect) {
let bg = NSBezierPath(roundedRect: bounds, xRadius: 12, yRadius: 12) let bg = NSBezierPath(roundedRect: bounds, xRadius: 12, yRadius: 12)
@@ -472,6 +716,12 @@ final class MeterView: NSView {
break break
} }
if let preview = previewBlock() {
let top = textY - 6
preview.text.draw(with: NSRect(x: 16, y: top - preview.height, width: bounds.width - 32, height: preview.height),
options: [.usesLineFragmentOrigin, .usesFontLeading])
}
guard mode == .recording else { return } guard mode == .recording else { return }
let segs = 20 let segs = 20
@@ -496,7 +746,7 @@ final class MeterView: NSView {
final class RecordingHUD { final class RecordingHUD {
private var window: NSWindow? private var window: NSWindow?
private let meter = MeterView(frame: NSRect(x: 0, y: 0, width: 320, height: 72)) private let meter = MeterView(frame: NSRect(x: 0, y: 0, width: 420, height: 72))
func show() { func show() {
meter.reset() meter.reset()
@@ -512,6 +762,13 @@ final class RecordingHUD {
layout() layout()
} }
func setPreview(_ text: String?) {
meter.previewText = text
layout()
}
var hasPreview: Bool { meter.hasPreview }
// Stay up after recording while a slow (model-loading) transcription runs // Stay up after recording while a slow (model-loading) transcription runs
func showTranscribing(notice: MeterView.Notice?) { func showTranscribing(notice: MeterView.Notice?) {
makeWindowIfNeeded() makeWindowIfNeeded()
@@ -533,7 +790,7 @@ final class RecordingHUD {
private func makeWindowIfNeeded() { private func makeWindowIfNeeded() {
if window == nil { if window == nil {
let w = NSPanel(contentRect: NSRect(x: 0, y: 0, width: 320, height: 72), let w = NSPanel(contentRect: NSRect(x: 0, y: 0, width: 420, height: 72),
styleMask: [.borderless, .nonactivatingPanel], styleMask: [.borderless, .nonactivatingPanel],
backing: .buffered, defer: false) backing: .buffered, defer: false)
w.isOpaque = false w.isOpaque = false
@@ -786,6 +1043,7 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
var settingsWindowController: NSWindowController? var settingsWindowController: NSWindowController?
var whisperServer: WhisperServer? var whisperServer: WhisperServer?
let recordingHUD = RecordingHUD() let recordingHUD = RecordingHUD()
let livePreview = LivePreview()
var meterTimer: Timer? var meterTimer: Timer?
var silenceStart: Date? var silenceStart: Date?
weak var dictTable: NSTableView? weak var dictTable: NSTableView?
@@ -814,6 +1072,11 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
set { UserDefaults.standard.set(newValue, forKey: Defaults.playSounds) } set { UserDefaults.standard.set(newValue, forKey: Defaults.playSounds) }
} }
var livePreviewEnabled: Bool {
get { UserDefaults.standard.object(forKey: Defaults.livePreview) as? Bool ?? true }
set { UserDefaults.standard.set(newValue, forKey: Defaults.livePreview) }
}
// Custom dictionary: ordered list of ["from": recognised, "to": replacement]. // Custom dictionary: ordered list of ["from": recognised, "to": replacement].
var replacements: [[String: String]] { var replacements: [[String: String]] {
get { (UserDefaults.standard.array(forKey: Defaults.replacements) as? [[String: String]]) ?? [] } get { (UserDefaults.standard.array(forKey: Defaults.replacements) as? [[String: String]]) ?? [] }
@@ -823,11 +1086,16 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
// Whisper initial prompt built from the desired ("to") terms, to bias // Whisper initial prompt built from the desired ("to") terms, to bias
// recognition toward the user's names/jargon. Returns nil if empty. // recognition toward the user's names/jargon. Returns nil if empty.
func customPrompt() -> String? { func customPrompt() -> String? {
let terms = replacements.compactMap { $0["to"] } let terms = dictionaryTerms()
return terms.isEmpty ? nil : terms.joined(separator: ", ")
}
// The desired ("to") dictionary terms, also given to the live preview model
func dictionaryTerms() -> [String] {
replacements.compactMap { $0["to"] }
.map { $0.replacingOccurrences(of: "\\n", with: " ") } .map { $0.replacingOccurrences(of: "\\n", with: " ") }
.map { $0.trimmingCharacters(in: .whitespaces) } .map { $0.trimmingCharacters(in: .whitespaces) }
.filter { !$0.isEmpty } .filter { !$0.isEmpty }
return terms.isEmpty ? nil : terms.joined(separator: ", ")
} }
// Apply the dictionary find/replace pairs to a transcript. "\n" in a // Apply the dictionary find/replace pairs to a transcript. "\n" in a
@@ -858,6 +1126,7 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
} }
checkAccessibilityPermission() checkAccessibilityPermission()
if livePreviewEnabled { LivePreview.installModelIfNeeded(languageCode: language) }
NSLog("WhisperDictate started. Press ⌃⌥D to toggle recording.") NSLog("WhisperDictate started. Press ⌃⌥D to toggle recording.")
} }
@@ -1091,7 +1360,7 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
func createSettingsWindow() -> NSWindow { func createSettingsWindow() -> NSWindow {
let window = NSWindow( let window = NSWindow(
contentRect: NSRect(x: 0, y: 0, width: 450, height: 540), contentRect: NSRect(x: 0, y: 0, width: 450, height: 580),
styleMask: [.titled, .closable], styleMask: [.titled, .closable],
backing: .buffered, backing: .buffered,
defer: false defer: false
@@ -1102,7 +1371,7 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
let contentView = NSView(frame: window.contentView!.bounds) let contentView = NSView(frame: window.contentView!.bounds)
var y: CGFloat = 498 var y: CGFloat = 538
let labelWidth: CGFloat = 120 let labelWidth: CGFloat = 120
let controlX: CGFloat = 140 let controlX: CGFloat = 140
let controlWidth: CGFloat = 280 let controlWidth: CGFloat = 280
@@ -1180,6 +1449,15 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
loginCheck.state = isLaunchAtLoginEnabled() ? .on : .off loginCheck.state = isLaunchAtLoginEnabled() ? .on : .off
contentView.addSubview(loginCheck) contentView.addSubview(loginCheck)
y -= 40
// Live preview
let previewCheck = NSButton(checkboxWithTitle: "Live preview while recording", target: self, action: #selector(livePreviewChanged(_:)))
previewCheck.frame = NSRect(x: controlX, y: y, width: controlWidth, height: 24)
previewCheck.state = livePreviewEnabled ? .on : .off
previewCheck.toolTip = "Shows a rough live transcript in the recording overlay using macOS's on-device dictation model. The pasted text still comes from Whisper."
contentView.addSubview(previewCheck)
// Custom dictionary (recognised -> replacement) // Custom dictionary (recognised -> replacement)
let dictLabel = NSTextField(labelWithString: "Dictionary (heard → replacement):") let dictLabel = NSTextField(labelWithString: "Dictionary (heard → replacement):")
dictLabel.frame = NSRect(x: 20, y: 304, width: 300, height: 20) dictLabel.frame = NSRect(x: 20, y: 304, width: 300, height: 20)
@@ -1247,6 +1525,7 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
@objc func languageChanged(_ sender: NSPopUpButton) { @objc func languageChanged(_ sender: NSPopUpButton) {
if let code = sender.selectedItem?.representedObject as? String { if let code = sender.selectedItem?.representedObject as? String {
language = code language = code
if livePreviewEnabled { LivePreview.installModelIfNeeded(languageCode: code) }
NSLog("Language changed to: \(language)") NSLog("Language changed to: \(language)")
} }
} }
@@ -1330,6 +1609,11 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
} }
} }
@objc func livePreviewChanged(_ sender: NSButton) {
livePreviewEnabled = sender.state == .on
if livePreviewEnabled { LivePreview.installModelIfNeeded(languageCode: language) }
}
@objc func playSoundsChanged(_ sender: NSButton) { @objc func playSoundsChanged(_ sender: NSButton) {
playSounds = sender.state == .on playSounds = sender.state == .on
} }
@@ -1562,6 +1846,7 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
NSLog("Recording started") NSLog("Recording started")
startMeter() startMeter()
prepareModelForDictation() prepareModelForDictation()
startLivePreview()
} catch { } catch {
NSLog("Recording failed: \(error)") NSLog("Recording failed: \(error)")
if playSounds { NSSound(named: "Basso")?.play() } if playSounds { NSSound(named: "Basso")?.play() }
@@ -1603,6 +1888,16 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
checkModelLoaded() checkModelLoaded()
} }
// MARK: - Live preview
func startLivePreview() {
guard livePreviewEnabled else { return }
livePreview.onText = { [weak self] text in
guard let self = self, self.isRecording else { return }
self.recordingHUD.setPreview(self.applyReplacements(text))
}
livePreview.start(languageCode: language, contextualStrings: dictionaryTerms())
}
// MARK: - Model warm-up // MARK: - Model warm-up
// Start getting the model into memory as soon as recording starts, so loading // Start getting the model into memory as soon as recording starts, so loading
// overlaps with the user talking instead of following it. Only if that turns // overlaps with the user talking instead of following it. Only if that turns
@@ -1659,10 +1954,13 @@ class AppDelegate: NSObject, NSApplicationDelegate, URLSessionDownloadDelegate,
audioRecorder?.stop() audioRecorder?.stop()
isRecording = false isRecording = false
isTranscribing = true isTranscribing = true
// Model still loading and already announced: keep explaining the wait livePreview.stop()
let keepHUD = modelLoading && loadNoticeShown // Keep the HUD up while Whisper works if there is something to show:
// the live preview text, or why this transcription is slower
let explainLoading = modelLoading && loadNoticeShown
let keepHUD = explainLoading || recordingHUD.hasPreview
stopMeter(hideHUD: !keepHUD) stopMeter(hideHUD: !keepHUD)
if keepHUD { recordingHUD.showTranscribing(notice: .loadingModel) } if keepHUD { recordingHUD.showTranscribing(notice: explainLoading ? .loadingModel : nil) }
statusItem.button?.title = "⏳" statusItem.button?.title = "⏳"
updateStatus("Transcribing...") updateStatus("Transcribing...")
if playSounds { NSSound(named: "Pop")?.play() } if playSounds { NSSound(named: "Pop")?.play() }