Stop Whisper from appending repetition loops to local dictation

My local voice-to-text app had an ugly failure mode. A useful dictation would end with hundreds of copies of On, page, or a short phrase.

One recording produced 89 words I wanted, then more than 500 words of repetition. A few silent recordings became 223 copies of On and nothing else.

What the logs showed

I replayed the cleaner against 3,484 entries in my local WhisperAlone history:

The failures clustered around long pauses and accidental recordings. They also had the same suspicious sizes. Several separate recordings returned the exact same 223-token On loop.

The fake streaming path skipped VAD

The app called its local path “streaming,” but it did not run streaming inference. The renderer sent a WebM chunk every 100 milliseconds. The Python server stored every chunk, joined them at the end, and then called Whisper once.

That path had two problems. It made hundreds of local HTTP requests per recording, and it skipped the voice activity detection trimming in the batch path. Whisper received the trailing silence that followed my speech.

I removed those endpoints. The renderer now sends one complete WebM file with the first and last speech timestamps from its local VAD. The server passes that range to MLX Whisper through clip_timestamps.

Sending the complete file matters. Cutting early MediaRecorder chunks can remove the WebM header and produce an invalid file.

The app now skips a recording when VAD finds no speech. It also stops recordings after five minutes, which prevents an accidental open microphone from transcribing the room.

Stop one bad window from prompting the next

MLX Whisper uses condition_on_previous_text=True by default. Each 30-second window can prompt the next window with its output.

The MLX Whisper parameter documentation says that disabling this option makes the model less likely to get stuck in repetition loops or lose timestamp sync. WhisperAlone now calls:

mlx_whisper.transcribe(
    audio_file,
    path_or_hf_repo=model_name,
    clip_timestamps=f"{speech_start},{speech_end}",
    condition_on_previous_text=False,
)

The installed mlx-whisper version was 0.4.3. Its fallback code also had the high no-speech behavior described in MLX issue 1427. That issue explains how a high no-speech probability can make MLX accept a decode even when its compression ratio marks the output as repetitive.

A final filter catches obvious loops

Audio-side fixes reduce the chance of a bad decode. They cannot promise that a generative model will never repeat itself.

I added a deterministic filter at the point where all transcription backends meet. It detects only consecutive repeats above fixed limits:

• Eight copies of one word.
• Four copies of a two-word or three-word phrase.
• Three copies of a phrase with four to twelve words, with at least twelve repeated words total.

Short emphasis such as no, no, no stays intact. The filter removes a long loop at the end, keeps valid text around a loop in the middle, and rejects loop-only output before anything reaches the clipboard.

A versioned migration runs the same filter over stored history once. It updates word counts and rebuilds usage totals. The raw application logs stay untouched.

Why I did not add a tiny LLM

My first idea was to put a small local language model after Whisper. I already had mlx-community/Qwen2.5-1.5B-Instruct-4bit on the machine, so I tested it on six clean and six corrupted transcripts.

The model summarized some dictations, dropped filler words, and sometimes stopped with the loop still present. Those are bad properties for dictation cleanup. I need the words I said, not a more polished guess.

The deterministic filter preserved all clean samples in the same check. It also ran against the full local history in under one second. No new model download was needed.

Reproduce the history check

The repository now includes a read-only replay command:

npm run eval:cleanup

It reads the local history, runs the filter, and prints aggregate counts. It does not change the history or print transcript text.

I also tested the audio boundary with a generated sentence wrapped in seven seconds of silence. The new HTTP path used the VAD clip timestamps and returned the sentence exactly in 2.07 seconds from a new server process.

The useful fix turned out to be smaller than a 500-million-parameter model. Keep silence away from the decoder, stop bad windows from feeding later windows, and reject only the repetition pattern you can identify with certainty.