Skip to main content

Consider supporting the coreML version of Parakeet v3 for local ASR

BoltAI Mobile

Whisper is extremely slow by comparison, and while the ASR on macOS is actually decent now, those upgrades don’t seem to be coming to iOS devices (including the 16gb ipad pros). Even though I am a native english speaker and IRL people have no problem understanding me, the Apple ASR model on iOS literally can’t understand me well enough to finish 2 sentences properly.

I currently use the ~490mb distilled Parakeet ASR model with a 3rd party app on macOS, and it is absolutely sufficient. If we extrapolate based on my known transcription speed on M3 Max (~250x realtime) and relative Geekbench Metal performance, an iPhone 15 Pro could transcribe audio 47x faster than realtime. So if a user was yapping for 2 whole minutes, parakeet would transcribe it in 2.5s, it would probably take an extra ~2-3s for the first time loading it with app in foreground,

https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v2-coreml
It claims max memory usage is 800mb

Posted by kernkraft ·
2 comments

Log in to comment and vote

Comments2

  • Daniel Nguyen

    Team•

    Oct 28, 2025

    Thanks for the suggestion. I actually added support for Parakeet for the mac app thanks to your suggestion. It’s trickier for the iOS app, so stay tuned.

    • kernkraft

      •

      Oct 30, 2025

      Congrats, you now have a better TTS than the ChatGPT app!

      I see that you added a “Instant dictation” mode. This is a feature I will watch with great interest. If you are looking to expand on this feature path, I suggest looking at the MacWhisper app. They detect which app you are in when dictation is triggered and enable you to set custom prompts to have your dictation processed by an LLM of your choice before giving it back as text input (or clipboard fallback).

      One way you can take this idea from MacWhisper even further is if you made an option for “attach app window screenshot as context” (MacWhisper doesn’t do this). This would make my “Messages” app-specific prompt MUCH more powerful because the LLM could post-process my raw dictation from parakeet with the actual conversation context. It might be possible to also just scrape context from active app via Accessibility API, but IDK enough about it to say for sure.

      - My biggest complaint with MacWhisper’s implementation is that you have to set the full prompt per custom prompt. So I have 20 custom prompts for different apps and they all contain the same “boilerplate” before I get into custom instructions for each. This means when I change the LLM I use or need to tweak the boilerplate, I have to manually make 20 edits.

      - My 2nd biggest complaint with macwhisper is that it doesn’t provide any way to access the raw dictation too. You only can access dictation history AFTER the LLM post-processing