Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor (get-inscribe.com)

Article summary

Apple's new SpeechAnalyzer API has been benchmarked against Whisper and its predecessor, SFSpeechRecognizer, on a standard test dataset of 5,559 utterances. The results show that SpeechAnalyzer is the most accurate on-device speech engine, beating Whisper Small and SFSpeechRecognizer on both clean and noisy speech. SpeechAnalyzer also runs roughly three times faster than Whisper Small. The benchmark suggests that developers should migrate from SFSpeechRecognizer to SpeechAnalyzer for improved accuracy.

Main themes

• Speech recognition accuracy
• On-device vs cloud-based models
• Apple SpeechAnalyzer vs Whisper
• Model benchmarking
• Language support
• Device compatibility

What commenters say

• Some commenters suggest that newer models like Parakeet and Granite Speech 4.1 should be considered as alternatives to Whisper.
• Others argue that Apple's SpeechAnalyzer is not usable due to its limited language support and accuracy issues.
• There is a debate about the feasibility of reverse-engineering Apple's SpeechAnalyzer model and making it available on non-Apple platforms.
• Some commenters believe that Apple's new Siri is slow and not as effective as other conversational AI models like ChatGPT.
• Others argue that the new Siri is an improvement over the old one, but still has room for improvement.
• There are concerns about the copyright and trade secret laws surrounding AI models and their weights.
• Some commenters think that benchmarking against older models like Whisper Small is not sufficient and that newer models should be used for comparison.