โ† All examples

ModelEthio-ASR Amharic (600M)2Ethio-ASR Multi (300M)5Ethio-ASR Multi (600M)4Ethio-ASR Multi (94M)6Meta MMS-1B16OmniASR-CTC (1B)5OmniASR-CTC (300M)9OmniASR-CTC (3B)8OmniASR-LLM (1B)10OmniASR-LLM (300M)12OmniASR-LLM (3B)9OpenAI Whisper-small88
Audio qualityClean Studio16Ambient Room Noise30Harsh Environment88Distant / Low-Volume Mic16Wideband Phone (VoLTE)16Randomized Phone Channel16Narrowband 2G + Packet Loss16

What did the model get wrong?

OpenAI Whisper-small Harsh Environment   Example 17 of 20  ยท  speaker: Male

๐ŸŽง Listen โ€” Harsh Environment
Very heavy background noise โ€” much harder than typical audio.
โœ… What was actually said
แ‰ แˆ˜แ‰ฐแŒแ‰ แˆชแ‹ซแ‹ แ‹แˆตแŒฅ แˆฎแˆšแŠ•แŒ แˆฐแˆญแ‰ชแˆต แ‹จแˆšแˆ แˆšแŠ’ แˆ˜แ‰ฐแŒแ‰ แˆชแ‹ซ แ‹ซแŒˆแŠ›แˆ‰ แŠฅแ‹› แ‹แˆตแŒฅ แŒˆแ‰ฅแ‰ฐแ‹ แŠ แŒˆแˆแŒแˆŽแ‰ฑแŠ• แˆ›แˆตแŒ€แˆ˜แˆญแˆ แˆ†แŠ แˆ›แ‰‹แˆจแŒฅ แ‹ญแ‰ฝแˆ‹แˆ‰
๐Ÿค– What this model heard
sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd s
Correct
Wrong (different word)
Missed (skipped)
Extra (added)
0
Correct
16
Wrong
0
Missed
72
Extra
0%
Words right
Out of 16 spoken words, this model got 0 right (0%). Errors: 16 words wrong; 72 extra.

Try switching Audio quality โ€” see the same model degrade as the audio gets noisier.