misk@sopuli.xyz to Technology@lemmy.worldEnglish · 18 days agoConcerns about medical note-taking tool raised after researcher discovers it invents things no one said — Nabla is powered by OpenAI's Whisperwww.tomshardware.comexternal-linkmessage-square7fedilinkarrow-up189arrow-down11
arrow-up188arrow-down1external-linkConcerns about medical note-taking tool raised after researcher discovers it invents things no one said — Nabla is powered by OpenAI's Whisperwww.tomshardware.commisk@sopuli.xyz to Technology@lemmy.worldEnglish · 18 days agomessage-square7fedilink
minus-squareRobotToaster@mander.xyzlinkfedilinkEnglisharrow-up6arrow-down3·18 days agoHow can it be that bad? I’ve used zoom’s ai transcriptions, for far less mission critical stuff, and it’s generally fine, (I still wouldn’t trust it for medical purposes)
minus-squarehuginn@feddit.itlinkfedilinkEnglisharrow-up15arrow-down1·18 days agoZoom ai transcriptions also make things up. That’s the point. They’re hallucination engines. They pattern match and fill holes by design. It doesn’t matter if the match isn’t perfect, it will patch it over with nonsense instead.
minus-squareElPussyKangaroo@lemmy.worldlinkfedilinkEnglisharrow-up6·18 days agoIt’s not the transcripts that are the issue here. It’s that the transcripts are being interpreted by the model to give information.
minus-squareGrimy@lemmy.worldlinkfedilinkEnglisharrow-up4·edit-218 days agoWhisper has been known to hallucinate during long moments of silence. Most of their examples though are most likely due to bad audio quality. I use whisper quite a bit and it will fumble a word here or there but never to the extent that is being shown in the article.
How can it be that bad?
I’ve used zoom’s ai transcriptions, for far less mission critical stuff, and it’s generally fine, (I still wouldn’t trust it for medical purposes)
Zoom ai transcriptions also make things up.
That’s the point. They’re hallucination engines. They pattern match and fill holes by design. It doesn’t matter if the match isn’t perfect, it will patch it over with nonsense instead.
It’s not the transcripts that are the issue here. It’s that the transcripts are being interpreted by the model to give information.
Whisper has been known to hallucinate during long moments of silence. Most of their examples though are most likely due to bad audio quality.
I use whisper quite a bit and it will fumble a word here or there but never to the extent that is being shown in the article.