In healthcare, accuracy is important. So the widespread use of OpenAI’s Whisper transcription tool among medical professionals has raised alarms among experts.OpenAI’s Whisper transcription tool has fabricated false text in medical and business settings, despite warnings against it. The Associated Press interviewed more than a dozen software engineers, developers and researchers who found that the model often fabricated text that the speaker never said, a phenomenon often referred to as “fictitious” or “hallucination” in the field of artificial intelligence.
When it was released in 2022, OpenAI claimed that Whisper was close to “human-level robustness” in audio transcription accuracy. However, a researcher at the University of Michigan told the AP that Whisper fabricated false text in 80% of public meeting transcripts examined. Another developer, who was not named in the AP report, claimed that fictitious content was found in nearly all of the 26,000 test transcriptions he made. These fabrications are particularly dangerous in medical settings. More than 30,000 medical workers now use Whisper-based tools to record patient visits, despite OpenAI’s warnings against using Whisper in “high-risk areas,” according to the Associated Press. Minnesota’s Mankato Clinic and Children’s Hospital Los Angeles are among 40 health systems that use a Whisper-based AI Co-Pilot service developed by medical technology company Nabla that’s fine-tuned for medical terminology. Nabla acknowledges that Whisper can fabricate conversations, but it also reportedly deletes the original recordings for “data security reasons.” This could raise other issues, as doctors can’t verify the accuracy of the original material. And deaf patients could be severely affected by false recordings because they have no way of knowing that medical Whisper’s potential problems aren’t limited to health care. Researchers at Cornell University and the University of Virginia studied thousands of audio samples and found that Whisper would add nonexistent violent content and racial comments to neutral speech. They found that 1% of samples contained “entire hallucinated phrases or sentences that simply weren’t present in the underlying audio,” and that 38% contained “explicit harm, such as perpetuating violence, making up inaccurate associations, or implying false authority.”
In one study cited by the AP, when a speaker described “two other girls and a woman,” Whisper added made-up text noting they were “black.” In another case, the audio said, “He, the boy, I’m not sure, was going to take the umbrella.” Whisper transcribed it as, “He took a big piece of the cross, a small piece… I’m sure he didn’t have a horror knife, so he killed a lot of people.”
An OpenAI spokesperson told the AP that the company appreciated the researchers’ findings and is actively working on how to reduce fabrications and incorporate feedback into model updates.
Laptop Battery Online Shop- Replacement Laptop Battery, Cell phone Battery, Tablet Battery, Power Tools Battery Wholesale and Retailer in United Kingdom. All products are brand new with one year warranty and factory price!”
