Questioning Your Vision? Don’t Go Online
Searching for online advice about suspected vision problems? Doing so could put you at increased risk for vision loss, according to research from the University of California, San Diego.
Presenting at the 127th annual meeting of the American Academy of Ophthalmology, the researchers cautioned that these models are not fine-tuned to provide accurate eye health information.
To conduct their project, the researchers asked a group of ophthalmologists to assess the utility of the most popular generative artificial intelligence (AI) programs – ChatGPT, Google Bard, and Bing Chat – as an educational resource for patients with eye conditions and diseases.
The ophthalmologists compared the ability of these programs to answer common patient questions and create educational resources, as well as recommend ophthalmologists practising in the 20 largest cities in the United States. The ophthalmologists evaluated the information on a scale of one to four.
Majority of Online Responses Inaccurate
They found the majority of responses were inaccurate. Furthermore, two of the three chatbots demonstrated a significant bias against female ophthalmologists.
Google Bard scored the highest for quality and accuracy of responses to patient questions, with an average rating of 2.3 out of four. ChatGPT had the highest rating for patient educational resources, three out of four.
All three chatbots struggled when asked to recommend practising ophthalmologists or to accurately locate ophthalmologists in or near a specific city. Google Bard and Bing Chat recommended female ophthalmologists less than 2% of the time, even though 27% of the ophthalmologists in the United States are women.
“Given the substantial bias and inaccuracy demonstrated in this study, we warn against reliance on AI chatbots when seeking health-related information until improvements in algorithms are achieved and validated in the future,” said researcher Michael Oca, BS of the University of California, San Diego. “A poor recommendation from a chatbot could further delay a patient’s treatment.”
“Relying on online tools for quick advice may be tempting, but we urge the public to remember that this is not a replacement for a comprehensive eye exam with an ophthalmologist. Seeing a medical doctor for preventative exams and examining any sudden change in vision is the best way to protect your eye health,” said senior author Sandy Zhang-Nunes, MD, associate professor of clinical ophthalmology, and director of oculofacial plastic surgery at the University of Southern California.
Do you have eye health concerns?
If you have concerns about your eye health, please visit an eye specialist. If you would like to discuss permanent vision correction or cataract surgery, contact my office by email or call us on 02 8080 2180. With over 20 years of experience as a practising ophthalmologist I can provide you with personalised advice and care.
Reference.
- Beware of Dr. Chatbot: Generative AI Often Gives Unreliable, Biased Medical Advice. American Academy of Ophthalmology. November 3, 2023. Accessed November 6, 2023. www.aao.org